Better JSON output from Rails with the Jsonifier plugin

In: JavaScript|Open Source|Ruby|Ruby on Rails|Web development

11 Jul 2007

Update: Edge Rails now does whatever Jsonifier does (and more). Check out my blog post on JSON serialization maturity in edge Rails. This means that this plugin is virtually obsolete if you’re using edge Rails.

If you’ve tried to output JSON from your Rails applications before, you’d probably have noticed how inadequate it all seems. Let’s look at the kind of output (pretty-printed for easier reading) you get from calling to_json on your typical User model:

{
  attributes: {
    id: 1,
    name: "Konata",
    awesome: true,
    created_at: "07/01/2007"
  }
}

Some of the common problems developers who write applications that speak JSON have with this are:

  • The attributes part is often unnecessary cruft when consuming ActiveRecord objects as JSON.
  • There’s no obvious way to control which attributes are shown in the JSON (e.g. you may want to hide private or lengthy attributes). The way to workaround this is either to write your own to_json instance method for your ActiveRecord model, or to use a Rails plugin (like acts_as_json) or a gem.
  • Including 2nd (or higher order) associations is tricky (as above, you can achieve this by defining your own to_json instance methods).

Under the hood of the current ActiveRecord#to_json

The to_json method for ActiveRecord objects falls back on using that defined in ActiveSupport’s JSON classes. The way it’s currently done, there are a bunch of JSON encoder classes defined for various types – when you call to_json on an ActiveRecord object it simply uses Object#to_json.

By the way, see that created_at: "07/01/2007" bit in the example above which is the JSON output for a Date attribute? That’s in MM/DD/YYYY format. Yup, 1st July 2007 not 7th January 2007 you “rest of the world you”. Ouch, but at least you can use JavaScript’s Date.parse() on it directly as detailed in this patch. And before you ask, yes, Date.parse() doesn’t understand the YYYY-MM-DD format.

Can has Rails patch?

So about 2 days ago, I wrote a Rails patch to boost ActiveRecord#to_json with an options hash. Yep much like what you can do with ActiveRecord#to_xml.

I hope it gets committed into the core some day. I mean, gosh, isn’t Rails all about Web 2.0 mashups. It even has that fancy render :json thing. I do believe JSON output needs to be treated more seriously in Rails rather than the half-hearted attempt at JSON encoding. No? Well, if the patch doesn’t get accepted I’ll still have my plugin (see below).

Anyway, let’s see some examples:

Some examples, see?

Assuming User and Post models where User has_many Posts:

david = User.find(1)
david.to_json  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007"}

No more attributes cruft!

:only and :except work the same way that as for ActiveRecord#to_xml:

david.to_json(:only => :name)                 # {name: "David"}
david.to_json(:only => [:id, :name])          # {id: 1, name: "David"}
david.to_json(:except => :created_at)         # {id: 1, name: "David", awesome: true}
david.to_json(:except => [:id, :created_at])  # {name: "David", awesome: true}

You can use the :methods options as well to include any methods on the object.

david.to_json(:methods => :permalink)
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", permalink => "1-David"}
david.to_json(:methods => [:permalink, :interestingness])
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", \
  #   permalink => "1-David", :interestingness => 666}

The :include option lets you include associations.

david.to_json(:include => :posts)
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", \
  #    posts: [{id: 1, author_id: 1, title: "Welcome to the weblog"}, \
  #            {id: 2, author_id: 1, title: "So I was thinking"}]}

:only, :except, and :methods works on the included associations as well:

david.to_json(:include => { :posts => { :only => :title } })
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", \
  #    posts: [{title: "Welcome to the weblog"}, \
  #            {title: "So I was thinking"}]}

Of course, 2nd level (and higher order) associations work too:

david.to_json(:include => { :posts => { \
                              :include => { :comments => { \
                                              :only => :body } }, \
                              :only => :title } })
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", \
  #    posts: [{comments: [{body: "1st post!"}, {body: "OMGWTFBBQ!"}], \
  #             title: "Welcome to the weblog"}, \
  #            {comments: [{body: "Don't think too hard"}], \
  #             title: "So I was thinking"}]}

Please do give any feedback at the Rails patch for ActiveRecord#to_json or here. I wanna know what you feel about adding this functionality into the Rails core!

Jsonifier plugin

I’ve rolled the patch into a plugin: Jsonifier. The Jsonifier Trac is at http://trac.codefront.net/jsonifier/.

Git repository:

git://github.com/chuyeow/jsonifier.git

Github project page:

http://github.com/chuyeow/jsonifier/

To install the plugin:

ruby script/plugin install git://github.com/chuyeow/jsonifier.git

or git clone the repository manually into the vendor/plugins directory.

A note on valid JSON

The JSON Rails spits out by default is not strictly valid JSON since the JSON specifications require keys to be double quoted. To get strictly valid JSON, add

ActiveSupport::JSON.unquote_hash_key_identifiers = false

in your environment.rb (or in the Rails initializers directory if you're on edge). See my blog post on how to get strictly valid JSON from Rails for more info.

30 Responses to Better JSON output from Rails with the Jsonifier plugin

Avatar

choonkeat

July 11th, 2007 at 5pm

I’d found myself wanting the :only and :except but never took the effort to write it up. Good work!

Btw, :methods sound intrusive to the underlying mechanism.. and makes me wonder if :only => [:name] uses self[:name] instead of self.name() ?

Avatar

Manuzhai

July 11th, 2007 at 6pm

I guess I just don’t understand why Rails doesn’t generate valid JSON by default. The JSON spec is pretty clear, so why does Rails generate invalid JSON for every default installation out there?

Avatar

Chu Yeow

July 11th, 2007 at 7pm

Hey Choon Keat, the same thing bugged me for the longest time (I use too much JSON) but I procrastinated on the patch until recently ;)

The way I see it is, :only and :except applies only to attributes (i.e. whatever is returned by AR#attributes). :methods then allows you to output any methods you define (for computations, etc.).

Manuzhai: Yeah I feel the same way too. Open a ticket! (Maybe I will.)

Avatar

Michael Mahemoff

July 13th, 2007 at 6pm

Yes, it’s not good how Rails (de-serializes) using “attributes” – definitely not the plain, simple, and DRY one becomes accustomed to in Rails.

Avatar

Software As She’s Developed » Blog Archive » Rails JSON, JSON Rails

July 14th, 2007 at 2am

[…] I just discovered this extremely helpful library – Jsonifier, which also ships as a Rails plugin. It deals with both of the above problems. There’s no […]

Avatar

Ryan Hanks

July 27th, 2007 at 4am

Hi Chu,

I like what you’ve done here, and agree it’d be nice to have this functionality in core. One suggestion for making that happen: support collections of AR objects. I want to be able to:

User.find(:all).to_json(:include => :posts)

How do we get that? XmlSerialization makes use of to_xml methods defined in ActiveSupport::CoreExtensions to be able to do it.

The more I think about it, the more it’s seems odd that the JSON encoding stuff ended up in ActiveSupport::JSON in the first place, as opposed to in ActiveSupport::CoreExtensions::*::Conversions. For example, ActiveSupport::CoreExtensions::Array::Conversions seems like a more appropriate place to implement a to_json method for arrays that optionally takes a hash argument, but reasons for that decision could easily be beyond me.

I might mess around with this some more. Shoot me an email if you’d like some help working on this.

Ryan Hanks
ryandhanks is my google mail username

Avatar

namxam

August 8th, 2007 at 1am

Well it almost works… but i have this one model where it always raises an CircularReferenceError when including the plugin. Very strange!

Avatar

Alex Le

August 16th, 2007 at 1pm

Works as a charm for me. Thanks!
( this feature would be cool to be included in the rails core)

Avatar

Pierre

August 17th, 2007 at 1am

Can’t get this to work:(
I get a strange “Wrong number of arguments (1 for 0)” when i try to use any of the options.

Works fine besides that… no “attributes” in the string’n’stuff;)

Can anyone help me?

Avatar

Vinh Tran

August 20th, 2007 at 9pm

Jsonifier options seem to work only with ActiveRecord, calling to_json on an array or hash seem to cause the “wrong number of argument…” error

Avatar

Brandon Beacher

August 20th, 2007 at 9pm

This gets me past the wrong number of arguments error:


format.json { render(:json => @customers.collect { |customer| customer.to_json(:only => :name) } ) }

Avatar

Alex Egg

August 24th, 2007 at 4am

Hi,

Why does DateTime.now.to_json throw a ActiveSupport::JSON::CircularReferenceError exception?

Then if you call to_json on type Time it returns {}

How can I use to_json with any type of date?

Avatar

Chu Yeow

August 24th, 2007 at 10am

Rails 1.2.3 and earlier do not encode Date, Time and DateTime? fields into JSON properly (they become an empty hash when to_json is called on them). Naxnam has reported a similar error with DateTimes over here: http://trac.codefront.net/jsonifier/ticket/3 If you can provide me enough information to replicate it that’d be great!

Also, Vinh Tran is correct in saying that Jsonifier only works for ActiveRecord objects since that’s I’ve only overridden the to_json method for ActiveRecord::Base. Work is under way to extend options support to all other Ruby types though (if anyone wants to help to speed up the process do drop me an email!)

Avatar

Alex Egg

August 24th, 2007 at 3pm

I have this structure

>> pp c
#”test”,
“thumbnail”=>nil,
“body”=>””,
“upload_time”=>nil,
“notes”=>nil,
“subject”=>””,
“id”=>”15”,
“billing_code”=>nil,
“source_video_clip”=>nil,
“category_id”=>”8”,
“valid_after”=>nil,
“ad_id_source”=>nil,
“birth”=>”2007-08-23 23:40:37″},
@status=”Ready”>

How can I include @status in the json string when I call:

c.to_json

???

c.to_json(:include => @status) doesn’t work.

Any idea?

Avatar

Alex Egg

August 24th, 2007 at 3pm

seems like my brackets were stripped from my previous post, try this:

#”test”, “thumbnail”=>nil, “body”=>””, “upload_time”=>nil, “notes”=>nil, “subject”=>””, “id”=>”15”, “billing_code”=>nil, “source_video_clip”=>nil, “category_id”=>”8”, “valid_after”=>nil, “ad_id_source”=>nil, “birth”=>”2007-08-23 23:40:37”}>

Avatar

Alex Egg

August 24th, 2007 at 3pm

haha, here’s one more try with html escaped:

#<ContentCampaign:0x363b8a4 @status=\"Ready\", @attributes={\"name\"=>\"test\", \"thumbnail\"=>nil, \"body\"=>\"\", \"upload_time\"=>nil, \"notes\"=>nil, \"subject\"=>\"\", \"id\"=>\"15\", \"billing_code\"=>nil, \"source_video_clip\"=>nil, \"category_id\"=>\"8\", \"valid_after\"=>nil, \"ad_id_source\"=>nil, \"birth\"=>\"2007-08-23 23:40:37\"}>

Avatar

Chu Yeow

August 24th, 2007 at 3pm

You can output the status attribute if you provide a reader method to it like so:

class ContentCampaign < < ActiveRecord::Base
  attr_reader :status
end

And then pass an additional :methods option to to_json:

my_campaign.to_json(:methods => :status)

Avatar

Vinh Tran

August 26th, 2007 at 10am

Can I also suggest that when we do something like

customer.to_json(:include=>[resources….

if the resources is empty we should get a empty array instead of nothing

Thanks

Avatar

?????Ruby Library?Rails Plugin

August 29th, 2007 at 4pm

[…] Jsonifier????to_json?????????????????jsonifier??????encoder.rb???Time????to_json???http://blog.codefront.net/2007/07/11/better-json-output-from-rails-with-the-jsonifier-plugin/ […]

Avatar

Maximiliano

August 31st, 2007 at 1am

I changed this plugin to has to_json method accessible in Arrays too.

I just do like this:

class Array
def to_json(options = {})
self.each {|o| o.to_json(options) }
end
end

// and in my model:
MyModel.find(:all).to_json()

tonight I’ll send for you a svn patch.

Avatar

Alex Egg

September 20th, 2007 at 11am

“Rails 1.2.3 and earlier do not encode Date, Time and DateTime? fields into JSON properly (they become an empty hash when to_json is called on them). Naxnam has reported a similar error with DateTimes over here: http://trac.codefront.net/jsonifier/ticket/3 If you can provide me enough information to replicate it that’d be great!”

So it’s simply not possible to get any db datetimes into json?

Avatar

Chu Yeow

September 21st, 2007 at 6pm

You guys may be interested in a recent commit to Rails trunk that adds ActiveRecord::Base#to_json. Awesome news, even if it does mean that this plugin is now obsolete (unless you want to use the :include option, which the built-in AR#to_json doesn’t support).

Avatar

Alex Egg

September 24th, 2007 at 3am

@chu

How’s this any different than the current JSON support in rails?

Avatar

Chu Yeow

September 24th, 2007 at 10am

Well, it doesn’t print out “attributes” anymore, and it supports the :methods option. I’m sure someone will patch it to support :include for adding fields from associations as well!

Avatar

Tim

September 29th, 2007 at 5pm

I also get the “wrong number of arguments (1 of 0)” error when I try to use any arguments. I’m stuck, any help would be much appreciated.

Avatar

alex

October 4th, 2007 at 2am

I’m confused…. So, before I add jsonifier. I get output with the ‘attributes’ tag. I add jsonifier and now I get an error that “object references iitself” from the same controller def that just worked. No other changes. All the controller does is:

@projects = Project.find(:all)
render :text => @projects.to_json

Avatar

Software As She’s Developed » Blog Archive » Rails JSON, JSON Rails

February 24th, 2008 at 3am

[…] I just discovered this extremely helpful library – Jsonifier, which also ships as a Rails plugin. It deals with both of the above problems. There’s no […]

Avatar

brian

March 16th, 2008 at 2am

I want to be able to provide a link in a page via a third party application that will call back to my site and post data. Javascript gets caught in the cross browser issue. I see examples where Json can pull data back via cross-browser, but how do you pull page a link, that could POST data back to the server site if clicked? Are their any good examples of doing this?

Avatar

fb

March 28th, 2008 at 10am

I also get the “wrong number of arguments (1 of 0)” error when I try to use any arguments. This happens if I use rails 1.8.6. But everything is fine when I use rails 2.0.1. Can somebody help me please? I need to make this work under 1.8.6. Thanks!

Avatar

Rendering plists from your Rails app - redemption in a blog

June 14th, 2010 at 11am

[…] was kind enough to let me know he “stole” my old Jsonifier plugin (github) code in […]