Better JSON output from Rails with the Jsonifier plugin

Update: Edge Rails now does whatever Jsonifier does (and more). Check out my blog post on JSON serialization maturity in edge Rails. This means that this plugin is virtually obsolete if you’re using edge Rails.

If you’ve tried to output JSON from your Rails applications before, you’d probably have noticed how inadequate it all seems. Let’s look at the kind of output (pretty-printed for easier reading) you get from calling to_json on your typical User model:

{
  attributes: {
    id: 1,
    name: "Konata",
    awesome: true,
    created_at: "07/01/2007"
  }
}

Some of the common problems developers who write applications that speak JSON have with this are:

  • The attributes part is often unnecessary cruft when consuming ActiveRecord objects as JSON.
  • There’s no obvious way to control which attributes are shown in the JSON (e.g. you may want to hide private or lengthy attributes). The way to workaround this is either to write your own to_json instance method for your ActiveRecord model, or to use a Rails plugin (like acts_as_json) or a gem.
  • Including 2nd (or higher order) associations is tricky (as above, you can achieve this by defining your own to_json instance methods).

Under the hood of the current ActiveRecord#to_json

The to_json method for ActiveRecord objects falls back on using that defined in ActiveSupport’s JSON classes. The way it’s currently done, there are a bunch of JSON encoder classes defined for various types - when you call to_json on an ActiveRecord object it simply uses Object#to_json.

By the way, see that created_at: "07/01/2007" bit in the example above which is the JSON output for a Date attribute? That’s in MM/DD/YYYY format. Yup, 1st July 2007 not 7th January 2007 you “rest of the world you”. Ouch, but at least you can use JavaScript’s Date.parse() on it directly as detailed in this patch. And before you ask, yes, Date.parse() doesn’t understand the YYYY-MM-DD format.

Can has Rails patch?

So about 2 days ago, I wrote a Rails patch to boost ActiveRecord#to_json with an options hash. Yep much like what you can do with ActiveRecord#to_xml.

I hope it gets committed into the core some day. I mean, gosh, isn’t Rails all about Web 2.0 mashups. It even has that fancy render :json thing. I do believe JSON output needs to be treated more seriously in Rails rather than the half-hearted attempt at JSON encoding. No? Well, if the patch doesn’t get accepted I’ll still have my plugin (see below).

Anyway, let’s see some examples:

Some examples, see?

Assuming User and Post models where User has_many Posts:

david = User.find(1)
david.to_json  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007"}

No more attributes cruft!

:only and :except work the same way that as for ActiveRecord#to_xml:

david.to_json(:only => :name)                 # {name: "David"}
david.to_json(:only => [:id, :name])          # {id: 1, name: "David"}
david.to_json(:except => :created_at)         # {id: 1, name: "David", awesome: true}
david.to_json(:except => [:id, :created_at])  # {name: "David", awesome: true}

You can use the :methods options as well to include any methods on the object.

david.to_json(:methods => :permalink)
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", permalink => "1-David"}
david.to_json(:methods => [:permalink, :interestingness])
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", \
  #   permalink => "1-David", :interestingness => 666}

The :include option lets you include associations.

david.to_json(:include => :posts)
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", \
  #    posts: [{id: 1, author_id: 1, title: "Welcome to the weblog"}, \
  #            {id: 2, author_id: 1, title: "So I was thinking"}]}

:only, :except, and :methods works on the included associations as well:

david.to_json(:include => { :posts => { :only => :title } })
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", \
  #    posts: [{title: "Welcome to the weblog"}, \
  #            {title: "So I was thinking"}]}

Of course, 2nd level (and higher order) associations work too:

david.to_json(:include => { :posts => { \
                              :include => { :comments => { \
                                              :only => :body } }, \
                              :only => :title } })
  # {id: 1, name: "David", awesome: true, created_at: "07/01/2007", \
  #    posts: [{comments: [{body: "1st post!"}, {body: "OMGWTFBBQ!"}], \
  #             title: "Welcome to the weblog"}, \
  #            {comments: [{body: "Don't think too hard"}], \
  #             title: "So I was thinking"}]}

Please do give any feedback at the Rails patch for ActiveRecord#to_json or here. I wanna know what you feel about adding this functionality into the Rails core!

Jsonifier plugin

I’ve rolled the patch into a plugin: Jsonifier. The Jsonifier Trac is at http://trac.codefront.net/jsonifier/.

Subversion repository:

svn://svn.codefront.net/jsonifier/trunk

To install the plugin:

ruby script/plugin install svn://svn.codefront.net/jsonifier/trunk

A note on valid JSON

The JSON Rails spits out by default is not strictly valid JSON since the JSON specifications require keys to be double quoted. To get strictly valid JSON, add

ActiveSupport::JSON.unquote_hash_key_identifiers = false

in your environment.rb (or in the Rails initializers directory if you’re on edge). See my blog post on how to get strictly valid JSON from Rails for more info.

29 Comments & TrackBacks (Add yours)

The paper doll icon that precedes each comment is an idea conceived by Vanessa Tan.

Paper doll icon
choonkeat's Gravatar

I’d found myself wanting the :only and :except but never took the effort to write it up. Good work!

Btw, :methods sound intrusive to the underlying mechanism.. and makes me wonder if :only => [:name] uses self[:name] instead of self.name() ?

Posted by: choonkeat on July 11, 2007 5pm

Paper doll icon
Manuzhai's Gravatar

I guess I just don’t understand why Rails doesn’t generate valid JSON by default. The JSON spec is pretty clear, so why does Rails generate invalid JSON for every default installation out there?

Posted by: Manuzhai on July 11, 2007 6pm

Paper doll icon
Chu Yeow's Gravatar

Hey Choon Keat, the same thing bugged me for the longest time (I use too much JSON) but I procrastinated on the patch until recently ;)

The way I see it is, :only and :except applies only to attributes (i.e. whatever is returned by AR#attributes). :methods then allows you to output any methods you define (for computations, etc.).

Manuzhai: Yeah I feel the same way too. Open a ticket! (Maybe I will.)

Posted by: Chu Yeow on July 11, 2007 7pm

Paper doll icon
Michael Mahemoff's Gravatar

Yes, it’s not good how Rails (de-serializes) using “attributes” - definitely not the plain, simple, and DRY one becomes accustomed to in Rails.

Posted by: Michael Mahemoff on July 13, 2007 6pm

Paper doll icon
Software As She’s Developed » Blog Archive » Rails JSON, JSON Rails's Gravatar

[…] I just discovered this extremely helpful library - Jsonifier, which also ships as a Rails plugin. It deals with both of the above problems. There’s no […]

Posted by: Software As She’s Developed » Blog Archive » Rails JSON, JSON Rails on July 14, 2007 2am

Paper doll icon
Ryan Hanks's Gravatar

Hi Chu,

I like what you’ve done here, and agree it’d be nice to have this functionality in core. One suggestion for making that happen: support collections of AR objects. I want to be able to:

User.find(:all).to_json(:include => :posts)

How do we get that? XmlSerialization makes use of to_xml methods defined in ActiveSupport::CoreExtensions to be able to do it.

The more I think about it, the more it’s seems odd that the JSON encoding stuff ended up in ActiveSupport::JSON in the first place, as opposed to in ActiveSupport::CoreExtensions::*::Conversions. For example, ActiveSupport::CoreExtensions::Array::Conversions seems like a more appropriate place to implement a to_json method for arrays that optionally takes a hash argument, but reasons for that decision could easily be beyond me.

I might mess around with this some more. Shoot me an email if you’d like some help working on this.

Ryan Hanks
ryandhanks is my google mail username

Posted by: Ryan Hanks on July 27, 2007 4am

Paper doll icon
namxam's Gravatar

Well it almost works… but i have this one model where it always raises an CircularReferenceError when including the plugin. Very strange!

Posted by: namxam on August 8, 2007 1am

Paper doll icon
Alex Le's Gravatar

Works as a charm for me. Thanks!
( this feature would be cool to be included in the rails core)

Posted by: Alex Le on August 16, 2007 1pm

Paper doll icon
Pierre's Gravatar

Can’t get this to work:(
I get a strange “Wrong number of arguments (1 for 0)” when i try to use any of the options.

Works fine besides that… no “attributes” in the string’n’stuff;)

Can anyone help me?

Posted by: Pierre on August 17, 2007 1am

Paper doll icon
Vinh Tran's Gravatar

Jsonifier options seem to work only with ActiveRecord, calling to_json on an array or hash seem to cause the “wrong number of argument…” error

Posted by: Vinh Tran on August 20, 2007 9pm

Paper doll icon
Brandon Beacher's Gravatar

This gets me past the wrong number of arguments error:


format.json { render(:json => @customers.collect { |customer| customer.to_json(:only => :name) } ) }

Posted by: Brandon Beacher on August 20, 2007 9pm

Paper doll icon
Alex Egg's Gravatar

Hi,

Why does DateTime.now.to_json throw a ActiveSupport::JSON::CircularReferenceError exception?

Then if you call to_json on type Time it returns {}

How can I use to_json with any type of date?

Posted by: Alex Egg on August 24, 2007 4am

Paper doll icon
Chu Yeow's Gravatar

Rails 1.2.3 and earlier do not encode Date, Time and DateTime? fields into JSON properly (they become an empty hash when to_json is called on them). Naxnam has reported a similar error with DateTimes over here: http://trac.codefront.net/jsonifier/ticket/3 If you can provide me enough information to replicate it that’d be great!

Also, Vinh Tran is correct in saying that Jsonifier only works for ActiveRecord objects since that’s I’ve only overridden the to_json method for ActiveRecord::Base. Work is under way to extend options support to all other Ruby types though (if anyone wants to help to speed up the process do drop me an email!)

Posted by: Chu Yeow on August 24, 2007 10am

Paper doll icon
Alex Egg's Gravatar

I have this structure

>> pp c
#”test”,
“thumbnail”=>nil,
“body”=>”",
“upload_time”=>nil,
“notes”=>nil,
“subject”=>”",
“id”=>”15″,
“billing_code”=>nil,
“source_video_clip”=>nil,
“category_id”=>”8″,
“valid_after”=>nil,
“ad_id_source”=>nil,
“birth”=>”2007-08-23 23:40:37″},
@status=”Ready”>

How can I include @status in the json string when I call:

c.to_json

???

c.to_json(:include => @status) doesn’t work.

Any idea?

Posted by: Alex Egg on August 24, 2007 3pm

Paper doll icon
Alex Egg's Gravatar

seems like my brackets were stripped from my previous post, try this:

#”test”, “thumbnail”=>nil, “body”=>”", “upload_time”=>nil, “notes”=>nil, “subject”=>”", “id”=>”15″, “billing_code”=>nil, “source_video_clip”=>nil, “category_id”=>”8″, “valid_after”=>nil, “ad_id_source”=>nil, “birth”=>”2007-08-23 23:40:37″}>

Posted by: Alex Egg on August 24, 2007 3pm

Paper doll icon
Alex Egg's Gravatar

haha, here’s one more try with html escaped:

#<ContentCampaign:0×363b8a4 @status=\"Ready\", @attributes={\"name\"=>\"test\", \"thumbnail\"=>nil, \"body\"=>\"\", \"upload_time\"=>nil, \"notes\"=>nil, \"subject\"=>\"\", \"id\"=>\"15\", \"billing_code\"=>nil, \"source_video_clip\"=>nil, \"category_id\"=>\"8\", \"valid_after\"=>nil, \"ad_id_source\"=>nil, \"birth\"=>\"2007-08-23 23:40:37\"}>

Posted by: Alex Egg on August 24, 2007 3pm

Paper doll icon
Chu Yeow's Gravatar

You can output the status attribute if you provide a reader method to it like so:

class ContentCampaign < < ActiveRecord::Base
  attr_reader :status
end

And then pass an additional :methods option to to_json:

my_campaign.to_json(:methods => :status)

Posted by: Chu Yeow on August 24, 2007 3pm

Paper doll icon
Vinh Tran's Gravatar

Can I also suggest that when we do something like

customer.to_json(:include=>[resources….

if the resources is empty we should get a empty array instead of nothing

Thanks

Posted by: Vinh Tran on August 26, 2007 10am

Paper doll icon
?????Ruby Library?Rails Plugin's Gravatar

[…] Jsonifier????to_json?????????????????jsonifier??????encoder.rb???Time????to_json???http://blog.codefront.net/2007/07/11/better-json-output-from-rails-with-the-jsonifier-plugin/ […]

Posted by: ?????Ruby Library?Rails Plugin on August 29, 2007 4pm

Paper doll icon
Maximiliano's Gravatar

I changed this plugin to has to_json method accessible in Arrays too.

I just do like this:

class Array
def to_json(options = {})
self.each {|o| o.to_json(options) }
end
end

// and in my model:
MyModel.find(:all).to_json()

tonight I’ll send for you a svn patch.

Posted by: Maximiliano on August 31, 2007 1am

Paper doll icon
Alex Egg's Gravatar

“Rails 1.2.3 and earlier do not encode Date, Time and DateTime? fields into JSON properly (they become an empty hash when to_json is called on them). Naxnam has reported a similar error with DateTimes over here: http://trac.codefront.net/jsonifier/ticket/3 If you can provide me enough information to replicate it that’d be great!”

So it’s simply not possible to get any db datetimes into json?

Posted by: Alex Egg on September 20, 2007 11am

Paper doll icon
Chu Yeow's Gravatar

You guys may be interested in a recent commit to Rails trunk that adds ActiveRecord::Base#to_json. Awesome news, even if it does mean that this plugin is now obsolete (unless you want to use the :include option, which the built-in AR#to_json doesn’t support).

Posted by: Chu Yeow on September 21, 2007 6pm

Paper doll icon
Alex Egg's Gravatar

@chu

How’s this any different than the current JSON support in rails?

Posted by: Alex Egg on September 24, 2007 3am

Paper doll icon
Chu Yeow's Gravatar

Well, it doesn’t print out “attributes” anymore, and it supports the :methods option. I’m sure someone will patch it to support :include for adding fields from associations as well!

Posted by: Chu Yeow on September 24, 2007 10am

Paper doll icon
Tim's Gravatar

I also get the “wrong number of arguments (1 of 0)” error when I try to use any arguments. I’m stuck, any help would be much appreciated.

Posted by: Tim on September 29, 2007 5pm

Paper doll icon
alex's Gravatar

I’m confused…. So, before I add jsonifier. I get output with the ‘attributes’ tag. I add jsonifier and now I get an error that “object references iitself” from the same controller def that just worked. No other changes. All the controller does is:

@projects = Project.find(:all)
render :text => @projects.to_json

Posted by: alex on October 4, 2007 2am

Paper doll icon
Software As She’s Developed » Blog Archive » Rails JSON, JSON Rails's Gravatar

[…] I just discovered this extremely helpful library - Jsonifier, which also ships as a Rails plugin. It deals with both of the above problems. There’s no […]

Posted by: Software As She’s Developed » Blog Archive » Rails JSON, JSON Rails on February 24, 2008 3am

Paper doll icon
brian's Gravatar

I want to be able to provide a link in a page via a third party application that will call back to my site and post data. Javascript gets caught in the cross browser issue. I see examples where Json can pull data back via cross-browser, but how do you pull page a link, that could POST data back to the server site if clicked? Are their any good examples of doing this?

Posted by: brian on March 16, 2008 2am

Paper doll icon
fb's Gravatar

I also get the “wrong number of arguments (1 of 0)” error when I try to use any arguments. This happens if I use rails 1.8.6. But everything is fine when I use rails 2.0.1. Can somebody help me please? I need to make this work under 1.8.6. Thanks!

Posted by: fb on March 28, 2008 10am

You can subscribe to the RSS feed for comments on this post.

Post a comment

(required)

(required, but never displayed)


You can format your comments using XHTML. Your email address will not be displayed or used for nefarious purposes.

Only following tags are allowed:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>