Software As She's Developed » JSON

Rails JSON, JSON Rails

mahemoff — Fri, 13 Jul 2007 18:56:29 +0000

p style="clear:both; margin-top: 2em; "> Man, JSON has come out of nowhere! I first came across it amid the Javascript hype in early '05, now it's everywhere. Not only in your Ajax apps, but in the On-Demand Javascript APIs of the world.

Sharing an ethic of simplicity with Rails, it's not surprising these technologies have come closely together, with JSON support now baked into the Rails core. (They would have been even closer together, if only anyone had noticed JSON==YAML early on and merged the two completely.) And yet, the current implementation misses a couple of tricks.

Use of "attributes". According to Rails, a JSON object generally has just a single key, "attributes". Instead of the simple { :name=>"Pac-Man", :creator=>"Namco" }, we get (and have to give) { :attributes => { :name=>"Pac-Man", :creator=>"Namco" }}. That's not DRY. Not that it matters, but the reason for this is because ActiveRecords internally store persistable data fields in a hashmap called "attributes". The JSON serialization is therefore faithful to Rails' internal implementation, while being unfaithful to intuition. Maybe you could argue for "attributes" on the basis that you might one day have a transient attribute with the same name as a persistent attribute, in which case you'd get a clash. But Convention Over Configuration rules this argument out; you can always override JSON serialization behavior in boundary situations.
Child attributes This is probably an issue with XML as well. Basically, there is no attention paid to child attributes, i.e. JSON/XML serialization of a HABTM or has_many relationship. You want something like { :name=>"Pac-Man", :creator=>"Namco", :high_scores=>[{ :player_id=>1, :score=>99999 }, {:player_id=>2, :score=>88888 }] }, but all you get is the top-level attributes :name and :creator.

What to do?

First, I just discovered this extremely helpful library - Jsonifier, which also ships as a Rails plugin. It deals with both of the above problems. There's no "attributes" chaff and it deals with associated records, even going recursive and letting you navigate from end of a many-to-many into the other. (In my example, you could show the names and ages of Pac-Man high scorers.) You also get to whitelist or blacklist attributes matches Rails' associations syntax, i.e. nly, :except. Highly recommended, check it out.

Jsonifier, however, is pure to_json - it doesn't (yet) handle deserialization. To my knowledge, there's no Rails library that persists both parent and children in one go. There are lots of forum posts about handling multiple checkboxes and the like, and the answer usually involves overriding "children=" or "child_ids=". I want to do this automatically!

I've been working out how to extend ActiveRecord to persist automatigically. I have the following very rough code working in basic situations, although it would need to do some more validation to work in production, among other things. It works on data structures such as the recurisve one above. You must first use a before_filter to convert the incoming JSON string into a Rails structure.

PLAIN TEXT

RUBY:

class <
def recursive_create(params)
transaction do
collections_by_children = pluck_hash_collections_by_children params
new_model = self.create(params)
populate_children new_model, collections_by_children
new_model
end
end
def pluck_hash_collections_by_children(params)
hash_collections_by_children = Hash.new
params.each_pair { |name, value|
if value.class==Array
hash_collections_by_children[name] = value
params.delete(name)
end
}
hash_collections_by_children
end
def populate_children(new_model, hash_collections_by_children)
hash_collections_by_children.each_pair { |children_name, hash_collection|
new_model.send("#{children_name}").destroy_all
child_class_name = children_name.singularize.camelize
child_class = Object.const_get(child_class_name)
hash_collection.each { |child_hash|
child = child_class.create(child_hash)
new_model.send("#{children_name}") <
}
}
end
end
module ActiveRecord
class Base
def recursive_update_attributes(params)
transaction do
collections_by_children = self.class.pluck_hash_collections_by_children params
self.update_attributes(params)
self.class.populate_children self, collections_by_children
save
end
end
end
end

Ajax Programming Patterns – Podcast 1 of 4: Web Service Patterns

mahemoff — Fri, 31 Mar 2006 18:30:57 +0000

Whereupon a new podcast series begins ...

As promised, a new series of Ajax pattern podcasts. This is the first of four podcasts on the Ajax programming patterns.

In this 73 minute podcast, we look at the seven patterns of web services as they relate to Ajax clients.

RPC Service Expose web services as Remote Procedural Calls (RPCs). (Note: In the book and wiki, REST appears before RPC.) (6:55)
RESTful Service Expose web services according to RESTful principles. (13:25 )
HTML Response Have the server generate HTML snippets to be displayed in the browser. (44:45)
Semantic Response Have the server respond with abstract, semantic, data. (49:00)
Plain-Text Message Pass simple messages between server and browser in plain-text format. (56:05)
XML Message Pass messages between server and browser in XML format. (57:20)
JSON Message Pass messages between server and browser in Javascript Object Notation (JSON) format. (59:55)

Thanks for your feedback since last time. Good, bad, or ugly, it's all welcome - in the comments for this podcast or michael@mahemoff.com.

New Ajax Demos

mahemoff — Fri, 16 Sep 2005 19:37:26 +0000

Further to the previous post on new Ajax programming patterns, it's also worth noting there's a new bunch of corresponding online Ajax demos too. Being programming patterns, they tend to be "pure refactorings" rather than enhancements, thus the user experience is mostly the same - you have to look under the covers to see what's changed. The demos are mentioned in individual patterns, but here's a sampling:

The wiki has a bunch of timeout-related refactorings. These refactorings are user-visible. (They're not actually part of the programming patterns, but they were recently added.)
2 other wiki refactorings involve On-Demand Javascript . (No UI change.)
A JSON playground.
The Portal Drilldown Demo has been refactored to render with XSLT and browser templating tools. (No UI change.)
A couple of refactorings added to demonstrate REST (broken right now, probably due to a PHP config issue) and RPC on the Ajax Shop demo. (No UI change.)

As with all the demos, they're tested on FF and IE6, please let me know if anything breaks.

18 New Ajax Programming Patterns

mahemoff — Fri, 16 Sep 2005 19:09:05 +0000

I've uploaded full text for 18 new Ajax patterns, completing a first-cut draft for all Programming Patterns content, which will be one part of the book. This section bridges the gap between the very basics of Ajax - XMLHttpRequest, DOM, etc - and the high-level stuff like widgets and visual effects. For instance, how do you do design the services accessed by XMLHttpRequest? Or how do you deal with a lots of Javascript? The focus here is on traditional tech concerns such as maintainability and understandability; and less about usability.

Breaking it down:

Web Services patterns are about overall browser-server architecture - what kinds of services and what kinds of messages? It's relevant to the conversation of Jon Tiersen and others about what sort of responses are possible - XML, JS, HTML, JSON. It's also relevant to another recent conversation, on [http://www.theserverside.com/news/thread.tss?thread_id=35979 Ajax and webservices], as it covers REST and RPC.
Browser-Server Dialogue covers a mixed bag of patterns about XMLHttpRequest handling and what you can do with it, such as loading JS on demand.
DOM Population is about transforming incoming XML into DOM content, e.g. XSLT.
Performance Optimisation covers a variety of performance patterns such as caching and pre-fetching.
Breakout (can you think of a better name) is about going beyond standard Ajax constraints, namely using a server-side mediator to access external domains and introducing a plugin to go where no Ajax app is allowed to go.
Code Generation and Reuse is strongly related to some of the emerging frameworks like Echo2 and SAJAX.

This is all early days, so any feedback on the structure, names, or content will be taken into account and acknowledged too (really!). OK, here's the lot ...

Programming Patterns

Web Services

RESTful Service Expose web services according to RESTful principles.
RPC Service Expose web services as Remote Procedural Calls (RPCs).
HTML Response Have the server generate HTML snippets to be displayed in the browser.
Semantic Response Have the server respond with abstract, semantic, data.
Plain-Text Message Pass simple messages between server and browser in plain-text format.
XML Message Pass messages between server and browser in XML format.
JSON Message Pass messages between server and browser in Javascript Object Notation (JSON) format.

Browser-Server Dialogue

Call Tracking Accommodate busy user behaviour by allocating a new XMLHtpRequest object for each request. See Richard Schwartz's blog entry (http://smokey.rhs.com/web/blog/poweroftheschwartz.nsf/d6plinks/RSCZ-6CEQAR).Note: Pending some rewrite to take into account request-locking etc.
Distributed Events Keep objects synchronised with an event mechanism.
On-Demand Javascript Download Javascript as and when required, instead of downloading it all on page load.

DOM Population

XML Data Island Retain XML responses as "XML Data Islands", nodes within the HTML DOM.
Browser-Side XSLT Apply XSLT to convert XML Messages into XHTML.
Browser-Side Templating Produce browser-side templates and call on a suitable browser-side framework to render them as HTML.

Performance Optimisation

Fat Client Create a rich, browser-based, client by performing remote calls only when there is no way to achieve the same effect in the browser.
Browser-Side Cache Maintain a local cache of information.
Guesstimate Instead of grabbing real data from the server, make a guesstimate that's good enough for most user's needs.
Submission Throttling Instead of submitting upon each Javascript event, retain the data in a local buffer and upload it periodically.
Explicit Submission Instead of submitting upon each Javascript event, require the user to explicitly request it, e.g. submit upon clicking a button.
Multi-Stage Download Quickly download the page structure with a standard request, then populate it with further requests.
Predictive Fetch Anticipate likely user actions and pre-load the required data.

Breakout

Cross-Domain Mediator Allow the browser to communicate with other domains by server-based mediation.
Richer Plugin Make your application "more Ajax than Ajax" with a Richer Plugin.

Code Generation and Reuse

Ajax Stub Use an "Ajax Stub" framework which allows browser scripts to directly invoke server-side operations, without having to worry about the details of XMLHttpRequest and HTTP transfer.
Server-Side Code Generation Automatically generate HTML and Javascript from server-side code.
Cross-Browser Component Create cross-browser components, allowing programmers to reuse them without regard for browser compatibility.

Aside I was interviewed yesterday for a Japanese magazine about how I'm using the wiki. So maybe some people will be interested to know that I always write the patterns offline because I am more creative in Vim and also to avoid textarea hell. (So much for Writely, will anyone create Vimly, there's gotta be more money it than cloning MS-Word online ). I also use the Mozex or ViewSourceWith extensions to make partial edits using Vim.

This time, I decided not to upload once or twice at a time; but instead all 18 at once. There's serious overhead of introducing each new pattern to the wiki (from Vim, the sequence is: 2mins spell-check, 5-10mins markup massaging, 2 mins fixing the link and description on the homepage; sometimes exacerbated by server lag.) Uploading all at once at least allowed me to focus fully on the task and also made some aspects more efficient, particularly updating the homepage.)