WebWorkers, Cross-Domain JSONP, and CoffeeScript

Today’s buzzword gotcha is brought to you by WebWorkers, Cross-Domain JSONP, and CoffeeScript. TL;DR: Always use “self” for WebWorker “globals”. And a library.

My Twitter app’s locking the main thread a bit, so thought I could quickly push polling functionality into a WebWorker. Not so fast …

Firstly, WebWorkers don’t do JSONP the normal way. You can’t inject a script because there’s no DOM. This means you can’t even use standalone JSONP libraries, let alone jQuery (which doesn’t even run in a WebWorker, as it assumes DOM presence, and would be a waste of bandwidth anyway).

Fortunately, we can use importScripts(). Conveniently enough, importScripts() can be used anytime and accepts any old string. So the API would actually be useful in the main thread too, but I digress …

So in Plain Old JavaScript, you would do:

javascript

  1. function go() { doSomething(); }
  2. importScripts("api.twitter.com?callback=go");

Works just fine. But this won’t work in the CoffeeScript equivalent, i.e.:

  1. go = -> doSomething()  ### NOT CALLED
  2. importScripts "api.twitter.com?callback=go"

In fact, Chrome gives a mysterious error “Uncaught undefined”, while Firefox cuts to the chase: “go is not defined”.

The reason is that the CoffeeScript compiler wraps the entire thing in a closure. This is tremendously useful default behaviour, as you don’t have to worry about adding new globals, but can cause problems in situations like this. If the script comes back from Twitter and calls “go()”, where does that actually end up? With the main thread, a CoffeeScript can explicitly bust properties out of the closure using the global namespace, i.e. “window.foo” (or “global.foo” in the case of a server-side Node script). In fact, there’s a solution to this with WebWorkers too. have their own global identity, “self”.

  1. self.go = -> doSomething()  ### FTFY
  2. importScripts "api.twitter.com?callback=go"

Works.

Be sure to use “self” for other WebWorker “globals” too, i.e.:

  1. self.onmessage = (ev) -> doSomething()

Here’s a tiny jsonp library to support this:

So you can say:

Rails JSON, JSON Rails

<

p style=”clear:both; margin-top: 2em; “> Man, JSON has come out of nowhere! I first came across it amid the Javascript hype in early ’05, now it’s everywhere. Not only in your Ajax apps, but in the On-Demand Javascript APIs of the world.

Sharing an ethic of simplicity with Rails, it’s not surprising these technologies have come closely together, with JSON support now baked into the Rails core. (They would have been even closer together, if only anyone had noticed JSON==YAML early on and merged the two completely.) And yet, the current implementation misses a couple of tricks.

  • Use of “attributes”. According to Rails, a JSON object generally has just a single key, “attributes”. Instead of the simple { :name=>”Pac-Man”, :creator=>”Namco” }, we get (and have to give) { :attributes => { :name=>”Pac-Man”, :creator=>”Namco” }}. That’s not DRY. Not that it matters, but the reason for this is because ActiveRecords internally store persistable data fields in a hashmap called “attributes”. The JSON serialization is therefore faithful to Rails’ internal implementation, while being unfaithful to intuition. Maybe you could argue for “attributes” on the basis that you might one day have a transient attribute with the same name as a persistent attribute, in which case you’d get a clash. But Convention Over Configuration rules this argument out; you can always override JSON serialization behavior in boundary situations.
  • Child attributes This is probably an issue with XML as well. Basically, there is no attention paid to child attributes, i.e. JSON/XML serialization of a HABTM or has_many relationship. You want something like { :name=>”Pac-Man”, :creator=>”Namco”, :high_scores=>[{ :player_id=>1, :score=>99999 }, {:player_id=>2, :score=>88888 }] }, but all you get is the top-level attributes :name and :creator.

What to do?

First, I just discovered this extremely helpful library – Jsonifier, which also ships as a Rails plugin. It deals with both of the above problems. There’s no “attributes” chaff and it deals with associated records, even going recursive and letting you navigate from end of a many-to-many into the other. (In my example, you could show the names and ages of Pac-Man high scorers.) You also get to whitelist or blacklist attributes matches Rails’ associations syntax, i.e. :only, :except. Highly recommended, check it out.

Jsonifier, however, is pure to_json – it doesn’t (yet) handle deserialization. To my knowledge, there’s no Rails library that persists both parent and children in one go. There are lots of forum posts about handling multiple checkboxes and the like, and the answer usually involves overriding “children=” or “child_ids=”. I want to do this automatically!

I’ve been working out how to extend ActiveRecord to persist automatigically. I have the following very rough code working in basic situations, although it would need to do some more validation to work in production, among other things. It works on data structures such as the recurisve one above. You must first use a before_filter to convert the incoming JSON string into a Rails structure.

class << ActiveRecord::Base

def recursive_create(params) transaction do collections_by_children = pluck_hash_collections_by_children params new_model = self.create(params) populate_children new_model, collections_by_children new_model end end

def pluck_hash_collections_by_children(params) hash_collections_by_children = Hash.new params.each_pair { |name, value| if value.class==Array hash_collections_by_children[name] = value params.delete(name) end } hash_collections_by_children end

def populate_children(new_model, hash_collections_by_children) hash_collections_by_children.each_pair { |children_name, hash_collection| new_model.send(“#{children_name}”).destroy_all child_class_name = children_name.singularize.camelize child_class = Object.const_get(child_class_name) hash_collection.each { |child_hash| child = child_class.create(child_hash) new_model.send(“#{children_name}”) << child } } end

end

module ActiveRecord class Base

def recursive_update_attributes(params)
  transaction do
    collections_by_children = self.class.pluck_hash_collections_by_children params
    self.update_attributes(params)
    self.class.populate_children self, collections_by_children
    save
  end
end

end

end [/ruby]

Ajax Programming Patterns – Podcast 1 of 4: Web Service Patterns

Whereupon a new podcast series begins …

As promised, a new series of Ajax pattern podcasts. This is the first of four podcasts on the Ajax programming patterns.

In this 73 minute podcast, we look at the seven patterns of web services as they relate to Ajax clients.

Click to download the Podcast. You can also subscribe to the
feed if you want future podcasts automatically downloaded - check out the
podcast FAQ at http://podca.st.

  • RPC Service Expose web services as Remote Procedural Calls (RPCs). (Note: In the book and wiki, REST appears before RPC.) (6:55)
  • RESTful Service Expose web services according to RESTful principles. (13:25 )
  • HTML Response Have the server generate HTML snippets to be displayed in the browser. (44:45)
  • Semantic Response Have the server respond with abstract, semantic, data. (49:00)
  • Plain-Text Message Pass simple messages between server and browser in plain-text format. (56:05)
  • XML Message Pass messages between server and browser in XML format. (57:20)
  • JSON Message Pass messages between server and browser in Javascript Object Notation (JSON) format. (59:55)

Thanks for your feedback since last time. Good, bad, or ugly, it’s all welcome – in the comments for this podcast or [email protected]

New Ajax Demos

Further to the previous post on new Ajax programming patterns, it’s also worth noting there’s a new bunch of corresponding online Ajax demos too. Being programming patterns, they tend to be “pure refactorings” rather than enhancements, thus the user experience is mostly the same – you have to look under the covers to see what’s changed. The demos are mentioned in individual patterns, but here’s a sampling:

  • The wiki has a bunch of timeout-related refactorings. These refactorings are user-visible. (They’re not actually part of the programming patterns, but they were recently added.)
  • 2 other wiki refactorings involve On-Demand Javascript . (No UI change.)
  • A JSON playground.
  • The Portal Drilldown Demo has been refactored to render with XSLT and browser templating tools. (No UI change.)
  • A couple of refactorings added to demonstrate REST (broken right now, probably due to a PHP config issue) and RPC on the Ajax Shop demo. (No UI change.)

As with all the demos, they’re tested on FF and IE6, please let me know if anything breaks.

18 New Ajax Programming Patterns

I’ve uploaded full text for 18 new Ajax patterns, completing a first-cut draft for all Programming Patterns content, which will be one part of the book. This section bridges the gap between the very basics of Ajax – XMLHttpRequest, DOM, etc – and the high-level stuff like widgets and visual effects. For instance, how do you do design the services accessed by XMLHttpRequest? Or how do you deal with a lots of Javascript? The focus here is on traditional tech concerns such as maintainability and understandability; and less about usability.

Breaking it down:

  • Web Services patterns are about overall browser-server architecture – what kinds of services and what kinds of messages? It’s relevant to the conversation of Jon Tiersen and others about what sort of responses are possible – XML, JS, HTML, JSON. It’s also relevant to another recent conversation, on [http://www.theserverside.com/news/thread.tss?thread_id=35979 Ajax and webservices], as it covers REST and RPC.

  • Browser-Server Dialogue covers a mixed bag of patterns about XMLHttpRequest handling and what you can do with it, such as loading JS on demand.

  • DOM Population is about transforming incoming XML into DOM content, e.g. XSLT.

  • Performance Optimisation covers a variety of performance patterns such as caching and pre-fetching.

  • Breakout (can you think of a better name) is about going beyond standard Ajax constraints, namely using a server-side mediator to access external domains and introducing a plugin to go where no Ajax app is allowed to go.

  • Code Generation and Reuse is strongly related to some of the emerging frameworks like Echo2 and SAJAX.

This is all early days, so any feedback on the structure, names, or content will be taken into account and acknowledged too (really!). OK, here’s the lot …

Programming Patterns

Web Services

  • RESTful Service Expose web services according to RESTful principles.
  • RPC Service Expose web services as Remote Procedural Calls (RPCs).
  • HTML Response Have the server generate HTML snippets to be displayed in the browser.
  • Semantic Response Have the server respond with abstract, semantic, data.
  • Plain-Text Message Pass simple messages between server and browser in plain-text format.
  • XML Message Pass messages between server and browser in XML format.
  • JSON Message Pass messages between server and browser in Javascript Object Notation (JSON) format.

Browser-Server Dialogue

  • Call Tracking Accommodate busy user behaviour by allocating a new XMLHtpRequest object for each request. See Richard Schwartz’s blog entry (http://smokey.rhs.com/web/blog/poweroftheschwartz.nsf/d6plinks/RSCZ-6CEQAR).Note: Pending some rewrite to take into account request-locking etc.
  • Distributed Events Keep objects synchronised with an event mechanism.
  • On-Demand Javascript Download Javascript as and when required, instead of downloading it all on page load.

DOM Population

Performance Optimisation

  • Fat Client Create a rich, browser-based, client by performing remote calls only when there is no way to achieve the same effect in the browser.
  • Browser-Side Cache Maintain a local cache of information.
  • Guesstimate Instead of grabbing real data from the server, make a guesstimate that’s good enough for most user’s needs.
  • Submission Throttling Instead of submitting upon each Javascript event, retain the data in a local buffer and upload it periodically.
  • Explicit Submission Instead of submitting upon each Javascript event, require the user to explicitly request it, e.g. submit upon clicking a button.
  • Multi-Stage Download Quickly download the page structure with a standard request, then populate it with further requests.
  • Predictive Fetch Anticipate likely user actions and pre-load the required data.

Breakout

  • Cross-Domain Mediator Allow the browser to communicate with other domains by server-based mediation.
  • Richer Plugin Make your application “more Ajax than Ajax” with a Richer Plugin.

Code Generation and Reuse

  • Ajax Stub Use an “Ajax Stub” framework which allows browser scripts to directly invoke server-side operations, without having to worry about the details of XMLHttpRequest and HTTP transfer.
  • Server-Side Code Generation Automatically generate HTML and Javascript from server-side code.
  • Cross-Browser Component Create cross-browser components, allowing programmers to reuse them without regard for browser compatibility.

Aside I was interviewed yesterday for a Japanese magazine about how I’m using the wiki. So maybe some people will be interested to know that I always write the patterns offline because I am more creative in Vim and also to avoid textarea hell. (So much for Writely, will anyone create Vimly, there’s gotta be more money it than cloning MS-Word online :-D). I also use the Mozex or ViewSourceWith extensions to make partial edits using Vim.

This time, I decided not to upload once or twice at a time; but instead all 18 at once. There’s serious overhead of introducing each new pattern to the wiki (from Vim, the sequence is: 2mins spell-check, 5-10mins markup massaging, 2 mins fixing the link and description on the homepage; sometimes exacerbated by server lag.) Uploading all at once at least allowed me to focus fully on the task and also made some aspects more efficient, particularly updating the homepage.)