Advanced Feature Wishlist for Ajax Frameworks

Looking at the best practice/process patterns gave me some ideas for Ajax frameworks. Here’s a few thoughts. I know some frameworks already do some of these things, though most don’t, and none do all of them.

Logging frameworks should provide a sensible default, ie log to a div, but allow for more flexiblity on the logging strategy, as in log4j. For example, it would be nice to hook into a local storage solution like AMASS. I know we can all “spy on users”, but a local storage solution lets you amass lots more data, helps solve some privacy issues, and will survive network problems … could even help restore lots data. I’m obviously thinking more of intranets than public internet apps.

Mock object frameworks should, um ,exist. As I mentioned last week, there’s definitely a good case for a JS mock object library like JMock.

Web Remoting frameworks should:

  • Provide a high-level, technology-free, interface, and then implement it in as many ways necessary to support the greatest amount of browsers possible. Gloves off – IFrames, Images, Stylesheets … whatever works.
  • Make access to 3rd-party domains virtually transparent. e.g. CPaint does this by allowing a proxy option, which points to a proxy running on your server. That aside, calls are same as normal.
  • Support simulation. Let a caller prep the remoting framework with the response it should provide to a given query. Testing in that way, I could see very little reason to ever go back to the real server if you could do this.
  • Support logging. Rather flabbergasted that I couldn’t find a framework that supports logging of remote calls. The new Network Sniffing pattern describes other tricks for logging browser-server traffic, but the JS remoting frameworks ought to take care of it.
  • Support mocking. In addition to prepping with the response, the caller should be able to say “throw an exception if you don’t get this query”. Note that this wouldn’t need to be the responsibility of the remoting framework if there was a general-purpose mocking framework available.

8 New Ajax Patterns (Diagnosis and Testing)

Cool! The Best Practices/Processes Patterns are now complete. They are the final eight Ajax Patterns for now – “final” in the sense of “the list is not yet finalised”. The patterns had been sitting there unattended for about four months now.

More details on the new patterns later, but here’s a quick summary …

First, there’s a new demo – the Ajax Patterns Reader – the best version to try is at The reader grabs the content and presents them Ajax-style. You actually queue up patterns in a playlist and click “Next” to “play” them. Yeah, a bit contrived, but it helped illustrate quite a few patterns! If I have time, I’d like to enhance it into a proper reader, and also offer an easy interface to leave feedback, which would be automatically appended to the wiki’s Discussion tab for that pattern.

BTW This further refactoring of the Ajax Patterns Reader illustrates the Scriptaculous GhostTrain tool. If you haven’t seen GhostTrain, have a look – the Javascript will track your activity and build up a test case dynamically (covered in the new System Test pattern). All within the browser. I’ve been in contact with the developers (Thomas and Jon), and discovered it’s still proof-of-concept, but if they can tie it all together, it will be an excellent way to create a system/functional test.

Next, there’s four patterns on diagnosis:

And finally, four patterns on testing:
  • Simulation Service Develop the browser application against “fake” web services that simulate the actual services used in production.
  • Browser-Side Test Create automated tests of browser-side Javascript components.
  • Service Test Build up automated tests of web services, using HTTP clients to interact with the server as the browser normally would.
  • System Test Build automated tests to simulate user behaviour and verify the results.

HTTP Streaming: An Alternative to Polling the Server

If Ajax apps are to be rich, there must be a way for the server to pass new information to the browser. For example, new stock quotes or an instant message someone else just sent you. But the browser’s not a server, so the server can’t initiate an HTTP connection to alert the browser. The standard way to deal with this dilemma is Periodic Refresh, i.e. having the browser poll the server every few seconds. But that’s not the only solution.

The recent podcast on Web Remoting includes a discussion of the HTTP Streaming pattern. By continuing to stream information from the server, without closing the connection, you can keep the browser content fresh. I wasn’t aware that it was being used much on the public web, since it can be costly, but I recently discovered JotLive (which is only semi-public since it requires registration) is indeed using it. Do you know any other examples?’s interview with Abe Fettig of JotLive:

How do you handle the “live” part? Polling?

We’re using a (very slightly modified) version of LivePage, which Donovan Preston wrote as part of Nevow, a Python library for building web applications using the Twisted networking framework (which I just wrote a book on: Twisted Network Programming Essentials). LivePage doesn’t use polling. Instead, it uses a clever technique where each browser keeps an open XMLHTTP request to the server at all times, opening a new connection each time the old one closes. That way every client viewing the page is constantly waiting for a response from the server. When the server wants to send a message to a client, it uses the currently open request. So there’s no waiting.

A few (edited) extracts from the HTTP Streaming pattern:

Alternative Pattern: Periodic Refresh is an obvious alternative to HTTP Streaming. It fakes a long-lived connection by frequently polling the server. Generally, Periodic Refresh is more scaleable and easier to implement in a portable, robust, manner. However, HTTP Streaming can deliver more timely data, so consider it for systems, such as intranets, where there are less simultaneous users, you have some control over the infrastructure, and each connection carries a relatively high value.

Refactoring Illustration: The Basic Wiki Demo, which uses Periodic Refresh, has been refactored to use [](HTTP Streaming).

Stream server data in the response of a long-lived HTTP connection. Most web services do some processing, send back a response, and immediately exit. But in this pattern, they keep the connection open by running a long loop. The server script uses event registration or some other technique to detect any state changes. As soon as a state change occurs, it pushes new data to the outgoing stream and flushes it, but doesn’t actually close it. Meanwhile, the browser must ensure the user-interface reflects the new data. This pattern discusses a couple of techniques for Streaming HTTP, which I refer to as “Page Streaming” and “Service Streaming”.
“Page Streaming” involves streaming the original page response. Here, the server immediately outputs an initial page and flushes the stream, but keeps it open. It then proceeds to alter it over time by outputting embedded scripts that manipulate the DOM. The browser’s still officially writing the initial page out, so when it encounters a complete <script> tag, it will execute the script immediately. A simple demo is available at
…(illustration and problems)…
“Service Streaming” is a step towards solving these problems, though it doesn’t work on all browsers. The technique relies on XMLHttpRequest Call (or a similar remoting technology like IFrame_Call). This time, it’s an XMLHttpRequest connection that’s long-lived, instead of the initial page load. There’s more flexibility regarding length and frequency of connections. You could load the page normally, then start streaming for thirty seconds when the user clicks a button. Or you could start streaming once the page is loaded, and keep resetting the connection every thirty seconds. Having a range of options helps immeasurably, given that HTTP Streaming is constrained by the capabilities of the server, the browsers, and the network.

Experiments suggest that the Page Streaming technique does work on both IE and Firefox ([1]), but Service Streaming only works on Firefox, whether XMLHTTPRequest ([2]) or IFrame ([3]) is used. In both cases, IE suppresses the response until its complete. You could claim that’s either a bug or a feature; but either way, it works against HTTP Streaming.

Donovan Preston explains the technique he uses in Nevow, which overcomes this problem:

When the main page loads, an XHR (XMLHttpRequest) makes an “output conduit” request. If the server has collected any events between the main page rendering and the output conduit request rendering, it sends them immediately. If it has not, it waits until an event arrives and sends it over the output conduit. Any event from the server to the client causes the server to close the output conduit request. Any time the server closes the output conduit request, the client immediately reopens a new one. If the server hasn’t received an event for the client in 30 seconds, it sends a noop (the javascript “null”) and closes the request.

Basics of Ajax 2 of 3: Web Remoting (XMLHttpRequest etc) (Podcast)

Ajax Basics 2 of 3

This is the second of three podcasts on the basic Ajax patterns.

  • Podcast 1: Display Patterns and the DOM.
  • Podcast 2: Web Remoting – XMLHttpRequest, IFrame Call, HTTP Streaming.
  • Podcast 3: Dynamic Behaviour – Events and Timing.

Podcast 2: Web Remoting (XMLHttpRequest, IFrame, HTTP Streaming)

Click to download the Podcast. You can also subscribe to the
feed if you want future podcasts automatically downloaded - check out the
podcast FAQ at

This 75 minute podcast covers web remoting concepts and the following specific patterns:

  • XMLHttpRequest Call Use XMLHttpRequest objects for browser-server communication. (05:00)
  • IFrame Call Use IFrames for browser-server communication. (31:45)
  • HTTP Streaming Stream server data in the response of a long-lived HTTP connection. (47:00)

Please ignore this bit, I’m claiming My Odeo Channel (odeo/bdd768d9acefcad9)