AjaxPatterns Tutorial: Demos are Online

Here’s a heads-up on the tutorial to be included with “Ajax Design Patterns”. I’ll let you know when the tutorial text is available, but for now, the demos are running:

Demos for the AjaxPatterns Tutorial

The first half is a fairly standard intro to Ajax, using the Foundational Technologies patterns (Display Manipulation, Web Remoting, Dynamic Behaviour).

The second half is more interesting, as it documents the Ajaxification of Ajaxagram, an anagram finder (finds all combinations of a word, no dictionary used so as to keep the business logic simple). It begins as a conventional web app First, into a basic Ajax version. Then with a number of patterns progressively applied: first, Live Search, Progress Indicator, and One-Second Spotlight (Yellow Fade Technique) (about 3 lines, 8 lines, and 3 lines of code respectively). Followed by Submission Throttling. And finally, I want to state loudly that most of the patterns don’t have to be implemented by hand, thanks to the many great Ajax libraries and frameworks available. For that reason, the final refactoring is not about one particular pattern, but is instead about introducing a library. In this case, the AjaxPatterns library.

Following the Ajaxification, I’ll give some suggestions for further enhancements and refactorings. Some people like to learn by hacking an existing app rather than building it from scratch. Taking the code base established in the final version of Ajaxagram, it should be easy to play around with many of the Ajax patterns. For example, use an XML Message instead of the custom-format Plain-Text Message that’s used. Or try a different visual effect in place of the Yellow-Fade Technique.

One thing you’ll notice is there’s only five characters due to the exponential nature of anagrams. A nice refactoring would be to use a Virtual Workspace so the anagrams are loaded gradually.

Host-Proof Authentication?

Abe Fettig’s done some important experimenting to arrive at a direct remoting technique, one which bypasses the need for a Cross-Domain Proxy and doesn’t rely on cross-domain On-Demand Javascript. Compared to the latter technique, Abe’s idea is more functional, because you get the power, expressivity, and bidirectional capability of XMLHttpRequest, as opposed to the On-Demand Javascript hack, which only allows downloading, though you could perhaps pass CGI arguments with the script request, or use one of the image/stylesheet hacks to get information in the other direction.

If we can bypass the server. then we can consider the idea of Host-Proof Authentication. It’s based on Richard Schwartz’s Host-Proof Hosting idea, where encrypted data is decrypted on the fly in the browser. In similar vein, if you needed third-party authentication, these remoting hacks are one way to keep your password away from the prying eyes of the server host. A while back, one of the internet banks (egg?) copped it for asking users to give them all their cusomter IDs, passwords, etc., so they could provide a one-stop-shop service. Maybe Host-Proof Authentication would be a better approach – if not automated, a portal could be set up to allow users to shuffle funds around within the browser.

Back here on Earth, I wouldn’t in reality use Host-Proof Authentication for a critical application – not without a lot more consideration – because there are two reality checks:

  • Host-Proof Hosting is far from perfect– Alex Russell has noted it’s vulnerable to script injection attacks. See the comments in the above links for more on that. Similar comments apply to Host-Proof Authentication.
  • All these direct remoting techniques rely on some form of co-operation with the external server. e.g. Abe’s technique requires it to explicitly set the document.domain property; On-Demand Javascript requires it to expose appropriate scripts; image remoting requires the script to recognise any variables and output an invisible pixel; etc. The external API would have to explicitly let the browser perform remote authentication.

HTTP Streaming: An Alternative to Polling the Server

If Ajax apps are to be rich, there must be a way for the server to pass new information to the browser. For example, new stock quotes or an instant message someone else just sent you. But the browser’s not a server, so the server can’t initiate an HTTP connection to alert the browser. The standard way to deal with this dilemma is Periodic Refresh, i.e. having the browser poll the server every few seconds. But that’s not the only solution.

The recent podcast on Web Remoting includes a discussion of the HTTP Streaming pattern. By continuing to stream information from the server, without closing the connection, you can keep the browser content fresh. I wasn’t aware that it was being used much on the public web, since it can be costly, but I recently discovered JotLive (which is only semi-public since it requires registration) is indeed using it. Do you know any other examples?

Ajaxian.com’s interview with Abe Fettig of JotLive:

How do you handle the “live” part? Polling?

We’re using a (very slightly modified) version of LivePage, which Donovan Preston wrote as part of Nevow, a Python library for building web applications using the Twisted networking framework (which I just wrote a book on: Twisted Network Programming Essentials). LivePage doesn’t use polling. Instead, it uses a clever technique where each browser keeps an open XMLHTTP request to the server at all times, opening a new connection each time the old one closes. That way every client viewing the page is constantly waiting for a response from the server. When the server wants to send a message to a client, it uses the currently open request. So there’s no waiting.

A few (edited) extracts from the HTTP Streaming pattern:

Alternative Pattern: Periodic Refresh is an obvious alternative to HTTP Streaming. It fakes a long-lived connection by frequently polling the server. Generally, Periodic Refresh is more scaleable and easier to implement in a portable, robust, manner. However, HTTP Streaming can deliver more timely data, so consider it for systems, such as intranets, where there are less simultaneous users, you have some control over the infrastructure, and each connection carries a relatively high value.

Refactoring Illustration: The Basic Wiki Demo, which uses Periodic Refresh, has been refactored to use [http://ajaxify.com/run/wiki/streaming](HTTP Streaming).

Stream server data in the response of a long-lived HTTP connection. Most web services do some processing, send back a response, and immediately exit. But in this pattern, they keep the connection open by running a long loop. The server script uses event registration or some other technique to detect any state changes. As soon as a state change occurs, it pushes new data to the outgoing stream and flushes it, but doesn’t actually close it. Meanwhile, the browser must ensure the user-interface reflects the new data. This pattern discusses a couple of techniques for Streaming HTTP, which I refer to as “Page Streaming” and “Service Streaming”.
“Page Streaming” involves streaming the original page response. Here, the server immediately outputs an initial page and flushes the stream, but keeps it open. It then proceeds to alter it over time by outputting embedded scripts that manipulate the DOM. The browser’s still officially writing the initial page out, so when it encounters a complete <script> tag, it will execute the script immediately. A simple demo is available at http://ajaxify.com/run/streaming/.
…(illustration and problems)…
“Service Streaming” is a step towards solving these problems, though it doesn’t work on all browsers. The technique relies on XMLHttpRequest Call (or a similar remoting technology like IFrame_Call). This time, it’s an XMLHttpRequest connection that’s long-lived, instead of the initial page load. There’s more flexibility regarding length and frequency of connections. You could load the page normally, then start streaming for thirty seconds when the user clicks a button. Or you could start streaming once the page is loaded, and keep resetting the connection every thirty seconds. Having a range of options helps immeasurably, given that HTTP Streaming is constrained by the capabilities of the server, the browsers, and the network.

Experiments suggest that the Page Streaming technique does work on both IE and Firefox ([1]), but Service Streaming only works on Firefox, whether XMLHTTPRequest ([2]) or IFrame ([3]) is used. In both cases, IE suppresses the response until its complete. You could claim that’s either a bug or a feature; but either way, it works against HTTP Streaming.

Donovan Preston explains the technique he uses in Nevow, which overcomes this problem:

When the main page loads, an XHR (XMLHttpRequest) makes an “output conduit” request. If the server has collected any events between the main page rendering and the output conduit request rendering, it sends them immediately. If it has not, it waits until an event arrives and sends it over the output conduit. Any event from the server to the client causes the server to close the output conduit request. Any time the server closes the output conduit request, the client immediately reopens a new one. If the server hasn’t received an event for the client in 30 seconds, it sends a noop (the javascript “null”) and closes the request.

Basics of Ajax 2 of 3: Web Remoting (XMLHttpRequest etc) (Podcast)

Ajax Basics 2 of 3

This is the second of three podcasts on the basic Ajax patterns.

  • Podcast 1: Display Patterns and the DOM.
  • Podcast 2: Web Remoting – XMLHttpRequest, IFrame Call, HTTP Streaming.
  • Podcast 3: Dynamic Behaviour – Events and Timing.

Podcast 2: Web Remoting (XMLHttpRequest, IFrame, HTTP Streaming)

Click to download the Podcast. You can also subscribe to the
feed if you want future podcasts automatically downloaded - check out the
podcast FAQ at http://podca.st.

This 75 minute podcast covers web remoting concepts and the following specific patterns:

  • XMLHttpRequest Call Use XMLHttpRequest objects for browser-server communication. (05:00)
  • IFrame Call Use IFrames for browser-server communication. (31:45)
  • HTTP Streaming Stream server data in the response of a long-lived HTTP connection. (47:00)

Please ignore this bit, I’m claiming My Odeo Channel (odeo/bdd768d9acefcad9)

Basics of Ajax 1 of 3: DOM and Display Manipulation (Podcast)

The Basics of Ajax: A 3-Part Podcast Series

I’m beginning to podcast about specific Ajax patterns. To start with, a three-part series on the basics of Ajax development, covering all the Foundational Technologies patterns:

  • Podcast 1: Display Patterns and the DOM.
  • Podcast 2: Web Remoting – XMLHttpRequest, IFrame Call, HTTP Streaming.
  • Podcast 3: Dynamic Behaviour – Events and Timing.

These will be useful if your familiar with basic forms and CGI-type web development, and wanting an overview on Ajax development techniques.

If you do know enough Ajax to at least write the obligatory Ajax chat application, there’s probably not much new info here. The most interesting pattern will be HTTP Streaming, covered in the next podcast.

Note that the pattern names and structure are subject to change, but the basic ideas will remain valid.

Podcast 1: Display Patterns and the DOM

Click to download the Podcast. You can also subscribe to the
feed if you want future podcasts automatically downloaded - check out the
podcast FAQ at http://podca.st.

This 42 minute podcast covers the DOM and the following patterns:

  • Display Morphing: Morph display elements by altering styles and values in the DOM, such as text and colour properties.

  • Page Rearrangement: Add, remove, move, and overlay elements by manipulating the DOM.

Credits and Production Notes

  • The podcast concludes with the world’s first HCI Rap, “We Got It”, from the multi-talented team at OK-Cancel (the website with the funniest usability cartoons around).
  • The theme, My Morning Jacket’s “One Big Holiday”, is back too.
  • For the podcasters among you, the podcast was produced on a Powerbook running the excellent Audio Hijack Pro, with Bias SoundSoap and Apple’s new AUAudioFilePlayer plugin to cue audio. For the podcasters among you, this is a nice, easy, setup: allows recording directly to MP3, real-time noise reduction, ability to cue up sound clips, tag the MP, run a script at the end (which could FTP the file somewhere). In theory, you could have a podcast up a few minutes after you’ve clicked the stop button. Rogue Amoeba, creators of Audio Hijack Pro, know how to make software that’s intuitive and seriously useful. There’s detailed instructions for podcasters in the manual and also a blog entry on podcasting with Audio Hijack Pro.

Pros and Cons of Ajax

I just updated the ajaxpatterns.org “What’s Ajax” page to include more info on Ajax Benefits, and added a new section on Downsides. Have I missed any?

Benefits of Ajax

  • Web as a Platform: The web is no longer just about websites that expose some information; it’s increasingly being used for full-blown applications. These applications demand richer styles of interaction.
  • User Frustration: The standard “click-and-wait” form-based model no longer suffices. Most people are now au fait with the basics of the web and are ready for something more.
  • Web 2.0 Zeitgeist: Ajax sits well in the “Web 2.0” umbrella, with many of the Web 2.0 poster-child projects – such as flickr and technorati – incorporating Ajaxian features (having started before the “Ajax” term came about). Web 2.0 is about new ideas and the $$$ to make them happen, and Ajax is certainly creating opportunities for new breeds of applications, as well as fodder for improving existing systems. Ajax is the “web” in web 2.0.
  • Multiple Access Points: Many people use more than one computer. People are using computers at home, at work, at school, in cafes. Laptops and portable hard drives offer one solution, but they are inconvenient, at risk of theft, and often not able to be plugged in. Hosting the data online is a more straightforward solution and accessing it from a rich web application is a natural fit.
  • Web as the only Platform Thanks to the widespread adoption of public internet access, the so-called technology gap between countries and between socioeconomic groups is closing. Many people don’t actually own a PC, but do have regular access to the web at internet cafes or schools or friends’ homes. For this diverse category of user, there’s no point installing applications and keeping their data locally. The web is their only platform.
  • Better Infrastructure: Many homes now have broadband, and server capacities have continued to improve in the past few years. Due to frequent back-and-forth between client and server, Some Ajax applications can be hungry for both bandwidth and server capacity. Furthermore, server-side storage is cheap enough for many individuals and companies to prefer holding data online, which makes it feasible to access via a web application.
  • Productive Development: For developers a web programming model can be far more productive than a the conventional GUI alternative. Developers only have to produce a single product for all platforms, they can upgrade the application frequently rather than in “big bang” style (have you ever seen a website version number?), they can choose whatever programming language and libraries they care for.
  • Browser Improvements: While the basics of Ajax have been present for several years, browser improvements in areas such as XML-processing and debugging support, as well as adherence to standards, have eased the pain for the Ajax development style.
  • Security Concerns: As security concerns have heightened, companies are now quicker to lock down desktops and forbid browser plugins. Web applications hosted on a company’s intranet are therefore a boon for security, and allow for tighter monitoring and access control.
  • Platform Diversity: The rise of Apple, combined with the ever-present desktop Linux, means that many applications must be developed in a portable way, and choosing the web platform is one popular way to achieve portability.
  • Browser Diversity: Flock, Safari, and Firefox have joined Mozilla and Opera in offering serious alternatives to Internet Explorer which can no longer be ignored by mainstream developers. Most Ajax techniques, being based on standard browser features – albeit more modern ones – can generally be applied on all the recent browsers.

It’s also worth noting why this information has spread through the community so quickly:

  • The ‘G’ Factor: Several prominent Google projects have been thoroughly Ajaxian, including Google Maps and GMail. When Google talks, people listen.
  • Simple Message: Ajax is a straightforward message: all you have to do is point someone at Google Maps. It’s a lot easier than explaining “the long tail” or “the folksonomy” :-P.
  • Rapid Dissemination: Due to RSS, blogs, and podcasts, any worthy information is rapidly disseminated. For the reasons above, Ajax struck a chord in the community. Once a name was there for this desirable style of development, everyone wanted to talk about it.

Downsides of Ajax?

Ajax is a trade-off. Any developer considering its adoption should be aware of the downsides, such as:

  • Limited Capabilities: Some Ajax applications are certainly doing things people never dreamed were possible on the web, but there are still substantial restrictions of the web platform. For example: multimedia capabilities, local data storage, real-time graphics, interaction with hardware such as printers and webcams. Support for some of these are improving in recent browsers, some can be achieved by delegating to Flash, but many are simply not possible, and if required, would rule out Ajax.
  • Performance Concerns: Constant interaction between browser and server can make an application feel unresponsive. There are, however, quite a few well-known patterns for performance optimisation such as browser-side caching. These usually suffice, even for fast-paced applications like stock trading, but Ajax still might not work for really time-critical applications such as machine control.
  • Internet Access Required: The user can’t access an Ajax application in the absence of a network connection.
  • Second Programming Language: Serious Ajax applications require some knowledge of Javascript. Many developers are discovering that Javascript is actually a more capable language than at first assumed, but there is nevertheless an imposition to use a language different to that on the server-side.
  • Easily Abused: As with any powerful technology, Ajax concepts can be abused by careless programmers. The patterns on this site are intended to guide developers towards more usable solutions, but the fact remains that Ajax isn’t always be used in a manner that supports usability.

Blummy: The Mother of All Bookmarklets

Check out Alexander Kirk’s new website: Blummy. A blummy is a kind of bookmarklet that opens up a kind of pop-up portal, giving you access to various web services. Just like a portal is made up of Portlets, a Blummy is made of Blummlets, which essentially do the kind of things bookmarklets do. e.g. A blummlet can let you subscribe to this page with Bloglines, change the browser location, or show an image. Where the blummlet is interactive, the action takes place within the popup as a Live Form.

Here’s a few interesting things about the site:

  • Because the Blummlet lives in a bookmarklet, XMLHttpRequest can access any domain. Scott Isaacs recently suggested it would be worth the risk for browsers to drop the same-domain policy, though it’s unlikely to happen anytime soon. A bookmarklet, I’m guessing, gets around that constraint.
  • The site lets users share Blummlets, which might lead to Cross-Site Scripting (XSS) attacks like the now-famous myspace effort. Alexander’s well aware of this risk, which is why there’s a “Report Concern” checkbutton. It will be interesting to see how this kind of moderation works out. I still think users might need something stricter, like a whitelist approach, where Blummlets have to be explicitly approved by “the community”, i.e. guilty until proven innocent. (This is still vulnerable, but I think it’s a better trade-off overall.)
  • There are shades of yubnub here, as well as the widget/gadget/startlet idea seen on start.com. Leading to a repository of Blummlets. I mentioned to Alexander it would be nice to see an RSS feed, where users could somehow drag or a Blummlet into their existing Blummy.
  • Speaking of gadgets/widgets, the popup feels something like Dashboard and Konfabulator. Another example of tha Ajax Desktop, and a pretty useful one in this case. There are various popup bookmarklets like JSCalc, which offer one particular function. They are convenient , but there’s no central management point, and a lot of duplication and inconsistency in how the manage the popup experience. So Blummy does to the browser what Dashboard does the windows system: provide a common structure for individual applets. I haven’t looked into the programming model, but there’s probably good scope for a JS util library to further facilitate Blummlet development.

Ajax Myths (Podcast and Text)

It’s that time in a technology’s lifecycle when myths abound and someone wheels out a collection of “myths” and retorts. Here’s my contribution to that time-honoured genre. Nine myths in 37 minutes.

Click to download the Podcast. You can also subscribe to the
feed if you want future podcasts automatically downloaded - check out the
podcast FAQ at http://podca.st.

Myth: “AJAX”
Reality: Ajax

Myth: Ajax is rocket science
Reality: It’s an incremental progression

Myth: Javascript sucks
Reality: It doesn’t

Myth: The URL’s always the same
Reality: Unique URLs are possible

Myth: Ajax==XMLHttpRequest
Reality: There are other remoting technologies, and some Ajax apps don’t need any at all

Myth: The server must output XML
Reality: Server can output plain-text or specialised formats like JSON

Myth: Ajax lets websites spy on users.
Reality: The same “spying” techniques have already been possible using images and form submissions.

Myth: Ajax will crush Flash.
Reality: Flash can augment Ajax.

Myth: Ajax will rule the world!
Reality: Ajax still has many challenges ahead: usability, accessibility, portability, scaleabiity …

State of the Ajax Frameworks

The publicly-editable Ajax Frameworks Page got a nice kick along in the past few days, presumably due to a recent link from Ajaxian. If this list is anything to go by, the most common language is pure-Javascript, and Java is, as you might have guessed, highest on the server-side, followed by .Net and PHP. Sections for Python and Perl were opened up during this time. Thanks for your contributions!

1 Pure Javascript: Application Frameworks

  • 1.1 Isomorphic (Commercial RIA Toolkit)
  • 1.2 Bindows (Since 2003)
  • 1.3 BackBase (from 2003)
  • 1.4 DOJO (Under development; from September, 2004)
  • 1.5 Open Rico (Under development; from May, 2005; based on earlier proprietary framework)
  • 1.6 qooxdoo (Under development; from May, 2005)
  • 1.7 Tibet (Under development; from June, 2005)
  • 1.8 AJFORM (Since June 2005)
  • 1.9 ThyAPI (Under development; Since end of 2004)

2 Pure Javascript: Infrastructural Frameworks

  • 2.1 AjaxCaller (Alpha; from May 2005)
  • 2.2 Flash JavaScript Integration Kit ()
  • 2.3 Google AJAXSLT (Released June 2005)
  • 2.4 HTMLHttpRequest (Beta; from 2005)
  • 2.5 Interactive Website Framework (from May 2005)
  • 2.6 LibXMLHttpRequest (Released; June 2003)
  • 2.7 MAJAX (Released; August 2005)
  • 2.8 RSLite (x)
  • 2.9 Sack (In development; from May 2005)
  • 2.10 Sarissa (Released; from February, 2003)
  • 2.11 XHConn (Released; from April, 2005)

3 Server-Side: Multi-Language

  • 3.1 Cross-Platform Asynchronous INterface Toolkit (May 2005)
  • 3.2 SAJAX (Workable but not 1.0; from ?March 2005)
  • 3.3 Javascipt Object Notation (JSON) and JSON-RPC
  • 3.4 Javascript Remote Scripting (JSRS) (from 2000)
  • 3.5 Bitkraft for ASP.NET

4 Server-Side: Python

  • 4.1 CrackAJAX

5 Server-Side: Java

  • 5.1 WebORB for Java (from August 2005)
  • 5.2 Echo 2 (from March 2005)
  • 5.3 WidgetServer (2004)
  • 5.4 Direct Web Remoting (DWR) (2005)
  • 5.5 SWATO (2005)
  • 5.6 AJAX JSP Tag Library
  • 5.7 AJAX Java Server Faces Framework
  • 5.8 ThinkCAP JX: RAD Environment for AJAX, J2EE, and Open Source (Commercial Framework)
  • 5.9 Struts-Layout

6 Server-Side: Lisp

  • 6.1 CL-Ajax

7 Server-Side: .NET

  • 7.1 MonoRail (from May 2005)
  • 7.2 WebORB for .NET (from August 2005)
  • 7.3 Ajax.NET (from March 2005)
  • 7.4 ComfortASP.NET (from August2005)
  • 7.5 AjaxAspects (from August 2005)

8 Server-Side: Perl

  • 8.1 CGI::Ajax – Export perl methods to javascript for AJAX

9 Server-Side: PHP

  • 9.1 AjaxAC (From April, 2005)
  • 9.2 JPSpan
  • 9.3 XAJAX
  • 9.4 PEAR::HTML::Ajax
  • 9.5 CPAINT

10 Server-Side: Ruby

  • 10.1 Ruby On Rails

Ajax, not “AJAX”: A User-Centered Definition

Many equate Ajax to “Is it using XMLHttpRequest?”, which I think is taking the acronym too literally. There’s a reason why I’ve learned to say “Ajax” rather than “AJAX”: the term is user-centric, not techno-centric, and best defined in terms of what it gives users rather than how you deliver it. And what it gives users is a rich, continuous, experience to rival the desktop, with standards-based technologies.

Jesse James-Garret, who coined the term, doesn’t define Ajax in terms of specific technologies. Here’s what he said in the recent Ajaxian.com podcast interview: “In my opinion, Ajax refers simply to using browser-native technologies, open standard technologies, in ways that depart from the traditional interaction model of the web – the kind of call-and-response interaction model where every user action is tied to some kind of server communication, and while that server communication is going on, no user actions can take place. Any time you’re decoupling the flow of user interaction with the application from the flow of server communication, and you’re doing that with browser-native standard technologies, I think that’s Ajax.”

If you can pull off a rich browser client using HTML 1.0, you got Ajax! In reality, you’ll end up using a mix of technologies suggested by the acronym (“Asynchronous Javascript and XML”) and some other things too (“DOM/DHTML/CSS”), but not necessarily all of those things. Having peeked under the covers of various high-profile Ajax sites, it’s clear that many are using custom-format messages, rather than XML for transfer. Similarly, XMLHttpRequest is not the only valid means of transfer. Google Maps uses IFrames, I’ve come across various enterprise systems that do the same, and I expect other Ajax apps do it too (and still on the lookout for examples BTW).