Basics of Ajax 1 of 3: DOM and Display Manipulation (Podcast)

The Basics of Ajax: A 3-Part Podcast Series

I’m beginning to podcast about specific Ajax patterns. To start with, a three-part series on the basics of Ajax development, covering all the Foundational Technologies patterns:

  • Podcast 1: Display Patterns and the DOM.
  • Podcast 2: Web Remoting – XMLHttpRequest, IFrame Call, HTTP Streaming.
  • Podcast 3: Dynamic Behaviour – Events and Timing.

These will be useful if your familiar with basic forms and CGI-type web development, and wanting an overview on Ajax development techniques.

If you do know enough Ajax to at least write the obligatory Ajax chat application, there’s probably not much new info here. The most interesting pattern will be HTTP Streaming, covered in the next podcast.

Note that the pattern names and structure are subject to change, but the basic ideas will remain valid.

Podcast 1: Display Patterns and the DOM

Click to download the Podcast. You can also subscribe to the
feed if you want future podcasts automatically downloaded - check out the
podcast FAQ at http://podca.st.

This 42 minute podcast covers the DOM and the following patterns:

  • Display Morphing: Morph display elements by altering styles and values in the DOM, such as text and colour properties.

  • Page Rearrangement: Add, remove, move, and overlay elements by manipulating the DOM.

Credits and Production Notes

  • The podcast concludes with the world’s first HCI Rap, “We Got It”, from the multi-talented team at OK-Cancel (the website with the funniest usability cartoons around).
  • The theme, My Morning Jacket’s “One Big Holiday”, is back too.
  • For the podcasters among you, the podcast was produced on a Powerbook running the excellent Audio Hijack Pro, with Bias SoundSoap and Apple’s new AUAudioFilePlayer plugin to cue audio. For the podcasters among you, this is a nice, easy, setup: allows recording directly to MP3, real-time noise reduction, ability to cue up sound clips, tag the MP, run a script at the end (which could FTP the file somewhere). In theory, you could have a podcast up a few minutes after you’ve clicked the stop button. Rogue Amoeba, creators of Audio Hijack Pro, know how to make software that’s intuitive and seriously useful. There’s detailed instructions for podcasters in the manual and also a blog entry on podcasting with Audio Hijack Pro.

Blummy: The Mother of All Bookmarklets

Check out Alexander Kirk’s new website: Blummy. A blummy is a kind of bookmarklet that opens up a kind of pop-up portal, giving you access to various web services. Just like a portal is made up of Portlets, a Blummy is made of Blummlets, which essentially do the kind of things bookmarklets do. e.g. A blummlet can let you subscribe to this page with Bloglines, change the browser location, or show an image. Where the blummlet is interactive, the action takes place within the popup as a Live Form.

Here’s a few interesting things about the site:

  • Because the Blummlet lives in a bookmarklet, XMLHttpRequest can access any domain. Scott Isaacs recently suggested it would be worth the risk for browsers to drop the same-domain policy, though it’s unlikely to happen anytime soon. A bookmarklet, I’m guessing, gets around that constraint.
  • The site lets users share Blummlets, which might lead to Cross-Site Scripting (XSS) attacks like the now-famous myspace effort. Alexander’s well aware of this risk, which is why there’s a “Report Concern” checkbutton. It will be interesting to see how this kind of moderation works out. I still think users might need something stricter, like a whitelist approach, where Blummlets have to be explicitly approved by “the community”, i.e. guilty until proven innocent. (This is still vulnerable, but I think it’s a better trade-off overall.)
  • There are shades of yubnub here, as well as the widget/gadget/startlet idea seen on start.com. Leading to a repository of Blummlets. I mentioned to Alexander it would be nice to see an RSS feed, where users could somehow drag or a Blummlet into their existing Blummy.
  • Speaking of gadgets/widgets, the popup feels something like Dashboard and Konfabulator. Another example of tha Ajax Desktop, and a pretty useful one in this case. There are various popup bookmarklets like JSCalc, which offer one particular function. They are convenient , but there’s no central management point, and a lot of duplication and inconsistency in how the manage the popup experience. So Blummy does to the browser what Dashboard does the windows system: provide a common structure for individual applets. I haven’t looked into the programming model, but there’s probably good scope for a JS util library to further facilitate Blummlet development.

Ajax Myths (Podcast and Text)

It’s that time in a technology’s lifecycle when myths abound and someone wheels out a collection of “myths” and retorts. Here’s my contribution to that time-honoured genre. Nine myths in 37 minutes.

Click to download the Podcast. You can also subscribe to the
feed if you want future podcasts automatically downloaded - check out the
podcast FAQ at http://podca.st.

Myth: “AJAX”
Reality: Ajax

Myth: Ajax is rocket science
Reality: It’s an incremental progression

Myth: Javascript sucks
Reality: It doesn’t

Myth: The URL’s always the same
Reality: Unique URLs are possible

Myth: Ajax==XMLHttpRequest
Reality: There are other remoting technologies, and some Ajax apps don’t need any at all

Myth: The server must output XML
Reality: Server can output plain-text or specialised formats like JSON

Myth: Ajax lets websites spy on users.
Reality: The same “spying” techniques have already been possible using images and form submissions.

Myth: Ajax will crush Flash.
Reality: Flash can augment Ajax.

Myth: Ajax will rule the world!
Reality: Ajax still has many challenges ahead: usability, accessibility, portability, scaleabiity …

State of the Ajax Frameworks

The publicly-editable Ajax Frameworks Page got a nice kick along in the past few days, presumably due to a recent link from Ajaxian. If this list is anything to go by, the most common language is pure-Javascript, and Java is, as you might have guessed, highest on the server-side, followed by .Net and PHP. Sections for Python and Perl were opened up during this time. Thanks for your contributions!

1 Pure Javascript: Application Frameworks

  • 1.1 Isomorphic (Commercial RIA Toolkit)
  • 1.2 Bindows (Since 2003)
  • 1.3 BackBase (from 2003)
  • 1.4 DOJO (Under development; from September, 2004)
  • 1.5 Open Rico (Under development; from May, 2005; based on earlier proprietary framework)
  • 1.6 qooxdoo (Under development; from May, 2005)
  • 1.7 Tibet (Under development; from June, 2005)
  • 1.8 AJFORM (Since June 2005)
  • 1.9 ThyAPI (Under development; Since end of 2004)

2 Pure Javascript: Infrastructural Frameworks

  • 2.1 AjaxCaller (Alpha; from May 2005)
  • 2.2 Flash JavaScript Integration Kit ()
  • 2.3 Google AJAXSLT (Released June 2005)
  • 2.4 HTMLHttpRequest (Beta; from 2005)
  • 2.5 Interactive Website Framework (from May 2005)
  • 2.6 LibXMLHttpRequest (Released; June 2003)
  • 2.7 MAJAX (Released; August 2005)
  • 2.8 RSLite (x)
  • 2.9 Sack (In development; from May 2005)
  • 2.10 Sarissa (Released; from February, 2003)
  • 2.11 XHConn (Released; from April, 2005)

3 Server-Side: Multi-Language

  • 3.1 Cross-Platform Asynchronous INterface Toolkit (May 2005)
  • 3.2 SAJAX (Workable but not 1.0; from ?March 2005)
  • 3.3 Javascipt Object Notation (JSON) and JSON-RPC
  • 3.4 Javascript Remote Scripting (JSRS) (from 2000)
  • 3.5 Bitkraft for ASP.NET

4 Server-Side: Python

  • 4.1 CrackAJAX

5 Server-Side: Java

  • 5.1 WebORB for Java (from August 2005)
  • 5.2 Echo 2 (from March 2005)
  • 5.3 WidgetServer (2004)
  • 5.4 Direct Web Remoting (DWR) (2005)
  • 5.5 SWATO (2005)
  • 5.6 AJAX JSP Tag Library
  • 5.7 AJAX Java Server Faces Framework
  • 5.8 ThinkCAP JX: RAD Environment for AJAX, J2EE, and Open Source (Commercial Framework)
  • 5.9 Struts-Layout

6 Server-Side: Lisp

  • 6.1 CL-Ajax

7 Server-Side: .NET

  • 7.1 MonoRail (from May 2005)
  • 7.2 WebORB for .NET (from August 2005)
  • 7.3 Ajax.NET (from March 2005)
  • 7.4 ComfortASP.NET (from August2005)
  • 7.5 AjaxAspects (from August 2005)

8 Server-Side: Perl

  • 8.1 CGI::Ajax – Export perl methods to javascript for AJAX

9 Server-Side: PHP

  • 9.1 AjaxAC (From April, 2005)
  • 9.2 JPSpan
  • 9.3 XAJAX
  • 9.4 PEAR::HTML::Ajax
  • 9.5 CPAINT

10 Server-Side: Ruby

  • 10.1 Ruby On Rails

Ajax, not “AJAX”: A User-Centered Definition

Many equate Ajax to “Is it using XMLHttpRequest?”, which I think is taking the acronym too literally. There’s a reason why I’ve learned to say “Ajax” rather than “AJAX”: the term is user-centric, not techno-centric, and best defined in terms of what it gives users rather than how you deliver it. And what it gives users is a rich, continuous, experience to rival the desktop, with standards-based technologies.

Jesse James-Garret, who coined the term, doesn’t define Ajax in terms of specific technologies. Here’s what he said in the recent Ajaxian.com podcast interview: “In my opinion, Ajax refers simply to using browser-native technologies, open standard technologies, in ways that depart from the traditional interaction model of the web – the kind of call-and-response interaction model where every user action is tied to some kind of server communication, and while that server communication is going on, no user actions can take place. Any time you’re decoupling the flow of user interaction with the application from the flow of server communication, and you’re doing that with browser-native standard technologies, I think that’s Ajax.”

If you can pull off a rich browser client using HTML 1.0, you got Ajax! In reality, you’ll end up using a mix of technologies suggested by the acronym (“Asynchronous Javascript and XML”) and some other things too (“DOM/DHTML/CSS”), but not necessarily all of those things. Having peeked under the covers of various high-profile Ajax sites, it’s clear that many are using custom-format messages, rather than XML for transfer. Similarly, XMLHttpRequest is not the only valid means of transfer. Google Maps uses IFrames, I’ve come across various enterprise systems that do the same, and I expect other Ajax apps do it too (and still on the lookout for examples BTW).

Chat: The “Hello World” of Ajax?

Day Barr:

Chat is not quite the Hello World of Ajax, but it’s one of the simplest yet useful things I could do. I didn’t learn very much by writing an Ajax Hello World example and it’s completely pointless :-)

As many are learning, an Ajax “Hello World” is pretty easy, provided you’ve already got a grounding in web/JS programming. So beyond that, what’s the quintessential Ajax tutorial app? Some of the Ajax texts are beginning to pour out, so maybe a pattern will emerge. But, for now, the learning apps of choice seem to be:

I’m surprised we haven’t seen more people playing with Ajax wikis and RSS aggregators, but I’m sure they’re coming.

BTW, a basic “Hello World” might be easy, but it’s also a useful springboard for some important variants. e.g. Add a long delay in your server and see how the browser script handles a timeout (or simulate a delay with Julien’s GM script. e.g. Host the service on another domain and have the server act as a proxy. e.g. Apply a visual effect like the famous Yellow Fade Technique when the message comes in. These variants aren’t just interesting exercises – all are important for real-world Ajax development.

Another DHTML “Game”

A new DHTML boxing “game” from the Man In Blue. Not quite DHTML Lemmings or Super Maryo World, but still more fun than it should be. A bit like the Ruby On Rael thing in reverse.

Also from the same presentation is an eerily lifelike OSX clone (Firefox-only). More evidence that Ajax might be useful in prototyping desktop apps. And since prototypes usually go on to become the real thing …

Thanks to the Web Essentials organisers for podcasting the talks. Safran on Sunday will have to go on hold for a while.

Ajax can *Improve* Performance Too

Recent Ajax apps like Kiko are sluggish according to Alexander Kirk’s “Rise of Slow Ajax Applications (via AjaxDeveloper):

Pages get more voluminous because so much code has to be loaded to the browser (which makes the browser slow again) so you could just begin to use the application. This somehow reminds me of all the flash apps. Waiting for hours to load the page and you’ll stick to that page for half a minute.

The initial delay is due to loading of Javascript and data too. Fast startup is certainly a big motivation for Ajax, so it’s a problem if that’s not happening.

Alexander gives four tips:

  • Keep it bookmarkable. Don’t load everything to one page, let users return to a certain section of your app via their bookmarks.

This is sort of mixing two concepts together. Using Unique URLs, it’s possible for an Ajax app to be bookmarkable, but still the same “page”. I think the advice here is to use real page refreshes rather than making an entire website a single Ajax application. I think that’s fair for a big corporate website, but for a standalone app like Kiko, it’s best seen as a last resort. Saying that, a reasonable use might be in switching between a general login/menu page and the core app itself. Or, in Writely, switching between the different tabs – editing, blog it, etc.

  • Don’t overuse AJAX. Often simple Javascript without server interaction will do. Try to reduce the server callbacks.

Agree with the tip, though not necessarily the implicit definition that Ajax == server interaction. Rich, standards-based, Javascript is Ajax in my book, and certainly a good way to improve performance.

  • Minimize the code to be loaded. When you don’t have any other choice, consider code downloading.

Code downloading – or On-Demand Javascript – is actually quite easy to do. It involves loading some basic Javascript immediately and continue to load the rest in the background or as needed. You can load the JS by tweaking the head element, or alternatively by pulling it down with XMLHttpRequest and eval’ing it. As an extension of multi-stage code loading, there’s also Multi-Stage Download – downloading an initial structure upfront and populating it with further calls.

  • Speed up your apps with AJAX. Use AJAX for what it was meant for: tune your existing application at points where postbacks need to reload the same page with only little change.

Great point. <rant>Too often, Ajax is blamed for making the web slow, unaccessible, unusable, etc. But what if we stop a minute and ask “What if Ajax could improve performance/accessibility/usability? How could that happen?” By attempting to answer these questions as effectively as possible, even if we disagree with the premise, we’re better equipped to weigh both sides of the argument.</rant>

In the case of performance, there are plenty of ways Ajax actually improves the situation. For starters, you don’t need to download an entire HTML page every time you talk to the server. More specifically, smart Javascript can do stuff like caching, guesstimating, and pre-fetching.

New Ajax Demos

Further to the previous post on new Ajax programming patterns, it’s also worth noting there’s a new bunch of corresponding online Ajax demos too. Being programming patterns, they tend to be “pure refactorings” rather than enhancements, thus the user experience is mostly the same – you have to look under the covers to see what’s changed. The demos are mentioned in individual patterns, but here’s a sampling:

  • The wiki has a bunch of timeout-related refactorings. These refactorings are user-visible. (They’re not actually part of the programming patterns, but they were recently added.)
  • 2 other wiki refactorings involve On-Demand Javascript . (No UI change.)
  • A JSON playground.
  • The Portal Drilldown Demo has been refactored to render with XSLT and browser templating tools. (No UI change.)
  • A couple of refactorings added to demonstrate REST (broken right now, probably due to a PHP config issue) and RPC on the Ajax Shop demo. (No UI change.)

As with all the demos, they’re tested on FF and IE6, please let me know if anything breaks.

18 New Ajax Programming Patterns

I’ve uploaded full text for 18 new Ajax patterns, completing a first-cut draft for all Programming Patterns content, which will be one part of the book. This section bridges the gap between the very basics of Ajax – XMLHttpRequest, DOM, etc – and the high-level stuff like widgets and visual effects. For instance, how do you do design the services accessed by XMLHttpRequest? Or how do you deal with a lots of Javascript? The focus here is on traditional tech concerns such as maintainability and understandability; and less about usability.

Breaking it down:

  • Web Services patterns are about overall browser-server architecture – what kinds of services and what kinds of messages? It’s relevant to the conversation of Jon Tiersen and others about what sort of responses are possible – XML, JS, HTML, JSON. It’s also relevant to another recent conversation, on [http://www.theserverside.com/news/thread.tss?thread_id=35979 Ajax and webservices], as it covers REST and RPC.

  • Browser-Server Dialogue covers a mixed bag of patterns about XMLHttpRequest handling and what you can do with it, such as loading JS on demand.

  • DOM Population is about transforming incoming XML into DOM content, e.g. XSLT.

  • Performance Optimisation covers a variety of performance patterns such as caching and pre-fetching.

  • Breakout (can you think of a better name) is about going beyond standard Ajax constraints, namely using a server-side mediator to access external domains and introducing a plugin to go where no Ajax app is allowed to go.

  • Code Generation and Reuse is strongly related to some of the emerging frameworks like Echo2 and SAJAX.

This is all early days, so any feedback on the structure, names, or content will be taken into account and acknowledged too (really!). OK, here’s the lot …

Programming Patterns

Web Services

  • RESTful Service Expose web services according to RESTful principles.
  • RPC Service Expose web services as Remote Procedural Calls (RPCs).
  • HTML Response Have the server generate HTML snippets to be displayed in the browser.
  • Semantic Response Have the server respond with abstract, semantic, data.
  • Plain-Text Message Pass simple messages between server and browser in plain-text format.
  • XML Message Pass messages between server and browser in XML format.
  • JSON Message Pass messages between server and browser in Javascript Object Notation (JSON) format.

Browser-Server Dialogue

  • Call Tracking Accommodate busy user behaviour by allocating a new XMLHtpRequest object for each request. See Richard Schwartz’s blog entry (http://smokey.rhs.com/web/blog/poweroftheschwartz.nsf/d6plinks/RSCZ-6CEQAR).Note: Pending some rewrite to take into account request-locking etc.
  • Distributed Events Keep objects synchronised with an event mechanism.
  • On-Demand Javascript Download Javascript as and when required, instead of downloading it all on page load.

DOM Population

Performance Optimisation

  • Fat Client Create a rich, browser-based, client by performing remote calls only when there is no way to achieve the same effect in the browser.
  • Browser-Side Cache Maintain a local cache of information.
  • Guesstimate Instead of grabbing real data from the server, make a guesstimate that’s good enough for most user’s needs.
  • Submission Throttling Instead of submitting upon each Javascript event, retain the data in a local buffer and upload it periodically.
  • Explicit Submission Instead of submitting upon each Javascript event, require the user to explicitly request it, e.g. submit upon clicking a button.
  • Multi-Stage Download Quickly download the page structure with a standard request, then populate it with further requests.
  • Predictive Fetch Anticipate likely user actions and pre-load the required data.

Breakout

  • Cross-Domain Mediator Allow the browser to communicate with other domains by server-based mediation.
  • Richer Plugin Make your application “more Ajax than Ajax” with a Richer Plugin.

Code Generation and Reuse

  • Ajax Stub Use an “Ajax Stub” framework which allows browser scripts to directly invoke server-side operations, without having to worry about the details of XMLHttpRequest and HTTP transfer.
  • Server-Side Code Generation Automatically generate HTML and Javascript from server-side code.
  • Cross-Browser Component Create cross-browser components, allowing programmers to reuse them without regard for browser compatibility.

Aside I was interviewed yesterday for a Japanese magazine about how I’m using the wiki. So maybe some people will be interested to know that I always write the patterns offline because I am more creative in Vim and also to avoid textarea hell. (So much for Writely, will anyone create Vimly, there’s gotta be more money it than cloning MS-Word online :-D). I also use the Mozex or ViewSourceWith extensions to make partial edits using Vim.

This time, I decided not to upload once or twice at a time; but instead all 18 at once. There’s serious overhead of introducing each new pattern to the wiki (from Vim, the sequence is: 2mins spell-check, 5-10mins markup massaging, 2 mins fixing the link and description on the homepage; sometimes exacerbated by server lag.) Uploading all at once at least allowed me to focus fully on the task and also made some aspects more efficient, particularly updating the homepage.)