An Ajax Framework a Day!

Today’s Ajax framework is JsRia. Yesterday’s was ZK, with the Backbase entries updated too. In the past week, there were Smartclient, Ajax JSP Taglib, Ajax JSF Framework, Cajax. Here’s the diff. The week prior to that saw introduction of XOAD, Rialto, and Lotus Notes info.

Have the Ajax frameworks entered the enlightened age of singularity? (I’ve been listening to a lot of Ray Kurzweil podcasts lately, forgive me.) To some extent, yes, there is some pretty explosive growth here, because several of these frameworks really have been released in the past couple of weeks, as far as I can tell. In addition, many of the project owners and users have presumably become aware of the page, and seek to add their project or update my original description.

So I’m thinking the frameworks page needs to be split before it bursts at the seams, what do you reckon? I wish there was a way to keep them all on the same page, with a bit more Ajaxy dynamism, to let you manage and personalise things better. As I alluded to yesterday on Ajaxian, of all the Ajax projects being announced – wikis are somewhat lacking. Surprising to me, since it was one of the most obvious Ajax examples to me – one I mentioned in the original Ajax Podcast, and one of the first proof-of-concept demos I created for the patterns. One thing I’d like to see is wikis take more of a web service approach – let a thousand Ajax/Flash/Desktop wikipedia clients bloom. Sure, there are mashups now, but they’re mostly read-only, and require manual scraping. The idea I had with the Ajax Patterns Reader was to eventually let people leave feedback. There’s another demo – the portal, which grabs content from ajaxpatterns.org, and there will be a further demo coming soon. To do all that properly, I’ll likely create a web service to expose the wiki content as an RESTful API.

In closing off this tangent, here’s a question: How would it be creating a wiki from the ground up, with no UI? Just a collection of web services for managing content. I’ve found MoinMoin is more configurable and pluggable than most, but it still starts with the unnecessary premise that the UI lives in the same process as the content.

An Ajax Framework a Day!

Today’s Ajax framework is JsRia. Yesterday’s was ZK, with the Backbase entries updated too. In the past week, there were Smartclient, Ajax JSP Taglib, Ajax JSF Framework, Cajax. Here’s the diff. The week prior to that saw introduction of XOAD, Rialto, and Lotus Notes info.

Have the Ajax frameworks entered the enlightened age of singularity? (I’ve been listening to a lot of Ray Kurzweil podcasts lately, forgive me.) To some extent, yes, there is some pretty explosive growth here, because several of these frameworks really have been released in the past couple of weeks, as far as I can tell. In addition, many of the project owners and users have presumably become aware of the page, and seek to add their project or update my original description.

So I’m thinking the frameworks page needs to be split before it bursts at the seams, what do you reckon? I wish there was a way to keep them all on the same page, with a bit more Ajaxy dynamism, to let you manage and personalise things better. As I alluded to yesterday on Ajaxian, of all the Ajax projects being announced – wikis are somewhat lacking. Surprising to me, since it was one of the most obvious Ajax examples to me – one I mentioned in the original Ajax Podcast, and one of the first proof-of-concept demos I created for the patterns. One thing I’d like to see is wikis take more of a web service approach – let a thousand Ajax/Flash/Desktop wikipedia clients bloom. Sure, there are mashups now, but they’re mostly read-only, and require manual scraping. The idea I had with the Ajax Patterns Reader was to eventually let people leave feedback. There’s another demo – the portal, which grabs content from ajaxpatterns.org, and there will be a further demo coming soon. To do all that properly, I’ll likely create a web service to expose the wiki content as an RESTful API.

In closing off this tangent, here’s a question: How would it be creating a wiki from the ground up, with no UI? Just a collection of web services for managing content. I’ve found MoinMoin is more configurable and pluggable than most, but it still starts with the unnecessary premise that the UI lives in the same process as the content.

Redundant Design is Worth Fighting For

Matt @ 37Signals discusses new countdowns being used at pedestrian crossings (crosswalks). Did you ever count how many redundant messages are available at a pedestrian crossing? Good, let’s be sad together and count them, then. At a workshop one time, various attendees from different countries came up with a list of cues, something like the six below:

  • The walking man (is there a walking woman anywhere in the world?) or “Walk”/”Don’t Walk” message.
  • The main traffic lights for drivers.
  • Countdown displays.
  • Display next to the button, indicating if it’s already been pushed (in which case, currently in “Don’t Walk” mode).
  • Sound. (A continuous noise to indicate whichever phase they’re in, and/or a transition sound.)
  • Cars and pedestrians. (Not actually designed and not reliable, but certainly an indication.)

The redundancy is presumably to cope with different sets of disabilities, as well as improve safety for everyone. Software developers don’t always like redundancy – it goes against just about every fundamental design principle you care to name – but users generally benefit from it. So it’s a matter of architecting things so that redundant UI doesn’t lead to redundant code. e.g. point two event handlers to the same Command object.

Yeah, another funny thing about crossings is the button. In one place (Singapore?), I was told not to push it, because it’s only for disabled or elderly people (and of course, ignorant tourists). Everyone else just waits and it will turn green eventually.

Blummy: The Mother of All Bookmarklets

Check out Alexander Kirk’s new website: Blummy. A blummy is a kind of bookmarklet that opens up a kind of pop-up portal, giving you access to various web services. Just like a portal is made up of Portlets, a Blummy is made of Blummlets, which essentially do the kind of things bookmarklets do. e.g. A blummlet can let you subscribe to this page with Bloglines, change the browser location, or show an image. Where the blummlet is interactive, the action takes place within the popup as a Live Form.

Here’s a few interesting things about the site:

  • Because the Blummlet lives in a bookmarklet, XMLHttpRequest can access any domain. Scott Isaacs recently suggested it would be worth the risk for browsers to drop the same-domain policy, though it’s unlikely to happen anytime soon. A bookmarklet, I’m guessing, gets around that constraint.
  • The site lets users share Blummlets, which might lead to Cross-Site Scripting (XSS) attacks like the now-famous myspace effort. Alexander’s well aware of this risk, which is why there’s a “Report Concern” checkbutton. It will be interesting to see how this kind of moderation works out. I still think users might need something stricter, like a whitelist approach, where Blummlets have to be explicitly approved by “the community”, i.e. guilty until proven innocent. (This is still vulnerable, but I think it’s a better trade-off overall.)
  • There are shades of yubnub here, as well as the widget/gadget/startlet idea seen on start.com. Leading to a repository of Blummlets. I mentioned to Alexander it would be nice to see an RSS feed, where users could somehow drag or a Blummlet into their existing Blummy.
  • Speaking of gadgets/widgets, the popup feels something like Dashboard and Konfabulator. Another example of tha Ajax Desktop, and a pretty useful one in this case. There are various popup bookmarklets like JSCalc, which offer one particular function. They are convenient , but there’s no central management point, and a lot of duplication and inconsistency in how the manage the popup experience. So Blummy does to the browser what Dashboard does the windows system: provide a common structure for individual applets. I haven’t looked into the programming model, but there’s probably good scope for a JS util library to further facilitate Blummlet development.

New Patterns: XMLHttpRequest Call and IFrame Call

Full drafts are now online for the two big Web Remoting patterns: XMLHttpRequest Call and IFrame Call. There are also a couple of new demos to illustrate GETting and POSTing with both call types: XMLHttpRequest Example, IFrame Example.

A few extracts follow, but first, let me ask you: Do you know of any public IFrame remoting examples? Google Maps is the only public example I know. All the other examples here are tutorials and framework demos.

The XMLHttpRequest alternatives was the most fun thing here (headings shown below). I went through the various techniques I’ve seen, read about, and tried out, to list as many ways as possible to programmatically call the server. As you might expect for these “Foundational Technology” patterns, there’s a lot more code in the Solutions than in later patterns, where the focus shifts more towards the -ilities, particularly usability and maintainability.

As usual, feedback much appreciated. (Mainly on the content, rather than woeful spelling and outright grammatical disasters at this stage.)

XMLHttpRequest Call

Solution: Use XMLHttpRequest objects for browser-server communication. XMLHttpRequest is a Javascript class capable of calling the server and capturing its response. Just like a browser or a command-line web client, an XMLHttpRequest issues standard HTTP requests and grabs responses. Note: The code shown throughout this demo is closely based on an online companion demo (http://ajaxify.com/run/xmlHttpRequestCall/).

The above example hopefully illustrates that the fundamental technology is pretty simple. However, be aware that it’s a very basic usage, not yet fit for production. Fundamental questions remain:

* How can XMLHttpRequests be created?
* How does an asynchronous call work?
* How do our code detect errors?
* What if the service requires a "POST" or "PUT" request rather than a "GET"?
* Which URLs can be accessed?
* How can you deal with XML responses?
* What's the API? 

The following sections address these questions and show how the code above needs to be modified. …

Decisions

6.1 What kind of content will web services provide?
6.2 How will caching be controlled?
6.3 How will you deal with errors?

Alternatives:

9.1 IFrame Call
9.2 Persistent Connection
9.3 Richer Plugin
9.4 On-Demand Javascript
9.5 Import XML Document
9.6 Image-Cookie Call
9.7 Stylesheet Call

IFrame Call

Solution: Use IFrames for browser-server communication. IFrames are page-like elements that can be embedded in other pages. They have their own source URL, distinct from their parent page, and it can change dynamically. IFrame Calls work by making the IFrame point to a URL we’re interested in, then reading the IFrame’s contents once the new content has loaded.

Ajax can *Improve* Performance Too

Recent Ajax apps like Kiko are sluggish according to Alexander Kirk’s “Rise of Slow Ajax Applications (via AjaxDeveloper):

Pages get more voluminous because so much code has to be loaded to the browser (which makes the browser slow again) so you could just begin to use the application. This somehow reminds me of all the flash apps. Waiting for hours to load the page and you’ll stick to that page for half a minute.

The initial delay is due to loading of Javascript and data too. Fast startup is certainly a big motivation for Ajax, so it’s a problem if that’s not happening.

Alexander gives four tips:

  • Keep it bookmarkable. Don’t load everything to one page, let users return to a certain section of your app via their bookmarks.

This is sort of mixing two concepts together. Using Unique URLs, it’s possible for an Ajax app to be bookmarkable, but still the same “page”. I think the advice here is to use real page refreshes rather than making an entire website a single Ajax application. I think that’s fair for a big corporate website, but for a standalone app like Kiko, it’s best seen as a last resort. Saying that, a reasonable use might be in switching between a general login/menu page and the core app itself. Or, in Writely, switching between the different tabs – editing, blog it, etc.

  • Don’t overuse AJAX. Often simple Javascript without server interaction will do. Try to reduce the server callbacks.

Agree with the tip, though not necessarily the implicit definition that Ajax == server interaction. Rich, standards-based, Javascript is Ajax in my book, and certainly a good way to improve performance.

  • Minimize the code to be loaded. When you don’t have any other choice, consider code downloading.

Code downloading – or On-Demand Javascript – is actually quite easy to do. It involves loading some basic Javascript immediately and continue to load the rest in the background or as needed. You can load the JS by tweaking the head element, or alternatively by pulling it down with XMLHttpRequest and eval’ing it. As an extension of multi-stage code loading, there’s also Multi-Stage Download – downloading an initial structure upfront and populating it with further calls.

  • Speed up your apps with AJAX. Use AJAX for what it was meant for: tune your existing application at points where postbacks need to reload the same page with only little change.

Great point. <rant>Too often, Ajax is blamed for making the web slow, unaccessible, unusable, etc. But what if we stop a minute and ask “What if Ajax could improve performance/accessibility/usability? How could that happen?” By attempting to answer these questions as effectively as possible, even if we disagree with the premise, we’re better equipped to weigh both sides of the argument.</rant>

In the case of performance, there are plenty of ways Ajax actually improves the situation. For starters, you don’t need to download an entire HTML page every time you talk to the server. More specifically, smart Javascript can do stuff like caching, guesstimating, and pre-fetching.

New Ajax Demos

Further to the previous post on new Ajax programming patterns, it’s also worth noting there’s a new bunch of corresponding online Ajax demos too. Being programming patterns, they tend to be “pure refactorings” rather than enhancements, thus the user experience is mostly the same – you have to look under the covers to see what’s changed. The demos are mentioned in individual patterns, but here’s a sampling:

  • The wiki has a bunch of timeout-related refactorings. These refactorings are user-visible. (They’re not actually part of the programming patterns, but they were recently added.)
  • 2 other wiki refactorings involve On-Demand Javascript . (No UI change.)
  • A JSON playground.
  • The Portal Drilldown Demo has been refactored to render with XSLT and browser templating tools. (No UI change.)
  • A couple of refactorings added to demonstrate REST (broken right now, probably due to a PHP config issue) and RPC on the Ajax Shop demo. (No UI change.)

As with all the demos, they’re tested on FF and IE6, please let me know if anything breaks.

18 New Ajax Programming Patterns

I’ve uploaded full text for 18 new Ajax patterns, completing a first-cut draft for all Programming Patterns content, which will be one part of the book. This section bridges the gap between the very basics of Ajax – XMLHttpRequest, DOM, etc – and the high-level stuff like widgets and visual effects. For instance, how do you do design the services accessed by XMLHttpRequest? Or how do you deal with a lots of Javascript? The focus here is on traditional tech concerns such as maintainability and understandability; and less about usability.

Breaking it down:

  • Web Services patterns are about overall browser-server architecture – what kinds of services and what kinds of messages? It’s relevant to the conversation of Jon Tiersen and others about what sort of responses are possible – XML, JS, HTML, JSON. It’s also relevant to another recent conversation, on [http://www.theserverside.com/news/thread.tss?thread_id=35979 Ajax and webservices], as it covers REST and RPC.

  • Browser-Server Dialogue covers a mixed bag of patterns about XMLHttpRequest handling and what you can do with it, such as loading JS on demand.

  • DOM Population is about transforming incoming XML into DOM content, e.g. XSLT.

  • Performance Optimisation covers a variety of performance patterns such as caching and pre-fetching.

  • Breakout (can you think of a better name) is about going beyond standard Ajax constraints, namely using a server-side mediator to access external domains and introducing a plugin to go where no Ajax app is allowed to go.

  • Code Generation and Reuse is strongly related to some of the emerging frameworks like Echo2 and SAJAX.

This is all early days, so any feedback on the structure, names, or content will be taken into account and acknowledged too (really!). OK, here’s the lot …

Programming Patterns

Web Services

  • RESTful Service Expose web services according to RESTful principles.
  • RPC Service Expose web services as Remote Procedural Calls (RPCs).
  • HTML Response Have the server generate HTML snippets to be displayed in the browser.
  • Semantic Response Have the server respond with abstract, semantic, data.
  • Plain-Text Message Pass simple messages between server and browser in plain-text format.
  • XML Message Pass messages between server and browser in XML format.
  • JSON Message Pass messages between server and browser in Javascript Object Notation (JSON) format.

Browser-Server Dialogue

  • Call Tracking Accommodate busy user behaviour by allocating a new XMLHtpRequest object for each request. See Richard Schwartz’s blog entry (http://smokey.rhs.com/web/blog/poweroftheschwartz.nsf/d6plinks/RSCZ-6CEQAR).Note: Pending some rewrite to take into account request-locking etc.
  • Distributed Events Keep objects synchronised with an event mechanism.
  • On-Demand Javascript Download Javascript as and when required, instead of downloading it all on page load.

DOM Population

Performance Optimisation

  • Fat Client Create a rich, browser-based, client by performing remote calls only when there is no way to achieve the same effect in the browser.
  • Browser-Side Cache Maintain a local cache of information.
  • Guesstimate Instead of grabbing real data from the server, make a guesstimate that’s good enough for most user’s needs.
  • Submission Throttling Instead of submitting upon each Javascript event, retain the data in a local buffer and upload it periodically.
  • Explicit Submission Instead of submitting upon each Javascript event, require the user to explicitly request it, e.g. submit upon clicking a button.
  • Multi-Stage Download Quickly download the page structure with a standard request, then populate it with further requests.
  • Predictive Fetch Anticipate likely user actions and pre-load the required data.

Breakout

  • Cross-Domain Mediator Allow the browser to communicate with other domains by server-based mediation.
  • Richer Plugin Make your application “more Ajax than Ajax” with a Richer Plugin.

Code Generation and Reuse

  • Ajax Stub Use an “Ajax Stub” framework which allows browser scripts to directly invoke server-side operations, without having to worry about the details of XMLHttpRequest and HTTP transfer.
  • Server-Side Code Generation Automatically generate HTML and Javascript from server-side code.
  • Cross-Browser Component Create cross-browser components, allowing programmers to reuse them without regard for browser compatibility.

Aside I was interviewed yesterday for a Japanese magazine about how I’m using the wiki. So maybe some people will be interested to know that I always write the patterns offline because I am more creative in Vim and also to avoid textarea hell. (So much for Writely, will anyone create Vimly, there’s gotta be more money it than cloning MS-Word online :-D). I also use the Mozex or ViewSourceWith extensions to make partial edits using Vim.

This time, I decided not to upload once or twice at a time; but instead all 18 at once. There’s serious overhead of introducing each new pattern to the wiki (from Vim, the sequence is: 2mins spell-check, 5-10mins markup massaging, 2 mins fixing the link and description on the homepage; sometimes exacerbated by server lag.) Uploading all at once at least allowed me to focus fully on the task and also made some aspects more efficient, particularly updating the homepage.)

Cross-Domain Portlets with Start.com Gadgets/Widgets/Startlets

Scott Isaacs on a new Start.com feature (quoted on Ajaxian):

DHTML-based Gadgets: Start.com consumes DHTML-based components called Gadgets. These Gadgets can be created by any developer, hosted on any site, and consumed into the Start.com experience. The model is completely distributed. You can develop components derived from other components on the web.

Remote portlets take the Ajax portals to another level. In fact, the whole concept is one of the biggest developments in the history of the web. A9 pre-empted it with their “Open Search” capability, which allows developers to hook into the A9 interface with live search results. Now MS is going a step further, with Konfabulator-style interaction directly on the web. It’s likely Yahoo has similar plans, given that it’s now the owner of Konfabulator. What Google does is anyone’s guess. Strategically, MS has a big edge here because of its experience working with Developers, Developers, Developers! (YES!)

The idea combines [Cross-Domain Mediator](http://ajaxpatterns.org/Cross-Domain Mediator) with Portlet. Those patterns have subsumed a now-deleted pattern that previous covered this kind of thing:

Cross-Domain Portlet: Introduce Interactive Portlets to facilitate a conversation between the user and a third-party, e.g. for services or “advertising as conversation” (http://radar.oreilly.com/archives/2005/05/remaking_advert.html)

Ajax makes it possible because there’s no page refresh. Conceptually, each portlet is autonomous and capable of conducting a conversation with another domain in parallel with other portlets. When a result comes back to one portal, the other portlets are unaffected. (You can obviously yoke portals together if you want to.) In practice, there are a couple of finer implementation points:

  • You can’t make an XMLHttpRequest Call to an external domain. Hence, the portal server must mediate.
  • Conducting parallel conversations, you’ll need some form of Call Tracking. Hopefully, you’re using a library that abstracts those details from you.

Now that a happening, here are a few things I’d like to see:

  • An open standard for conducting cross-domain conversations, one that protects the end-user and the portal host, but provides sufficient functionality to the
  • Useful ads (the advertising as conversation) concept. There’s definitely a business model here for opt-in advertising. That is, users would be willing to add advertising portlets that offer genuinely useful information. e.g. a travel portlet that lets you check flight availability.
  • RSS aggregator integration The nature of a portal can change, right? So RSS aggregators should be able to include remote portlets.
  • End-user-created portlets. Make it easy for end-users to construct their own gadgets. This idea is inspired by Charlene Li’s comments on Konfabulator: “The problem is I already know what the ultimate widget would be me … There’s only one small problem — I don’t know JavaScript! So I am at the mercy of app developers …”

A Few Ajax Gotchas At Jalecode

Andrew Sutherland offers a few Ajax Gotchas/Tips. I’ll add some comments.

  • Escape content with encodeURIComponent() which is superior to escape.

  • XMLHttpRequest’s readyState tells you how far the request has progressed. MM: If you’re confused about readyState‘s transition from 0 to 4, you have good reason to be. Read the recent posting and comments on David Flanagan’s blog, and you’ll learn that 2 and 3 are ambiguous to the point of being unusable. Essentially, you want to wait for either 4 or timeout, and probably ignore everything else.

  • Permission Denied” for XMLHttpRequest is usually due to trying to call another domain. MM: The standard security policy is that requests can only be sent to the originating server, just like the traditional policy for Java applets. To get to another domain, you can set up a Cross-Domain Mediator. This security issue has become interesting with the growing popularity of Single Page Applications (SPA). What can an HTML page sitting on you hard drive access? All domains or no domains? It would certainly be convenient if it could access the web at large. I don’t think it can access any domains on standard browsers, but it’s still possible if the user wants it to happen. Here’s what Steve Yen (TrimPath) says on this issue: “I’m shooting for now to have explicity user-driven synchronization working, which my experiments lead me to believe is workable.”

  • MM: Finally, I’ll add another gotcha-inspired tip to Andrew’s collection: Set content type to XML (in the case where you want to treat the response as XML), e.g. in PHP, header("Content-Type: text/xml");.