In his book The Mathematician’s Apology (1941), the Cambridge mathematician GH Hardy expressed his reverence for pure maths, and celebrated its uselessness in the real world. Yet one of the branches of pure mathematics in which Hardy excelled was number theory, and it was this field which played a major role in the work of his younger colleague, Alan Turing, as he worked first to crack Nazi codes at Bletchley Park and then on one of the first computers. Melvyn Bragg and guests explore the many surprising and completely unintended uses to which mathematical discoveries have been put. These include: The cubic equations which led, after 400 years, to the development of alternating current – and the electric chair. The centuries-old work on games of chance which eventually contributed to the birth of population statistics. The discovery of non-Euclidean geometry, which crucially provided an ‘off-the-shelf’ solution which helped Albert Einstein forge his theory of relativity. The 17th-century theorem which became the basis for credit card encryption.

The relevance to the topic of this blog should be clear.

Show me someone doing something cool or experimental and I’ll show you someone who sniffs, “Why would anyone want to do this?”. The answer may be because it was fun, because they wanted to see if it was possible, or because they wanted to learn something. But whatever their primary reason, one thing’s for sure: there will be unintended consequences. The examples above show it’s happened in mathematics, and we’ve seen the same thing happen time and again in web development.

The rich interactivity we see today wouldn’t have been possible if people hadn’t been willing to fool around with the not-so-always-obvious features of web browsers. Take cross-domain iframe messaging, for example. Not something people knew much about until a few years ago, when James Burke documented his experiments. A few years later, it’s a fundamental technology in OpenSocial (which means iGoogle, the Yahoo! homepage, among many other sites), Facebook’s official Javascript client, and evidently has much interest from elsewhere.

So next time you see some wit ask “Why would anyone ever need this?”, just stay schtoom, sit back, and wait six months.

@premasagar is visiting the Osmoplex today (thanks @jayfresh for arranging it) and is taking us through his work on iframes, widgets, and sandboxing. I’ve realised we could perhaps be collaborating as my jquery-iframe plugin is so close to his. Different emphases, but much overlap.

GitHub is where you can see what he’s been working on. Basically, this guy is a guru on all things iFrame. In particular, all the quirks around squirting dynamic content into iFrames, as opposed to pointing them elesewhere using “src”.

QUOTE OF THE DAY

“sqwidget is my tiddly”

BACKGROUND – THE EMBEDDING PROBLEM

In patternesque speak, the basic problem is:

Problem: You want to embed 3rd party content into another site.

Forces:

• You want the 3rd party content to have its own style
• BUT it will inherit style from the parent page

SOLUTION 1

IFrame

Works great (in fact, I’m using it in my TiddlyWiki playground app, to be documented at some point, and is similarly used in Jon Lister’s TiddlyTemplating.

The problem is you sometimes want the widget to jump out of the iframe, e.g. a ligthboxed video. So …

SOLUTION 2

CleanSlateCSS

Basically a CSS reset, but whereas CSS resets will only handle browser defaults, cleanSlate blats everything! This is exactly the kind of thing I was looking for when I was trying to cancel out tiddlywiki styling. In that case, I was flipping the entire page back and forth, so I could just cheat by removing stylesheets and re-add them. (Prem pointed out there’s a “disabled” attribute on style tags – true, so I should really use this instead, assuming it’s portable, which he thinks it is.)

Problems: - Difficult to maintain CleanSlate library, because new CSS stuff and browser quirks keep coming up - IE6 and IE7 don’t support “inherit”, so need CSS expression. - When using Javascript to interact with CSS style properties, e.g. slideDown(), these will override CleanSlate. The solution is to set the “style” attribute with !important, but it becomes an arms race! - Doesn’t solve iFrame security model

SOLUTION 3

Inject (aka squirt, inject; summary) content into a fresh iframe.

The content comes from the widget site. The sqwidget library injects it. This resolves the tension with wanting independent CSS on the same page. If the sqwidget library is running on the host page, it could even (potentially) lock down capabilities, i.e. do Caja-style sanitisation.

Sqwidget also does templating, using Resig’s micro-templating. (That thing’s getting to be very popular; I’m using it myself in Scrumptious via the UnderScoreJS library after @fnd gave us a talk about them.)

Also, prem is playing around with the idea of a div, with custom (data-*) attribute pointing to the third-party URL. You could put inside it “now loading” and then the script tag will pick those things up and load them.

Various points:

• for content-squirting into the iframe, you should be setting doctype, otherwise IE will load in.

• worth investigating use of

1. My web apps receive some <svg>...<svg> from an Ajax call
2. ???
3. Super, there's a drawing on the page!!!

What is the secret sauce in step 2?

After a merry frolic through the majority of the internets and a dozen prototypes, I finally found the answer. In essence, you just create an "object" element with type and data settings:

It works in Firefox, and the article explains how to get it working in IE too, which I don't need just yet.

A few other things I learned and tried:

• I initially, naievely, tried just adding the SVG via innerHTML. As Jeremy explained to me, this doesn't work because the browser uses a different compiler for HTML compared to pure XHTML. (And TiddlyWiki, like most things on the web, is HTML.) Even with an <object> tag around it, it won't just switch over.
• The easiest way to do this is to use a .svg suffix (or probably set mime type to svg, but this is tiddlywiki and I'm working from file:// URLs, where it's not possible to set mime type). But then you can't embed it on the page - you'd have to point to it from an embed tag or object tag.
• You can inline the SVG if you write "pure" xhtml, and for this, you have to give the file a .xhtml suffix. As in this example.
• I tried the Inject HTML into an iframe technique. I was hoping that with the right XML declaration and HTML type, I could convince the iframe it was hosting XHTML. But no, I could not. I'm still interested to know if there's any way I could convince a dynamically generated iframe, with dynamic content, about the type of content it contains.

This is all part of some recent prototyping on an exciting TiddlyWiki project involving rich text editing, among other things. In a later blog post, I'll explain how we've used this SVG stuff to mash up an online chart drawing tool with TiddlyWiki. I'm currently packaging it into a plugin.

The code takes a string and injects into an IFrame. I had talked to Jon a little while ago about a similar problem and was wondering about it ever since. The technique here looks like this:

It wraps the text with tags to make it an HTML document:

It then introduces a new iframe to the page.

Now comes the tricky part. You would think you could just create a new iframe element and set its innerHTML, but with iframes you must use an internal doc ument property.

Now the content is in our new iframe. We can then manipulate the content using standard Javascript...but ensuring a call like getElementById is executed against the iframe document, not the global document. (i.e. don't use the usual document.getElementById()).

With this technique, you can take an arbitrary HTML string and delegate its parsing to the browser's built-in DOM engine.

Background: Cross-Domain Communication

Ironic that in this world of mashups and Ajax, it's not very easy to do both of them together. Ajax applications run in the browser and such applications were never intended to talk to anything but the server from whence they came. So it's not easy to mash content from multiple sources, when everything must be squeezed through the originating web server. A few hacks have arisen over the years to deal with this, such as On-Demand Javascript, and the most recent one is a hack involving iframes, which I'll explain in this article. As we'll see later, the iframe technique is arguably more secure than On-Demand Javascript, and it's also better places for communication within the browser, i.e. from one iframe to another.

Related to this article is a demo application and a couple of variants.

The first mention I've seen of this hack originated on James Burke's Tagneto blog in June, 2006, though I'm fairly certain it's been used in some quarters long before that. It's now used in production by Google in Mapplets. It's also used in Shindig for widget-container communication. The technique also happens to be the best way to make safe cross-domain calls from the browser directly to a third-party server, which is why it is employed by Facebook's new Javascript Client Library.

The Demo

First, let's see what we can do with this hack.

Demo

In this demo, we have a control on the top-level document affecting something in the iframe and vice-versa. This shows you can run communication in both directions with this technique. Typical of the technique, the communication is between two browser-side components from different domains (as opposed to browser-to-server communication, although there is actually server communication involved in making this happen).

The Laws of Physics: What you can do with IFrames

To understand the hack, we need to understand the "laws of physics" as they apply to iframes and domain policies within the browser. Once you appreciate the constraints in place, the pattern itself becomes trivial. This demo was created to explore and illustrate these constraints, and contains some simple code examples.

Definition I: A "window" refers either to an iframe or the top-level window (i.e. the "main" page). In our model, then, we have a tree-like hierarchy of windows.

Law I: Any window in the hierarchy can get a handle to any other window in the hierarchy. It doesn't matter where they live within the hierarchy or which domain they come from - with the right commands, a window can always refer to any other window. Parent windows are accessed as "parent", "parent.parent", etc., or "top" for the top-level. Child windows are accessed as "window.frames[0]" or "window.frames[name]". Note in this case that the name is not the iframe's id, but rather the iframe's name. (This reflects the legacy nature of all this stuff, relating back to ugly late-90s frames and framesets.) Thus, to get a sibling handle, you might use "parent.frames[1]".

Law II: Windows can only access each others' internal state if they belong to the same domain. This rather puts a kibosh on the whole cross-domain cross-iframe thing. All this would be so easy if iframe scripts could talk to each other directly, but that would cause all manner of security shenanigans. HTML 5 does define explicit communication between iframes, but until wide adoption, we have to think harder ...

Law III: Any window in the hierarchy can set (but not read) any other window's location/URL, even though (from Law II) browser security policies prevent different-domain iframes from accessing each other's internal state. Note: Exact details for this law needs further investigation Again, it doesn't matter which domain it comes from or its position in the hierarchy. It can always get a handle on another window and can always set the window's URL, e.g. "parent.frames[1].location.href". This establishes window URLs as the one type of information on the page which is shared across all windows, regardless of the domain they come from. It seems sensible that a parent can change its child windows' URLs, BUT not vice-versa; how strange that a child window is allowed to alter its parent's (or uncle's, sibling's, etc.) URLs! The only justification I know of is the old technique of escaping the frame trap, where a website, upon loading, ensures it's not inside a frame by simply setting the top-level URL - if it's different to itself - to its own URL. This would then cause the page to reload to its own URL. However, that's a special case and hardly seems worth justifying this much leeway. So I don't really know why you can do this, but lucky for us, you can!

Law IV: When you change the URL's fragment identifier (the bit on the end starting with a #, e.g. http://example.com/blah#fragmentID), the page doesn't reload. This will already be familiar to you if you're familiar with another Ajax hack, Unique URLs to allow for bookmarkability and page history. Normally, changing a document's "href" property causes it to reload, but if you only change the fragment identifier, it doesn't. You can use this knowledge to change the URL symbolically - in a manner which allows a script to inspect it and make use of it - without causing any noticeable change to the page content.

Exploiting the Laws of Physics for Cross-Domain Fun and Profit - The Cross-Domain Hack (URL Polling version)

The laws above are all we need to get cross-domain communication happening. The technique is simply this, assuming Window A wants to control Window B:

• Window A changes Window B's fragment identifier.
• Window B is polling the fragment identifier and notices the change, and updates itself according to the fragment identifier.

Ta-da!!! That's the whole thing, in its glorious entirety. Of course, you had to know the laws of physics in order to understand why all this works. It simply relies on the fact that both Window A and Window B have one common piece of state - the URL - and the fact that we can change the URL unintrusively by manipulating only the fragment identifier. For example, in the demo, the iframe's URL changes to http://ajaxpatterns.org/crossframe/#orange and once the iframe script notices it, it updates the colour.

A few observations:

• This works in either direction. Parent to child, child to parent. As the demo illustrates.
• It requires co-operation from both parties; it's not some magic way to bypass browser security mechanisms. Once Window A changes Window B's fragment identifier, it's up to Window B to act on the change; and it's up to Window B to be polling the fragment identifier in the first place.
• Polling the fragment identifier happens to be exactly the same technique used in the Unique URLs pattern.

There are a couple of downsides: (a) Polling slows down the whole application; (b) Polling always involves some lag time (and there's always a trade-off a and b - the faster the response, the more cycles you application uses up); (c) The URL visibly changes (assuming you want to manipulate the top-level window). We'll now consider a second technique that addresses these (albeit in a way that introduces a different downside).

The Cross-Domain Hack (Marathon version)

Here's a variant which no longer involves polling or changing any URLs. I learned of it from Juliene Le Comte's blog, and he's even packaged it as a library.

Looking back at Law II: "Windows can only access each others' internal state if they belong to the same domain". At the time, I made this sound like a bad thing, but as David Brent likes to say, "So, you know, every cloud ...". The law is bad if you state it as "cross-domain iframes can't play with each others' toys" (paraphrasing the informal version of Demeter's law). But it's good if you spin it as "well, at least same-domain iframes can play with each others' toys". That's what we're going to exploit here.

As for the demo, the functionality is the same, but since this one involves spawning iframes, I've left them intact, and made them visible, for your viewing delight. Normally, of course, they'd be invisible, and the application would look exactly the same as the previous demo.

Here's how this technique works:

• Every time Window A wants to call Window B, it spawns a child iframe, "Window B2" in the same domain as Window B. The URL includes the command being issued (as a CGI parameter, fragment identifier, or any other URL pattern which will be recognised by the destination script).
• When window B2 starts up, its Javascript inspects the URL, gets a handle on Window B, and updates Window B according to the URL (e.g. a CGI parameter).
• Window B2 destroys itself in a puff of self-gratified logic.

So in this case, we create a new, short-lived, iframe for every message being passed. Because the iframe comes from the same domain as the window we're trying to update, it's allowed to change the window's internal state. It's only useful to us on startup, because after that we can no longer communicate with it (apart from by the previous fragment identifier trick, but we could do that directly on the original window).

Window B2 is sometimes called a proxy because it accepts commands from Window A and passes them to Window B. I like to think of it as Pheidippides of fame; it passes on a message and then undergoes a noble expiration. Its whole mission in life is to deliver that one message.

This technique comes with its own downside too. Quite obviously, the downside is that you must create a new iframe for every call, which requires a trip to the server. However, with caching in place, that could be avoided, since everything that must happen will happen inside the browser. So it would simply be the processing expense of creating and deleting an iframe element. Note that the previous variant never changed the DOM structure or invoked the server.

Also, note that in either versions of the hack, there is the awkward matter of having to express the request in string form, since in either pattern, you are required to embed the request on the window URL. There is an inspired extension of this hack that also has some untapped promise in this area. It involves setting up a subdomain and updating its DNS to point to a third-party website. When combined with the old document.domain hack, you end up with a situation where your iframe can communicate with a cross-domain iframe, without relying on iframe. (The technique described in the article is about browser-to-server communication, but I believe this iframe-to-iframe is possible too.)

A Third Hack Emerges: Window Size Monitoring

A newer third hack by Piers Lawson is based around the porous nature of window sizes and the use of window.resize(). Fragment IDs are used like in the first technique here, but instead of polling, window resize events are used to cause a more direct trigger.

Applications

Cross-Domain IFrame-to-IFrame Calls ... and Widgets/Gadgets

In the world of mashups, iframes are a straightforward way to syndicate content from one place to another. The problem, though, is limited interaction between iframes; in pure form, you end up with a few mini web browsers on a single page. It gets better when the iframes can communicate with each other. For example, you can imagine having iGoogle open, with a contacts widget and a map widget. Clicking on a contact, the map widget notices and focuses on the contact's location. This is possible via Gadget-To-Gadget communication, a form of publish-subscribe which works on the iframe hack described here. And speaking of maps, check out Google Mapplets, which are a special form of gadget that work on Google Maps, and also rely on this technique.

In terms of gadgets, another application is communication between a gadget and its container, and this is something I've been looking at wrt Shindig. For example, there is a dynamic-height feature gadgets can declare. This gives the gadget developer an API to say "I've updated, now please change my height". Well, an iframe can't change its own height; it must tell its parent to do that. And since the gadget lives in an iframe, on a different domain as the container (e.g. iGoogle), this requires a cross-domain, cross-iframe, message. And so, it uses this technique ("rpc" - remote procedural call - in shindig terminology) to pass a message to the container.

Cross-Domain Browser-to-Server Calls

The best known technique for calls from the browser to an third-party server is On-Demand Javascript, aka Javascript APIs aka JSON/JSONP. This was obscure in 2005, with Delicious being the best example. Now, it's big time in Web 2.0 API land, and Yahoo! has exposed almost all of its APIs this way, and Google also provides data such as RSS content via JSON.

It works by spawning a new script element programmatically, an element pointing to an external Javascript URL. Since there's no restriction on third-party Javascript running, the browser will faithfully fetch and execute the script, and so the script will typically be written to "return" by updating variable and/or calling an event handler.

There are two major security issues with On-Demand Javascript. Firstly, you have to trust the API provider (e.g. Yahoo!) a lot because you are letting them run a script on your own web page. And there's no way to sanitise it, due to the script tag mechanism involved. If they are malicious or downright stoopid, your users may end up running an evil script which could ask for their password, send their data somewhere, or destroy their data altogether. That's because whatever your web app can do, so can the third party's script, even if you're only trying to get a simple value back. The mechanism forces you to hand over the keys to the Ferrari when all you want is a new bumper sticker. Secondly, what's to stop other websites also making use of the external Javascript? If your own site can embed the script to call a third-party JS API, so too can a malicious fourth-party. This is fine for public, read-only, data, but what if you're relying on the user's cookies to make privileged calls to the third-party? Then the fourth-party's web app will be just as capable of issuing calls from the browser to the third-party, and they might well be more evil calls than you're making, e.g. "transferFunds()" instead of "getBankBalance()". The moral is: Javascript APIs can only be used for serving public data.

Whoa!!! Public data only? That's a tragic restriction on our cross-domain API! For mashups to be truly useful, it must be personal. We'll increasingly have OAuth-style APIs where users will tell Site X that Site Y is allowed to read its data. But how can that work in the browser? How can Site Y expose its data so that it's usable from the browser, but only when Site X is running? It can't work with On-Demand Javascript. Site Y could try reading the referrer headers to see where the call is coming from, but anyone could write a command-line client with fake headers.

In fact, the answer is to use the iframe hack described in this article. As I mentioned earlier, this is how Facebook gets the job done, with what is essentially the same "power of attorney" delegation model as OAuth (BTW thanks to my colleague Jeremy Ruston for the "power of attorney" OAuth analogy - albeit it was stated in a slightly different context from OAuth).

I haven't looked too much into the mechanism involved with the Facebook API, but it looks like it's essentially using a variant of the Marathon technique. From memory, there's an ever-present invisible facebook.com iframe. Each time your web app make a Facebook call, the Facebook JS library spawns a new "proxy" iframe, which passes the message on to its same-domain ever-present frame, which makes a bog-standard XHR call to Facebook. So now we're making an XHR call to another domain, which we can get away with because it's coming from a separate iframe. Once the XHR call returns, I think the message is returned to your application (this happens via another same-domain iframe you must host on your server, though I think that's unnecessary) and the proxy iframe disappears.

Note that all Facebook ever exposes is a standard web service that relies on the user being logged into Facebook - there's no Javascript involved. The user must be logged in and must have given permission for the application to access its Facebook details. Effectively, the user is allowing a particular website URL to make Facebook calls, since the application developer must register the URL. If you look back at the iframe algorithms I described earlier, you'll see that it's straightforward for Facebook to ensure that only this application (and any other application the user trusts) can access the data. The Facebook.com iframe (whose behaviour is controlled by IFrame and can't be tampered) simply has to inspect the URL of the parent window and pass it to the server as part of the XHR call. The server can then check that the logged-in user has authorised this application, using the URL to identify it.

As for the first concern of cross-domain Javascript - having to trust the third-party API provider - I believe the iframe technique overcomes this concern too. Facebook.com never gets to run arbitrary Javascript on your server. Of course, you have to trust the Facebook library and the Facebook-provided iframe you're required to host on your domain, but those could be audited prior to installation. All those things are set up to do is call callback methods inside your top-level application; you could inspect the library and ensure that's all that will ever happen.

Thus, cross-domain iframe-based communication solves both problems which have plagued On-Demand Javascript. It is slightly more complicated, however.

Conclusions

Yes, this is a somewhat complicated technique. Actually understanding the problem it solves is really the hard part! Once you understand that, and once you understand those laws of physics, the trick is actually quite straightforward (either version of it).

The technique will be critical for gadget containers such as Shindig. As OAuth takes off, we'll also see the technique used a lot more in mainstream applications and APIs.

With HTML 5, cross-frame messaging will render the hack unnecessary for iframe-to-iframe communication. Indeed, the aforementioned Cross-Domain library uses that technique already for Opera, in a fortuitous twist of fate since Opera doesn't actually support everything this hack requires. However, the notion of using iframes for cross-domain calls will still be present, no matter how the windows talk to each other.

Update (2 days later): The site’s been popular - 10k+ views yesterday. Hit Delicious Popular and somehow got caught up in the German blogosphere, the greatest source of hits. Technorati it. There’s a good discussion in Ajaxian of the strengths and weaknesses of this technique. As with AjaxPatterns, which also reached Delicious Popular, it failed to attract Digg users somehow. (Digg was supposedly inspired by Delicious Popular. Incidentally, Digg doesn't let you submit URLs with fragment identifiers such as http://webwait.com#digg.com, which rules out any Ajax site attempting to allow bookmarks.) Go figure. Or better, go Digg .

Here's another new website - WebWait. I wanted a portable, consistent, way to benchmark Ajax web apps, that would show how long the wait is (though it's useful for any app, especially if there were a lot of images, for instance). Using a command-line tool like curl is an improper simulation and doesn't cut it as a proper simulation. WebWait has the following benefits:

• Runs in a browser. You get actual load times in the same client web users are running, not simulated times.
• Runs in multiple browsers. There are plugins that do this, but as well as the installation overhead, they are usually specific to one browser. With WebWait, you can just cut-and-paste the same URL into different browsers. (No Safari yet as it doesn't listen to iframe onload ???.)
• Respects your cookies and authentication - If you can access a URL in a web page, you can benchmark it with WebWait. Trying to set up cookies for use with a command-line tool like Curl is hard work. Doing it with a plugin is usually impossible. Doing it with a third-party website is dangerously insecure.

Quick feature list as it stands right now:

• Basic functionality: Type a URL, see how long it takes to load.
• Option: Set the delay between calls. WebWait will call the website multiple times and provide an average load time.
• Option: Set the number of calls before ceasing activity.
• Ability to pause.
• Partially transparent lightbox eye candy.
• Unique URLs - it's Ajax, but that shouldn't stop you from bookmarking and sending URLs with details of the website being tested. Incidentally, implementing this rare but highly useful feature took three lines of Javascript.

Have fun. Any comments/suggestions, please let me know!

See the FAQ for more info.

Comet - or HTTP Streaming, if you will - is a little sensitive when it comes to portability, and I'll give you four guesses which major browser is causing the grief? Yeah, IE makes it difficult for two reasons: (a) IE's XMLHttpRequest component doesn't tell you anything about the response until the connection has closed - even if you try polling it instead of relying on onReadyStateChange, you'll still get an empty string (Try it); (B) Okay, switch to plan B and inspect IFrame content - we can't rely on onload, which is only called once at the end, so we must poll. But no, polling won't help either, as the IFrame content remains empty until (you guessed it) the connection is closed. (Try it).

We're Going Back, Way Back: Inline Script Tags

Don't give up on the IFrame so fast ... we're closer than we think. Actually, the solution harkens back to one of the original Push techniques: outputting script tags with inline code (what the HTTP Streaming pattern calls "page streaming"). If you do that in your main document, the code will be executed immediately, and the same is true if you do that inside an IFrame.

I'm talking about a service that outputs stuff like this:

So Make the Server Spit Out Inline Script Tags

The portable solution is this: Have the server continuously output script tags that call a known function in the parent frame. When you set the child IFrame's source to point to this service, it will start evaluating the inline scripts as they pop out of the server. This happens to be one technique people have used for remoting for many years (I think Brent Ashley recently told me he was doing it in ?1999). The twist with Comet is that you *keep* doing it, and don't actually close the connection for a while. (Again, I'm sure some people were doing that for a long time too!).

Is it elegant? No. It means the remote service is suddenly coupled to the client-side of the web app - it has to know something about what's in the parent frame, whereas you'd often like it to be generic and just deliver a response in XML, JSON or whatever. Like most things Ajax, we're using a hack because it nonetheless push all the right buttons for our users.

Try It. Portable Comet for the Masses

Here's Some Code I Prepared Earlier

In the example above, the remote service outputs script tags in a loop - it's a simple countdown service:

And in the browser's initial Javascript, there's a run-of-the-mill onNewValue() function, it looks something like this.

See what I mean about coupling? The server-side service had to know that there's a Javascript function defined in the parent called onNewValue(). At least we've minimised the coupling by using an Observer/Event style indirection approach - evidenced by the simple call to "onNewValue()". It would be worse if it was the script that actually performed application logic directly, repainting the DOM with the new value.

IFrame is the new XMLHttpRequest

Whoever put the X in Ajax ought to put an I in Comit. IE's XHR component doesn't provide the response until the connection has closed, and I don't think version 7 changes that. Assuming (graciously) it happens in IE8, you'll need to use IFrame for the next five years if you're doing any Comet work. And of course, you won't need to do that, because libraries will do it for you.

BTW You could argue that IE is doing the right thing by preventing access to the content until the connection is closed. Maybe they are, maybe they aren't. From a pragmatic perspective, though, portable Comet requires Ajax. Alternatively, use IFrame with IE and XmlHttpRequest with the others, though I'm not sure if there's much mileage to be gained from this hybrid strategy.

A little while back, I mentioned that some people in the patterns community have been noticing the Ajax Patterns. In particular, there have been a series of discussions about the patterns by the Software Architecture Group in the University of Illinois Computer Science Dept (home of Netscape forerunner Mosaic btw). The SAG is led by Ralph Johnson (Gang Of Four "Design Patterns" author) and the group also includes Brian Foote, who blogged about Ajax as a pattern earlier on and has kindly been keeping me updated on the MP3s emerging from these discussions.

The feedback has been very helpful and I've been able to incorporate it in time for the physical publication - thanks again to everyone in the group.

While listening to the audio, I've been taking notes and writing some comments. With the permission of Ralph and Brian, I'm going to be posting these, each discussion as a separate post. It's an opportunity to see how a group of very intelligent people without much Ajax experience respond to Ajax and the Ajax patterns. You'll notice two conventions here: "TODO" is a note to myself that some action needs to be taken. "MM" signals my ideas, views, and comments back to the group.

Jan-30-2006 First Ajax Patterns Discussion

30/1/2006 Ajax Workshop

9:45 XMLHttpRequest Call

What it's about. - Probably missing in old browsers if you can't use Ajax on them. - Remote call to server without refreshing a whole page. - I assumed in JS you can open a socket, you could have done this yourself. - Depends on what the browser provides. But not cross-browser. [TODO Mention HTTP restriction and what JS can do, cf Richer Plugin] - Ironically, because JS isn't general-purpose (due to security), it's wound up being a better citizen. Also because it was kind of low-brow, everyone kind of ignored it. - The big idea is this is a way to call the server.

Interesting characteristic of XHR: - Asynchronous - Built-in security (originating server)

Did you find this pattern easy to understand? - One thing that troubles me (not particular to this pattern), pretty soon the code becomes spaghetti-like. Nd good patterns on how to manage code. [MM Agree, we need a JS patterns book! That's not the aim here, the book will make that explicit]. - Part of the problem here is JS itself. - Haven't seen the word "simple" "elegant" or "pretty" to describe JS architect. This is a Rube Goldberg solution, duct-tape ... at it's best. [MM Sort of true, but there's a lot that can be done to improve it] - There are libraries that help. (AjaxCaller, Prototype).

Writing here, even though it doesn't address these problems, could understand/follow it? - I don't like the Problems section. "How can the browser communicate with the server?" But this pattern is more specific than server communication. The next 1-2 patterns have the same problem. - But the problem might be the same, but different Solution. What I dislike is the forces are also the same. How does it mitigate the forces? Maybe there should be some different forces. - Only difference between IFrame and XHR is only restricted to same host, so maybe okay other than that. [MM Actually, even this difference doesn't exist, since you can't read a remote IFrame's content] [TODO Possibly update forces to be different, back-reflect the solution]

22:30 Doesn't say anything about being asynchronous at the start - Ajax should be highly responsive. Distributed system, so you want to minimise communications. ie Must be asynchronous [TODO Revise forces. HOWEVER, note that XHR doesn't have to be async.] - "Conventional means ... slow." He's trying to rule out Solutions, before we get to the solution. [MM This sort of goes against the previous suggestion that IFrame and XHR need different Forces. Maybe suggests different people have different views on this issue of the forces.] Doesn't so much talk about the forces as take potshots at existing solutions. [MM This is a fair point, more prevalent in the initial patterns as they're arising as a reaction to conventional web dev, but it's true you don't have to formulate them that way.]

26:30 It's a long pattern, is that okay? - Yes, longer than all the others, problem if all patterns were this long, but given it's so core to Ajax, it's fine. - Length is fine, but a lot of code there. - I would like more examples of the old way of doing things. [Covered in the Anagrams tutorial, perhaps reference it] - The general idea is if you have a big object and only small things change at a time, then you keep going back to the server and grabbing small bits of it. I think of Google Maps. - Pattern could be shorter if PHP wasn't written. - So you'd like less example code, others want more. - It's a Chimera/Frankenstein ie PHP (or whatever serverside) on the one side, and even JS is a kind of Frankenstein language. So it's important to have the PHP, reminds me that's the game I'm playing. I didn't mind it, seeing all the pieces together reminds you we're receiving small chunks etc. - The pattern is really introducing XHR, not how to use it. - Disagree with people who are trying to call everything a pattern. - Well, these particular patterns are what he calls foundational. He says they're not really patterns, they're just how the technology works. - I know what you're saying about that, and it bugs me too, but I've been trying to come up with a rationalisation for admitting that this kind of design exposition is a contribution to our software architecture literature ... and if designs recur, if a lot of people come up with the same solution to a problem ... my mind cries out to keep distinct from Visitor and ... Composite, but I don't have the vocab to keep them distinct, and I want to maintain this notion that the patterns community is talking about good ideas that keep coming up over and over that we haven't come up with ourselves ... true, it's a different kind of discussion from the GoF, without having disclaimer kind of nags at me [MM Agree wholeheartedly, why I've sounded a bit apologetic describing these as patterns, but they just fit into the overall language well. See earlier blog post.] [TODO Needs better "disclaimer" in the book] - So then what are the patterns around XHR? - Event handling, Asynchronous call - Lots of people dealing with SOA, problems s.a. async smell the same but with different names [MM Later patterns, e.g. Distr'd events] - Error detection - Invesion-of-control/DepInj/Observers. People patternising closures. There's an aspect of dealing with callbacks. Callbacks are part of the discussion here, and that's an idea that comes up here. - Feedback from the call. Or using poll. Fire-and-forget. Typical remote invocation styles: What does XHR do? - In JS: Callbacks used here (XHR), also used for user interops. We want to follow flow-of-control, but if everything is event handlers, hard to follow. (ie JS hard to follow because of this style.) A lot of the time, "callbacks" are basically continuations. It's a general pattern discussion we could have.

41:30 Real-World Examples and Code. - These systems are not thoroughly responsive as claimed. Google Suggest surprisingly fast. Others like Backpackit and Kiko not. [MM See perf optimisation, also comments in HTML Message, etc. Alex Kirk mentioned Kiko a while ago as a problem due to too much browser-side rendering.] - Need to avoid too many requests - Curious didn't mention the most famous examples (Google Suggest etc) Maybe can't see the code. But it's JS, have to be able to. But maybe unclear. [MM Yes, obfuscated. Also, tried to avoid the cliche references too much, they're mentioned elsewhere] - Discussion about mint stats package, security issues in uploading data. - Next, ways of telling your web browser what to report back. - If trying to keep people from attacking you. It's interesting to me. - Applets got crucified for some of the security problems that JS has. People moved on from panicking about them, but got away with it because browser wars finished etc.

=============================================================== 49:00 IFrame Call - Poor Man's version of the previous one. - More like a hack - What can you do with one that you can't do with the other? XHR and not with IFr? - I only know the other way round...IFrame is more compat with older browsers. - Long discussion about relative benefits etc. - Comment about Google using IFrame. Not sure if it's true as you can zoom in.[TODO More specific about how it's being used. (I think book version already does that)] - IFrame Call doesn't talk about hidden frames. [MM XHR Pattern alternatives has a detailed comparison)] [TODO XHR comparison should briefly explain how the two are diff, not just compare the benefits] - IFrame has no timeout function. (But could fake this using polling) - Calls IFrame a hack ... Pot calling the kettle black, since Ajax itself is sort of a hack. Is it bad just because it's old? [MM Again, comes down to the comparison. XHR is a more intention-based, intuitive, API, although it's true that you won't care about that because you should use a wrapper lib anyway. Better argument is the extra functionality such as call tracking.] - Would like better explanation on what's so bad about IFrames. [TODO Include x-reference in IFrame Solution. Also mention the portable wrappers in the Solution, ie you shouldn't actually care/know which you're using is the more pertinent issue here.] - I'm feeling historical. The room we used to sit in is the room where the original Mosaic web browser developers sat. Continuously astonished in the web industry, using things that in ways they weren't invented for. Go down this list, frames, tables ... Ungodly mess, but really impressed, poster child for pragmatism and "worse is better". IFrame is a typical example. - I didn't get any feel on whether I would use one or the other. - Intrigued by the history of JS. Knowing that IFrame predated XHR because features that made it easier to do the second came along in 2003. Whether it needs to be here, not sure. [TODO Sidebar?]

=============================================================== 49:00 HTTP Streaming

Is it competition for the other two?  
  - Different problem (push/streaming)
  - "How can the browser communicate with the server?" Same
    problem as the other two.
    - I don't know, it turns the table. Little pattern device
      of using the same problem with different solutions may
      be okay here. Here, the forces would be different.
      Could make the case for having the same problem.  [TODO
      Change the problem statement, just enough to make it a
      little different from previous two.]

What would make you choose this over the others?
  - Changes coming from lots of other sources, not just the
    client.
  - e.g. Chat system.
  - I don't want to do this. Don't want to grow the system
    ...
    - But scaleability is oversold. You'll never be like
      EBay, don't need to scale up like that.  [TODO Good
      point, mention in the pattern.]
    - Our wiki - 4-5 hits per second - can handle that, but
      not scaleable.  Wikipedia is not a wiki in that sense,
      pages don't change because I believe something like 90%
      of all edits are done by 200 people, these are official
      wikipedia authors. Specific process.
    - Doesn't fit into proxies. Caching etc doesn't ddeal
      with longer responses.

Long refactoring illustration here. Do the others have one?
  - Streaming wiki demo.
  - Seemed nice to refactor in this way.
  - The group is looking at the live version on the web -
    "There's about a half dozen laptops in it for the
    author's information". It worked.  [MM Phew]

=============================================================== 1:20 - I didn't get the feel of which one to use. - Was hidden in the Alternatives section [MM TODO Emphasise this in the partintro, maybe in the solutions]

====== Next time: Different patterns - Problem with web version [MM Incidentally, the wiki publishing hasn't worked out ideally, didn't get the full benefit as I didn't open up the wiki (and still haven't due to spam). Ideally would have better PS format. (Blogged about printing from the web a yr ago as it happens.)] - Being on the web doesn't seem to affect (ie drop) sales. [MM We'll soon find out...:-)]

This is the second of three podcasts on the basic Ajax patterns.

• Podcast 1: Display Patterns and the DOM.
• Podcast 2: Web Remoting - XMLHttpRequest, IFrame Call, HTTP Streaming.
• Podcast 3: Dynamic Behaviour - Events and Timing.

Podcast 2: Web Remoting (XMLHttpRequest, IFrame, HTTP Streaming)

This 75 minute podcast covers web remoting concepts and the following specific patterns:

• XMLHttpRequest Call Use XMLHttpRequest objects for browser-server communication. (05:00)
• IFrame Call Use IFrames for browser-server communication. (31:45)
• HTTP Streaming Stream server data in the response of a long-lived HTTP connection. (47:00)

Please ignore this bit, I'm claiming My Odeo Channel (odeo/bdd768d9acefcad9)

A few extracts follow, but first, let me ask you: Do you know of any public IFrame remoting examples? Google Maps is the only public example I know. All the other examples here are tutorials and framework demos.

The XMLHttpRequest alternatives was the most fun thing here (headings shown below). I went through the various techniques I've seen, read about, and tried out, to list as many ways as possible to programmatically call the server. As you might expect for these "Foundational Technology" patterns, there's a lot more code in the Solutions than in later patterns, where the focus shifts more towards the -ilities, particularly usability and maintainability.

As usual, feedback much appreciated. (Mainly on the content, rather than woeful spelling and outright grammatical disasters at this stage.)

XMLHttpRequest Call

Solution: Use XMLHttpRequest objects for browser-server communication. XMLHttpRequest is a Javascript class capable of calling the server and capturing its response. Just like a browser or a command-line web client, an XMLHttpRequest issues standard HTTP requests and grabs responses. Note: The code shown throughout this demo is closely based on an online companion demo (http://ajaxify.com/run/xmlHttpRequestCall/).

The above example hopefully illustrates that the fundamental technology is pretty simple. However, be aware that it's a very basic usage, not yet fit for production. Fundamental questions remain:

* How can XMLHttpRequests be created?
* How does an asynchronous call work?
* How do our code detect errors?
* What if the service requires a "POST" or "PUT" request rather than a "GET"?
* Which URLs can be accessed?
* How can you deal with XML responses?
* What's the API? 

The following sections address these questions and show how the code above needs to be modified. ...

Decisions

6.1 What kind of content will web services provide?
6.2 How will caching be controlled?
6.3 How will you deal with errors?

Alternatives:

9.1 IFrame Call
9.2 Persistent Connection
9.3 Richer Plugin
9.4 On-Demand Javascript
9.5 Import XML Document
9.6 Image-Cookie Call
9.7 Stylesheet Call

IFrame Call

Solution: Use IFrames for browser-server communication. IFrames are page-like elements that can be embedded in other pages. They have their own source URL, distinct from their parent page, and it can change dynamically. IFrame Calls work by making the IFrame point to a URL we're interested in, then reading the IFrame's contents once the new content has loaded.