HTTP Streaming: An Alternative to Polling the Server

If Ajax apps are to be rich, there must be a way for the server to pass new information to the browser. For example, new stock quotes or an instant message someone else just sent you. But the browser’s not a server, so the server can’t initiate an HTTP connection to alert the browser. The standard way to deal with this dilemma is Periodic Refresh, i.e. having the browser poll the server every few seconds. But that’s not the only solution.

The recent podcast on Web Remoting includes a discussion of the HTTP Streaming pattern. By continuing to stream information from the server, without closing the connection, you can keep the browser content fresh. I wasn’t aware that it was being used much on the public web, since it can be costly, but I recently discovered JotLive (which is only semi-public since it requires registration) is indeed using it. Do you know any other examples?

Ajaxian.com’s interview with Abe Fettig of JotLive:

How do you handle the “live” part? Polling?

We’re using a (very slightly modified) version of LivePage, which Donovan Preston wrote as part of Nevow, a Python library for building web applications using the Twisted networking framework (which I just wrote a book on: Twisted Network Programming Essentials). LivePage doesn’t use polling. Instead, it uses a clever technique where each browser keeps an open XMLHTTP request to the server at all times, opening a new connection each time the old one closes. That way every client viewing the page is constantly waiting for a response from the server. When the server wants to send a message to a client, it uses the currently open request. So there’s no waiting.

A few (edited) extracts from the HTTP Streaming pattern:

Alternative Pattern: Periodic Refresh is an obvious alternative to HTTP Streaming. It fakes a long-lived connection by frequently polling the server. Generally, Periodic Refresh is more scaleable and easier to implement in a portable, robust, manner. However, HTTP Streaming can deliver more timely data, so consider it for systems, such as intranets, where there are less simultaneous users, you have some control over the infrastructure, and each connection carries a relatively high value.

Refactoring Illustration: The Basic Wiki Demo, which uses Periodic Refresh, has been refactored to use [http://ajaxify.com/run/wiki/streaming](HTTP Streaming).

Solution:
Stream server data in the response of a long-lived HTTP connection. Most web services do some processing, send back a response, and immediately exit. But in this pattern, they keep the connection open by running a long loop. The server script uses event registration or some other technique to detect any state changes. As soon as a state change occurs, it pushes new data to the outgoing stream and flushes it, but doesn’t actually close it. Meanwhile, the browser must ensure the user-interface reflects the new data. This pattern discusses a couple of techniques for Streaming HTTP, which I refer to as “Page Streaming” and “Service Streaming”.
“Page Streaming” involves streaming the original page response. Here, the server immediately outputs an initial page and flushes the stream, but keeps it open. It then proceeds to alter it over time by outputting embedded scripts that manipulate the DOM. The browser’s still officially writing the initial page out, so when it encounters a complete <script> tag, it will execute the script immediately. A simple demo is available at http://ajaxify.com/run/streaming/.
…(illustration and problems)…
“Service Streaming” is a step towards solving these problems, though it doesn’t work on all browsers. The technique relies on XMLHttpRequest Call (or a similar remoting technology like IFrame_Call). This time, it’s an XMLHttpRequest connection that’s long-lived, instead of the initial page load. There’s more flexibility regarding length and frequency of connections. You could load the page normally, then start streaming for thirty seconds when the user clicks a button. Or you could start streaming once the page is loaded, and keep resetting the connection every thirty seconds. Having a range of options helps immeasurably, given that HTTP Streaming is constrained by the capabilities of the server, the browsers, and the network.

Experiments suggest that the Page Streaming technique does work on both IE and Firefox ([1]), but Service Streaming only works on Firefox, whether XMLHTTPRequest ([2]) or IFrame ([3]) is used. In both cases, IE suppresses the response until its complete. You could claim that’s either a bug or a feature; but either way, it works against HTTP Streaming.

Donovan Preston explains the technique he uses in Nevow, which overcomes this problem:

When the main page loads, an XHR (XMLHttpRequest) makes an “output conduit” request. If the server has collected any events between the main page rendering and the output conduit request rendering, it sends them immediately. If it has not, it waits until an event arrives and sends it over the output conduit. Any event from the server to the client causes the server to close the output conduit request. Any time the server closes the output conduit request, the client immediately reopens a new one. If the server hasn’t received an event for the client in 30 seconds, it sends a noop (the javascript “null”) and closes the request.

9 thoughts on HTTP Streaming: An Alternative to Polling the Server

  1. I would like to compliment you on a fantastic analysis, Michael. I am extremely glad to find such a profound, detailed and methodical classification of these patterns. Your book will be a “must have”. I’ve been involved in this research field for five years and also coined some terms to refer to some of the patterns that you mention. Recently I used the “streaming AJAX” term is some posts and press releases to refer to a state-of-the-art HTTP streaming paradigm. And the “asynchronous polling” term to refer to the mechanism adopted by solutions such as LivePage (in which the polling timing is asynchronously controlled by the server).

    I would really like to introduce Lightstreamer here. But it is a commercial product and I’m conscious this may not be appreciated in such discussions. But if it fits so well in your theory and in the examples, then as its architect and for the sake of completeness I really don’t want it to be excluded.

    The Lightstreamer project began five years ago where we decided to take the HTTP streaming paradigm to the extreme. We were working with banks that needed some push technology to update the stock quotes in real-time on their customer’s client applications. We had to solve all of the issues that you mention and many more, from scalability to cross-browser compatibility. After five years of continuous reengineering and improvement, now Lightstreamer is being used in mission critical systems to deliver real-time data to any type of client. I want to skip all the marketing stuff here and get to the point of what Lightstreamer really is.

    First of all, I strongly suggest that you take a look at our online free demos. I think they speak for themselves. You can find them at .

    So, basically Lightstreamer is a push/streaming server that has been optimised for the real-time distribution of textual data through HTTP connections. In addition to supporting traditional clients (thick/desktop applications), Lightstreamer is also able to stream data to normal Web pages, without the use of Java applets, ActiveX controls, plug-ins or Flash components. The streaming channel is handled by a hidden frame. Notwithstanding iframes are also supported too, the use of a permanent hidden frame allows the user to navigate the site without closing the stream connection at each page switch. The updates destined to multiple frames and windows are multiplexed in a single HTTP (or HTTPS) connection, on order to avoid saturation of the of the browser connection pool. The JavaScript libraries handle the subscription and unsubscription operations through short-lived control connections that affect the content of the stream connection. Five years of continuous improvement in the JavaScript code have led to a framework that is highly reliable and really cross-browser compatible. For example, if you open the Stock List Demo in Netscape 4.7, you will see it works, because each time the JavaScript layer chooses the best approach to use by taking into account the actual capabilities supported by the browser. Let me mention some other features of the JavaScript layer of Lightstreamer. For example it’s compatible with the Opera Mobile browser too, so that you can see HTTP streaming in action even on mobile phones. But we also implemented streaming graphs in pure HTML. You can refer to the Chart Demo or to the Stock List Demo (clicking on a stock name): real-time charts are plotted pixel by pixel based on the data received from the server. Well, even the streaming charts work on the mobile browser as well… (that’s something that makes people go crazy when they see the demos on their mobile phones ;-)). Then, client-side dynamic sorting and paging are available (please refer to the Portfolio demos).

    Now permit me to spend a couple of words about the server side technology. Lightstreamer Server is a stand-alone Java process. It does not rely on a Web server or Application server, because it needs to have the direct control over the TCP/IP stack to optimize the data transmission. For mission critical applications or systems with a lot of concurrent users, you cannot simply dump on the network what you receive from your data feed and forget all the rest. You need bandwidth management, congestion control, filtering, throttling, etc. For example it is possible to individually assign a maximum bandwidth for each user’s streaming channel. The data will be dynamically filtered in such a way that the used bandwidth remains within that allocated. It is also possible to assign a maximum update frequency for each subscribed data item. Lightstreamer Server is also able to detect any network congestion and then adaptively modulate the data transmission in such a way that it will never send more data than the network is able to handle at any given instant (if you are on a mobile network and the bandwidth shrinks, Lightstreamer does not buffer old data but automatically starts sending less updates, if MERGE subscription mode is being used).

    To finish (or I really risk to go into a marketing mode…) it is important to mention the performance and scalability topic. Lightstreamer Server is based on a staged event driven architecture. This means that there is no relation between the number of threads used and the number of concurrent connections supported. In practical terms, this translates in the ability to support 10,000 concurrent streaming users with a single Pentium 4 CPU running at 2.4 GHz (in a benchmarking scenario in which 1 event per second is transmitted to each client).

    Based on your experience, I would love to have a feedback from you regarding our technology. Please also feel free to use our online demos in the example sections of your material.

    For any further information you can see the web site (where a white paper is also available): . Or contact me directly at .

    Thank you very much and congratulations for your great work!

    Alessandro Alinone

  2. It’s worth noting that this aproach is neither new or novel. KnowNow (and the offshoot mod_pubsub project) have had implmentations of this as early as ’99 or thereabouts.

    The real change here is Twisted Python, which allows large numbers of these “zombie” connections to be handled efficiently. The kinds of workloads that these “streaming” solutions involve would melt your normal apace configuration down to a pile of slag.

    Regards

  3. Hi

    I want to make page which have live quotes from the stock market. How can i make this, I think u people r the right candidate to ask help for this.

    Regards Jais

  4. Excellent work, and we were testing your app and we have a couple comments:

    1.- Memory leak at the browser level, and some of your demos are designed to run for a long period of time using JS code.

    2.- And about the server scalabity the limitation will be about the simultaneous concurrent connections, every connection will be a socket and probably a thread in the server side, regarless of the frecuency. Our of curiosity how much in RAM that machine you refers it has?

    Congratulations for the great work.

  5. I am very interested in the server pushing technology which is implemented by XMLHttpRequest object, like Comet. I have read many materials about that, one of which is ‘Http Streaming’ by Ajaxpatterns.org. But the method forward by it seems only work on firefox and many people think It is unappropriate to use the readystate 3 of XMLHttpRequest.

    I am eager to know whether there are other ways to do that, I know you are expert of that, could you please give me some directions? I will be honored to receive your advice.

Leave a Reply