Software As She’s Developed

Mahemoff’s Podcast/Blog – Web, Programming, Usability from the Author of ‘Ajax Design Patterns’ (AjaxPatterns.org)

Software As She’s Developed header image 1

Another HTML5 Tidbit: Web SQL Database

May 27th, 2010 · SoftwareDev

I coded up a small demo of Web SQL Database. As the demo points out, notice you can hit reload and the data remains, being that it’s one of the several techniques for offline storage.

There’s not much to say about Web SQL – you either know SQL already, in which case it’s fairly straightforward and just a matter of squeezing your SQL calls into the conventions of the API, like any other SQL framework as, lamentably, there’s never been any kind of standardisation on the wrappers that go around SQL – or you don’t know SQL, in which case there’s your bottleneck child if you intend to use Web SQL Database for offline storage.

Web SQL Database lies at the other end of the complexity spectrum from Web Storage (aka localStorage and sessionStorage), which is simply a big ol’ hashmap for your domain. The complexity is fairly manageable, though, as there’s literally a generation of SQL know-how out there, and hopefully ActiveRecord-like libraries on their way.

IndexedDB promises to offer a third way – more like the hashmap approach, but with some of the indexing-based performance gains SQL has to offer. No browsers have implemented it yet, though you can build an experimental windows version using a third-party Firebreath implementation.

→ No CommentsTags: ····

Upcoming: UXCampEurope and SWDC

May 22nd, 2010 · SoftwareDev

Google I/O is over and I’ll post a bit about the HTML5 hack session I ran later, but here I want to highlight a couple of upcoming sessions in Europe:

UXCampEurope. If this is anything like the Bay Area and London UX camps I’ve been fortunate to attend, it will be huge, and being Europe-wide and in Berlin, this expectation might just be met. I was sitting on the browser along with @mattlucht hitting refresh 10 times a second when the tickets were released at the start of the year! I’ll set up a slot called “What did HTML5 ever do for users, anyway?”. I’m planning to overview some of the features of HTML5 and its evolution from Ajax, and ask how it might be used to improve UX. It’s a camp, so I’ll also be hoping to collect contributions and writing them up somewhere.

SWDC. The first Scandinavian Web Developer Conference and Peter Svensson is organising, so I know it will be an awesome event, and the sessions speak for themselves. I’ve bumped into the guy twice in the past month – he travelled from Stockholm to DC for JSConf, and again to SF for I/O. My session is on the mobile day, called “HTML5 Gives You Wings”, focusing on HTML5 techniques for performance and app-like behaviour. Here’s the summary:

Welcome to the dynamic world of mobile development, where new browsers stay close to the edge and HTML5 is already a reality. Despite the impressive advances, many mobile apps are still bottlenecked by the network and compact processors continue to lag behind their desktop counterparts. So how can HTML5 help? This talk will focus on those features of HTML5 that are interesting for performance optimisation and the techniques for emulating native apps, such as offline data storage.

→ No CommentsTags: ········

Offline Apps with Application Cache: Quickstart, Tips, and Deep Dive

May 19th, 2010 · SoftwareDev

I've been mucking with AppCache, aka ApplicationCache. It's the secret sauce that lets you build offline apps, which is great for performance and fabulous for pretending to be an iPhone app when you're not.

Quickstart an Offline App

As with most HTML5 technologies, the basic usage is quite simple:

  1. Create a trivial trio of HTML, CSS, and JS files, maybe with an image or two. (Or reuse an existing toy app.)
  2. Link to a "cache manifest" file from your HTML file's html tag like so:
    HTML:
    1. <html manifest="clock.manifest">

  3. Make the manifest file. It's just the literal phrase "CACHE MANIFEST" on the top line (I know, bit random), followed by the list of files in your app:
    CACHE MANIFEST
    clock.html
    clock.css
    clock.js
    clock.png
    
  4. Mark it as MIME type text/cache-manifest. If you're SSHing to a virtual host (like I did with Dreamhost), all you need here is a one-line .htaccess file:
    AddType text/cache-manifest manifest
    

That's it. Now those files above will be cached locally and won't need to be downloaded again next time. To test it, load the site, shut down your net connection, and reload. It's still there innit.

It works the same on your app-phone - on iPhone or Android (or others maybe), load the site, switch to airport mode, go back to the browser and reload ... the site's still there. Where it gets really fun is making it into a regular app:

  • On iPhone, bookmark the site with the Safari "+" button, choose "Add to Home Screen", and the app will be sitting alongside all your other apps on the home screen.
  • On Android, bookmark the app in the browser. Hold your finger down somewhere empty on the home screen, choose add "Shortcuts" | "Bookmark", and choose the bookmark you've made.

You could do the same with any website of course, but it's particularly cool when the website is an offline one...it works just like a regular app!

Practical Tips and Troubleshooting

Purging the Cache

In practical terms, you are quickly going to come up the no. 1 problem of developing against a cache. How do you purge it whenever you update your code?

Simple. You only need to make an arbitrary change to the cache manifest file, so just include a version number in the comment:

CACHE MANIFEST
# v1
clock.html
clock.css
clock.js
clock.png

Keep revving it up every time you change any file, because changing those files alone won't have any effect. And if it's getting serious, you'll want to practice the principles of DRY and continuous integration by automating this process.

Ridding yourself of Errors

An unfortunate thing happens when there's a downloading error: The whole thing silently fails. Until debugging tools get better, if you find your app's not updating, you ought to check each of the resources specified in the manifest really, really, carefully.

Cached Resources

Cache manifests are not exempt from standard caching rules, so the manifest itself, as well as other files might be cached. Pain. A good solution is to use the following Apache directives to .htaccess:

ExpiresActive On
ExpiresDefault

That MIME Type

Things might fail if the Cache Manifest MIME type is wrong, so use your browser's debugging tools or a service like Web Sniffer to check it (thanks CSS Ninja).

The Wording

The wording can also trip you up. Be really sure it's exactly "CACHE MANIFEST" at the start of the file. Again, better debugging tools in the future will help here.

The Fancy Stuff

All of the above is enough to get up and running with an offline app, and maintaining it too. But wait, there's complexity too!

Fine Control Over What Gets Cached

The cache manifest file actually allows three kinds of resources:

  • Online resources that can be cached (CACHED). As shown in the example above, it's the stuff that's available for caching.
  • "Fallback" resources in case a file isn't cached (FALLBACK). These are like catch-all email addresses; any resource that's not cached, is a candidate for a fallback. There's a pattern-matching algorithm to map resources (which mightn't have been cached) to other resources (which hopefully have been cached). For example, you probably wouldn't cache every wikipedia article, so the fallback section would ensure a special "offline.html" page is shown for those missing articles.
  • Online-only resources (NETWORK) Resources which should never be cached, e.g. it would be a good idea not to cache stock prices if people were making million-dollar decisions on the back of what they see on your possibly-cached web application.

The syntax (adapted from the WHATWG example):

CACHE MANIFEST
# a comment
# v37
# (version number unnecessary
# but recommended for cache purposes)

# the following "CACHE:" is optional; # a special case because top of file # is cache by default # CACHE: images/sound-icon.png images/background.png # note that each file has to be put on its own line

FALLBACK: / /sorry-i-am-offline.html

NETWORK: /stocks/*

Checking if We're Online

window.navigator.onLine ... try it now in your console (and try it again with your connection off).

Monitoring State and State Changes

What do you notice about all the app cache stuff above? It's all declarative, no Javascript. But you can go on to do extra stuff with Javascript too, via the Application Cache API. (I am somewhat skeptical about the applications for this API, since simple and declarative is the way to go for something as potentially hairy as offline...but that's for another rant.)

The window is effective tied one-to-one to the cache object, so you can just access the cache with window.applicationCache. For example, you can get cache state with window.applicationCache.status, try it now in your console. It will probably be zero...the possible values range from 0 to 5, representing: UNCACHED, IDLE CHECKING, DOWNLOADING, UPDATEREADY, OBSOLETE. (These constants are also available on the cache object, i.e. window.applicationCache.UNCACHED is 0.)

There are also event handlers to monitor event state: onchecking, onerror, onnoupdate, ondownloading, onprogress, onupdateready, oncached, onobsolete.

Forcing Updates

applicationCache.update() will force an update, i.e. start downloading the application again if the manifest has changed. However, it won't suddenly switch over after downloading. To do so would be to pull the rug from under your live, running, application. It will be there next time. But if you do want it right away, use applicationCache.swapCache().

Offline Mobile Apps: It's More than Application Cache

Application cache certainly is a critical feature of mobile apps; it's a basic user expectation that a mobile app starts right away, unlike a complex web app which must be downloaded. However, there are other things too. For example, offline storage. That's another feature of HTML5, comes in many incaranations, and something for another day. The other really big deal is the UI. Of course, you need to simulate certain UI features of the native apps if you want your web app to go native. Hence, libraries like IUI, jQTouch, and Apple's secret PastryKit. On the input side, your UI must also handle touch events, something some of the aformentioned brand of libraries will support. Finally, there will be specific features of the platform that can help; for example, setting the home screen icon and going full-screen so the browser chrome doesn't show up.

→ 5 CommentsTags: ··

Cached Positions in the Geolocation API

May 4th, 2010 · SoftwareDev

The Geolocation API is surprisingly short and simple. A simple lookup is a doddle:

JAVASCRIPT:
  1. navigator.geolocation.getCurrentPosition(function(position) {
  2.   alert("You're at " + position.coords.latitude + ","
  3.           + position.coords.longitude);
  4. });


About the most complicated thing is "cached positions", and even those are quite straightforward. So what's that all about?

A cached position is what the API returns when you don't need a live, "this is where the user is right now", position. Geo-lookups can be time-consuming, so sometimes you might opt for a slightly stale location instead of making the user wait for a lookup. You might also be helping the user to reduce battery life or bandwidth consumption.

In the simplest usage, a real lookup will always occur:

JAVASCRIPT:
  1. navigator.geolocation.getCurrentPosition(showMap); // no options? no cache for you.


However, you can also say "I'll happily accept a location from the past minute (60,000 ms) if you know it"

JAVASCRIPT:
  1. navigator.geolocation.getCurrentPosition
  2.   (showMap, handleError, {maximumAge:60000});


Set maximumAge as Infinity (a valid Javascript literal) to say "just give me a cached value you cached any time, I don't care how far back".

Using a second timeout parameter, you can force the position to be cached, i.e. prevent a lookup altogether, even if there's no cached value, in which case you get an error.

JAVASCRIPT:
  1. navigator.geolocation.getCurrentPosition
  2.   (showMap, handleError, {maximumAge:60000, timeout:0});


(The spec uses "position" and "location" interchangeably; ideally a spec like this would use a single term, but at least in this case they do mean almost the same thing.)

→ No CommentsTags: ·

HTML5 is a Brand

May 3rd, 2010 · SoftwareDev

HTML5 has conflicting definitions.

If you go by the official WHAT-WG spec, HTML5 is a list of specific features, including Canvas, Video, Audio, the new form controls, the new semantic markups, and microdata.

If you go by the media and Steve Jobs, HTML5 is simply a new platform that lets us do rich, interactive, applications without the need for the desktop, or browser plugins. This definition implies a superset of the WHAT-WG HTML5 features. As the HTML5 spec notes, the following are frequently confused as being part of HTML5:

  • Web Workers
  • Web Storage
  • WebSocket API
  • WebSocket protocol
  • Server-sent Events
  • Web SQL Database
  • Geolocation API
  • SVG
  • MathML
  • XMLHttpRequest

The spec says it's the media who confuse those items; I'd say it's just about everyone who doesn't read Last Week In HTML5 (NSFW) religiously, i.e. most people. Furthermore, I would add the following delicious ingredients to the HTML5 mix:

  • CSS 3
  • ECMAScript 5

And now that we're getting real, as far as most people are concerned, HTML5 is about what the technologies do for end-users. So I'll add the following, which all have an impact on programmer productivity and hence indirectly improve the user experience by making development more agile and effective:

  • Javascript libraries and frameworks
  • Web Development Tooling
  • Web Development Theory (guidelines, principles, hacks, "browser physics", patterns)

And I'll also add the containers in which web apps run, where features such as add-ons and improved security are also making things more splendid for users:

  • Browsers
  • App wrappers (PhoneGap, Titanium, Air)
  • Operating Systems (WebOS, ChromeOS, Yahoo! TV widgets, Web Widgets

And in addition, there are the anticipated features, which the HTML5 spec says will come with a later version, but are still considered by many to be the promise of HTML5:

  • Device APIs (microphone, webcams - these can't come sooner!)
  • Ping attribute

If you want, you can stick your head in the sand and declare "HTML5 is the limited subset of technologies addressed in the WHAT-WG specification". However, the ship has already set sail and thanks to the ongoing Apple vs Adobe saga, HTML5 has been deeply etched into the fabric of the industry as "the technology that makes rich web applications". HTML5 is now a brand referring to all of the things mentioned above. Better get used to it.

→ 5 CommentsTags: ··

Audio/Video Tag: Don’t Forget to load() before you play()

April 30th, 2010 · SoftwareDev

This is a gotcha that will catch a lot of people out because it goes against the expectation that you just change an image's appearance by setting its "src", done. And also, it's a deviation from the more general convention that you change DOM properties declaratively to make stuff happen. Probably for good reason, given the temporal nature of audio and video playback, but just beware it's a different model.

Bottom line is - load() between changing src and play()ing:

JAVASCRIPT:
  1. function play(url) {
  2.   var audio = document.querySelector("audio");
  3.   audio.src = url;
  4.   audio.load(); // !HUL|_O! PAY ATTENTI0N!
  5.   audio.play();
  6. }


A lot of online sources talk about play() and omit to mention this vital fact, because they're only play()ing a single track...and the first track works fine. This is an oddity - you can just call play() the first time and it will automatically load. This happens whether you set "src" in the initial HTML's "audio" tag or whether you set it programatically. I did some fiddling and it looks like the behaviour is the same in Chrome and Firefox. It's like the track is auto-load()ed the first time you set its source. (And note that Firefox doesn't auto-buffer so that's nothing to do with it.)

devx has you covered on this topic, and points out the subtle issues which you would need to look at for a quality production:

However, media by its very nature is a time-spanning process. You're working with both the need to load and cache audio and video files and the network connection, which itself can prove to be a challenge that's usually not a factor for most non-temporal resources. For these reasons, when dealing with video and audio on the web, if you want to take control of the process in any way, you have to work asynchronously, tying into the various events that are passed back to the client from the media-serving website. Table 4 (taken directly from the HTML 5 documentation) provides a full listing of the various events and when they are dispatched. ... Of all the events in Table 4, perhaps the most useful are the canplay and canplaythrough events. The first, canplay, will fire when enough data has been loaded for the video player to start actually rendering content constructively, even if not all of the data has been loaded. The canplaythrough event, on the other hand, will fire when the data has essentially completely loaded into the browser's buffer, making it possible to play the video all the way through without the need to pause and retrieve more content.

→ 2 CommentsTags: ···

500th Blog Post

April 29th, 2010 · SoftwareDev

That's all.

→ No CommentsTags:

Yoink: Extracting All Scripts and Stylesheets on the Page

April 26th, 2010 · SoftwareDev

This here is a script that will put the code of all scripts and stylesheets into a single variable. Usage:

JAVASCRIPT:
  1. yoink(function(all) {
  2.   console.log("scripts", all.scripts); // string array
  3.   console.log("stylesheets", all.stylesheets); // string array
  4. });


As you can see, you have to provide a callback because this stuff is asynchronous. It downloads the files one at a time to keep things simple and without using any fancy-pants Ajax queueing library. And right now, timeouts won't be too graceful. Also, browsers will typically allow scripts and stylesheets to work cross-domain, but of course XHR won't. So any external scripts and stylesheets are unfortunately out of reach and will not be captured. This is an especially tragic circumstance in the case where they are hosted on a different subdomain. But, as they say, "hey". I don't know what that means, but, as they say, "hey".

Yoink will download inline scripts (script tags with bodies), remote scripts (script tags having a "src" attribute), inline style attributes (elements having a "style" attribute), inline style tags (style tags with bodies), and remote stylesheets (links of type "text/css").

There are several purposes. I did a quick hack version of this a while back to improve logging info; in principle, a logging method could introspect on the source tree to show extra context info (locate the log message I'm currently outputting, and then say which line it's coming from, for example).

In the case of Single Page Applications a la TiddlyWiki, a plugin might use Yoink to inline all the remote content.

Right now, I'm more interested in using it for browser extensions, such as the Feature Creep extension I suggested, to show what's happening on the page. Some of that stuff you can work out by inspecting the DOM, but other stuff ... show me the code!

Get it here - http://gist.github.com/378850. Shown below for convenience:

JAVASCRIPT:
  1. function yoink(complete) {
  2.  
  3.   var all={
  4.     scripts: [],
  5.     stylesheets: []
  6.   }
  7.  
  8.   function downloadResources(urls, onComplete) {
  9.  
  10.     var resources = [];
  11.  
  12.    (function inlineResource(index) {
  13.  
  14.       if (index==urls.length) {
  15.         onComplete(resources);
  16.         return;
  17.       }
  18.  
  19.       var xhr = new XMLHttpRequest();
  20.       xhr.open("GET", urls[index], true);
  21.       xhr.onreadystatechange = function() {
  22.         if (xhr.readyState!=4) return;
  23.         if (xhr.status==200) resources.push(xhr.responseText);
  24.         inlineResource(index+1);
  25.       }
  26.       xhr.send();
  27.     })(0);
  28.  
  29.   }
  30.  
  31.   var scripts = document.getElementsByTagName("script");
  32.   var scriptSources = [];
  33.   for (var i=0; i<scripts.length; i++) {
  34.     if (scripts[i].src) scriptSources.push(scripts[i].src); // remote
  35.     else all.scripts.push(scripts[i].innerHTML); // inline
  36.   };
  37.   downloadResources(scriptSources, function(resources) {
  38.     all.scripts = all.scripts.concat(resources);
  39.  
  40.     var styles = document.getElementsByTagName("styles");
  41.     for (var i=0; i<styles.length; i++) { all.stylesheets.push(styles[i]); };
  42.     var allElements = document.getElementsByTagName("*");
  43.     for (var i=0; i<allElements.length; i++) {
  44.       var inlineStyle = allElements[i].getAttribute("style")
  45.       if (inlineStyle) all.stylesheets.push(inlineStyle);
  46.     };
  47.     var links = document.getElementsByTagName("link");
  48.     var cssHrefs = [];
  49.     for (var i=0; i<links.length; i++) {
  50.       if (links[i].type=="text/css") cssHrefs.push(links[i].href);
  51.     };
  52.     downloadResources(cssHrefs, function(resources) {
  53.       all.stylesheets = all.stylesheets.concat(resources);
  54.       complete(all);
  55.     });
  56.  
  57.   });
  58.  
  59. }


→ No CommentsTags: ··

SPA Hacks: Hacks Emerging From the World of Single-Page Web Apps

April 14th, 2010 · SoftwareDev

This is a permalink for my JSConf.US talk. Full slides are online here:

THE LOST HACKS: Ridiculous browser tricks from the world of single-page Applications

The talk will overview TiddlyWiki and Single-Page Apps, and then cover eight specific hacks:

  • File access without browser extensions
  • Javascript-HTML chameleon files
  • SVG-VML chameleon files
  • Inline SVG
  • iFrame squriting
  • Script islands
  • Embedded images
  • Fragment IDs

I'll be posting a link to the slideshow notes from here. I'm pleased to say the nascent TiddlySlidy app has been a pleasure to dogfood it in.

[Image credits]

→ 2 CommentsTags: ····

Web-O-Random is New

April 14th, 2010 · SoftwareDev

Web-O-Random is a new site I made, a simple random number generator with simple URLs. The About Page tells the story:

Random Numbers

Random Lists

Programming Formats

(The old Web-O-Random is late. Deceased. Run down the curtain and joined the bleedin' choir. This is another of those things I was planning to do for several years and finally got around to it.)

→ No CommentsTags: