Embedded Images in TiddlyWiki Under IE6 via MHTML – Proof-of-concept

tiddlywiki mhtml images (by mahemoff)

I only came across the MHTML image hack over the weekend, while listening to @jeresig on the new jQuery podcast (incidentally not the only jQuery podcast to be launched in the past week or two).

The MHTML image hack lets you embed images with text, just like ye olde data: URI hack, but in a way that works for IE6. MHTML is MIME for HTML.

Of course, I immediately wondered if it could work in a single-page web app like tiddlywiki, and it turns out it can, though my quick exercise still has some problems.

IE6 TiddlyWiki images demo here

As I wrote on the demo itself:

Normally, images are contained in a separate location, pointed at from HTML IMG tags or from CSS background-image properties. However, Tiddlywiki is a style of web app where everything resides in one file. So how to include images?

The usual hack is to embed data: URIs (http://en.wikipedia.org/wiki/Data_URI_scheme). However, no go for IE6 and IE7. Hence, a “newer” technique – newer meaning recently discovered. That is, MHTML (http://www.phpied.com/mhtml-when-you-need-data-uris-in-ie7-and-under/).

I was curious if MHTML worked in single-file HTML pages, reading off a file URI, and to my surprise it does. That said, it’s not perfect at all. Firstly, I had to hard-code the location, because I don’t know how to refer to “the current file” within the MHTML link. (I suppose a workaround would be to output the image file and refer to that with a relative URL, but we lose the benefit of everything being in one file.) Secondly, I played around with various base-64 images and this arrow one (from the phpied.com demo) was the only that worked :(.

So it’s a proof-of-concept with many gaps left for the reader to fill!

Hopefully people play with this further. At 3002 days old and counting, your grandpa’s browser isn’t going anywhere fast.

Whoa! jQuery’s Adding User Comments to their Docs

I just heard on the new jQuery podcast that jQuery docs will be getting comments. jQuery just got a whole – even more – useful. This is direct from the horse’s mouth, an interview with jQuery head honcho John Resig.

I’ve felt for a long time jQuery has been ahead of the pack with its documentation, and since then both jQuery and its docs have gone from strength to strength.

One thing I was still missing though is user comments.

PHP leads the way here, with the official API docs allowing user comments at the bottom (example). No-one can anticipate all the issues that will arise, so why not open up docs to the developer community? Allowing user comments is also an easy way to help keep the docs up to date (if something of a compromise.) It could also be done as a wiki, if the wiki is managed well. (The Rails wiki is a good example of a well-maintained wiki that receives frequent contributions from people in the know.)

I was missing it until I heard that announcement. Again, jQuery’s leading the way.

I realise there are spam complications, but user comments are a no-brainer for making documentation useful. I continue to derive huge benefit from user comments. (To wit: The user comments about random number seeding were of vital importance in the making of the site I realised earlier today. I use them pretty much every time I have a PHP problem.)

I’ve even thought of making such a project myself, so much did I miss doc based comments.

The docs are being migrated from MediaWiki to WordPress. FWIW I think the OWF could benefit from a similar architecture if the wiki is going to engage with the community. As I’ve learned from AjaxPatterns and maintaining the tiddlywiki comments plugin, most people would rather append comments than updating a wiki. Updating a wiki is intimidating for some, and in any event, requires much more thought in order to make it coherent. Comments tend to be easier to moderate too because of their atomic nature.

Anyway, great news for jQuery and the community!

It starts with a Yellow Fade: The need for a more comprehensive understanding of visual effects on the web

Around the time Ajax got coined, one of the already-known patterns was 37Signals’ Yellow Fade Effect. As techniques were shared and visual effects libraries emerged, we began to see visual effects become commonplace on the web. I documented four of them in Ajax Design Patterns: One-Second Spotlight, One-Second Mutation, One-Second Motion, Highlight. (I wish I called them something else, e.g. “Spotlight Effect”.) And all of these are around today, and even in the jQuery core which makes t hem pretty darn accessible to any developer. It’s as simple as $(“message”).fadeIn().

Where we are now, most developers are stuck with trial-and-error. In an ideal world, you’d have a SWAT team of UX experts to analyse each visual effect, test with users, and maybe they’d get it right, but most UI design in the real world and developers often just have to play around a bit and then move on to something else. So I think we could do with some simple guidelines, ideally based on empirical data, but even just based on pattern mining the web would be handy.

I’ll give some examples of the kinds of problems I’ve faced as a developer building in visual effects:

  • Choosing Visual Effect. I want to update some content that’s just been received from the server. e.g. the football score changed. Do I show a yellow highlight, make it blink, make it vibrate, do nothing?
  • Choosing Effect Parameters. I’ve decided to use a Slide Down effect. How longshould it last, i.e. the duration of time from not shown to show? Should it slide down at constant pace or accelerate and bounce at the bottom?
  • Composing Effects I have a block of content with the user can edit. When switching it between edit and view modes, there are two simultaneous effects taking place – one thing is being hidden while the other is being shown. How do I handle those? Do I slide one up and when that’s done, slide down the other? Do I fade one out and simultaneously fade one in? Do I immediately close one and transition in the other, or vice-versa? Many options come into play.

It should go without saying that there’s no right answer for any of these problems. But instead of developers always working from first principles, I think there are some very useful guidelines and discussions that could be had around the design patterns involved. Maybe even leading to a high level effects library supporting those patterns.

InfoBoxPlugin: A TiddlyWiki Plugin for InfoBoxen

InfoBoxPlugin - Lists tiddlers in a table (by mahemoff)

G’Day, here’s a new tiddlywiki plugin I’ve been working on: InfoBoxPlugin. It’s based on the equally-monikered infoBox in MediaWiki/Wikipedia, which you’ll see in any article that is marked “current event” or “controversial”, for example, on the big W. I find infoBoxes elegant, as they are unobtrusive enough to let you get on reading the article, and are easily ignored in the same way as web ads, but when you do focus on them, they are clear in meaning and support pattern recognition, with each type of infoBox having its own look and distinct icon.

The infoBox macro makes infoBoxen that look as shown in the diagram above. In the simplest case, you type this into a tiddler:

<<infoBox>>This is good stuff – pay attention mkay.>>

Pretty straightforward. One funniness here is the bracket asymmetry. We’re trying to do something similar to XML tags, what with their start tag and attribs, followed by body, followed by closing tag, so you’d expect to see:

<<infoBox>>This is good stuff – pay attention mkay.<</infoBox>>

or sumptink like dat. But one of my lessons was that tiddlywiki has a somewhat unusual convention, enshrined only in the gradient macro to my knowledge, for the former syntax. The only way to do something else would be to refactor or replicate or hijack the core code, and I don’t fancy it, and in any event it would go against the standard already set by gradient. In any event, I was pleased enough when Jeremy showed me the example of the gradient tag, which shows that this kind of “macro body” is even possible, so I settled with that.

Okay, so that’s nice, you can do a simple message, but where it gets more to the point is where you build up a family of infoBox types for your TiddlyWiki, each having a separate definition tiddler. For example, you will often want a “warning” infoBox to appear in various places. So you make a “warningInfoBox” tiddler, with the following text:

|heading|Danger, Will Robinson!!!|
|message|Please follow these instructions carefully.|

And then in a tiddler, you just write <<infoBox warning>>>> The association between “warning” and “warningInfoBox” is an enforced convention; my macro just appends “InfoBox” to the type you declare. The double-double closing bracket is a consequence of what I said above, combined with the fact that the definition has a pre-defined “message”. This is often going to be what we want, but not always. In fact, all of the fields above are optional, so you could leave out “message” and then the warning macro would have to specify it (if you wanted a message, that is). i.e. <<infoBox warning>>This Is Serious Mum.>> (… because the undeniably-talented, always-controversial, Aussie band This Is Serious Mum – aka TISM – popped into my head as an instructive example for this warning.)

Some other things to say

  • This plugin is part of a greater effort to pull out the goodies from TiddlyGuv into reusable modules. TiddlyGuv was the first thing I worked on upon joining Osmosoft, and when I looked it at with fresh eyes recently, I realised how much of what I built is just generic TiddlyWiki functionality. I now have a better understanding of what makes an independent TiddlyWiki plugin, and indeed I have a much greater respect for the whole modularity concept in TiddlyWiki, because it really satisfies all the traditional software engineering principles of encapsulation, orthogonality, etc etc in a neat bite-size way, which I will have to explain more about elsewhere. infoBox is the first of several plugins that need to be exorcised from the TiddlyGuv base.
  • The “macro body” thing is actually quite easy – you just call wikifier.subWikify(domEl, ">>"); where domEl is some DOM element. This will keep reading from the end of the macro definition to the next occurrence of >>, and fill domEl with the results of wikifying that content. It would be nice if there was a function to let you just get the string, instead of sticking it in the DOM, but you could still achieve that yourself by shoving it into a hidden element and capturing its innerHTML. The only complication for me was a common one these days of jumping between JQuery-land and traditional DOM-land, since TiddlyWiki now ships with JQuery, but much of the infrastructure has not (yet) been retrofitted to talk JQuery. Once I did a JQuery-to-DOM conversion, it worked fine.
  • Another interesting detour that happened here concerned cross-referencing tiddlers. Something you will want often want to do with infoBoxes is use icons. In a single-file TiddlyWiki, often intended to be offline, the icons would need to be data: URIs. This applies to other plugins too, so I dedcided to look into it a bit. I wanted a way for the “iconURL” slice in the InfoBox definition to either be a regular URL, or a reference to another tiddler containing the data: URI, so as to isolate it away. This led me to learn more about transclusion. With thanks to @FND, and after some experimentation, I found the easiest way was to support linking to sections of another tiddler. See how it works in the demo tiddlywiki, where an “icons” tiddler contains the data:URI icons.

Wow, I didn’t expect to say so much about what is not a major plugin. It comes at a time when I’ve been making some realisations about tiddlywiki, hence the verbeage.

SimpleMessagePlugin: Unobtrusive TiddlyWiki Status Message

To simplify TiddlyGuv message rendering, I made “SimpleMessagePlugin”. It removes the message box 1 second after a message was shown (using displayMessage). In the event another message appears in that time, it appends the message (as it normally does) and extends the message box’s lifetime by a second. In other words, it always closes a second after the last message was shown. The algorithm is a pretty similar throttling deal as Ajaxagram.

Demo here.

Latest version from SimpleMessagePlugin.

State of jQuery 2009

Fred noticed Jon Resig’s State of jQuery slides are up:

Highlights for me:


  • Now using sizzle, the quick, generic (not just JQuery), selector engine. Slide 8 show s almost 2* performance improvement on IE6 from JQuery 1.2.6 -> 1.3.2.
  • Performance improvements are apparently dramatic (browser averages):
    • width() / height() ~4* speedup (good – I need these for http://caption.im8ge.com)
    • hide() / show() ~3* speedup (good – I use them everywhere!)
    • element insert: : ~6* speedup (likewise)
  • Test Swarm is go! Awesome, Awesome initiative. I hope we take advantage of it for TiddlyWiki testing, once it extends to other frameworks. (Or make something similar.)

  • Ridiculous growth and market share (p15-17), and new big sites using JQuery (p13): Whitehouse, Wikipedia, Amazon, Microsoft. Not too shabby eh. Stuff like this makes it much easier for advocates to sell in their organisations.

  • Setting up private CDN on Media Temple, who’s donating all the resources.


  • More performance improvements in 1.3.3 – p23 shows 3.5x improment over 1.3.2! (perhaps for a subset, but it’s still a big and useful set).
  • JQuery builds optimised for mobile.
  • Moving parts of JQuery UI widget -> core.
  • Four conferences in 2009 – online, London, SF, Boston.
  • Better community sites, inlcuding online forum. Plugin site rewrite.
  • All plugins on CDN..
  • Core -> Github.

JQuery IFrame Plugin

This article by Nick Zakas, covering some technical issues in iframe loading, triggered me to surface a JQuery IFrame plugin I made a little while ago, which supports loading IFrames. (I did tweet it at the time, though I’ve since changed the location.)

JQuery IFrame plugin

The plugin basically tells you, the programmer of the parent document, when your child iframe has loaded. It also has some other bells and whistles, like timing the load duration.

It’s inspired by the use cases of WebWait (http://webwait.com) and the Scrumptious trails player (http://scrumptious.tv). Both are applications whose whole mission in life is to open up an external page in an iframe, and do something when it has loaded.

There’s already an IFrame JQuery plugin, but it’s to do with controlling what’s inside the iframe, i.e. the situation which is only possible when the iframes are from the same domain. What I’m dealing with here is the parent knowing when the iframe is loaded, regardless of when it comes from and agnostic to what it does.

Modal Dialog JQuery Plugin

This has been a while coming, but I made a little “yet another modal dialog lightbox JQuery plugin” thing this week.

Demo and Download for Modal Dialog – JQuery Plugin

It was driven by TiddlyDocs, but I’ve been wanting one anyway for a while. Mostly because lightbox libraries generally do some hocus-pocus on page load, like applying to everything marked rel=”lightbox”, but don’t let you dynamically produce the lightbox yourself. That’s fine for pages that are just a static image gallery, but not useful to someone building a dynamic web app.

I’ve subsequently used nyromodal, on good advice, but wanted something smalller and with a simple API for my main use case, which is just showing some text.

The plugin also simplifies work by not requiring you to install a separate CSS file. Doing that, and linking to it, as well as installing any images, is something that always slows me down when I want to start using a graphical client. In keeping with the “happy path” You-Aint-Gonna-Need-It (YAGNI) mantra, I’d rather keep a library to a single Javascript file – be evil and do styling in the library by default, but still support users who want to put custom CSS in a stylesheet.

Towards A Single Page Application Framework

Tonight, I was thinking of making a Twitter app to manage my various accounts (I have ~dormant accounts related to projects like @webwait and @listoftweets). The app would be holding username and password details for each of these accounts, so it made sense to build it as a Single Page Application (SPA). This way, a more paranoid user could always keep the app in their local file system, while a less paranoid user could always stick the file on a protected server somewhere, having configured all the username-password details.

TiddlyWiki is a framework for developing SPAs. One might say it’s the framework for developing SPAs, since there are no prominent alternatives. So my obvious choice was a TiddlyWiki. However, not too long ago, the TiddlyWiki core guys extracted out the secret sauce for SPAs: the ingenius bit of code that saves files without requiring any browser extensions. (I’ve tried to explain this to people and it always leaves even the most brilliant minds a little dumbstruck, but yes TiddlyWiki demonstrates there are pragmatic hacks that can be used to read a file into the browser and then write it out again.) I was keen to explore this saving mechanism, so experimentation ensued.

The file management techniques ship conveniently in a JQuery plugin, jQuery.twFile. There is a basic demo which lets you edit the entire text and save it. The demo confused me at first – because it was editing the entire body of the file, I wasn’t sure how to translate that info into what I, as an application developer needed. The demo is useful for framework developers understanding the plugin, but less so for application developers. So I extracted it into a demo that is still minimalistic, but closer to the kind of thing you’d do in an application.

The SPA demo is here.

Once I did that, I realised what’s required is a SPA framework. In practice, most developers using twFile will be keeping all the HTML, CSS, and Javascript in a single file, so it’s possible to build a higher-level abstraction on twFile, so that developers can focus only on the content. I built the demo in a way that distinguished what was boilerplate framework code and what was application-specific HTML, CSS, and Javascript.

It was all still in one file, and that’s fine for many developers – you can give the developer the file, tell them “edit these bits”, and they can come up with something functional. I decided to extract things further though, and found the Jinja templating framework to be useful here. Jinja has a concept of template inheritance, so you can easily build up an “include” system. The net effect is I was able to encapsulate SPA logic in a single file, which was passed through a Jinja processor to produce the executable HTML document.

The basic SPA logic is shown below:

  1. {% extends "spa-template.html" %}
  2. {% block title %}JQuery SPA demo{% endblock %}
  3. {% block css %}
  4.   etc etc
  5.   body { background: black; padding: 0; margin: 0; font-family: Gill Sans, sans-serif; }
  6.   h1 { background: white; color: black; padding: 10px 10px 0; margin: 0; height: 52px; }
  7. {% endblock %}
  8. {% block html %}
  9.   <h1>
  10.   <img id="logo" src="data:image/jpeg,%FFetc etc"/>
  11.     <span id="title">JQuery-SPA Demo</span>
  12.   </h1>
  14.   <div id="main">
  16.     <ol>
  17.       <li>Save this file to your local file system and open your local copy in the browser (using a file:/// URI). The browser might ask for special permission for this document, and you will need to grant it.</li>
  19.       <li>Type your message: <input id="message" value="change me"></input></li>
  21.       <li>Save this page: <input id="saveButton" value="Save" type="button"></li>
  23.       <li>Hit shift-reload to perform a clean reload the page and observe that your message has been saved.</li>
  24.     </ol>
  26.     <h3>What is this?</h3>
  28.     <p>This is a demo of an experimental Single Page Application I am building atop <a href="http://jquery.tiddlywiki.org/twFile.html">jQuery.twFile</a>, the file saving plugin extracted from TiddlyWiki. Forked from <a href="http://jquery.tiddlywiki.org/twFileDemo.html">this twFile Demo</a>. Only tested on Firefox for now.</p>
  30.   </div>
  31. {% endblock %}
  32. {% block javascript %}
  33.     $.spa.save = function(text) {
  34.       return text.replace(/<input id="message".*?></input>/,
  35.               '<input id="message" value="'+$("#message").val()+'"></input>');
  36.     }
  37. {% endblock %}

So it contains separate HTML/CSS/Javascript blocks. Probably not a good engineering practice (though it doesn’t impact on the end-user experience, which is always a single file), but it’s convenient for now. The key SPA logic is here:

  1. <li>Type your message: <input id="message" value="change me"></input></li>
  2.       ....
  3.     $.spa.save = function(text) {
  4.       return text.replace(/<input id="message".*?></input>/,
  5.               '<input id="message" value="'+$("#message").val()+'"></input>');
  6.     }

As an app developer, all you have to do is override $.spa.save (which in retrospect should be renamed as it doesn’t actually perform the save). This function receives the text of the file as it was stored on disk when the page loaded. It must then return the text that should be saved to disk. Thus, it must store application state in HTML, probably by performing some kind of substitution.

Having put this together, I’m keen to proceed with the envisioned Twitter app. It’s not yet clear if there’s any mileage here over a regular TiddlyWiki, but as someone who is more familiar with starting apps from a blank HTML page (or a simple Project Deploy* template), it might end up being a more familiar way to kick an app off. Watch this space.

ColourLovers API – JSON Demos

About a year ago, I was excited to discover ColourLovers had an API. There is some great data about colours and palettes on that site, and some great possibilities for mashups – some as general-purpose art projects, and some as tools for designers.

However, I noticed there was no JSON-P interface, so there was no way to write a pure browser-based mashup. I got in touch with them, and heard back from Chris Williams, who to my pleasant surprise was willing to introduce JSON-P. And so we had some email discussion about the various options for the API interface (e.g. do you call it “jsonp” versus “callback”; do you use “real JSONP” versus just a callback function; do you hardcode the name of the callback function versus let the caller specify it as a parameter), and ultimately Chris went ahead and implemented it a short time later and published the API on January 26, 2009. Fantastic!

I offered to make some demos illustrating code usage, and did around that time, but never got around to publishing them until now. I’ve released the demos here. There’s a couple of JQuery demos, and one raw Javascript demo. I had fun making them and have plans to do a lot more with this API.