The official olympics medal tally is broken. Let’s fix it.

This is how the official olympics medal tally looks:

Overall Medal Standings - The official website of the BEIJING 2008 Olympic Games

It’s not only the official tally, but the one linked from google each time you type “olympics” and terms like “australia olympics”. Thus making it an absurdly popular page at this time. As you can see, the design is reedeeeculous. The thing you want to see the most – total medal tally – is several miles away from the country. It’s good they’ve used alternative shading for odd and even rows, but even so, it’s still difficult to see the connection. Especially since the gold/silver/bronze tallies are shown in a different colour.

Did you notice even “open/mixed” gets preferential treatment? Look, I can’t even think of an open/mixed event at the olympics. Ballroom dancing? Friendly game of trumps? All I’m saying is, I’d rather see the headline figures first. I don’t know how many gazillion dollars the official IT contract went for, but this is what the tally should look like according to table design 101:

Overall Medal Standings - The official website of the BEIJING 2008 Olympic Games

(BTW I also pulled total to the front and emphasised it. Matter of opinion whether to go by total number versus number of golds.)

(The following images are just to record the fact it’s linked from Google since it won’t be linked much longer.)

olympics - Google Search

australia olympics - Google Search

Can’t really get excited about Ecmascript enhancements. Can’t really get disappointed about them either. When it comes to Ecmascript enhancements, I’m all “it doesn’t really affect me much”.

There’s some tweets and such like about Ecmascript Harmony today. I also saw a nice demo of Ecmascript 1.7 goods today.

Really, I can’t get excited about future versions of Ecmascript and can’t get emotionally all in a tangle about the trials and tribulations of Ecmascript. I mean, Javascript is arguably the most popular language in the world, so it’s cool to know it’s maturing over time. But, you know, I’m only interested in learning about a language (version/dialect) when at least one of the following applies:

  • It can be used in practice, today.
  • Learning it will broaden my mind in some way.

Using it in practice, today? No.

New versions of Ecmascript cannot really be used today, not by me anyway. I mostly write public web apps and enterprise web apps, and in both cases, it’s essential to target Internet Explorer. Ecmascript version N is only worth learning when most of the world has abandoned every version of IE prior to that which supports version N. When you see a new specification being mooted, you have about 10 years of thinking music before the spec is actually released (2-4 years), integrated into all major browsers (0-5 years if ever), and moreover, previous versions have died out (5+ years).

Maybe as society accelerates to the singularity, these lengths will become shorter and you’ll be able to deploy a new version of Ecmascript three times a week. For now, though, it’s fairly pointless from a practical perspective. If you take an agile attitude, why bother investing the time and focusing on something that end up changing or being abandoned altogether?

Okay, this is exaggerated – there are people who can put in practice new versions fairly quickly. Firefox is not only quick to incorporate new versions of Ecmascript, but also happens to be a platform where browser-specific code is sometimes warranted. I refer to browser extensions (and, relatedly, Greasemonkey scripts). If you happen to be writing Firefox browser extensions, you might choose to use Ecmascript as soon as it’s baked into Firefox. You’ll still be breaking compatibility with previous versions, but Firefox users constantly update anyway. This will continue to be the case for as long as most global enterprises force their workers to install Firefox on the sly. Which will be a long time. So you can happily use the latest Ecmascript if you’re writing extensions.

There are also people targeting the Flash platform for whom all this stuff is more important today. Not me, not right now.

The other case, of course, is when you’re using Ecmascript outside of the browser. e.g. server-side Javascript (yum!). Entirely valid. But this is still rare.

Learning it will broaden my mind? Not much.

Javascript is a quite interesting language, so new features will be interesting too. But not hugely, since Javascript isn’t that different from other dynamic languages. And most Ecmascript enhancements are either trying to mould JS into Python or the Java/C++/C* OO model. So they’re not about creating interesting “native” features.

You’d be better off learning something completely different in order to expand your coding brain. Try one of these.

I mean, don’t get me wrong

Don’t get me wrong. The thought leaders need to be thinking about this stuff and making the long term efforts. But for the average Ajaxer on the street, it’s all thirsty fish in the desert sand.

Using 280Slides (Ajax slideshow maker): Real World Experiences

I used 280slides.com for a quick presentation today at the London Javascript meetup. (The usual Jobsesque slides – usually one phrase plus an optional image.)

The tool is very cool, as initial reviews suggested, but most of those folks had not used it in a real world situation. The tool has it’s complications when you use it for real.

What’s good:

  • Looks cool and OSX/Keynote like, but not slavishingly following the desktop in a way which would detract from usability
  • Great sharing and embedding options – can download as ppt, pptx (powerpoint 2007), pdf, and open document…and moreover, can view it full-screen, send it to slideshare, and embed it on a web page.
  • Has all the basic tools you need
  • Has undo and redo with a long history.
  • Supports OSX key bindings, i.e. uses the command key (ie the apple key) instead of what most web apps do, which is to use the ctrl-key even on OSX. (On the whole, there’s a need for a really great Ajax/Javascript key event library to do stuff like this automagically.)
  • Can easily do a slick full-screen mode presentation, using the PlainView browser, as noted on the 280slides blog.
  • Supports themes. (Themes is one rare area where the Ajax/web model makes life very easy for developers of desktop-like apps – just change the CSS style. If only life was so simple for the developers of the original desktop-like apps, where CSS-like functionality is usually lacking.)

What could be improved – this is all intended as constructive criticism. The team’s effort to date shows they are extremely talented Ajaxers and I have no doubt every one of the following can potentially be rectified in the coming months, provided they have enough will and bandwidth.

  • Cut-and-paste – this one really got me. Cut-and-paste is something we do all the time when making slideshows. The problem here is that elements are pasted directly on top of each other, so you can’t tell if it’s been pasted or how many times it’s been pasted (and you may well have several, as it’s likely you’ll be hitting command-v a few times, since there’s no feedback). This is exacerbated by the delay involved – sometimes takes a second or two to make the paste, so when you try to start dragging the copy, you might instead end up dragging the sole, original, element.
  • AFAICT cut/paste/copy are available only as keyboard shortcuts. There’s no toolbar/menu item. This exacerbates the previous problem. With “paste” lacking immediate visual feedback, you’d at least want a button you can click instead of relying on the keyboard shortcut, which feels less assuring (mainly because keyboard shortcuts are still a bit of a novelty on the web). And of course, the other problem is that users aren’t aware the keyboard shortcuts are available.
  • Saving – it’s not clear if auto-save is present. It certainly should be. They need some sort of status message for this, or an auto-save checkbox (though I think it should always be on).
  • Undo isn’t perfect. Sometimes the undo skips the most recent action and instead undoes an action before that. This often happens to be an action on a different slide – it’s good that it jumps back to that slide, but you can easily lose your place and makes it difficult to then go and manually undo the action you actually wanted to undo.
  • It’s fairly processor-intensive. This isn’t really a fair criticism, because it’s too much to expect an app like this to be gentle on resources. Still, I did struggle running it on a 3 year old powerbook and that will affect some users. On the bright side, it’s very snappy on a newly booted Firefox running on a more modern Macbook.
  • When you select text to change it, the text becomes grey to highlight it – thus making it impossible to see any new colour scheme you’ve chosen. Whenever I tweaked background or foreground colour, I had to keep unselecting the text, viewing it, and then re-selecting the text and change the colour again.
  • Text blocks don’t auto-expand/shrink when you change the text size. You must manually resize.
  • As you’d expect, there’s no support for a lot of the more advanced features. Some people would say those features are unnecessary, but personally I do find they do make an impact in live presentations and can often help to convey info in subtle ways. I’m talking about features like shadows, gradients, round corners. I know, it’s all just “bling-bling web 2.0 eye candy noise”, but really they’re essential if your aim is to keep your slides in the visual – rather than literary – realm. Also, there are some more basic features not yet implemented, e.g. borders for images and shapes, and the ability to crop.
  • Images can load quite slowly, and again, there’s no progress indicator. Unloaded images are shown just as a grey block, so viewers mightn’t be aware that an image is loading in that spot.
  • As I was writing this post, I opened up the slideshow and somehow it added a blank textboxes to every slide. Don’t ask me how or why. The existing text somehow had shifted up to the top of the page, and had been resized to one line. I think it somehow had tried to apply a standard slide template (title and main text area), whereas my slides are mostly just one line in the middle of the slide.

Anyway, it’s already usable and has huge potential. Long may 280slides rock! I only hope that if they’re acquired, the innovation keeps happening.

Tiddlywiki internals 3 of 3: Key Javascript Classes and Files

(This is part 3 of a 3-part series. Part 1 introduces the internals and highlights some of the key patterns and concepts. Part 2 introduces each Javascript file. Part 3 focuses on the internals of the more important classes and files.)

Concluding this series, below is a list of all core Javascript files, organised into functional groups.

main.js

main() is the function that runs onload.

Key functions:

  • creates a new tiddlywiki data store (new TiddlyWiki()) – this is the collection of tiddlers users are exposed to. The store is populated using TiddlyWiki.prototype.loadFromDiv(), which loads all the tiddlers from the “storeArea” div, which is an invisible block on the page (and rendered back in nice – and visible – manner later on).
  • creates a second TiddlyWiki data store to hold “shadow tiddlers” – these are “meta”/config tiddlers holding data such as CSS styling. Populated from invisible “shadowArea” div (which at compile time is defined in the Shadow/ directory).
  • creates a new “Story div”, a div which will show tiddlers to the user, and themes it according to config.options.txtTheme
  • sets up Popup.onDocumentClick (removes popup menus when user clicks outside of the menu)
  • sets up event propagation – certain tiddlers are notified when certain actions occur. The mappings are defined in refresh.js (e.g. {name: “StyleSheetLayout”, notify: refreshStyles})
  • sets up and renders backstage
  • loads plugins (plugins are evidently supposed to set a global “plugin problem” value if a problem occurs)

General:

  • calls several lifecycle event handlers as it loads – the wiki config can provide hook functions which run upon particular lifetime events
  • benchmarks most of the above (the benchmarking was possibly a quick fix – relies on variables t1,t2…t10 -> this code could be optimised for conciseness using function wrappers, but maybe startup would be too slow that way).
  • After initial setup ensures tiddlywiki data structures and other initialisation/config pieces are in place, it blats and shows the display with restart() and refreshDisplay().

Plugins

Tiddlywiki has a strong plugin architecture. Each plugin is included as a regular (non-shadow) tiddler, one that must be tagged “systemConfig”. (For all intents and purposes, “systemConfig” is synonymous with “plugin”.) There’s an example shipping with the default tiddlywiki instance on tiddlywiki.com (and a more detailed example in the source code – association/plugins/SparklinePlugin/SparklinePlugin.js). (Also of interest, the latest plugin template at the tiddlywiki.org wiki.)

  1. <div title="ExamplePlugin" modifier="JeremyRuston" created="200607271914" modified="200609212329" tags="systemConfig">
  2. <pre>/***
  3. |''Name:''|ExamplePlugin|
  4. |''Description:''|To demonstrate how to write TiddlyWiki plugins|
  5. |''Version:''|2.0.3|
  6. |''Date:''|Sep 22, 2006|
  7. |''Source:''|http://www.tiddlywiki.com/#ExamplePlugin|
  8. |''Author:''|JeremyRuston (jeremy (at) osmosoft (dot) com)|
  9. |''License:''|[[BSD open source license]]|
  10. |''~CoreVersion:''|2.1.0|
  11. |''Browser:''|Firefox 1.0.4+; Firefox 1.5; InternetExplorer 6.0|
  12. ***/
  13.  
  14. //{{{
  15.  
  16. // Uncomment the following line to see how the PluginManager deals with errors in plugins
  17. // deliberateError();
  18.  
  19. // Log a message
  20. pluginInfo.log.push(&quot;This is a test message from &quot; + tiddler.title);
  21.  
  22. //}}}</pre>
  23. </div>

A plugin is essentially just a Javascript block which gets executed on page load. All the biosketch info is optional (although in some cases, it does effect processing, e.g. there is a check against the required TiddlyWiki version). “Just some Javascript” did you say? This post on JQuery plugins by JQuery daddy John Resig is instructive. His point is that a plugin architecture needs explicit points for plugins to hook into – i.e. an API – and the existence of a plugin catalogue. Tiddlywiki doesn’t have a plugin API per se, but is structured with plenty of extension points to naturally hook into. As for the catalogue, there’s also a plugin wiki area, with a grander-scale plugin repo project in progress.

Incidentally, note that you don’t have to register the Javascript block as you might do in some other frameworks (e.g. runOnInit(myPlugin); ). It executes automatically when plugins are loaded.

Okay, so about those plugin extension points. I’m still learning that. In the case of sparklines, the purpose is to create a new macro (e.g. <<sparkline 100 200 300>>), so it defines config.macros.sparkline.handler(place,macroName,params), and its “output” is to populate the place element with sparkline content.

Another popular pasttime for plugin developers is szhushing the global Formatter object to shape how stuff gets rendered. e.g. if your formatter locates the built-in formatter named “heading”, it could easily overwrite its handler method to MAKE ALL THE HEADINGS SHOUT AT UNSUSPECTING READERS.

To install a plugin, users usually use the Import dialog, accessible from Backstage. It’s also possible to manually include plugins via cut-and-paste into Tiddlywiki.

There’s much more to be said about plugins. The bottom line is that Tiddlywiki’s architecture lets you bend the core product into many things. (By “architecture”, I refer to both the plugin mechanism and the flexible nature in which the overall architecture is structured.)

Tiddlers

Tiddlers are the atomic content blocks that make up a Tiddlywiki, typically about a paragraph in length. A Tiddler is simply a block of text, with extra info like a title, a list of tags, and timestamp data. There’s also a fields hash where you could store any arbitrary properties. (This seems suitable for plugins, but the core also makes use of it, and I don’t really get that. Even for plugins, why can’t they just make new fields dynamically?)

Tiddler is a Javascript class, so you get a new instance with new Tiddler(). Internally, it uses a publish-subscribe mechanism, where a changed() method is called after any mutation. This basically ensures the links property is up to date, as links is a redundant (and presumably there for performance) collection of links inside the tiddler.

A Tiddler also has a collection of “slices”, though the collection is managed by TiddlyWiki rather than Tiddler. (This relates to the fact that shadow tiddlers are mere text blocks – using Tiddlywiki to extract slices ensures shadow tiddlers can also be sliced up….and slices are a major feature of most shadow tiddlers, since they are config-related.)

There’s a string->string map from name to slice. This is similar to the fields hash, insofar as it’s a free-form map. In this case, though, it’s something that can easily be changed by the user in real time, as the slice collection is sync’d to the tiddler content. For example: |”slicename:”|”some slice content”|. Slices allow for easily edited meta-data, e.g. a stylesheet tiddler can have a slice called “backgroundColour”. Users then edit the backgroundColor slice content to set the background colour.

A Tiddler also has a set of notification handlers – this is also managed by TiddlyWiki rather than the Tiddlers themselves (again, this ensures the mechanism works for shadow tiddlers). These are listeners/observers that are notified each time tiddler is changed.

A file closely related to Tiddler is TiddlerFields.js. It actually alters the TiddlyWiki definition rather than the Tiddler definition, but in any event it deals with accessing the Tiddler’s fields map.

Shadow Tiddlers

Shadow tiddlers are a particular type of tiddler. There’s no separate “ShadowTiddler” class, but they are held in a separate store and treated in special ways. Indeed, shadow tiddlers aren’t actually of class Tiddler (which is slightly confusing). They are simply a title-text pairing; the data structure is a map from title to text. In contrast, regular Tiddlers are mapped from title to Tiddler.

In particular, TiddlyWiki has a fallback mechanism when asked to return a tiddler – if the tiddler doesn’t exist, it will attempt to revert to a shadow tiddler of the same name. Shadow tiddlers are immutable (unless you hack source code), whereas tiddlers are of course easily edited. You can override shasow tiddlers with regular tiddlers of the same name, but the original shadow still lurks (in a good way) in the background.

To see this, open an editable Tiddlywiki, choose a shadow tiddler from the right sidebar Contents menu (e.g. SiteUrl), edit it, and save it. Then re-open it to verify your changes were affected. Then delete it, and notice that it’s still in the list of shadow tiddlers. When you open it yet again, you’ll see it now contains the original content. (The shadow tiddler itself never changed.)

Shadow tiddlers are used for config stuff like stylesheets. The fail-safe mechanism ensures you can easily “restore factory defaults” at any time.

TiddlyWiki

A Tiddlywiki is essentially a hash of Tiddlers, keyed on their title. More precisely, it’s a wrapper around this hash. Here’s a (slightly refactored) look at the relevant code for managing tiddlers, which looks like any other hash wrapper:

javascript
< view plain text >
  1. function TiddlyWiki()
  2. {
  3.   var tiddlers = {}; // Hashmap by name of tiddlers
  4.   ...
  5.   this.clear = function() { tiddlers = {}; };
  6.   this.fetchTiddler = function(title) { return tiddlers[title]; };
  7.   this.deleteTiddler = function(title) { delete tiddlers[title]; };
  8.   this.addTiddler = function(tiddler) { tiddlers[tiddler.title] = tiddler; };
  9. }

There is also a set of similar methods which wrap around these to provide more intelligent behaviour. e.g. createTiddler() wraps addTiddler() to provide “Add or retrieve if exists” functionality. getTiddler() wraps fetchTiddler() to ensure null is returned if no such tiddler exists. removeTiddler() wraps deleteTiddler() to delete only if the tiddler exists, and also notifies the tiddler’s listeners. Most other methods also do “general stuff” with the tiddlers hash. A lot of them also run operations on behalf of Tiddlers themselves (this is mostly so it can endow shadow tiddlers – which are just strings – with certain behaviour, as mentioned in the previous section.)

Story

Story is the sovereign UI element in TiddlyWiki – its the container of all visible Tiddlers which you’ll usually see occupying the main, middle, column. Theoretically, there could be more than one Story instance on the page, but I’m told that there are some hard coding shenanigans that rule it out in the project’s current state. (Specifically, direct references to the “story” instance that main.js creates.) So Story is a singleton class in practice.

One gotcha here with the nomenclature – a “tiddler” inside Story.js is conceptually a DOM element, whereas in most other places its a data structure. Obviously, the tiddler UI element is a rendering of the tiddler data structure. However, the implementation isn’t entirely symmetrical because the data structure has a dedicated class (Tiddler), while the UI element doesn’t; tiddler rendering is handled purely by the Story class. In one case (displayTiddler()), either form is valid as the “tiddler” argument, similar to $() functions that accept either the element or the ID (title = (tiddler instanceof Tiddler) ? tiddler.title : tiddler.)

Story’s key properties are a container ID, which points to the container DOM element, and an idPrefix, the prefix for all tiddler IDs. The container already exists on the page when a Story object is created to manage it.

javascript
< view plain text >
  1. function Story(containerId,idPrefix)
  2. {
  3.   this.container = containerId;
  4.   this.idPrefix = idPrefix;
  5.   ...
  6. }

Each tiddler’s ID is simply idPrefix + title. You might expect an array of tiddler DOM elements, but Story doesn’t need it, as it can use the DOM itself to keep track of them; the direct descendents of the Story container are the Tiddler elements. It simply uses DOM traversal techniques to iterate through all such elements, when it needs to. (There’s a generic forEachTiddler function too; I could imagine there might be some value in other collection closure methods.)

Story contains the logic to display a tiddler. displayTiddler() decides if the tiddler is already being shown, and if not, creates a new child element with the tiddler content. It delegates to the animation engine for display.

There is also refreshTiddler() – the logic for rendering the tiddler – which is called from displayTiddler(). For flexibility, tiddlers are rendered using a template, a template which is generally contained in a shadow tiddler. There’s a ViewTemplate shadow tiddler and an EditTemplate shadow tiddler – it depends on whether the tiddler is being edited.

Furthermore, there is the concept of themes, which means you can use different templates. This is handled by switchTheme(). An example of different templates is illustrated here in the TiddlyPedia theme.


And that concludes the three-part series. Thanks again to those who helped me gather this info (see credits in first article). I’ve learned a lot about Tiddlywiki in writing it, but I still have a long way to go. There wil be more.

Tiddlywiki internals 2 of 3: List of Javascript Files

(This is part 2 of a 3-part series. Part 1 introduces the internals and highlights some of the key patterns and concepts. Part 2 introduces each Javascript file. Part 3 focuses on the internals of the more important classes and files.)

Continuing the series, below is a list of all core Javascript files, organised into functional groups.

Initialisation

  • main.js Runs the initialisation sequence.
  • Paramifiers.js Handles URL params.

Generic (Non-Animation)

  • BasicTypes.js Augments built-in Javascript Number and Array.
  • Crypto.js Crypto functions. (Tiddlers can generate fingerprints.)
  • Dates.js Augments built-in Javascript Date class.
  • Dom.js Supports DOM manipulation.
  • FileSystem.js
  • Strings.js Augments built-in Javascript Number and Array.
  • Http.js Supports XmlHttpRequest based remoting.
  • RGB.js CSS colour manipulation.

Generic (Specifically Animation)

See also (2005) TiddlyWiki animation write-up.

  • Animator.js Runs the dynamic flow of stepping through an animation, delegating to specific strategies.
  • Morpher.js Morphing animation strategy. Cool – smoothly animates between two CSS styles.
  • Scroller.js Scroller animation strategy. Scrolls window to show an element. (The way the page smoothly scrolls to show a tiddler when you click its link).
  • Slider.js Slider animation strategy. Slides elements opening and closed (e.g. Closing tiddlers or the Options box on right sidebar.).
  • Zoomer.js Zoomer animation strategy (the way a tiddler “jumps out” from its link).

Tiddlywiki-Specific Utilities

  • FormatterHelpers.js Utilities specifically for Formatters.
  • Refresh.js Mechanism for notifying and updating elements based on changes, e.g. if stylesheet shadow tiddler is updated.
  • Utilities.js Miscellaneous TiddlyWiki-specific utility functions.

Data Structures

  • Tiddler.js Data structure representing a tiddler, i.e. a block of text with a title.
  • TiddlerFields.js Augments TiddlyWiki to manage tiddler fields.
  • TiddlyWiki.js Data structure representing a collection of tiddlers.

Data Import/Export

  • AdaptorBase.js Adaptors convert from various wiki formats (e.g. Mediawiki) to TiddlyWiki. This is the base class for Adaptors.
  • FileAdaptor.js Subclass of AdaptorBase which reads the default/standard Tiddlywiki format.
  • Import.js Macro to import tiddlers from another Tiddlywiki.
  • LoaderSaver.js Converts between HTML and a list of tiddlers. (I think the main purpose is to get a clean HTML list of tiddlers.)
  • Saving.js Saves the Tiddlywiki – main case is serialising everything to DOM elements and saving to local file system.
  • SavingRSS.js Serves Tiddlywiki as RSS format (e.g. TiddlyWiki.com RSS feed) showing time-sorted list of recently updated tiddlers.
  • Sync.js Syncs
  • TW21Loader.js Standard implementation of LoaderBase (defined in LoaderSaver.js).
  • TW21Saver.js Standard implementation of SaverBase (defined in LoaderSaver.js).

Strategies

This is a broad category of options and control-type functions. The control-type functions are here because they are designed using flexible mechanisms which make them easily overrideable by plugin developers.

  • Config.js General Tiddlywiki config – controls capacities, names of shadow tiddlers, which options can be set, other stuff.
  • Commands.js Handlers for menus and toolbar.
  • Macros.js Defines built-in macros.
  • Formatter.js Formatters are strategies for locating regexp patterns in the wiki text (wiki words, image URLs, etc.) and rendering them.
  • Options.js Options are cookie-based preferences. The user can generally set them directly on the Tiddlywiki UI. This is in contrast to Config.js settings, which are fixed unless the uswer cares to dive into the source code.
  • Wikifier.js

UI Elements

  • Backstage.js The backstage space at the top of the page, with access to advanced features and acting as an escape route after over-enthusiastic bouts of customisation.
  • ListView.js A table-like list, e.g. shows all options when you hit Backstage|Tweak.
  • Manager.js Plugin manager (accessible from Backstage|Plugins)
  • Messages.js Simple status notifications.
  • NewTiddler.js Macro for a new tiddler, e.g. when user hits “New Tiddler” menu option, and also new journal.
  • Popup.js Popup menu (e.g. when you click on the name of a tiddler in the list of shadow tiddlers).
  • Search.js Search implementation – allows user to search for a term.
  • Sparkline.js Generates CSS based sparklines graphic.
  • Story.js Manages the container of all visible tiddler UI elements.
  • Tabs.js A UI element for handling tabs.
  • Toolbar.js The toolbar shown in the top of a tiddler (with “close”, “close others” etc controls – or “done”-”cancel”-”delete” if open).
  • Wizard.js Multi-step wizard UI framework.

Miscellaneous

  • Deprecated.js Deprecated functions.
  • Guide.js A short readme file.
  • Lingo.js internationalisation-localisation support – contains string keys and their English values.
  • Upgrade.js Support for upgrading Tiddlywiki vgersion.
  • Version.js Short file with info about this version of Tiddlywiki.

Tiddlywiki internals 1 of 3: Architectural Concepts

(This is part 1 of a 3-part series. Part 1 introduces the internals and highlights some of the key patterns and concepts. Part 2 introduces each Javascript file. Part 3 focuses on the internals of the more important classes and files.)

This is the first in a 3-part series on the internal design of Tiddlywiki. The series is more or less stream of consciousness – I’m a green Tiddlywiki developer, so I’ve been making these notes as I trawl through the source and learn it myself. Thanks to various people at Osmosoft for explaining some of this, and special thanks to Jeremy for some overviews and reviewing the writing here, Saq for a great overview on many feature, and Fred for reviewing the initially published version.

Overview

A Tiddlywiki is a collection of “tiddlers”, small blocks of content typically a paragaph or so in length. At any time, a subset of these tiddlers is displayed in the UI (between zero and the total number of stored tiddlers).

A special property of Tiddlywiki is that the entire application resides in a single HTML/Javascript/CSS file (it’s the quintessential SPA – Single-Page Application). This is why you can save a Tiddlywiki locally and run it off a file:// URL and stick it on your iPod or novelty hamburger USB stick.

In the file, all the tiddlers are stored inside invisible DIVs, which are read on startup into a “TiddlyWiki” data structure. When you invoke the save feature, for example by hitting the “save changes” control, the invisible DIVs are refreshed with latest content from memory, and the entire file is written out to the hard drive.

TiddlyWiki is much more than a specialised wiki – due to its flexible architecture and the possibility of plugins, it is more like a platform. Examples of apps built on Tiddlywiki.

TiddlyWeb, though not discussed specifically here, marks an important step in the future of TiddlyWiki development. It’s a RESTful server of Tiddlers which would allow for great flexibility in the kinds of UIs you end up with, as well as allowing non-UI clients.

Anatomy of a Tiddlywiki

The image below shows an Tiddlywiki in editable mode. As for the UI, you can see it consists of a main area with two sidebars. The main area is a “Story” – a story is a sequence of visible tiddlers.

A lot of this is configurable by changing special tiddlers. In particular, the tiddler called “PageTemplate” provides the overall structure, with references to other tiddlers, and “Stylesheet” the CSS styles.

Object-Oriented Concepts in Tiddlywiki

There are many ways to deal with classes, objects, and prototypes in Javascript – see “Javascript: The Good Parts” by Doug Crockford and “Pro Javascript Design Patterns” by Ross Harmes and Dustin Diaz.

Tiddlywiki’s OO relies on the constructor function pattern, where you create new objects using the new keyword.

javascript
< view plain text >
  1. var tiddler = new Tiddler();

In Javascript, new Fn() will magically does a couple of things that let us use the familiar (from C++, Java, etc.) idiom above. It sparks the creation of a blank object, then it conducts a special execution of Fn() in which this is superfrajalistically tied to the new-born object. This leads us to an idiom which is called a “constructor function” because it is a function that is both called and implemented as if it were, for the most part, a constructor in OO languages like C++ and Java. The Tiddler constructor function is defined as follows:

javascript
< view plain text >
  1. function Tiddler(title)
  2.   {  
  3.     this.title = title;
  4.     this.text = "";
  5.     ...
  6.     return this;
  7.   }

In addition, the new Tiddler has a number of standard Tiddler methods associated with it, so I can call them in typical OO fashion, e.g. tiddler.isTagged("happy"). The implementations refer to the specific instance using the this keyword. In Javascript, this can easily be achieved via prototypes. Therefore, subsequent to the constructor definition, we encounter in Tiddler.js a menagerie of method definitions like:

javascript
< view plain text >
  1. Tiddler.prototype.isTagged = function(tag)
  2. {
  3.   return this.tags.indexOf(tag) != -1;
  4. }

All of the attributes above are public, but Tiddlywiki also uses closures to ensure some attributes are only available externally via declared methods. For example, the tiddlers of a Tiddlywiki is a declared as a local variable, so there’s no direct reference to it outside the methods declared in the same scope.

javascript
< view plain text >
  1. function TiddlyWiki()
  2. {
  3.   var tiddlers = {}; // Hashmap by name of tiddlers
  4.   this.tiddlersUpdated = false;
  5.   ...
  6.   this.fetchTiddler = function(title) {
  7.     var t = tiddlers[title];
  8.     return t instanceof Tiddler ? t : null;
  9.   };
  10. }

The above methods will also be available on each instance created with new, just as with those declared using the prototype assignment. They are used in exactly the same way. The only difference is that all these functions are re-created with each new instance, so they will consume more memory. That’s the price we pay for the encapsulation.

You will also find static methods present (i.e. global functions attached to a constructor purely for the sake of namespacing them). For example:

javascript
< view plain text >
  1. TiddlyWiki.isStandardField = function(name)
  2. {
  3.   return TiddlyWiki.standardFieldAccess[name] != undefined;
  4. }

Typically, a class will be contained in a single, dedicated, Javascript file (within the source code from which a Tiddlywiki is built). However, the previous example was actually contained in TiddlerFields.js rather than Tiddlywiki.js, so it seems that class definitions may be distributed across multiple files in some limited cases.

And that’s how Tiddlywiki handles basic OO.

You’ll also see some parts of TiddlyWiki enhancing built-in Javascript types by extending their prototype – for example, BasicTypes.js endows all Arrays with a contains() method and Dates.js sticks a getAmPm() method onto each Date that’s created. Number, Array, and Date receive a dozen or so new methods.

Last but not least, there’s also a healthy dose of inheritance in Tiddlywiki. Javascript inheritance is a whole new can of worms. We see an example in AdaptorBase, which serves as the base class for server adaptor subclasses. AdaptorBase looks very normal, like Tiddler above. FileAdaptor, a subclass, looks like this:

javascript
< view plain text >
  1. function FileAdaptor() {
  2. }
  3.  
  4. FileAdaptor.prototype = new AdaptorBase();

Basically, Javascript has a concept of prototype chains. The assignment means that any instance of FileAdaptor will now have all methods present in a new instance of AdaptorBase. FileAdaptor goes on to define its own methods, using the standard prototype pattern. If so inclined, it can override AdaptorBase’s methods by defining them on its own prototype method. (This is why we say “new AdaptorBase()” – if we had assigned FileAdaptor.prototype to AdaptorBase.prototype, anything we set on FileAdaptor would also be set on AdaptorBase.)

URL Arguments

Tiddlywiki uses the fragment identifier pattern (described here) to provide flexible loading strategies.

Normally, the “DefaultTiddlers” shadow tiddler is used to specify which tiddlers are shown on startup. However, this can be overridden via URL params. For example, use http://www.tiddlywiki.com/#Examples to load with just the Examples tiddler showing. Or, for multiple tiddlers, just separate with a space (%20 in URL-5peak) http://www.tiddlywiki.com/#Examples%20Plugins. (An interesting possibility would be for Tiddlywiki to keep updating the URL to ensure its sync’d with the state of the app, so you could bookmark it at any time to save that configuration.)

But maybe you don’t want to manually list all the tiddlers – instead, you might want to show all tiddlers matching some criteria. Then you’d want an automated mechanism for auto-selecting those criteria (think iTunes Smart Playlist for dramatic effect.) This would make the URL shorter, easier to understand the true purpose of the configuation, and future-proof it against any changes to the set of tiddlers we’re interested in.

In Tiddlywiki, that mechanism is achieved with a URL “filter” prefix. For example, show all tiddlers with “systemConfig” tag – http://tiddlywiki.com/#filter:[tag[systemConfig]].

Other things you can do - http://tiddlywiki.com/#newTiddler:tiddlername – create a new tiddler, specifying the name

The URL is modelled as a map, i.e. key-value pairs. In the case of http://www.tiddlywiki.com/#Examples%20Plugins, that’s just an alias for the canonical map form, http://www.tiddlywiki.com/#open:Examples%20open:Plugins. All this is managed by the Paramifiers class.

Vim Macro for IDE-Like Behaviour

I find Vim easiest for browsing source and as I explore TiddlyWiki, I decided to use some fairly recent Vim features – vertical splitting and the explorer window. Combined, these give you the feeling you’re an 1988 edition of Eclipse. Just like a modern IDE, but navigation is ten times faster.

Here’s the macro – which assumes you just opened vim or have a single empty buffer:

map ]xx :Explore<cr>2jp<c-w>H20<c-w><

To switch files, navigate to the new file’s name in the explorer window, and hit “p” (the built-in command for “open this file in the previous location).

The macro:

  • Populates the current buffer with an explorer window
  • Jumps down two lines (to reach the second filename – the first name is often .svn)
  • Opens the file (“p”)
  • Places the explorer window on the left side (which also forces a re-orientation towards vertically split windows)
  • Resizes the explorer window to be 20 columns wide

While in the source window, you can still use commands like “:e filename” to open a new file (with tab completion) and ctrl-o/ctrl-i to jump back and forth.

The other helper is exuberant ctags, which I installed via Fink. I just run “ctags *.js” and then I can hyperlink from the file under my cursor using ctrl-], or visit any function using “:tag function-name”. (One remaining frustration is I can’t get it to hyperlink to OO methods established using the “abc.prototype.xyz = function() {…}” idiom, despite including the pattern in ~/.ctags.)

Shout – First TiddlyWiki Plugin

Okay, it’s remarkably lame, but here’s my first TiddlyWiki plugin. It defines a new shout macro – you include <<shout some-message>> in a tiddler and it includes A SHOUTING VERSION OF THE MESSAGE.

I adapted it from the sparklines plugin. The important code is at the end – you create a macro by defining a value config.macros.helloWorld, then you simply implement config.macros.helloWorld.handler. This function receives a place variable, defining where the macro will go, and the parameters that were passed into the macro (params).

javascript
< view plain text >
  1. /***
  2. |''Name''|ShoutPlugin|
  3. |''Description''|Make yourself heard - capitalises all text|
  4. |''Version''|1.0.0|
  5. |''Status''|stable|
  6. |''Source''|http://www.example.com/plugins.html#Shout|
  7. |''License''|[[BSD open source license]]|
  8. |''~CoreVersion''|2.4.0|
  9. |''Feedback''|[[TiddlyWiki community|http://groups.google.com/group/TiddlyWiki]] |
  10. |''Keywords''|exclamation|
  11. !Usage
  12. {{{
  13. <<shout message-to-be-shouted-about>>
  14. }}}
  15. Pretty simple really
  16. !!Examples
  17. {{{<<shout these tiddlers are making me thirsty!!!>>}}}
  18. !Code
  19. ***/
  20. //{{{
  21. if(!version.extensions.ShoutPlugin) {
  22. version.extensions.ShoutPlugin = {installed:true};
  23.  
  24. //--
  25. //-- Shout
  26. //--
  27.  
  28. config.macros.shout = {};
  29. config.macros.shout.handler = function(place,macroName,params)
  30. {
  31.  var greeting = createTiddlyElement(place,"span",null,"greeting",params.join(" ").toUpperCase());
  32.   place.appendChild(greeting);
  33. };
  34.  
  35.  
  36. }
  37. //}}}

Injecting HTML into an IFrame

Walking through Tiddlywiki source (write-up to follow), I noticed some interesting code in TiddlyWiki.js, importTiddlyWiki function.

The code takes a string and injects into an IFrame. I had talked to Jon a little while ago about a similar problem and was wondering about it ever since. The technique here looks like this:

It wraps the text with tags to make it an HTML document:

javascript
< view plain text >
  1. var content = "<html><body>" + text + "</body></html>";

It then introduces a new iframe to the page.

javascript
< view plain text >
  1. var iframe = document.createElement("iframe");
  2.     document.body.appendChild(iframe);

Now comes the tricky part. You would think you could just create a new iframe element and set its innerHTML, but with iframes you must use an internal doc ument property.

javascript
< view plain text >
  1. var doc = iframe.document;
  2.     if(iframe.contentDocument)
  3.         doc = iframe.contentDocument; // For NS6
  4.     else if(iframe.contentWindow)
  5.         doc = iframe.contentWindow.document; // For IE5.5 and IE6
  6.     // Put the content in the iframe
  7.     doc.open();
  8.     doc.writeln(content);
  9.     doc.close();

Now the content is in our new iframe. We can then manipulate the content using standard Javascript…but ensuring a call like getElementById is executed against the iframe document, not the global document. (i.e. don’t use the usual document.getElementById()).

javascript
< view plain text >
  1. // Load the content into a TiddlyWiki() object
  2.     var storeArea = doc.getElementById("storeArea");
  3. };

With this technique, you can take an arbitrary HTML string and delegate its parsing to the browser’s built-in DOM engine.

I, Osmosoftie

I’ve officially joined the Osmosoft team today. Osmosoft is all about open source and web technologies, so it’s a great fit. I’m looking forward to working in a talented, hands-on, team in a startup-like culture.

Osmosoft began as an acquisition, when BT brought in Jeremy Ruston and his TiddlyWiki product. The team continues to work on TiddlyWiki and related products such as RippleRap, in addition to guiding BT on open source strategy. Last week, Jeremy announced BT is joining FOSSBazaar, an open source community whose other partners include Google, HP, Novell, and SourceForge.

As for my limited TiddlyWiki experience, I have previously looked at some of the code – while writing Ajax Patterns in 2005 I mined parts of the code for Ajax patterns. At some point in 2006, I ported podca.st to use a modified TiddlyWiki I found somewhere, which allowed user edits, though spamming forced me to restore the original silly static HTML version (there are now other solutions for server-side TiddlyWiki hosting).

In its main form, TiddlyWiki runs off a file: URL, which means it enjoys extra privileges not afforded to typical web apps, and exploiting them effectively raises a lot of unusual challenges. I’m also very interested in TiddlyWeb: “a practical, readable, reference implementation of a rigorously designed headless wiki server that may use TiddlyWiki as a user interface”. This is effectively a RESTful wiki API.

What I’m doing at Osmosoft (unlikely to be OpenSocial related) – and to what extent it involves TiddlyWiki – is still being scoped. Watch this space. Regardless, I’ll still be involved – as a separate activity – with OpenSocial and BT’s OpenSocial and widgets strategy.