The Tiddly* Universe Leading Up to TiddlySpace

There have been a dizzying array of Tiddly* products spun off recently. I’m talking merely about the core infrastructure stuff rather than applications like TiddlyPocketBook. We’re starting to get some convergence within the team, as we’re working on a high-level product called TiddlySpace. It’s basically the manifestation of earlier discussion too.the ideas we were discussing earlier. Similar to WordPress MU or Wikia, you’ll be able to spin off new TiddlyWikis at the click of a button. The server itself is open source, so you can host it in your own enterprise or on your own server. And because TiddlyWiki is nowadays an application framework, not just a wiki thing, spinning off new instances means composing new applications.

Pretty much all the applications we’re working on right now are intended to run on this infrastructure. Many will run in standalone, file-based TiddlyWiki, as well; i.e. the whole server-sidedness and user authorisation is a completely separate deployment concern. Which is an awesome way to develop. I recently realised we’re about the only people in the world who primarily develop file:// based web apps! That they will magically work as multi-user, server-hosted, apps too is certainly a boon for the Tiddly* model.

Anyway, what are all these Tiddly* products Osmosoft’s been working on, and how do they lead up to TiddlySpace? Below is the universe of products we’ve been working on – if all this seems horrendously complicated at first glance, note that most users will only ever see one thing: either TiddlyWiki or TiddlySpace, and even that may be invisible to them if they are using certain high-level applications. Here, I’m just describing the various components we’re working on to pave that path. It’ll all be nice for users. Trust.

TiddlyWiki – This is the original client UI. It’s a web app fitting inside a single HTML file – HTML, CSS and Javascript all together. It can run on a file:/// URI, which means you can store TiddlyWikis on your local drive, a USB drive, or a share drive … or a http:// URI. In the former case, thanks to some very clever and still-obscure browser hacks, you can make edit the TiddlyWiki and save changes. TiddlyWiki is, at its heart, a personal scrapbook, where you can easily gather and manage free-form bits of content (“tiddlers”). However, some of these bits can include CSS or Javascript, meaning that you can actually start building powerful applications with it and using the underlying tiddlers as a kind of scaffolding. These applicatons can be useful for individuals, but we’re still limited to a single user working on their own hard drive. Hence the need for server-side products (see below).

TiddlyWiki 5 – This is a major in-progress upgrade of TiddlyWiki, a complete rewrite, the biggest overhaul in its 5 year existence. There are some much-requested features: follows graceful degradation principle so that the TiddlyWiki5 presents nicely to search engine bots and users with Javascript turned off; includes rich text editing control; SVG graphics; embedded images (with the IE6-compatible MHTML hack); better readiness for server-side integration (e.g. the possibility to track revisions). And yes, all of this in a single HTML file.

TiddlyWeb (low-level) – This is new RESTful server-side component we’ve been building at Osmosoft for persistence, where each Tiddler has its own URI; it’s similar to CouchDB and other NOSQL frameworks; the main difference is it provides extreme flexibility on all manner of things like authentication and storage and presentation formats, and will mostly be invisible to users as we’re building higher-level abstractions on top of it. Also, the underlying data model is that of tiddlers, which means a title, text, list of tags, creation/modification info, and key-value fields (many NOSQL frameworks will have similar document models, but none exactly matching the Tiddler data structure that has always been present in the TiddlyWiki client UI). For most people, TiddlyWeb is best considered as a powerful component further down the stack than what you actually interact with. You’d only need to know about it in detail if you were trying to do powerful admin things, or change the code, on something higher-level like TiddlySpace; or if you wanted to build something equivalent to TiddlySpace; or you were trying to use it in a kind of NOSQL server capacity.

TiddlyWebWiki (low-level) – This is a set of plugins for TiddlyWiki that help it talk to TiddlyWeb. It “hijacks” (intercepts) calls like store.saveTiddler() so that instead of saving to the local drive, it will upload to the TiddlyWeb server.

TiddlyHoster and TiddlyConsole These products will probably converge. TiddlyHoster is a first-cut at the TiddlySpace concept, an inspiring product mostly built by @cdent and cobbling together various plugins. You can try it out now at It’s a fairly direct exposure of the TiddlyWeb bags and recipes model. TiddlyConsole (formerly TiddlyRecon) is similarly a bag and recipe and user mangement tool, with more emphasis on admin usage than that from end-users. We hope the essence of these tools can be incorporporated into TiddlySpace to support power users.

TiddlySpace – This is a higher-level web app, building on TiddlyWeb, to make it easy for folks to spin off new multi-user (or single-user) TiddlyWiki instances. This is really the most important piece, and the one everything’s been leading up to, when it comes to Osmosoft’s mission to get all manner of web apps running inside the enterprise, and will hopefully be just as useful for other enterprises as well as ad hoc groups working on the broader internet. With TiddlySpace, you have an application running at for example. You click a “clone” button, type in “CameraGuide”, and suddenly you have a clone of that web app running at The neat thing is that the new space has copies of all the pocketbook tiddlers, which you can happily hack. The application tiddlers are “copied by reference” from a master application space, say, so you will inherit any future changes made to the application. All this sounds complicated, but will be seamless to the user, and is also easy for us to implement while building TiddlySpace, thanks to TiddlyWebs’ flexible bags and recipes model for containment of tiddlers. (See my earlier discussion too.) Pretty much all of TiddlySpace is driven by client-side code TiddlyWiki plugins, connecting to the TiddlyWeb server.

We’ve spent a while talking about the design of TiddlySpace and tomorrow we have a hackday with the goal of getting a v0.1 running.

Multi-User TiddlyWiki

TiddlyWiki MU is what I’m calling – in the absence of an official name – an effort within Osmosoft to pull together a bunch of work into something that will be very useful in the enterprise and beyond. You could also call it “tiddlywiki as a service” or “multi-room tiddlywiki”. Similar to Wikia or WordPress MU, you can spin off a new multi-user instance with a “single click”. This is “MU” in the sense that there’s already a TiddlyWeb-backed TiddlyWiki product (TiddlyWebWiki). The value-add of TiddlyWiki MU (or whatever it ends up being called) is that you can make a new instance of such a thing without being a system administrator and without going through the effort of building it on the server. This is exactly like the way WordPress MU lets you spin off a new blog without having to set up a new instance of WordPress on the server.

In addition to the ease of spinning off new instances, there’s an important side benefit from this architectural pattern: synergy. From a user’s perspective, they only have one URL to remember/bookmark/share/link; once at that URL, the system can helpfully guide them through the different instances they have access to. Also, it would be possible to make content that’s used across different rooms. For example, give each user a private bag of tiddlers and they could use it to set their global preferences (with the right UI).

The whole thing is an open-source server, so an enterprise can just download it, deploy it, and let a thousand flowers bloom as users spin off new instances and do what they will. The really important thing here is that tiddlywiki is not just a wiki, but a framework for web apps. Room admins can easily perform customisations like changing the stylesheet; or go so far as coding up Javascript plugins to radically alter look-and-feel. More importantly, they can update the room’s recipe to include a bags of tiddlers sitting elsewhere on the system. That bag might be the TiddlyDocs system for instance, if they are interested in collaborative document publication, or it might be TiddlyGuv if they want an open-source governance system. They can then fine-tune those systems according to their own needs.

Here is a rich user story (a rich user story is like a regular user story, but it deliberately includes detail that a “pure agile” developer might object to as “getting ahead of yourself” , You-Ain’t-Gonna-Need-It, or unverifiable; while those objections are valid reasons not to use the story as a direct input to coding or task breakdown, and while rich visions are inevitably too premature to turn out into accurate forecasts, they do help to prove you’re heading in the right direction, to motivate user-centric developers like myself, and to illustrate what you’re doing to people outside the project) …

(cue dream sequence)

Jimmy is a moderately experienced TiddlyWiki user and wants to make a room for the enterprise’s music club to collaborate on compositions. So he logs into TiddlyWiki MU, hits “New Room”, and creates a room called “musicians”. He then sees the new room – a vanilla TiddlyWiki with a list of users down the side: currently, just Jimmy, with an Admin icon beside his name. He ignores the “Invite Users” button for now so he can concentrate on setting up the room. The groups wants to put out compositions as collective anthologies, so he decides TiddlyDocs will form the basis of this room. Thus, he clicks on “Manage Room” and a lightbox appears. It shows him the technical detail of the room on one side of the dialog – the bags that make up the room’s recipe. And on the other side, he can search and browse for new bags. He navigates to “tiddlydocs” bag and hits the “Add” button beside it. It now appears on the top of the room recipe.

Dismissing the lightbox and reloading, the “musicians” room is now a vanilla TiddlyDocs. He clicks on the backstage button on the top of the page and edits a few tiddlers – SiteTitle, SiteSubTitle, and ColorPalette, to give the room its own identity. He also needs a way for the participants to enter musical compositions. Luckily, he already has a standalone TiddlyWiki for music composition, with a Music plugin in it. From backstage, he pulls up the Import dialog and uploads the Music plugin from there. For each imported tiddler, there’s a dropdown showing the bags he can import it into (“musicians-config”, “musicians-comments’ etc). He indicates it should go in the musicians-config bag. On reloading the page, he finds something is broken – he can’t add musical notes from the TiddlyDocs editor. It’s time to call a friend …

He clicks “Invite Users” and up pops a modal dialog where he enters the email address of Dwight Doomore, a TiddlyWiki expert he’s fortunate to know. He checks the “make this user an admin” box and in the optional message area, explains the problem he’s having. Dwight clicks on a link in the subsequent email and he’s up and running inside the slightly broken “musicians” room. Donning his cyber-shades with Matrix-like precision, he proceeds to create a new plugin tiddler and monkey patches the Music plugin functionality so it plays nicely with TiddlyDocs. Then he tests it by writing up the first new composition, pulls up the user admin panel to remove himself from the room, and replies to Jimmy that it’s all taken care of.

Pleased with the result, Jimmy writes a few compositions, re-arranges them using TiddlyDocs’ tree control, and adds a little text. Instead of using “Invite Users”, he just mails his colleagues the URL of the “musicians” room. They each visit the URL, indicate they wish to join, and a few days later, Jimmy jumps into the user admin panel, where he can accept their requests to join. Now they can all work together on different documents within the one room, and TiddlyDocs provides enough functionality to control how the final copies will be published.

Six months later, the first “enterprisey musicians” composition book is printed and bound.

A single TiddlyWiki MU server has the capacity to facilitate a thousand stories like this, each of them with its own unique characters and quirks. There are plugins for voting, blogging, commenting, structured writing, quizzes, graphics, social bookmarking … it will be fun to see how users deal with all that. Most likely, it will follow the usual pyramid structure:

  • Many users will just create a vanilla room – plain old (multi-user) tiddlywiki. Of these, some will just make it a private scratchpad for themselves and never invite anyone at all.
  • Some users will make the room become a vanilla edition of a specialised application for their group to work on. e.g. collaborate on publication-ready docs with TiddlyDocs; collect and annotate websites with Scrumptious; brainstorm and vote on innovations with New Ideas.
  • A smaller number of users will customise the config tiddlers; pull in extra plugins; build new room-specific plugins; or attempt to combine multiple bags.
  • An even smaller number of users will have to deploy a standalone customised server, perhaps deploying a customised edition of TiddlyWiki MU with certain server-side plugins and configurations. This is the kind of thing you might do if using TiddlyWiki MU for a hardcore, full-fledged, 110% uptime, supported to the teeth, enterprise web app. You’d perhaps run a pilot on the regular TiddlyWiki MU server and then perform a (relatively effortless) migration onto its own dedicated server. Or a power user might just hand-configure a TiddlyWeb instance from parts. (This is where we get into “TiddlyWeb as a generic RESTful server” territory.)

The first case is already well-supported by other wiki platforms, so it’s nice and all being open-source and TiddlyWiki-based, but not really the killer app. The second is really the sweet spot when it comes to optimising user experience in the short term. We’ve spent a lot of time at Osmosoft thinking about how to deploy “TiddlyDocs” as a standalone package. That’s still interesting, but I think it’s a lot more interesting to ask how to deploy TiddlyWiki MU as a standalone package, and then let users go into web app and make a new “TiddlyDocs” instance with one click. The third case is supported by the general TiddlyWeb model and doesn’t need any special optimisation at this stage.

[There’s a thread about some of this stuff on the mailing list, so I’ve switched off comments to keep the conversation in one place.]