Multi-User TiddlyWiki

TiddlyWiki MU is what I’m calling – in the absence of an official name – an effort within Osmosoft to pull together a bunch of work into something that will be very useful in the enterprise and beyond. You could also call it “tiddlywiki as a service” or “multi-room tiddlywiki”. Similar to Wikia or WordPress MU, you can spin off a new multi-user instance with a “single click”. This is “MU” in the sense that there’s already a TiddlyWeb-backed TiddlyWiki product (TiddlyWebWiki). The value-add of TiddlyWiki MU (or whatever it ends up being called) is that you can make a new instance of such a thing without being a system administrator and without going through the effort of building it on the server. This is exactly like the way WordPress MU lets you spin off a new blog without having to set up a new instance of WordPress on the server.

In addition to the ease of spinning off new instances, there’s an important side benefit from this architectural pattern: synergy. From a user’s perspective, they only have one URL to remember/bookmark/share/link; once at that URL, the system can helpfully guide them through the different instances they have access to. Also, it would be possible to make content that’s used across different rooms. For example, give each user a private bag of tiddlers and they could use it to set their global preferences (with the right UI).

The whole thing is an open-source server, so an enterprise can just download it, deploy it, and let a thousand flowers bloom as users spin off new instances and do what they will. The really important thing here is that tiddlywiki is not just a wiki, but a framework for web apps. Room admins can easily perform customisations like changing the stylesheet; or go so far as coding up Javascript plugins to radically alter look-and-feel. More importantly, they can update the room’s recipe to include a bags of tiddlers sitting elsewhere on the system. That bag might be the TiddlyDocs system for instance, if they are interested in collaborative document publication, or it might be TiddlyGuv if they want an open-source governance system. They can then fine-tune those systems according to their own needs.

Here is a rich user story (a rich user story is like a regular user story, but it deliberately includes detail that a “pure agile” developer might object to as “getting ahead of yourself” , You-Ain’t-Gonna-Need-It, or unverifiable; while those objections are valid reasons not to use the story as a direct input to coding or task breakdown, and while rich visions are inevitably too premature to turn out into accurate forecasts, they do help to prove you’re heading in the right direction, to motivate user-centric developers like myself, and to illustrate what you’re doing to people outside the project) …

(cue dream sequence)

Jimmy is a moderately experienced TiddlyWiki user and wants to make a room for the enterprise’s music club to collaborate on compositions. So he logs into TiddlyWiki MU, hits “New Room”, and creates a room called “musicians”. He then sees the new room – a vanilla TiddlyWiki with a list of users down the side: currently, just Jimmy, with an Admin icon beside his name. He ignores the “Invite Users” button for now so he can concentrate on setting up the room. The groups wants to put out compositions as collective anthologies, so he decides TiddlyDocs will form the basis of this room. Thus, he clicks on “Manage Room” and a lightbox appears. It shows him the technical detail of the room on one side of the dialog – the bags that make up the room’s recipe. And on the other side, he can search and browse for new bags. He navigates to “tiddlydocs” bag and hits the “Add” button beside it. It now appears on the top of the room recipe.

Dismissing the lightbox and reloading, the “musicians” room is now a vanilla TiddlyDocs. He clicks on the backstage button on the top of the page and edits a few tiddlers – SiteTitle, SiteSubTitle, and ColorPalette, to give the room its own identity. He also needs a way for the participants to enter musical compositions. Luckily, he already has a standalone TiddlyWiki for music composition, with a Music plugin in it. From backstage, he pulls up the Import dialog and uploads the Music plugin from there. For each imported tiddler, there’s a dropdown showing the bags he can import it into (“musicians-config”, “musicians-comments’ etc). He indicates it should go in the musicians-config bag. On reloading the page, he finds something is broken – he can’t add musical notes from the TiddlyDocs editor. It’s time to call a friend …

He clicks “Invite Users” and up pops a modal dialog where he enters the email address of Dwight Doomore, a TiddlyWiki expert he’s fortunate to know. He checks the “make this user an admin” box and in the optional message area, explains the problem he’s having. Dwight clicks on a link in the subsequent email and he’s up and running inside the slightly broken “musicians” room. Donning his cyber-shades with Matrix-like precision, he proceeds to create a new plugin tiddler and monkey patches the Music plugin functionality so it plays nicely with TiddlyDocs. Then he tests it by writing up the first new composition, pulls up the user admin panel to remove himself from the room, and replies to Jimmy that it’s all taken care of.

Pleased with the result, Jimmy writes a few compositions, re-arranges them using TiddlyDocs’ tree control, and adds a little text. Instead of using “Invite Users”, he just mails his colleagues the URL of the “musicians” room. They each visit the URL, indicate they wish to join, and a few days later, Jimmy jumps into the user admin panel, where he can accept their requests to join. Now they can all work together on different documents within the one room, and TiddlyDocs provides enough functionality to control how the final copies will be published.

Six months later, the first “enterprisey musicians” composition book is printed and bound.

A single TiddlyWiki MU server has the capacity to facilitate a thousand stories like this, each of them with its own unique characters and quirks. There are plugins for voting, blogging, commenting, structured writing, quizzes, graphics, social bookmarking … it will be fun to see how users deal with all that. Most likely, it will follow the usual pyramid structure:

  • Many users will just create a vanilla room – plain old (multi-user) tiddlywiki. Of these, some will just make it a private scratchpad for themselves and never invite anyone at all.
  • Some users will make the room become a vanilla edition of a specialised application for their group to work on. e.g. collaborate on publication-ready docs with TiddlyDocs; collect and annotate websites with Scrumptious; brainstorm and vote on innovations with New Ideas.
  • A smaller number of users will customise the config tiddlers; pull in extra plugins; build new room-specific plugins; or attempt to combine multiple bags.
  • An even smaller number of users will have to deploy a standalone customised server, perhaps deploying a customised edition of TiddlyWiki MU with certain server-side plugins and configurations. This is the kind of thing you might do if using TiddlyWiki MU for a hardcore, full-fledged, 110% uptime, supported to the teeth, enterprise web app. You’d perhaps run a pilot on the regular TiddlyWiki MU server and then perform a (relatively effortless) migration onto its own dedicated server. Or a power user might just hand-configure a TiddlyWeb instance from parts. (This is where we get into “TiddlyWeb as a generic RESTful server” territory.)

The first case is already well-supported by other wiki platforms, so it’s nice and all being open-source and TiddlyWiki-based, but not really the killer app. The second is really the sweet spot when it comes to optimising user experience in the short term. We’ve spent a lot of time at Osmosoft thinking about how to deploy “TiddlyDocs” as a standalone package. That’s still interesting, but I think it’s a lot more interesting to ask how to deploy TiddlyWiki MU as a standalone package, and then let users go into web app and make a new “TiddlyDocs” instance with one click. The third case is supported by the general TiddlyWeb model and doesn’t need any special optimisation at this stage.

[There’s a thread about some of this stuff on the mailing list, so I’ve switched off comments to keep the conversation in one place.]

TiddlyDocs User Authentication – Generic Design and Custom Features

We are working on a design that will keep TiddlyDocs as generic as possible. The core TiddlyDocs product will have a concept of “rooms”. This is the same kind of room you see in online forums or chat sites, where everyone inside the room contributes and sees content inside the room they’re in, and can’t see or do anything about rooms they’re not in.

Our design works like this. Each room is a recipe of several room-specific bags, along with a global config bag. Each room is associated with a unique user role.

Note the role is used inside the respective bags’ policy files, and they each vary somewhat. e.g. the comments bag says readable by everyone in ‘role’, appendable (“create tiddler” permission) by everyone in ‘role’, editable by “#(role)_admin”. (ie there is a second role called “role_admin”.); whereas the content bags says readable and writeable and editable by everyone in ‘role’; and the config bag says readable by everyone in ‘role’, appendable by ‘#{role}_admin”.

For now, there’s a “makeroom” script that generates the recipe, the bags and the policies. (I’d ideally like to see the recipe be “magic” in some respects, using a wildcard mechanism).

That’s the basic design. We also have some unique challenges in this particular installation – (a) users are authenticated using a custom, enterprise-specific, single-sign on solution, where the incoming header for each request identifies user’s ID and (effectively) company; (b) we have certain rooms whose composition is manually configured and are a subset of the company’s membership, and we also have a “everyone” room which is automatically composed of everyone in the company (e.g. we might have a “megacorp-everyone” room along with the manually configured “megacorp-accounting” room, and “megacorp-marketing” room); (c) to let users allocate specific TiddlyDocs to users, we need a dropdown containing real names of all users.

BTW I’m simplifying some of the finer details here to make the discussion more pertinent to the general topic of TiddlyWeb/TiddlyDocs, rather than our specific problem. All of the following will be developed as proprietary plugins for our own installation. It’s a somewhat complicated set of requirements because it’s a real-world enterprise problem, where we have specialised requirements, combined with integration issues where we can control some things, but can’t control others. (e.g. We can’t change the underlying user store, where we would ideally store which rooms a user is in, instead of holding secondary user records inside TiddlyDocs.)

We will be building a TiddlyWeb plugin that sets the “extractor” filter. This is one of the several filters the request is subject to, on the way into TiddlyWeb. It is usually paired with the challenger filter – where challenger might present a username/password form and set a cookie, and extractor will read the cookie and set usersign (aka user ID) and any other info like users’ roles. In this case, though, we already have the header set, so we don’t need a challenger at all. We simply need an extractor that sets the usersign from the incoming header, and also notes the user’s company. Verification will then check that (a) if the user is attempting to access the “everyone” company room, they are a member of the company; (b) if the user is attempting to access a “manual” company room, they are a member of the company and they have the role associated with the company. We use a naming convention to determine which company a manual room belongs to.

On (c), we need a tiddler in the room “config” bag containing a list of user names. So in addition to the custom extractor, we will also need a periodic script to poll the user store associated with the SSO system. This will give us all the info about users in each company, so we can build up a mapping from company to list of users, where each user record has an ID and name. company->list[{user ID, user name]}. The script will then (a) rewrite the list of user names for the “everyone room” based on the company list, by effectively “copying” the results for that company (b) rewrite the list of user names for each manual room, by walking through each user, checking their roles, and accumulating their user name into the room list for each room they’re in (c) do some cleanup – removing TiddlyWeb records for any users not in the list.

There’s nothing very elegant about these custom implementations, and I’ve primarily written these details for the benefit of myself and colleagues who are implementing these custom features. The more interesting, general, point here is about open-source and plugins. TiddlyWeb’s architecture allows us deliver a generic solution for TiddlyDocs rooms, which is applicable to any organisation, and we can then superimpose a custom feature using the plugin mechanism. Furthermore, the plugin mechanism is a great way to decouple concerns – plugins are moduels on steroids. The core TiddlyDocs developer can concentrate on building an open-source, functional, multi-room TiddlyDocs edition, using the default authentication mechanism. Meanwhile, other developers can work in parallel on custom plugins.

(This post is a refinement of this post; we have been doing further thinking as we get closer to the final product and we learn more about our external environment.)

TiddlyDocs, TiddlyCMS, and Permissioning Models

We’re designing a setup for TiddlyDocs (and potentially other “TiddlyCMS”s) where there will be an instance for each group of contributors. This is one TiddlyWeb design pattern, where you say “we all trust each other” (although there are still audit logs!) and so everyone can add, delete, or modify all the content freely. That is, there is only one content bag accessible to all. Likewise, anyone can append comments. There is still an admin role for setting up config stuffs, which only the admin can change; otherwise, any user could easily perform a phisihing attack, for example. Users also have private bags which they can do what they like with – they could use it for drafts, or for overriding shadow tiddlers setup by the admin in config.

A rough sketch looks like this:

(Some things need fixing, but the overall idea is conveyed.)

I discuss more about this architectural pattern with Jon in the following video:

TiddlyDocs Architecture from Michael Mahemoff on Vimeo.

Now that model alone is good enough for basic cases, where you have a group of people who work together and just want to set up an instance. TiddlyWeb handles that just fine, even using the basic cookie authentication technique. In more complex situations (read: “enterprises”), you have a many-to-many relationship between users and instances. I was talking with TiddlyWeb’s architect, Chris Dent, about possibilities, and we decided on one possible model:

A central user-to-pool mapping database, which is shared by three web apps: (a) all the TiddlyDocs’ servers, specifically their Challenger module; (b) an admin app for managing the mappings (typical Django/Rails app with forms generated from data model); (c) a user-facing app (possibly a gadget in our envisioned OpenSocial scenario) showing the user (already authenticated via single-sign-on) a dropdown of instances they can launch.

TiddlyDocsDocs – Design Docs for Tiddly*

TiddlyDocs - document collaboration by divide and conquer - FireMoff (-;

Since we’re planning to use TiddlyDocs internally, we’re in need of some high-level documentation for TiddlyDocs in order to have it approved for certain uses.

My starting point was to locate, solicit, or produce documentation for TiddlyWiki and TiddlyWeb, the technologies on which TiddlyDocs is based.

For TiddlyWiki, useful sources are:

  • Position Paper Although it’s technically a position paper for an event on device access, Paul Downey has done a great job of overviewing TiddlyWiki from both a high-level and technical perspective.
  • TiddlyWiki Internals Series I wrote this when I joined Osmosoft, so a caveat is that my knowledge of TiddlyWiki was at the time limited. OTOH it’s the only detailed dive of the internals I’m aware of.
  • Community Wiki Scouring around the official community wiki is another way to find a lot of useful info, particularly about the API.

For TiddlyWeb, there are less third-party sources, but fortunately its creator Chris Dent puts a lot of effort into writing detailed notes. These are notes shipped with the code itself, notes on the TiddlyWeb wiki, and even some very useful notes in commit messages. Most recently, Chris has been producing some excellent doco at here, inside the wiki.

For TiddlyDocs, Simon has produced a nice new screencast on TiddlyDocs with animations. Video quality needs some polishing, which we’ll do in later versions, but he’s done a great job motivating the product:

TiddlyDocs Intro from Simon McManus on Vimeo.

TiddlyDocs Prototype

Simon McManus has overviewed TiddlyDocs, a TiddlyWeb-based document editing and collaboration tool. It’s a great example of taking an existing framework – TiddlyWiki – adding in a bunch of components – TiddlyWiki plugins and other open source pieces – and putting some custom code around it to generate something useful in just a few weeks. Simon’s article has a good overview of all the moving parts inside TiddlyDocs.

I recorded a screencast to demo the initial version:

TiddlyDocs – Early Prototype from Michael Mahemoff on Vimeo.

I’m excited about the potential of this tool. In the simplest case, you can see it as an online word processor which, being open source, could therefore be deployed flexibly securely and freely behind any organisation’s firewall. But it can be much more than just a word-processor, because each section is a tiddler behind the scenes, and is therefore a first class data structure with its own set of fields. In particular, there are the seeds of a workflow system, with a field indicating the current completion status, and another field assigning the individual who will progress it. This means you can get an RSS feed, for example, of all sections in “under review” status, or a list of all sections assigned to a particular user.