TiddlyDocs, TiddlyCMS, and Permissioning Models

We’re designing a setup for TiddlyDocs (and potentially other “TiddlyCMS”s) where there will be an instance for each group of contributors. This is one TiddlyWeb design pattern, where you say “we all trust each other” (although there are still audit logs!) and so everyone can add, delete, or modify all the content freely. That is, there is only one content bag accessible to all. Likewise, anyone can append comments. There is still an admin role for setting up config stuffs, which only the admin can change; otherwise, any user could easily perform a phisihing attack, for example. Users also have private bags which they can do what they like with – they could use it for drafts, or for overriding shadow tiddlers setup by the admin in config.

A rough sketch looks like this:

(Some things need fixing, but the overall idea is conveyed.)

I discuss more about this architectural pattern with Jon in the following video:

TiddlyDocs Architecture from Michael Mahemoff on Vimeo.

Now that model alone is good enough for basic cases, where you have a group of people who work together and just want to set up an instance. TiddlyWeb handles that just fine, even using the basic cookie authentication technique. In more complex situations (read: “enterprises”), you have a many-to-many relationship between users and instances. I was talking with TiddlyWeb’s architect, Chris Dent, about possibilities, and we decided on one possible model:

A central user-to-pool mapping database, which is shared by three web apps: (a) all the TiddlyDocs’ servers, specifically their Challenger module; (b) an admin app for managing the mappings (typical Django/Rails app with forms generated from data model); (c) a user-facing app (possibly a gadget in our envisioned OpenSocial scenario) showing the user (already authenticated via single-sign-on) a dropdown of instances they can launch.

Scrumptious: Conversations About Websites (and other resources)

Scrumptious is a web framework I’ve begun working on at Osmosoft. It’s a web app and web service for sharing bookmarks and comments about websites, and pretty much anything else with a unique URL. Things it is related to: Delicious (bookmarking), Digg (threaded comments), JS-Kit and Disqus (embedded comments with common identity across multiple sites).

Scrumptious is open source, under the BSD license (meaning you can do just about anything you like with it). There are already many open-source clones of this sort of thing, so why make a new one? There are a few reasons:

  • Adherence to RESTful principles – Scrumptious is backed by TiddlyWeb, a RESTful data service.
  • A non-TiddlyWiki TiddlyWeb client – So far, people who have used TiddlyWeb as a server have used TiddlyWiki as a client. They do play nicely together, but TiddlyWeb is a powerful RESTful framework on its own. Part of the motivation for Scrumptious was to port the TiddlyWiki nest comments plugin to a generic JQuery plugin that could be used on any web page. (Comments are indeed implemented as a plugin right now, but more work needs to be done to extract it into something truly modular and reusable; for example, the plugin right now assumes comments are about a web page; also, they are tied to TiddlyWeb. Nevertheless, the app still does achieve the main purpose of demonstrating TiddlyWeb is a fine data service for generic web apps.)
  • Demonstrating the power of URLs – In evangelising web standards, a very practical piece of advice is simply to associate a unique URL with each distinct resource. That’s REST 101, but it’s something lacking in many web apps. With a tool like Scrumptious, you get a comment system “for free” as long as each resource in your system has a unique URL. We’ll be developing a similar framework for URL Trails in the future, and the same principle applies: use unique URLs, and people can put your stuff in trails “for free”.
  • Flexible security model – Again, TiddlyWeb offers flexible permissioning, so you can use the app in different ways. e.g. a private conversation between colleagues; a public conversation (as with the demo), a publicly readable conversation where only certain individuals can contribute, etc. Likewise, TiddlyWeb offers flexible authentication, so you could hook into an organisation’s LDAP system, use open ID, simple user-password pairs, or any other form of authentication you wish to use.

Scrumptious is still at an early stage. Future work includes:

  • Bookmarking – for now, there is only commenting rather than social bookmarking per se.
  • Nested comments UI needs work to give it the same kind of UI as in the TiddlyWiki comments UI. e.g. show info like creator and creation data, and use suitable rendering and indenting.
  • Graceful degradation – It would be possible to provide a basic system not requiring Javascript. This would be probably be done using TiddlyWeb plugins, though I’d also be interested in running the JQuery-powered web app on the server, using server-side Javascript.
  • TiddlyWiki comments plugin harmonisation – As TiddlyWiki now ships with JQuery, it would be ideal if there was a single code base for the comments plugin, running in and out of TiddlyWiki. Indeed, I hope TiddlyWiki moves towards a general microkernel architecture, in which all plugins are useful outside TiddlyWiki. This is certainly becoming the case, with generic JQuery plugins being extracted for core activities like file saving, CSS applying, and wikifying.
  • Browser extensions – instead of a bookmarklet, use a browser extension to show, for each page, if comments are available, as well as bookmarking info. (Similar to the StumbleUpon or Delicious Firefox extensions.) A good opportunity to get my hands dirty with JetPack.
  • Login and identity management – while TiddlyWeb already provides the security and permissioning model, work is required to handle this at the UI level. For example, let anonymous users enter their email address and homepage, and/or register.
  • User admin – for situations where users must be authenticated, some form of user management would be handy. Again, TiddlyWeb provides a good model for this – a “bag” of tiddlers has its permission specified in JSON, which may include a list of users for each access type, so you simply need to PUT and GET this if you are sufficiently permissioned (ie an admin). What’s missing right now is a UI (in TiddlyWiki too, let alone standalone web apps).

Scrumptious Screencast:

<

p>Scrumptious from Michael Mahemoff on Vimeo.

List of Interesting and Popular MediaWiki Instances

I’m currently at an Osmosoft hackathon event, working on a tiddlywiki vertical called mediawikiunplugged. No time to say more now, but we needed a bunch of mediawiki sites for testing. So in addition to the obWikipedia and obAjaxPatterns, here are more:

And a list of the biggest wikis according to Alexa (retrieved thanks to archive.org).

TiddlyGuv

I’ve just presented TiddlyGuv at the FOSSBazaar workshop, the event I wrote about yesterday. It’s a good time to introduce TiddlyGuv – I’ve mentioned it in some talks and tweeted about it on occasion, but this is its maiden appearance on Software As She’s Developed.

TiddlyGuv (that’s a code name we’ll use for a while) is a web app to support open source software governance. In promoting and supporting open source at Osmosoft, one of the activities is a governance process. This means establishing policies for open source usage and auditing projects against it. TiddlyGuv is the tool we’re developing to support this process – it will support community discussions, policy publication, project tracking, workflow, and alerts about potential problems.

A few things about TiddlyGuv:

  • TiddlyGuv is in progress, it’s not complete and still needs work around a clean installation package. (We’re currently re-working how all this is done with TiddlyWeb verticals; once that’s done, there will be a package so people can try it out.) In any event, the code in some form is online in Trac.
  • TiddlyGuv is a web app. No surprise there :).
  • TiddlyGuv motivated the comments plugin, which has now been used in various other projects, including the live TiddlyWeb documentation wiki.
  • TiddlyGuv is built on the TiddlyWiki-TiddlyWeb stack, like TiddlyDocs. We see a general trend of CMS apps emerging on this stack, which has been called TiddlyCMS. (TiddlyCMS is an architectural pattern.) The aim is that TiddlyCMS apps share a large suite of common components such as the comments plugin.
  • TiddlyGuv is open source. The license hasn’t been finalised, but will likely be BSD, same as TiddlyWiki. We hope other organisations use TiddlyGuv in ther own organisations; TiddlyWiki and TiddlyWeb offer highly flexible architectures which lend themselves to adaptability.

Functionally, the first thing we’re aiming for is a tool with document wiki, for general discussion about open source usage, and a license portal, showing a list of licenses and information about each one. The comments plugin ensures each document and each license can be commented on too. All of this is there today, albeit in an untested state. Later on, we will add info about projects and further still, a workflow tool – for example, “Once the project form has been submitted, it must be reviewed by two named reviewers. They will each receive an email notification when a project is ready for review.” We would want the system to follow rules like that, and we would want to allow site administrators to maintain such rules. The TiddlyWiki model is to encapsulate rules like that inside tiddlers. I foresee some kind of Javascript-powered Domain-Specific Language (DSL).

The FOSSBazaar slide pack:

The feedback from the workshop was positive; there’s interest in the project and there was also interest from one vendor of a commercial product about ways to introduce the wiki or commenting aspects into an existing product. A good example of this sort of thing is Amazon – the way it has a structured product page, which incorporates a forum and a wiki. It raises some interesting architectural issues about how a TiddlyWiki/TiddlyWeb app could be embedded onto a page as a widget-style component, with identity and permissioning handled too. I also had an extended discussion with Mark Donohoe about how TiddlyGuv could talk to FOSSology. Of which, more in the next post.

TiddlyDocsDocs – Design Docs for Tiddly*

TiddlyDocs - document collaboration by divide and conquer - FireMoff (-;

Since we’re planning to use TiddlyDocs internally, we’re in need of some high-level documentation for TiddlyDocs in order to have it approved for certain uses.

My starting point was to locate, solicit, or produce documentation for TiddlyWiki and TiddlyWeb, the technologies on which TiddlyDocs is based.

For TiddlyWiki, useful sources are:

  • Position Paper Although it’s technically a position paper for an event on device access, Paul Downey has done a great job of overviewing TiddlyWiki from both a high-level and technical perspective.
  • TiddlyWiki Internals Series I wrote this when I joined Osmosoft, so a caveat is that my knowledge of TiddlyWiki was at the time limited. OTOH it’s the only detailed dive of the internals I’m aware of.
  • Community Wiki Scouring around the official community wiki is another way to find a lot of useful info, particularly about the API.

For TiddlyWeb, there are less third-party sources, but fortunately its creator Chris Dent puts a lot of effort into writing detailed notes. These are notes shipped with the code itself, notes on the TiddlyWeb wiki, and even some very useful notes in commit messages. Most recently, Chris has been producing some excellent doco at here, inside the wiki.

For TiddlyDocs, Simon has produced a nice new screencast on TiddlyDocs with animations. Video quality needs some polishing, which we’ll do in later versions, but he’s done a great job motivating the product:


TiddlyDocs Intro from Simon McManus on Vimeo.

TiddlyWiki Drawing Plugin based on Project Draw

I’ve been creating a drawing plugin for TiddlyWiki (SVN trunk). It delegates the actual drawing editing to Project Draw (docs; demo), a browser-based drawing editor from Autodesk Labs.

Here’s a screencast:


TiddlyWiki Drawing Plugin based on Project Draw from Michael Mahemoff on Vimeo.

Autodesk Labs’ Project Draw editor looks like this:

… and the tool has a nice set of RESTful services we can make use of:

Project Draw doesn’t have any specialised cross-domain API, e.g. using JSON or OAuth, so normally you would have to proxy any calls via the server. However, a standard TiddlyWiki runs against a file:// URL and can therefore bypass cross-domain restrictions; hence, it was possible to build this plugin using the same kind of XMLHttpRequest calls as Project Draw does for its own RESTful services demo.

One likely use of Project Draw is in conjunction with TiddlyDocs, which will generally run as a standard website – off a http:// URL. Hence, we will need to do some proxying in order to get around cross-domain restrictions.

Also, the tool isn’t fully working in IE yet; I’m working on that.

You create a new drawing with the New Drawing menu item. This is simply a <<newDrawing>> macro and could appear anywhere in the page.

And you can set image dimensions in edit mode:

… which will cause images to be clipped depending on the dimensions:

By default, images are rendered as SVG, using the tricks I discussed earlier for inline SVG. However, this is only supported for non-IE browsers, or IE with Flash-based SVG support. I am still working on IE support. Without SVG, the image will fall back to a bitmap image (fortunately, Project Draw can output an image as SVG or bitmap). However, note that the bitmap image always shows the whole thing; so rather than being clipped, it will re-scale, which could lead to a lot of whitespace being shown in TiddlyWiki. For this reason, in Project Draw, you should set the page dimensions to avoid any whitespace:

TiddlyDocs Prototype

Simon McManus has overviewed TiddlyDocs, a TiddlyWeb-based document editing and collaboration tool. It’s a great example of taking an existing framework – TiddlyWiki – adding in a bunch of components – TiddlyWiki plugins and other open source pieces – and putting some custom code around it to generate something useful in just a few weeks. Simon’s article has a good overview of all the moving parts inside TiddlyDocs.

I recorded a screencast to demo the initial version:


TiddlyDocs – Early Prototype from Michael Mahemoff on Vimeo.

I’m excited about the potential of this tool. In the simplest case, you can see it as an online word processor which, being open source, could therefore be deployed flexibly securely and freely behind any organisation’s firewall. But it can be much more than just a word-processor, because each section is a tiddler behind the scenes, and is therefore a first class data structure with its own set of fields. In particular, there are the seeds of a workflow system, with a field indicating the current completion status, and another field assigning the individual who will progress it. This means you can get an RSS feed, for example, of all sections in “under review” status, or a list of all sections assigned to a particular user.

Inline SVG

Okay, I’m currently ripsnort delighted to have found a solution to this problem of rendering SVG element dynamically. As in:

  1. My web apps receive some <svg>...<svg> from an Ajax call
  2. ???
  3. Super, there’s a drawing on the page!!!

What is the secret sauce in step 2?

After a merry frolic through the majority of the internets and a dozen prototypes, I finally found the answer. In essence, you just create an “object” element with type and data settings:

javascript

  1. var svgObject = document.createElement('object');
  2. svgObject.setAttribute('type', 'image/svg+xml');
  3. svgObject.setAttribute('data', 'data:image/svg+xml,'+ svgCode); // the "<svg>...</svg>" returned from Ajax call
  4. $("#reportSVGDiv").append(svgObject);

It works in Firefox, and the article explains how to get it working in IE too, which I don’t need just yet.

A few other things I learned and tried:

  • I initially, naievely, tried just adding the SVG via innerHTML. As Jeremy explained to me, this doesn’t work because the browser uses a different compiler for HTML compared to pure XHTML. (And TiddlyWiki, like most things on the web, is HTML.) Even with an <object> tag around it, it won’t just switch over.
  • The easiest way to do this is to use a .svg suffix (or probably set mime type to svg, but this is tiddlywiki and I’m working from file:// URLs, where it’s not possible to set mime type). But then you can’t embed it on the page – you’d have to point to it from an embed tag or object tag.
  • You can inline the SVG if you write “pure” xhtml, and for this, you have to give the file a .xhtml suffix. As in this example.
  • I tried the Inject HTML into an iframe technique. I was hoping that with the right XML declaration and HTML type, I could convince the iframe it was hosting XHTML. But no, I could not. I’m still interested to know if there’s any way I could convince a dynamically generated iframe, with dynamic content, about the type of content it contains.

This is all part of some recent prototyping on an exciting TiddlyWiki project involving rich text editing, among other things. In a later blog post, I’ll explain how we’ve used this SVG stuff to mash up an online chart drawing tool with TiddlyWiki. I’m currently packaging it into a plugin.

TiddlyWeb user authentication

I’ve been getting to grips with TiddlyWeb and authentication lately.

The following plugin code will set the username in the TiddlyWiki client to match the username that was presented to the server. It’s simply a one-liner which delegates to quirksmodes’ cookie handling library. This all assumes tiddlyweb is using the authentication challenger, as there’s no standard on cookie names.

config.options.txtUserName = readCookie(“tiddlyweb_user”).split(“:”)[0].substr(1);

// http://www.quirksmode.org/js/cookies.html function readCookie(name) { var nameEQ = name + “=”; var ca = document.cookie.split(‘;’); for(var i=0;i < ca.length;i++) { var c = ca[i]; while (c.charAt(0)==’ ‘) c = c.substring(1,c.length); if (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length,c.length); } return null; } [/javascript]

BTW this isn’t essential. When a user uploads something, TiddlyWeb will automatically set the modifier to the username on the server anyway. So the next time the user sees it, it will have the right username anyway. But having the right username in the client will make things right from the start and it feels cleaner to upload the tiddler with the right username data, even if it’s ignored. I could imagine future versions/plugins on the server that allow any username to be uploaded, with some algorithm to check if it’s allowed. (ie if admin, allow all usernames.)

Guid0: A Javascript GUID Generator

Guid0.js

Guid0 is a GUID library for Javascript. Okay, it doesn’t yet do official, bona fide, 128-bit, GUIDs yet, mainly for API design reasons. But this is a library you might find useful if you want to generate a unique ID in your Ajax app.

Usage:

javascript

  1. guid = new Guid();
  2. guid.generate(); // Returns a unique ID, e.g. "dkvagrkx1rt"

With options:

javascript

  1. guid = new Guid(
  2.   {
  3.     chars: Guid.constants.base85,  // or you could say "abc" if you only wanted those chars to appear
  4.     epoch: "June 1, 2003",
  5.     counterSequenceLength: 2, // a counter field appended to the end
  6.     randomSequenceLength: 2 // a random field appended to the end
  7.   }
  8. )

The demo:

The demo generates a bunch of GUIDs. They look almost the same because most of the GUID is just a time representation in Base N (where N is the number of characters in the GUID’s configured charset).

I made Guid0 in conjunction with TiddlyWiki work. Within a store, tiddlers are keyed on their title. In the typical case, the user has entered a title to describe the content. But when we get into application territory – using TiddlyWiki as an application framework – there are reasons why we want the more familiar database approach of automagically generating the primary key. These are several reasons for this:

  • Tiddlers with no meaningful title. Sometimes, tiddlers are just containers for text someone has entered. A perfect example is the Comments Plugin I’ve been working on (separate blog post pending). A comment is a tiddler. When you have a blog or forum or whatever with comments, you usually don’t invite users title their comment…it’s not very enticing. So in this case, the comment is the tiddler text and the title must be auto-generated so we can put it in the store.
  • Renaming. As with the usual motivation for auto-generated IDs over meaningful IDs, renaming support is sometimes required. In one case, I have a user submitting a first cut of something, and an admin coming along later and cleansing it. We can’t expect the user to get the name right the first time, and there are several tiddlers referencing the initial tiddler (via custom fields) by the time it might be renamed.
  • Can’t ensure uniqueness. With multiple users, you don’t always want to force them into choosing different titles for each tiddler. If the comments plugin did include titles for instance, you wouldn’t want a uniqueness constraint. Using GUIDs, you could support multiple titles with the same display name.

The library is generic – not TiddlyWiki-specific – but I’ve wrapped it into a small TiddlyWiki plugin. In the future, we may expand the plugin. For example, each time a new tiddler is created, the plugin may intercept the call and give the tiddler a GUID title, and stick the title in a “name” field.

I’ll update this post when proper 128-bit GUID is supported.