When Resource IDs are URLs

Being a web app about websites and other resources, Scrumptious has a resource which is basically a URL, called “Pages”. A Scrumptious resource modelling google.com/about looks like:

http://scrumptious.tv/comments/bags/ pages/tiddlers/http%3A%2F%2Fgoogle.com%2Fabout

See? The resource ID is http://google.com/about, encoded. (encodeURIComponent("http://google.com/about")

This was working fine on my dev machine, running “twanager server comments.boz 8080” (comments.boz being an alias for my local machine). But on the server, and run through apache and TiddlyWeb’s apache.py, it failed:

The fix was twofold – both of the following were required:

  • AllowEncodedSlashes On in apache config. This option ensures encoded slashes (%2F) are passed through to the end-app.
  • PathInfoHack plugin As with any TiddlyWeb plugin, I downloaded it and added it to ‘system_plugins’ list in tiddlywebconfig.py. (Thanks Chris for the pointer.)

And now we can happily talk resources with URLs as IDs to the server.

TiddlyDocs, TiddlyCMS, and Permissioning Models

We’re designing a setup for TiddlyDocs (and potentially other “TiddlyCMS”s) where there will be an instance for each group of contributors. This is one TiddlyWeb design pattern, where you say “we all trust each other” (although there are still audit logs!) and so everyone can add, delete, or modify all the content freely. That is, there is only one content bag accessible to all. Likewise, anyone can append comments. There is still an admin role for setting up config stuffs, which only the admin can change; otherwise, any user could easily perform a phisihing attack, for example. Users also have private bags which they can do what they like with – they could use it for drafts, or for overriding shadow tiddlers setup by the admin in config.

A rough sketch looks like this:

(Some things need fixing, but the overall idea is conveyed.)

I discuss more about this architectural pattern with Jon in the following video:

TiddlyDocs Architecture from Michael Mahemoff on Vimeo.

Now that model alone is good enough for basic cases, where you have a group of people who work together and just want to set up an instance. TiddlyWeb handles that just fine, even using the basic cookie authentication technique. In more complex situations (read: “enterprises”), you have a many-to-many relationship between users and instances. I was talking with TiddlyWeb’s architect, Chris Dent, about possibilities, and we decided on one possible model:

A central user-to-pool mapping database, which is shared by three web apps: (a) all the TiddlyDocs’ servers, specifically their Challenger module; (b) an admin app for managing the mappings (typical Django/Rails app with forms generated from data model); (c) a user-facing app (possibly a gadget in our envisioned OpenSocial scenario) showing the user (already authenticated via single-sign-on) a dropdown of instances they can launch.

Scrumptious: Conversations About Websites (and other resources)

Scrumptious is a web framework I’ve begun working on at Osmosoft. It’s a web app and web service for sharing bookmarks and comments about websites, and pretty much anything else with a unique URL. Things it is related to: Delicious (bookmarking), Digg (threaded comments), JS-Kit and Disqus (embedded comments with common identity across multiple sites).

Scrumptious is open source, under the BSD license (meaning you can do just about anything you like with it). There are already many open-source clones of this sort of thing, so why make a new one? There are a few reasons:

  • Adherence to RESTful principles – Scrumptious is backed by TiddlyWeb, a RESTful data service.
  • A non-TiddlyWiki TiddlyWeb client – So far, people who have used TiddlyWeb as a server have used TiddlyWiki as a client. They do play nicely together, but TiddlyWeb is a powerful RESTful framework on its own. Part of the motivation for Scrumptious was to port the TiddlyWiki nest comments plugin to a generic JQuery plugin that could be used on any web page. (Comments are indeed implemented as a plugin right now, but more work needs to be done to extract it into something truly modular and reusable; for example, the plugin right now assumes comments are about a web page; also, they are tied to TiddlyWeb. Nevertheless, the app still does achieve the main purpose of demonstrating TiddlyWeb is a fine data service for generic web apps.)
  • Demonstrating the power of URLs – In evangelising web standards, a very practical piece of advice is simply to associate a unique URL with each distinct resource. That’s REST 101, but it’s something lacking in many web apps. With a tool like Scrumptious, you get a comment system “for free” as long as each resource in your system has a unique URL. We’ll be developing a similar framework for URL Trails in the future, and the same principle applies: use unique URLs, and people can put your stuff in trails “for free”.
  • Flexible security model – Again, TiddlyWeb offers flexible permissioning, so you can use the app in different ways. e.g. a private conversation between colleagues; a public conversation (as with the demo), a publicly readable conversation where only certain individuals can contribute, etc. Likewise, TiddlyWeb offers flexible authentication, so you could hook into an organisation’s LDAP system, use open ID, simple user-password pairs, or any other form of authentication you wish to use.

Scrumptious is still at an early stage. Future work includes:

  • Bookmarking – for now, there is only commenting rather than social bookmarking per se.
  • Nested comments UI needs work to give it the same kind of UI as in the TiddlyWiki comments UI. e.g. show info like creator and creation data, and use suitable rendering and indenting.
  • Graceful degradation – It would be possible to provide a basic system not requiring Javascript. This would be probably be done using TiddlyWeb plugins, though I’d also be interested in running the JQuery-powered web app on the server, using server-side Javascript.
  • TiddlyWiki comments plugin harmonisation – As TiddlyWiki now ships with JQuery, it would be ideal if there was a single code base for the comments plugin, running in and out of TiddlyWiki. Indeed, I hope TiddlyWiki moves towards a general microkernel architecture, in which all plugins are useful outside TiddlyWiki. This is certainly becoming the case, with generic JQuery plugins being extracted for core activities like file saving, CSS applying, and wikifying.
  • Browser extensions – instead of a bookmarklet, use a browser extension to show, for each page, if comments are available, as well as bookmarking info. (Similar to the StumbleUpon or Delicious Firefox extensions.) A good opportunity to get my hands dirty with JetPack.
  • Login and identity management – while TiddlyWeb already provides the security and permissioning model, work is required to handle this at the UI level. For example, let anonymous users enter their email address and homepage, and/or register.
  • User admin – for situations where users must be authenticated, some form of user management would be handy. Again, TiddlyWeb provides a good model for this – a “bag” of tiddlers has its permission specified in JSON, which may include a list of users for each access type, so you simply need to PUT and GET this if you are sufficiently permissioned (ie an admin). What’s missing right now is a UI (in TiddlyWiki too, let alone standalone web apps).

Scrumptious Screencast:

<

p>Scrumptious from Michael Mahemoff on Vimeo.

easy_install: Prerequisite for TiddlyWeb

Done a few TiddlyWeb installs now – here’s a tip. It’s easiest done with easy_install. This is a tool that installs or upgrades everything “in a click”, except that installing easy_install isn’t really easy; the Python community could learn a lot from the Rails community about how to make downloading and installing easy, for those who aren’t intimately familiar with the underlying technology. (Summary: I google for the tool, I click on the first result, and I see a big fat download button). If you try to look up easy_install, also called “easyinstall”, you’ll find it’s actually included with “setuptools”, and you’ll find a website that makes you go around in circles if you click the links you think you’re meant to click. What you might not find is the package you need as the downloads are buried down the bottom of

For a Mac at least, get easy_install with the following steps:

TiddlyGuv

I’ve just presented TiddlyGuv at the FOSSBazaar workshop, the event I wrote about yesterday. It’s a good time to introduce TiddlyGuv – I’ve mentioned it in some talks and tweeted about it on occasion, but this is its maiden appearance on Software As She’s Developed.

TiddlyGuv (that’s a code name we’ll use for a while) is a web app to support open source software governance. In promoting and supporting open source at Osmosoft, one of the activities is a governance process. This means establishing policies for open source usage and auditing projects against it. TiddlyGuv is the tool we’re developing to support this process – it will support community discussions, policy publication, project tracking, workflow, and alerts about potential problems.

A few things about TiddlyGuv:

  • TiddlyGuv is in progress, it’s not complete and still needs work around a clean installation package. (We’re currently re-working how all this is done with TiddlyWeb verticals; once that’s done, there will be a package so people can try it out.) In any event, the code in some form is online in Trac.
  • TiddlyGuv is a web app. No surprise there :).
  • TiddlyGuv motivated the comments plugin, which has now been used in various other projects, including the live TiddlyWeb documentation wiki.
  • TiddlyGuv is built on the TiddlyWiki-TiddlyWeb stack, like TiddlyDocs. We see a general trend of CMS apps emerging on this stack, which has been called TiddlyCMS. (TiddlyCMS is an architectural pattern.) The aim is that TiddlyCMS apps share a large suite of common components such as the comments plugin.
  • TiddlyGuv is open source. The license hasn’t been finalised, but will likely be BSD, same as TiddlyWiki. We hope other organisations use TiddlyGuv in ther own organisations; TiddlyWiki and TiddlyWeb offer highly flexible architectures which lend themselves to adaptability.

Functionally, the first thing we’re aiming for is a tool with document wiki, for general discussion about open source usage, and a license portal, showing a list of licenses and information about each one. The comments plugin ensures each document and each license can be commented on too. All of this is there today, albeit in an untested state. Later on, we will add info about projects and further still, a workflow tool – for example, “Once the project form has been submitted, it must be reviewed by two named reviewers. They will each receive an email notification when a project is ready for review.” We would want the system to follow rules like that, and we would want to allow site administrators to maintain such rules. The TiddlyWiki model is to encapsulate rules like that inside tiddlers. I foresee some kind of Javascript-powered Domain-Specific Language (DSL).

The FOSSBazaar slide pack:

The feedback from the workshop was positive; there’s interest in the project and there was also interest from one vendor of a commercial product about ways to introduce the wiki or commenting aspects into an existing product. A good example of this sort of thing is Amazon – the way it has a structured product page, which incorporates a forum and a wiki. It raises some interesting architectural issues about how a TiddlyWiki/TiddlyWeb app could be embedded onto a page as a widget-style component, with identity and permissioning handled too. I also had an extended discussion with Mark Donohoe about how TiddlyGuv could talk to FOSSology. Of which, more in the next post.

TiddlyDocsDocs – Design Docs for Tiddly*

TiddlyDocs - document collaboration by divide and conquer - FireMoff (-;

Since we’re planning to use TiddlyDocs internally, we’re in need of some high-level documentation for TiddlyDocs in order to have it approved for certain uses.

My starting point was to locate, solicit, or produce documentation for TiddlyWiki and TiddlyWeb, the technologies on which TiddlyDocs is based.

For TiddlyWiki, useful sources are:

  • Position Paper Although it’s technically a position paper for an event on device access, Paul Downey has done a great job of overviewing TiddlyWiki from both a high-level and technical perspective.
  • TiddlyWiki Internals Series I wrote this when I joined Osmosoft, so a caveat is that my knowledge of TiddlyWiki was at the time limited. OTOH it’s the only detailed dive of the internals I’m aware of.
  • Community Wiki Scouring around the official community wiki is another way to find a lot of useful info, particularly about the API.

For TiddlyWeb, there are less third-party sources, but fortunately its creator Chris Dent puts a lot of effort into writing detailed notes. These are notes shipped with the code itself, notes on the TiddlyWeb wiki, and even some very useful notes in commit messages. Most recently, Chris has been producing some excellent doco at here, inside the wiki.

For TiddlyDocs, Simon has produced a nice new screencast on TiddlyDocs with animations. Video quality needs some polishing, which we’ll do in later versions, but he’s done a great job motivating the product:


TiddlyDocs Intro from Simon McManus on Vimeo.

TiddlyWeb user authentication

I’ve been getting to grips with TiddlyWeb and authentication lately.

The following plugin code will set the username in the TiddlyWiki client to match the username that was presented to the server. It’s simply a one-liner which delegates to quirksmodes’ cookie handling library. This all assumes tiddlyweb is using the authentication challenger, as there’s no standard on cookie names.

config.options.txtUserName = readCookie(“tiddlyweb_user”).split(“:”)[0].substr(1);

// http://www.quirksmode.org/js/cookies.html function readCookie(name) { var nameEQ = name + “=”; var ca = document.cookie.split(‘;’); for(var i=0;i < ca.length;i++) { var c = ca[i]; while (c.charAt(0)==’ ‘) c = c.substring(1,c.length); if (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length,c.length); } return null; } [/javascript]

BTW this isn’t essential. When a user uploads something, TiddlyWeb will automatically set the modifier to the username on the server anyway. So the next time the user sees it, it will have the right username anyway. But having the right username in the client will make things right from the start and it feels cleaner to upload the tiddler with the right username data, even if it’s ignored. I could imagine future versions/plugins on the server that allow any username to be uploaded, with some algorithm to check if it’s allowed. (ie if admin, allow all usernames.)