Open-Source Hackathon Patterns: First Cut

This is my first cut at identifying process/organisational patterns we’ve used and evolved for running hackathons at Osmosoft. Many are regular agile patterns, specialised for the Osmosoft-style hackathon context.

I described one particular hackathon in detail a few months ago. We’ve done about half a dozen of them now,for a range of different customers, with requisite navel-gazing in between the events, so we have begun to get an idea of what works and what doesn’t. Hopefully, these patterns will help to broaden the conversation and help us further evolve our practices.

Preparation Patterns

  • Skeleton Project – A live skeleton project is prepared before the event.
  • Functional Plugin – Certain generic plugins required for the hackathon should be identified and built, if time permits.
  • User Stories – Team and customers agree on user stories, with priorities.
  • Fixed Duration – Team and customers agree upfront on finishing time.
  • Customer Agreement – Customers agree on their obligations and expectations.

Development Patterns

  • Open-Source Code – The hackathon develops open-source code, to be placed in a public repository throughout the day (proprietary features can be integrated via plugins and configuration at a later date).
  • Continuous Integration – Everyone checks into the central repo and builds/tests on their local machine.
  • Public Instance – An instance, continuously built from the repository, is available to the customer (speculative).
  • Dedicated Integrator – One developer supports code integration and continuous integration, instead of developing to user stories.
  • Customer Liason – One developer is primarily responsible for engaging with customers before, during, and after the hackathon.
  • Stand-Up Meetings – Periodic stand-up meetings throughout the day, with each meeting deciding the time for the next meeting.

Post-Development Patterns

  • Walkthrough – Developers walk through final product Screencast – Final product is captured as a screencast, to be published online.

Actions and Conclusions from TiddlyWeb Dev/Deploy Workshop


As everyone’s been developing in different ways, I wanted to arrange a session to share experiences and come up with best practices – to help us become (a) more consistent and (b) more productive going forward.

We agreed on objectives (20min), explained our current practices (1hr), had a general discussion and debate on deployment and development (1hr), and we then performed an interesting activity @cdent had suggested earlier on, which was to do a group coding session in front of a projector. You could think of it as an enhanced meeting more than “pair programming on steroids”. Part of it was for Chris to explain the tiddlyweb tools that we should be using more, and part of it was to walk through a project lifecycle and critique the stages. This was 2 hours, we concluded the event, and Chris, Fred (@FND), and I remained to refine some of the points into the following summary.

Best Practices and Required Tooling

Best Practices we agreed on:

  • Deployment – Release verticals as SetupTools modules [tarball is for people who want to play with the source egg is for people who just want to install it If you distribute the tarball, you’re effectively distributing the egg; you can build it from the tarball]

  • Deployment – Release mature plugins as SetupTools modules This way, verticals can refer to them as depencies for auto-update (“assemble and aggregate” as @FND put it); in the event of verticals using “immature” plugins (just “.py” files form the web), they can include it in their own package

  • Dev – develop in the dev store, which will be re-created/upgraded We’ll be discussing features for the new/upgraded dev store on irc tomorrow. It will need to offer a way to include files anywhere on the hard drive, and also in remote locations (http; ideally svn, git, etc)

  • Dev/Deploy – use bash scripts

    • “tiddlyspawn” (name TBA) creates a new project (probably using twinstance, the 2009 AD reincarnation to “twanager instance”, among other things);, inheriting config from tiddlywebwiki.config, make src dir, “parse” to find dependencies and run “easy_install” on them, perhaps make static dir
    • “” – inside the project to run updates, pull in remote plugins, build setuptools tarballs for release
  • Project dir structure (indicative)

    /  # python setuptools recipe - stub's created by
    / # the all-singing
    /component.yml #

TiddlyWeb Pow-Wow (by psd)

Osmosoft Collectively Exploring TiddlyWeb (by psd)

It starts with a Yellow Fade: The need for a more comprehensive understanding of visual effects on the web

Around the time Ajax got coined, one of the already-known patterns was 37Signals’ Yellow Fade Effect. As techniques were shared and visual effects libraries emerged, we began to see visual effects become commonplace on the web. I documented four of them in Ajax Design Patterns: One-Second Spotlight, One-Second Mutation, One-Second Motion, Highlight. (I wish I called them something else, e.g. “Spotlight Effect”.) And all of these are around today, and even in the jQuery core which makes t hem pretty darn accessible to any developer. It’s as simple as $(“message”).fadeIn().

Where we are now, most developers are stuck with trial-and-error. In an ideal world, you’d have a SWAT team of UX experts to analyse each visual effect, test with users, and maybe they’d get it right, but most UI design in the real world and developers often just have to play around a bit and then move on to something else. So I think we could do with some simple guidelines, ideally based on empirical data, but even just based on pattern mining the web would be handy.

I’ll give some examples of the kinds of problems I’ve faced as a developer building in visual effects:

  • Choosing Visual Effect. I want to update some content that’s just been received from the server. e.g. the football score changed. Do I show a yellow highlight, make it blink, make it vibrate, do nothing?
  • Choosing Effect Parameters. I’ve decided to use a Slide Down effect. How longshould it last, i.e. the duration of time from not shown to show? Should it slide down at constant pace or accelerate and bounce at the bottom?
  • Composing Effects I have a block of content with the user can edit. When switching it between edit and view modes, there are two simultaneous effects taking place – one thing is being hidden while the other is being shown. How do I handle those? Do I slide one up and when that’s done, slide down the other? Do I fade one out and simultaneously fade one in? Do I immediately close one and transition in the other, or vice-versa? Many options come into play.

It should go without saying that there’s no right answer for any of these problems. But instead of developers always working from first principles, I think there are some very useful guidelines and discussions that could be had around the design patterns involved. Maybe even leading to a high level effects library supporting those patterns.

InfoBoxPlugin: A TiddlyWiki Plugin for InfoBoxen

InfoBoxPlugin - Lists tiddlers in a table (by mahemoff)

G’Day, here’s a new tiddlywiki plugin I’ve been working on: InfoBoxPlugin. It’s based on the equally-monikered infoBox in MediaWiki/Wikipedia, which you’ll see in any article that is marked “current event” or “controversial”, for example, on the big W. I find infoBoxes elegant, as they are unobtrusive enough to let you get on reading the article, and are easily ignored in the same way as web ads, but when you do focus on them, they are clear in meaning and support pattern recognition, with each type of infoBox having its own look and distinct icon.

The infoBox macro makes infoBoxen that look as shown in the diagram above. In the simplest case, you type this into a tiddler:

<<infoBox>>This is good stuff – pay attention mkay.>>

Pretty straightforward. One funniness here is the bracket asymmetry. We’re trying to do something similar to XML tags, what with their start tag and attribs, followed by body, followed by closing tag, so you’d expect to see:

<<infoBox>>This is good stuff – pay attention mkay.<</infoBox>>

or sumptink like dat. But one of my lessons was that tiddlywiki has a somewhat unusual convention, enshrined only in the gradient macro to my knowledge, for the former syntax. The only way to do something else would be to refactor or replicate or hijack the core code, and I don’t fancy it, and in any event it would go against the standard already set by gradient. In any event, I was pleased enough when Jeremy showed me the example of the gradient tag, which shows that this kind of “macro body” is even possible, so I settled with that.

Okay, so that’s nice, you can do a simple message, but where it gets more to the point is where you build up a family of infoBox types for your TiddlyWiki, each having a separate definition tiddler. For example, you will often want a “warning” infoBox to appear in various places. So you make a “warningInfoBox” tiddler, with the following text:

|heading|Danger, Will Robinson!!!|
|message|Please follow these instructions carefully.|

And then in a tiddler, you just write <<infoBox warning>>>> The association between “warning” and “warningInfoBox” is an enforced convention; my macro just appends “InfoBox” to the type you declare. The double-double closing bracket is a consequence of what I said above, combined with the fact that the definition has a pre-defined “message”. This is often going to be what we want, but not always. In fact, all of the fields above are optional, so you could leave out “message” and then the warning macro would have to specify it (if you wanted a message, that is). i.e. <<infoBox warning>>This Is Serious Mum.>> (… because the undeniably-talented, always-controversial, Aussie band This Is Serious Mum – aka TISM – popped into my head as an instructive example for this warning.)

Some other things to say

  • This plugin is part of a greater effort to pull out the goodies from TiddlyGuv into reusable modules. TiddlyGuv was the first thing I worked on upon joining Osmosoft, and when I looked it at with fresh eyes recently, I realised how much of what I built is just generic TiddlyWiki functionality. I now have a better understanding of what makes an independent TiddlyWiki plugin, and indeed I have a much greater respect for the whole modularity concept in TiddlyWiki, because it really satisfies all the traditional software engineering principles of encapsulation, orthogonality, etc etc in a neat bite-size way, which I will have to explain more about elsewhere. infoBox is the first of several plugins that need to be exorcised from the TiddlyGuv base.
  • The “macro body” thing is actually quite easy – you just call wikifier.subWikify(domEl, ">>"); where domEl is some DOM element. This will keep reading from the end of the macro definition to the next occurrence of >>, and fill domEl with the results of wikifying that content. It would be nice if there was a function to let you just get the string, instead of sticking it in the DOM, but you could still achieve that yourself by shoving it into a hidden element and capturing its innerHTML. The only complication for me was a common one these days of jumping between JQuery-land and traditional DOM-land, since TiddlyWiki now ships with JQuery, but much of the infrastructure has not (yet) been retrofitted to talk JQuery. Once I did a JQuery-to-DOM conversion, it worked fine.
  • Another interesting detour that happened here concerned cross-referencing tiddlers. Something you will want often want to do with infoBoxes is use icons. In a single-file TiddlyWiki, often intended to be offline, the icons would need to be data: URIs. This applies to other plugins too, so I dedcided to look into it a bit. I wanted a way for the “iconURL” slice in the InfoBox definition to either be a regular URL, or a reference to another tiddler containing the data: URI, so as to isolate it away. This led me to learn more about transclusion. With thanks to @FND, and after some experimentation, I found the easiest way was to support linking to sections of another tiddler. See how it works in the demo tiddlywiki, where an “icons” tiddler contains the data:URI icons.

Wow, I didn’t expect to say so much about what is not a major plugin. It comes at a time when I’ve been making some realisations about tiddlywiki, hence the verbeage.

Open Source Challenge: The Search Problem

I’m back from the BT Open Source in Business conference. Will blog it later, but one thing to highlight was a panel question on threats and challenges to open source. I’m experiencing one challenge right now, so I thought I’d highlight it since it wasn’t mentioned by the panel. Come to think of it, I didn’t hear it in the two days of Open Web Forum either.

The challenge is finding what you’re looking for.

For many problems developers face, there is software out there that solves it. It’s free for them to use, and they – being the enlightened developer they are, see “not invented here” as a good thing, because it means there is potentially a community behind it and a proven code base. BUT what if they can’t find this thing? For them, it may as well not exist.

This is more of a problem than we might imagine, because by now, a whole lot of open sourceage exists out there, and a modern developer ought to be assembling and configuring and tweaking, as much as they should be coding new stuff.

There are several reasons why search is a challenge: * InfoScatter – Open source is hosted in a zillion different places on the web, from one-off web pages ( to tight-knit focused communities ( to big communities ( to massive general-purpose repos ( * Articulation – It’s not always easy to articulate what you want in a simple Google search. I could imagine smarter search where you tell it the kind of code signatures or structures you expect to see, but I suspect we’re a long way off from the kind of AI required for that to work at scale. * Clutter – Too many results mean too much to search for. We need better ways to filter, for example excluding obsolete projects.

The Seven Wonders of the Modern World

  • jQuery – A library whose elegance vastly exceeds anything else on the market to the extent it has redefined the Javascript landscape, has excellent performance, came on the scene with superb documentation.

  • Firebug – The tool that increased my web programming productivity by a factor of 3.

  • Web Developer Toolbar – Before Firebug, there was Web Developer Toolbar, an excellent way to inspect the page state that is still useful at times.

  • The Ajax Design Pattern – Yes, JJG didn’t come up with anything new and people were doing it before, but this was a “right time, right place” case where the name immediately spawned a massive community of people sharing ideas and experiences.

  • Firefox – Browser + Detailed Settings + Tabs (credit Opera) + Plugin Mechanism = developer’s best friend.

  • View Source – The ability to look under the covers, responsible for rapid accumulation of knowledge in web apps.

  • StackOverflow – Increasingly becoming the no-brainer search result for programming problems and a vast improvement over the cruddy SEOtastic train wrecks that went before it.

SimpleMessagePlugin: Unobtrusive TiddlyWiki Status Message

To simplify TiddlyGuv message rendering, I made “SimpleMessagePlugin”. It removes the message box 1 second after a message was shown (using displayMessage). In the event another message appears in that time, it appends the message (as it normally does) and extends the message box’s lifetime by a second. In other words, it always closes a second after the last message was shown. The algorithm is a pretty similar throttling deal as Ajaxagram.

Demo here.

Latest version from SimpleMessagePlugin.

Open World Forum Notes

As mentioned in the previous post, I was at Open World Forum in Paris these past couple of days. Previous notes covered today’s FOSSBazaar workshop, here’s a veritable panaply of miscellany from the other sessions.

Opening Keynotes

Risk of balkanisation in communities govt (mil.forge) commercial (gcode) ?developer (eclipse)

Worldwide IT spend 3.48T 18% of apps abandoned 55% “challenged”

Poprietary software quality 20-30 defects/kloc Open source 1-2 defects/kloc

Redhat vp (tieman) “free” means the product “ceases to exist” … It’s all about services

OSOR – Exit costs as or more important. If the cost to enter is free, exit costs become very important. Vendor lockin=no exit.

James Besson – Whiter Open Source

Open source isn’t new. e.g. Steam Engines – exchanged detailed information about their engines and what kind of efficiency. Personal exchanges, visits, publications, industry/engineering institutions.

Great inventors – had great PR. [similar comments in recent BBC IOT podcast on Leibnitz vs Newton]. We have “hero” status partly because of proprietary conditions. [Also it’s human nature].

The great innovations were limited in locale and time. e.g. steam workers in Cornwall. Shows examples of industries lasting 10-30 years.

Applies to open source? – Consolidation

  • User-friendliness? not a great concern; 2/3 of demand doesn’t require UI

  • Coexistence (commercial software, patent trolls) Uneasy, but institutions are forming to deal with patents and they’re not fatal threats. Also, there’s a “proprietary burden”; MS one of the most sued companies.

… FLOSS will be sticking around


Sourceforge talks stats (by mahemoff)

ohlo stats - language choice in open source repos (by mahemoff)

ohlo stats - jquery vs prototype in open source repos (by mahemoff)

Recently acquired ohlo – massive open source study

Git 25% / Subversion 63% / CVS 7%

Growing language – “it galls me to call Javascript a language, but

Fastest growing: Javascript Python C*

Most popular: Java Python PHP

W. Europe 48%, EE 11%, speaker points out AUSTRALIA is vastly over-represented as a contributor.

Communities Session

This was three talks from people running open source communities – Apace, Eclipse, Linux Foundations.

Apache talk

Covers how to get involved in the community, even simple things like submit a bug are a good start.

Know who you talk to – don’t lecture Roy fielding on the http spec


Was never intended to be just an IDE, the IDE was supposed to be the killer app for the platform, which was supposed to be tools around java.

After launching eclipse foundation in 2004, completely independent, unlike jcp you can’t find any small prit about IBM vetoing, and membership is set so IBM can’t control the vote.

Eclipse guy: If you’re in a commercially led ecosystem a la Microsoft, your only exit strategy is they eat you or they kill you. In an open source ecosystem, you’re working with a trusted partner.

Eidystem for innovation needs: * Licensing model * Project model * Governance model * Tech architecture • open source organisations like Linu, apache, eclipse give you these out of the box…so it’s crazy when people start their own – end up paying legal fees etc. Wastage.

Eclipse projects hidhlighted – the browser, modelling, CDT.

Jim Zemlin – Linux foundation (the best speaker of the conference IMO)

Different to apache with its alkos all volunteer staff. Budget of around $3M and 16 full time staff. Monthly conf calls, f2f once a quarter.

“everyone want linus’s autograph. I count myself as the only autograph Linus wants, he wants it every 2 weeks”. CEO, Linux foundation.

Runs big legal defence projects – patent commons, linux911 …

Linux roadmap – growing in every segment from embedded to supercomputers. Becoming de faci standar and supporting cross frtilisation eg a mobile manufacturer opted for Linux, contributed coed to reduce battery usage which went into the kernel, and saved power and cooling costs for supercomputers.

The Tweets

  • Brazil olympic victory overshadowed by today’s #OpenWorldForum #owf victory for most open sourcey government. The games relegated to page 7. Fri Oct 02 22:31:48 +0000 2009
  • Fossbazaar Conference — OpenWorldForum, October 2, 2009 Fri Oct 02 21:48:56 +0000 2009
  • Enlightened self-interest – Wikipedia, the free encyclopedia Fri Oct 02 15:42:16 +0000 2009
  • Australia is vastly over-represented as an open-source contributor :)) W Europe biggest region at 48%. #ohloh #owf #openworldforum Fri Oct 02 15:11:28 +0000 2009
  • no prizes for guessing fastest growing language. “it galls me to call javascript a language, but …” #owf #openworldforum #ohloh Fri Oct 02 15:08:15 +0000 2009
  • SourceForge speaker talking Ohloh stats: GIT now has 25% of commits! SVN 63%. CVS 7%. #owf #openworldforum Fri Oct 02 15:05:46 +0000 2009
  • report of open source used as an internal battle – product open sourced in a failed attempt to become the corporate standard #owf Fri Oct 02 13:58:12 +0000 2009
  • captchas drop conversions by 7.3% (and presumably bug the remaining 92.7%) (tx @usa2day)#fowa #ux Fri Oct 02 13:55:07 +0000 2009
  • removing barriers to access is key to a successful open source project – e.g. encouraging localisers, docs, feedback. #owf #openworldforum Fri Oct 02 13:49:58 +0000 2009
  • RT @wadje12: Q: How about countries or contributors who are not allowed to contribute? Such as Cuba vs US, or a 14 year old contribute #owf Fri Oct 02 13:43:23 +0000 2009
  • open source “community management” does not exist because you never manage people, you commit to them #owf Fri Oct 02 13:37:36 +0000 2009
  • @SiriusCorp i confirm #owf wifi is definitely of dubious stability. Fri Oct 02 08:55:26 +0000 2009
  • Shuttleworth on #ubuntu UI: want buddhist medidation style of attention: focus in one place, but aware of surroundings (==”ambient”?) #OWF Fri Oct 02 08:53:19 +0000 2009

    Mark Shuttleworth interviewed (by mahemoff)
    (getting interviewed, not his keynote from which the above tweet came)

  • Ingres CEO: not that all customers contribute code, but for those who do, that’s a strong vote for the feature they’ve built Thu Oct 01 10:27:58 +0000 2009
  • “Is Oracle an open source company?” if there was ever a moment to “pull a kanye” … l:safe-distance-from-the-stage #OWF #OpenWorldForum Thu Oct 01 09:57:27 +0000 2009
  • cloud speaker talking up and #OWF Thu Oct 01 09:41:17 +0000 2009
  • Lots of “gratis” and “libre” from our keynote speaker. Appears to be talking about beer. #owf #OpenWorldForum Thu Oct 01 08:55:58 +0000 2009
  • Keynoter announces he’s still in Paris, admonishes previous speakers for English usage, and proceeds with keynote en francais #owf Thu Oct 01 08:48:51 +0000 2009
  • no big surprise, but government IT is the big deal in the businessy end of the open source community #OWF #OpenWorldForum #NoOfficialHashTag Wed Sep 30 22:42:47 +0000 2009
  • back from #OpenWorldForum mingling in the fine setting of l’hôtel de ville, paris (“town hall” just doesn’t…well it just doesn’t.) #OWF Wed Sep 30 22:38:24 +0000 2009

    IMG_0024 (by mahemoff)

(via List Of Tweets)

IMG_0106 (by mahemoff)


p>(We thought the entire audience was going to be invited on stage, but they stopped short at surnames beginning with “K-N”.)

FossBazaar at Open World Forum (OWF), Paris, Oct-2009

Quick notes from today’s FOSSBazaar event at Open World Forum in Paris. Usual caveats on typos etc as I was writing these notes live.

Martin Michlmayr overviews FOSSBazaar

IMG_0069 (by mahemoff)

Procurement – might like to avoid, but they have a really important role to play. With open source, it’s easy to ignore procurement – why go through procurement when you can download it! Great, that saved me several weeks of work, I can go home now :). But then you ship the code a few months later with GPL.

Make FOSS “business as usual” – how acquired, chosen, used, supported, updated, project tracked, licensed, mature.

FOSSBazaar to help people manage these issues. Good to have members outside of the US (Europe and maybe some now joining from Asia) as issues are different; this is the first FOSSBazaar meeting here.

FOSSBazaar for experienced users – working together to define standards and best practices

FOSSBazaar for inexperienced users – learning about open source issues, reducing the fear

Discuss, resolve, and document the “hard” issues related to adopting FOSS in the enterprise.

Question – why is HP spending money supporting the community? Martin explains HP has some experience dealing with compliance issues. Want to share experiences, e.g. we explained how we explored Palamida; other companies might do the same with Black Duck. It’s been a really good experience, talking to people who we might otherwise have not talked to.

Open Source Compliance – View of Validos. Martin von Willebreand (lawyer basedin Finland).

IMG_0070 (by mahemoff)

Vaidos has DB of 130+ packages validated.

“Shall we use this package or not?” Validos adds legal validation

Shows DB report showing licenses being used in a package.

Talking about typical guidelines, e.g. distributing the licenses with the code. Putting license on your website isn’t really redistributing it.

Showing how they prepared a document for republication. It’s big, it’s scary, and it’s very likely necessary for legal purposes. [Kind of sad that open source gets this complex – makes me feel like just releasing all my code as public domain; losing attribution would be a shame, but the real downside would be the risk of liability if there’s no warranty. It also makes me think that licenses should be embracing the web more, instead of forcing all this stuff to be repackaged – cut-and-pasting a URL would be a good start to what is a long path.]

Martin comments FOSSBazaar has been thinking about standardising some of this license declaration stuff; and there was a parallel effort recently which he’s hoping to work with.

IP Tracking Methodology at INRIA

IMG_0074 (by mahemoff)

On the forge: 1600 projects; 400 open source.

Done various tracking and would be good to compare methodologies, work together on tools.

Discussion session – Working on a Creative Commons contract to establish relationship with contractors, ensuring they use open source the right way.

TiddlyGuv Tool (the bit with me in it)

Steve Barnes motivated the tool from his perspective as a governance administrator. I then gave a demo and explanation.

IMG_0081 (by mahemoff)


  • Possible to get metadata e.g. to get XML feed. [Yes. Showed the bags/tiddlers model in the browser and how you can easily get a list of JSON or txt. Made the mistake of not demoing xml, which would have taken all of 5 seconds. Take off the pro-json blinkers next time!)

  • Can I see which packages (components) are using a paricular license?

  • What does a software policy look like? Do we need them at all. [Steve explained we probably wouldn’t want to have a blanket policy like “don’t use GPL”, though others could; the tool lets you declare whatever policy you like.]

  • It should be more about training than official policies. [I explained TiddlyGuv is intended to be a tool for developers and governance authorities to work together, share information, leave comments, etc.; not a rigid policing-the-masses weapon]


IMG_0089 (by mahemoff)

Disclaimer that you need more than just tools and need other tools too.

Looks at every single file in a package – fuzzy match against a library of >400 known license.

Walks through demo. Can try online.

Many new features – email notifications, new licenses, tutorial section, cleanups.

Plans – new license analyser (based on phrases); concept of license categorisastion (e.g. “good license”/”commercial license”)

Question regarding international labour organisation – what happens if a child developed code and contributed it? Do they have the legal authority to donate the code etc. Is there a child labour question? Martin notes international treaties as a similar issue – notes you’re not allowed to distribute code from one country to another; how do you accept contributions across that boundary. It’s a potential FUD issue people can use against open source in government.

Community Management and Project Governance: A checklist with an attitude Charles-H Schultz

IMG_0094 (by mahemoff)

Good “HowTo”/tutorial presentation of governance on a particular projct.

Not just about making code available. but make it easy to find – e.g. when it’s on sourceforge, but not linked easily from project page.

You never manage your community; so “community management” doesn’t exist. – fair, transparent rules and governance – soft power – trust employees like you trust other contributors