TiddlyCMS Plugins from TiddlyGuv

TiddlyGuv has a few features which other “TiddlyCMS” apps could benefit from. If that’s the case, we ought to extract them into separate plugins (though I’d be reluctant to do it prematurely, until we know there is some demand for it). The point of this post is to outline some of those generic features of TiddlyGuv which are, or could be, extracted into plugins. Developers interested in this functionality can then let me know and help me extract the components from the current code base, and mould them in a way that will be useful to others too.

Comments

This one already is a plugin used in several apps: nested comments.

Public/Protected/Private Model

For general document tiddlers, a user may mark them as private or public. In addition, admin users can mark them as protected. All of this is already provided by TiddlyWeb, as long as you create the right bags, i.e. a private bag for each user, a bag that admins can edit but others can only read, and a public bag. In TiddlyGuv, the private user bag is created upon login, if it doesn’t already exist, using a server-side plugin.

Anyway, yes – all of this is already provided in the server by TiddlyWeb. The challenge here was to provide a UI for it. In the case of licenses, it’s like a “promotion” moving it from private to public. You promote by clicking on a button. Further stages are foreseen (e.g. reviewed, accepted) when TidldyGuv becomes more workflow-oriented.

In the case of documents, it’s more like a simple status – is it private, protected, or public. You choose the status with a dropdown, as shown below in edit and view modes, respectively. It’s slightly different, and I could possibly harmonise the two styles of access.

Edit View:

Read View:

All of this could be encapsulated as a UI-side plugin, whch accepts as parameters the names of the protected and public bags.

Mediawiki-Style Sections

A common TiddlyWiki pattern is to aggregate tiddlers together in a new tiddler. This is easy to do with markup, using the tiddler macro:

  1. <<tiddler intro>>
  2. <<tiddler arguments>>
  3. <<tiddler conclusion>>

It’s also easy to do programmatically:

javascript
< view plain text >
  1. var sections = store.getTaggedTiddlers("section");
  2.   sections = sortByPriority(sections);
  3.   /* for each section ... */
  4.      /* append HTML element containing section.text */

TiddlyGuv manages licenses, and each license has a standard set of sections such as plain English explanation, usage policy, and official license text. This could be generalised into “a set of configurable sections for each type of aggregate tiddler”. For example, an aggregate “cv” (resume) tiddler type would have sections like “education”, “work history”, “achievements”.

TiddlyGuv does aggregation, but also extends the pattern to include edit links on each section, similar to what you see in MediaWiki instances like the big W and the slightly less big A. You have an edit link next to each section.

Another feature, not shown here, is collapsing large bodies of text. If the text is over, say 1024 characters, it simply shows the first few words along with a word count and a toggle control to show and hide the entire text.

A further feature is the “policy” section. Only admins may set this section, and it is rendered with a warning to that effect. Again, this is generalisable; for some types of content, some of the content may be world-editable while others may only be edited by admins. This feature ties the sectioning aspect to the previously discussed aspect related to permissioning. Again, it could be encapsulated in a sectioning plugin.

GUIDs

Traditionally, tiddlers are identified by their title. e.g. you can programmatically grab a tiddler by saying var stylesheet = store.getTiddler("StyleSheet");. This is like having a database table whose primary key is a meaningful field, such as a name. There are many debates over this kind of key in database design, and most modern schema designs use a dedicated ID field instead. That’s what we need with TiddlyCMS apps, for two reasons: (a) tiddlers link to each other (such as aggregate tiddlers mentioned earlier); things would therefore get complicated if a tiddler was renamed, and that name was the primary key; (b) we sometimes want two tiddlers whose title is the same thing. For instance, in a single TiddlyDocs instance, we envision a “soup” of shared tiddlers which can be pulled into a document; a TiddlyDocs document is really just a hierarchical orchestration of existing tiddlers. For instance, a common “Disclaimer” section might appear in all documents. The problem arises when the user wants a deviation of the disclaimer’s contents; the title must appear the same, but the tiddler will be different. This is not possible when titles are used as primary keys; hence the need for a separate ID.

This will be achieved using the Guid0 library I built earlier. A TiddlyWiki plugin will be built, incorporating Guid0, intercepting certain calls like the call to create a new tiddler. It will store the user-meaningful title as a custom tiddler field, and make the tiddler’s title value become a newly-minted GUID. It will also intercept tiddler display mechanisms to show the custom tiddler field instead of the GUID, so that users will be none the wiser. I may also include a second “detailed title” field so that users can write a description of the tiddler, which won’t be rendered, but would be used to help them distinguish two tiddlers of the same title – when the tiddlers appear in a dropdown, for instance.

TiddlyGuv-FOSSology Collaboration

I will round out the FOSSBazaar workshop series with some comments on collaboration between TiddlyGuv and FOSSology.

First, what’s FOSSology? Simply put, FOSSology in its current state is a code/license scanner. You upload content, like a RedHat ISO, and it recursively explodes it into individual files, also recognising by way of checksums when a file is duplicated (across all the RPMs in a RedHat distro, you will probably see some files appear thousands of times). It then walks through each unique file/archive and tries to determine its license. At the end, you get a report telling you, for example, that the ISO contains 2120 GPL 3 licensed packages, 175 MIT licensed packages, and so on.

The underlying philosophy is to automate the license discovery process. Let developers get on with their job pulling in what they need to, in accordance with their companies policies, and then use the tool to audit what’s there. This is in preference to requiring developers to manually declare all that info – and even if they did declare that info, it provides a suitable audit mechanism to check against their report.

TiddlyGuv will eventually keep track of projects and as much as possible, we’d like to automate the process of tracking those projects’ components and licenses. Therefore, we’d like to call on FOSSology to capture this info. As a completely orthogonal side benefit, it would be a nice proof-of-concept if two open source projects, each primarily motivated by the needs of the developers’ own organisation, could talk to each other.

I was fortunate to meet Mark Donahoe from the FOSSology team at the workshop and discuss how we might get the tools talking to each other, along with my colleague Andrew Back. The first thing to say is we all want to just get some simple proof-of-concept integration happening. This is the agile process at work – a proof-of-concept would help us understand whether it’s worth proceeding and may inspire interest from others, as well as project sponsors (at least in the case of TiddlyGuv).

We see a proof-of-concept working like this. TiddlyGuv user creates a new project; server kicks off a FOSSology code scan and updates TiddlyGuv project record with a link to the code scan report. It’s as simple as that; in the future, there might be an email notification when the code scan is complete, and other niceties, but for now, this simple story is enough. All of this would take place in the same network by the way; both TiddlyGuv and FOSSology are tools intended to be run by a company inside their own estate (rather than in “the cloud”). If we implement this integration scenario, we will probably run TiddlyGuv and FOSSology on the same box. TiddlyGuv requires Python; FOSSology requires PHP and MySQL. Not the same, but both easily installable on a run-of-the-mill Linux server.

To get this proof-of-concept working, we each have some steps to take.

Osmosoft will need to introduce a project data model, using the TiddlyWeb data structure, and the UI to maintain it. This is the biggest step for us, because we haven’t really started that yet. We will then need to introduce a TiddlyWeb plugin that kicks off FOSSology when a new code repository is specified, and to store the resulting scan ID.

On FOSSology’s side, we need two things. First, FOSSology would need to allow “upload” of a URL rather than a physical file. i.e. you could just give it the URL of an archive or a SVN URL pointing to the project root within Subversion; and FOSSology would then pull the content from there and treat it as if you had uploaded it. Second, FOSSology’s command line mode would need to return the scan ID. I believe this is generated at kick-off time, so it could be returned even though the report isn’t yet present. From the scan ID, TiddlyWeb can calculate the scan report URL, which it can store and display in the project’s TiddlyGuv page.

Since the project functionality hasn’t even begun yet, all this will take a while even to build a rudimentary proof-of-concept, but from last week’s meeting, both parties are keen to make it happen.

TiddlyGuv

I’ve just presented TiddlyGuv at the FOSSBazaar workshop, the event I wrote about yesterday. It’s a good time to introduce TiddlyGuv – I’ve mentioned it in some talks and tweeted about it on occasion, but this is its maiden appearance on Software As She’s Developed.

TiddlyGuv (that’s a code name we’ll use for a while) is a web app to support open source software governance. In promoting and supporting open source at Osmosoft, one of the activities is a governance process. This means establishing policies for open source usage and auditing projects against it. TiddlyGuv is the tool we’re developing to support this process – it will support community discussions, policy publication, project tracking, workflow, and alerts about potential problems.

A few things about TiddlyGuv:

  • TiddlyGuv is in progress, it’s not complete and still needs work around a clean installation package. (We’re currently re-working how all this is done with TiddlyWeb verticals; once that’s done, there will be a package so people can try it out.) In any event, the code in some form is online in Trac.
  • TiddlyGuv is a web app. No surprise there :).
  • TiddlyGuv motivated the comments plugin, which has now been used in various other projects, including the live TiddlyWeb documentation wiki.
  • TiddlyGuv is built on the TiddlyWiki-TiddlyWeb stack, like TiddlyDocs. We see a general trend of CMS apps emerging on this stack, which has been called TiddlyCMS. (TiddlyCMS is an architectural pattern.) The aim is that TiddlyCMS apps share a large suite of common components such as the comments plugin.
  • TiddlyGuv is open source. The license hasn’t been finalised, but will likely be BSD, same as TiddlyWiki. We hope other organisations use TiddlyGuv in ther own organisations; TiddlyWiki and TiddlyWeb offer highly flexible architectures which lend themselves to adaptability.

Functionally, the first thing we’re aiming for is a tool with document wiki, for general discussion about open source usage, and a license portal, showing a list of licenses and information about each one. The comments plugin ensures each document and each license can be commented on too. All of this is there today, albeit in an untested state. Later on, we will add info about projects and further still, a workflow tool – for example, “Once the project form has been submitted, it must be reviewed by two named reviewers. They will each receive an email notification when a project is ready for review.” We would want the system to follow rules like that, and we would want to allow site administrators to maintain such rules. The TiddlyWiki model is to encapsulate rules like that inside tiddlers. I foresee some kind of Javascript-powered Domain-Specific Language (DSL).

The FOSSBazaar slide pack:

The feedback from the workshop was positive; there’s interest in the project and there was also interest from one vendor of a commercial product about ways to introduce the wiki or commenting aspects into an existing product. A good example of this sort of thing is Amazon – the way it has a structured product page, which incorporates a forum and a wiki. It raises some interesting architectural issues about how a TiddlyWiki/TiddlyWeb app could be embedded onto a page as a widget-style component, with identity and permissioning handled too. I also had an extended discussion with Mark Donohoe about how TiddlyGuv could talk to FOSSology. Of which, more in the next post.

FOSSBazaar Workshop

I’ve been at the FOSSBazaar workshop in SF for the past couple of days – an afternoon session and a session this morning. It’s part of the Linux Summit, FOSSBazaar being a workgroup of the Linux Foundation (though it’s not Linux-specific). (It was news to me that Good Friday isn’t an official public holiday in the US.) FOSSBazaar is an industry consortium focusing on open source usage in the enterprise, in particular open source governance. I attended because I have been developing a tool at Osmosoft to support enterprise open source governance, codenamed TiddlyGuv. I’ll say more about TiddlyGuv in the next post, but here are some notes I recorded at the session.

The notes here are mainly from yesterday’s session – brainstorming about where FOSSBazaar should be heading and what activities to perform. Today’s session was several presentations, one being mine on TiddlyGuv. For the record, the talks were: Krugle, Osmosoft overview (from my colleaugue Andrew Back), TiddlyGuv (me), INRIA, Qualypso, and Fossology (another open source project in this space which we’ve talked to previously and will hopefully collaborate with in the future; more in a later post).

These are iphone notes; spelling and grammar caution!

FossBazaar – Open source governance community – managing open source within an organisation – Share best practices …

started in jan 2008.

State of FOSSBazaar website: Blogs are healthy, forums need attention – would be good to have some discussion amongst ourselves. Whitepapers could be added to.

Qualypso project overview: an organisation encouraging open source adoption esp in Europe. Aiming to set up 6 competence centres -initially some funded by European commission and eventually private or regional funded.

mention of EUPL – recent European license – some unusual features – translated into multiple languages and gpl-compatible

Discussion about what kind of people are targeted by FOSSBazaar – important to think about developers not just managers/lawyers/etc because (a) they can often be an entry point into other parts of the organisation; (b) they often have policies forced on them so regardless of their view, good to target them too to support and promote open source usage. eg should I be using GPL?

Discussion about projects fossbazaar partner could collaborate on to add value and raise profile. The value compared to other organizations would be to offer expertise and experiences from the real world of organisations/companies. Most of the session involved brainstorming those projects. There was a pragmatic focus on packaging the info up neatly as handbooks, “top 10″ lists etc, and thinking about SEO. The notes from this brainstorming session were written up during the meeting and I can reproduce them below (thanks to Ragavan Srinivasan who wrote them up during the meeting and Martin Michlmayr for making them available).


Potential ideas:

  • Common patch contributor agreement – SFLC may be working on this. SFLC and LF are working on this
  • Authoritative source of license information: scan code; there are legal issues
  • Standardized way to communicate license to users (README? INSTALL.TXT?LICENSE.TXT? XML?) Document/Guidelines?
    • SFLC published a legal primer that could serve as a starting point. (Need link)
    • Maybe a “HOWTO” or cookbook could be added to primer
    • Get projects on board who will adopt the process
    • Potential projects to work with as pilot: Apache, Eclipse (may already be doing this?), KDE
    • Should this expand beyond just license text standardization to IPR policy standardization along the lines of what Eclipse seems to be doing?
    • Publish Wall of Fame with a badge.
  • Open source governance best practices guide book
    • Benefit if done by a neutral organization
    • Create editorial structure, then get content from each of the partners
    • e.g. creating an OSRB, what to include in a policy
    • FOSS Governance Fundamentals document is quite updated; this one could be more detail
    • Ask Andrew/Olliance for content
    • Create outline, then ask partners to contribute
  • General guide on the licenses, how to interpret them
    • Three books (Rosen, St. Laurent, Van Lindberg) – perhaps we should link to them on FB?
    • Icons to represent licenses ?
  • Survey data?
    • Governance focused (do we have an analyst company as a partner?)
    • Ask TonyW if a student who could do this
    • Start by making a survey of LF members; but that won’t include end-users
    • Make it repeatable; do it every quarter or year
    • Look at trends
    • Ask multiple people in one company and compare what they say (production, developer, manager etc view); show gaps in knowledge
  • Top 10 list of considerations (vulnerabilities, distributing software that contains GPL, license/legal papers; etc.); make sure solutions are included, not just problems or risks
  • Interviews with companies that have mature FOSS governance processes – HP, others? QualiPSo member
  • Common problems with popular FOSS libraries (tomcat, gSOAP etc.)?
    • Work with the projects to get answers (Ask FB)?
    • e.g. Project may have no clear license, work with the developer and remove ambiguity.
    • Magnifying the fine print on some projects/licenses.
  • World’s funniest license interpretations by developers/businesses (funny in a scary way)
  • What to do when you are undergoing some major business changes? (Mergers/acquisitions, divesting part of your company, etc.)
  • FOSS licensing for Dummies?
  • Read this before you do anything else list?
    • SFLC primer?
    • Books from above?
  • FTF (Freedom Task Force) has a lot of documents that are in draft stage, but perhaps we could host them on FB?
  • Grab fossbazaar on identica/twitter before someone squats!
  • Content syndication – cio.com, etc.
  • Outreach to India, China, Eastern Europe, etc.?
  • L10N of content?
    • How do we get the community to help with this?
    • Hugely important.
  • Hosted version of FOSSology? Legal issue? Competing technology?
    • Concern: You may end up with a comment board, all prefixed with IANAL. Problems, but no fixes
    • Why could we not adopt a Coverity/security remediation style approach?
    • Would BD/Palamida/Others be interested in doing this as part of FB (give open source projects access to their Tools behind closed doors as a way for the projects to earn the badge) – will need to address confidentiality, record keeping. We also need more lawyers to donate time to SFLC.
  • A policy builder that OpenLogic has built?
  • Some kind of tooling on FB (like an iPhone app people want to share, just as an example)
  • Start with a small set of core FOSS projects, see if we can get them to standardize on how they represent licenses, give them badges and use them as examples to go after more FOSS projects. This allows us to fix the license ambiguity problem at the root instead of every downstream user having to resolve this.
  • Perhaps we can have projects publish testimonials, maybe start with new projects or projects that don’t have a lot of adoption yet.
  • What are some non-license/legal issues we want to work on?
    • Have a better reputation and build more credibility.
  • Case studies gets another vote. Be generic so you don’t have to name names.
  • Boeing, Lockheed, BAE, BT, all have talked about their open source governance best practices in public. AFIS conference (DoD conf. On FOSS). We have presentations.
  • Forums: what are the strangest place you’ve found licenses in?
  • Forums: OSRB: to allow companies to have comparisons of OSRB

User categories:

  • My company won’t let me use FOSS – HELP!!!
  • My manager thinks we don’t use FOSS – how do I break the news?
  • What is this FOSS thing I keep hearing about?
  • I’ve been asked to create an OSRB/FOSS Center of Excellence for my company – what do I do?

Marketing * Top 10 theme related to governance * SEO * Social networking * Reach out to journalists * Webinar series (lead generation) * Press releases * Get more in-bound links: e.g. syndicating content * Create more partnerships & get links: e.g. with QualiPSo * Newsletter * Twitter: e.g. most active forum topic this week

Next steps:

  • Add “best practices” on FOSSBazaar
  • Create outline for Governance best practices document
  • Define use scenarios (e.g. “new to open source”) and show content depending on what they need
  • Top 10 lists
  • Interviews

Meeting with Aptana’s Kevin Hakman – Talking Jaxer

Regular readers of this blog will know I am a big fan of Javascript on the server. Through its Jaxer project, Aptana has been at the forefront of the server-side Javascript renaissance. As I’m in SF right now (mail or tweet me if you’d like to meet up), I was pleased today to be able to catch up with Kevin Hakman, a great talent in the Ajax world who runs community relations and evangelism for Aptana.

I wanted to get an overview of the state of Aptana and Jaxer, and also understand how it might be used in a TiddlyWiki context to help with accessibility.

We discussed a range of things Jaxer – I will list them below. Kevin reviewed a draft of these points and gave me some links and corrections.

  • Jaxer – as you probably know, Jaxer is an Ajax engine. It’s server-side Javascript, but more than that, you can perform DOM manipulation on the server, and then it spits out HTML from there. Very exciting for all the rasons I’ve mentioned in previous blog posts – using a single language, making proxied cals from client to server, etc. The great thing about Jaxer is it’s open source – you can download a version with Apache that “just works” – download, install, and start the server in one click and the sample apps are up and running. Aptana runs a cloud host for Jaxer (as well as Rails/PHP etc), so you can host with them if you don’t want to host it yourself. They have some interesting value-adds around their host, like support for different environments (dev/test/etc with different parameters, such as extra debugging), which you can easily spin out and turn off again.
  • ActiveJS – ActiveJS is a Jaxer-powered framework modelled on Rails and close to beta. The most mature part is ActiveRecordJS (ORM), but there is now ActiveView and ActiveController as well. As with Rails, you can use them independently. This is exactly what I’ve been hoping for and it sounds like it’s mature enough to start hacking against. Kevin is realistic about uptake and expects it to be a long process.
  • Microformats – Kevin directed me to this this Jaxer project which filters parts of a form depending on who is viewing it. You just use a different class name to indicate who can view what, and the server chooses which parts to output. This got me thinking about microformats and how Jaxer could help render them – with a single library, dynamically in the browser and statically from the server. There are a number of microformat libraries which might already be useful sitting in a Jaxer server.
  • TiddlyWiki – Taking TiddlyWiki based apps and using Jaxer to make them come statically from the server. Since Jaxer supports DOM manipulation, it should certainly be possible. Kevin said accessibility is certainly an argument for Jaxer, though there’s no significant demo to date, so a TiddlyWiki project would make a good demo. His intuition is that it will be a 90-10 – 90% straightforward, the remaining 10 will cause some complications. Would definitely make an interesting project to do a simple version, e.g. initially just get Jaxer to output each Tiddler, showing title and wiki markup.
  • Demo app – Aptana TV is implemented in Jaxer+ActiveJS and there are plans to open source the implementation and make it Jaxer’s “Pet Store”.
  • Interoperating – Jaxer can interoperate with other languages using browser extensions/plugins, since the server is basically Firefox; these are not extensions users have to install, but act as enhancements to the server. Just as a user can install a plugin to Firefox so web apps can make calls to Java for example, a developer can patch the server with the same plugin (or similar, but with the same API anyway?). Kevin said most of these kinds of components are XPCOM, but I guess Jaxer should accept regular Firefox extensions and even Greasemonkey scripts. DotNet COM example. In a simpler sense, you can call out to other languages using the OS command line. System commands have always been possible from the Jaxer server.
  • FF bugs – Jaxer shares the Firefox’s bug set and fixes, when you think about it!
  • Performance – Likewise, Jaxer benefits from all the stuff you read about sky-rocketing Javascript performance in the browser. Jaxer will be inheriting the performance boost from TraceMonkey promising to be silly-fast at 20x to 40x speed improvements for JS according to Moz.