Software As She’s Developed

I have the honour of delivering a keynote at the Tools Europe conference in Zurich next week, as well as at the “Mining Web 2.0 Patterns” workshop, which, as you can tell from its title, I’m looking forward to participating in. The workshop chairs are Dragos Manolescu and Joe Yoder.

The keynote is on OpenSocial, Gadgets, and Design Patterns; and I’ll have more to say about it later. I believe it will be recorded too. For now, I’m using my blog to post my response for the pre-workshop preparation.

Sites and materials relevant to my presentation

• What is OpenSocial?
• iGoogle
• Ning OpenSocial gadgets directory
• BBC weather gadget
• TODO gadget

Questions:

• Which features make the gadgets easy or hard to use, and how might the gadgets be improved?
• At UI level, which best practice patterns are apparent?
• What changes to the platform would improve user experience?

Sites and materials relevant to other workshop tracks

Site Navigation:

• SkyScanner Kayak Not sure if it qualifies as navigation, but the live filtering mechanisms are well designed.
• DZone The search results are an example of Virtual Workspace, aka infinite scrolling as opposed to pagination.
• Delicious The quintessential demo of tags and folksonomy.
• Mainly for the way it re-thought mail and how it’s used. Early example of navigation via Ajax calls; “star this”.

Services:

• OAuth Very important for service design in the future.
• Heroku If we’re including cloud hosting …
• Twitter Good example of an easy-to-use REST-ish API that’s led to a proliferation of apps.

I just realised there’s no good executive summary for OpenSocial. Every resource fluffs about the “social” aspect and forgets about the UI aspect. The UI aspect, essentially a gadget specification, is just as important as the social aspect, notwithstanding its absence from the “OpenSocial” brand name. I’m not a fan of the name “OpenSocial” for this reason, though I acknowledge it does have more attention-grabbing bling than “open social widget framework 1.0″. In any event, it’s regrettable that most commentators gobble it up hook, line, and sinker; as evidenced by their almost exclusive focus on the social angle.

With no further ado, here’s my attempt at a one-paragraph summary of OpenSocial …

OpenSocial Executive Summary

OpenSocial is a standard for “mini” web apps, called “gadgets”, which are designed to be embedded other websites. The standard informs developers who wish to write portable gadgets, i.e. those which work in any OpenSocial environment; and likewise, it informs developers who are creating such environments. It defines an XML format for applications, typically containing the user interface (HTML/CSS/Javascript) as well as metadata, e.g. author, gadget size, and preferences. In addition, OpenSocial defines a number of APIs, so that gadgets can invoke services such as changing the gadget height and fetching remote content. One particularly important class of APIs are those of a social nature. These enable a gadget to access data and run services related to the user who is viewing the gadget, or the user whose web page the gadget is embedded in. For example, the gadget can - subject to security settings - find a person’s name and location, traverse their social network, and invite them to install an application. Access to social data is not limited to gadgets, however, as OpenSocial also provides a RESTful variant of the social API, suitable for any kind of net-enabled client. OpenSocial was launched by Google in November, 2007, and is to be managed by the non-profit OpenSocial Foundation.

I’m a redundancy fanboy. In visualisation, different formats suit different personalities and different tasks. With version control, the usual format is just a text log. This is good if you’re scanning for specific terms, but pretty ordinary for other activities - e.g. to get a feel for general trends that have arisen, the pace of change, or the rise and fall of specific contributors.

It’s encouraging, then, to see demos like the following, which shows the evolution of the Python language project (via Dion’s tweet).

code_swarm - Python from Michael Ogawa on Vimeo.

It reminds me of one of the first screencasts by Jon Udell, a fascinating walkthrough of the evolution of a wikipedia page over a year or so. The page he chose for this demo is as memorable as the message of the video itself.

These visualisations are cool as tasters for what might be, but they are “here’s one we made earlier”. Where are the tools to automate all this? I have no doubt such tools have been created in academic research projects, but let’s see them in action. I’d love to see the source code hosts - sourceforge, google code, github, et al - integrate this technology to produce visualisations on the fly.

Firefox 3.0 ticker tape parade

Today is Firefox 3.0 landing day. Maybe tomorrow as the servers have been down for many hours now. Funny how Twitter is up, but Mozilla is down…it feels like today is the Juneth of 18. Anyway, this is great news as you could wile away a few lazy hours tweeting and plurking about the irony and how you’re chomping at the bit for FF 3.0, and watching other people say the same thing but with different yet equally charming emoticons. But anyway, FF 3.0 is pretty much here. Yay!

Firefox 3.0 is faster, more stable, and standards-based, and that’s no mean technical feat, but the upgrade is not so obvious from a consumer’s perspective. And my own ADD mindset craves more visible changes that will impact on usability and utility.

The modern browser is the application billions of users spend the most time in, many hours a day, and the trend is set to continue as desktop migrates to web. And Firefox is used by 18% of users (Net Applications via wikipedia, Q2 2008) - that’s 18% of some billions of users. A very big number indeed, and even more remarkable in certain markets, where Firefox dominates share, and even in the entire population of certain countries, where it appears to be neck-and-neck with IE. Furthermore, as a major player, the features of Firefox are clearly going to influence those in competing browsers. What’s in Firefox matters a lot. So, by deludedly illogical inference, the present article must also mean a lot to the future of our green planet. Here I will outline UI changes in FF3, then present my wishlist for FF4.

But first, what’s in 3.0 then?

This is a wishlist for the next version of Firefox, FF 4.0. But first, let’s review Firefox 3.0, ignoring those important but not too tasty “-ility” updates, and instead focusing on what users will actually see. I had to think about this. After several months of using both versions alongside each other (on different boxes), I really don’t notice much difference at all. as noted by wikipedia, are pretty trivial. (I noticed a better list here, though there are no other major features mentioned.)

• More native look and feel. Nice, but a small change.
• Redesigned download manager, can search for downloads Same.
• Redesigned add-on managerIt’s integrated with the main distro site This is by far the most important new feature. I’ll elaborate later, but FF is all about plugins and the plugin process has been pretty silly up to now. In the book, Art and Science of Javascript, my chapter on Firebug included a section on installing the Firebug extension. And I can tell you, looking at it step by step, you can see how complicated the whole thing is.
• Microformats are supported. Cool, we like microformats and it leads the way to the so-called “Web 3.0″!
• Introduced Places for bookmarks. Yes, this is the most obvious new feature, but the thing is, years of rudimentary bookmarking support, combined with Google’s almightiness and Delicious et al, has weaned me, and I suspect many others, off bookmarking in general. I don’t know, how many people are really going to tag their bookmarks? What I do like is the new smarter address bar with history auto-completion and better display. It has leap-frogged IE here, which has searched by title for some time. Handy little feature.
• OSX version supports Growl, spell check, and Aqua. Well, Growl is nice and it’s available to Add-on developers…but it should also be available to all websites, perhaps using a whitelist setup. (Similar to the Add-On trust system now - a notification panel that says “this application wishes to notify you of critical events via Growl. Do you agree?”.)
• Default icons change Cool but fairly basic again.

These are all changes you might expect in a point release - it would be hard to justify going 3.0 in the absence of all the other improvements to stability, etc. That’s why I’m not trying to take anything away from the overall FF3 effort … there’s a lot of changes outside the UI. But when you look at these UI changes, they’re very basic. I’m hoping FF4 is more focused on seriously evolving the browser, and that means taking some chances and making some bold improvements to the shiny Fox machine.

And now … The Firefox 4.0 Wishlist!

Shipping with Add-Ons

As I’ve said before, Firefox needs to ship with a proper suite of pre-installed extensions. The “thin kernel model with optional extensions” is architecturally sound, and of course many applications now incorporate a plugin style architecture. The highly successful Linux and Apache web server projects are based on this model too. However, Mozilla misses a trick by shipping without key extensions. The attitude seems to be “if users want it, they can ^@!#$ install it themselves! The Mozilla people are in general extremely concerned with usability and the human factor, but in this case, the attitude reeks of Comic Book Guy geekery. Leaving out basic tab functionality, for example, yields unecessary advantages to Opera and Safari. Yes, it’s available in TabMix if you care to download it, but if it’s that important, then ship FF with TabMix!!!

There should be a standard distro with 5 or 6 popular plugins, all chosen for utility, simplicity, security, and compatibility with each other. This should be the one you get when you click the big, fat, super-easy-to-use, Download button (which I have always admired and has since been copied by many other websites). A bit deeper in the recesses of the Firefox distro site, you could also download the minimal edition as well as specialised editions. Alternatively, you might select your distro the first time you run a new Firefox profile. I could imagine a wizard that (a) asks you which distro (mapped to a set of add-ons), and then (b) lets you customise the list. This is much better than the current situation for creating a new profile, where you have to go and (re-)add all your typical add-ons manually, then restart.

It would be cool to see specialised editions for developers, students of different ages, gamers.

Apart from usability, this would be a boon for security, which is arguably Mozilla’s primary concern and greatest source of pride when comparing Firefox to IE. Relieving most users from the need to install add-ons reduces the chance of a screw-up involving installation of a malicious plugin, or some unanticipated interaction between plugins that creates a hole. The default stack would be designed and tested for security, and any alerts involving an add-on in that stack will be highly visible.

Make it easier, much easier, to write Add-Ons

This was supposed to be an enhancement for FF3 iirc, but may have been dropped (?). Anyway, writing extensions in Firefox is a complicated business. You have to know XUL and deal with the whole packaging structure. It should be just as simple as writing Greasemonkey scripts IMO. I’d like to be able to say something like $(”addressBar”).style.backgroundColor = “red” for example. These things could be made possible with enhancements to Greasemonkey, but of course Greasemonkey isn’t built in. What I’m looking for here must work with the raw version of Firefox. This will lead to a proliferation of cool new add-ons. It will also increase the risk of security violations as there will be more plugins around and it will be harder for Mozilla to vet them all, but relying on a difficult API to slow down add-on … that’s not the way to ensure security.

Installation of Add-Ons, Themes, Search

Installing search modules should work the same way as the new way to install add-ons and themes, ie all inside the client instead of bumping you to a website. Installing themes is half integrated but still requires you to go to a website. Even Add-Ons requires you to go to a website to explore fully. I think this should be unnecessary.

There is also confusion of terminology, with references to both “Add-Ons” and “Extensions”. On the official add-on website, “Search” and “Themes” are combined with 10 other category of Add-On (Appearance, Bookmarks, etc.), whereas they are in fact special types of add-on. So in some cases, they are kept distinct, and in other cases, they are lumped together.

Improve Profile Management

I like the idea of profiles - as a developer, it would be nice to switch between a developer profile and a user profile. And also between multiple dummy user profiles while testing. But switching profiles requires a restart. Not only a restart, but you have to run Firefox with the “-p” flag, and the installer doesn’t come with a launcher for profile-switch mode. For that reason, I bet most users probably don’t even know this feature exists.

The simplest improvement would be for the installer to add a launcher for profile-switch mode to the Start menu (in windows at least), just like there is one for Safe Mode. Moreover, just integrate profile switching into the actual interface…just include a dropdown in the Advanced Preferences or add a new menu item somewhere which expands to show each profile ID. Easy! True, you’d still need a FF reboot. I don’t see that constraint changing anytime soon …

Tab-Mania! Sugoi yo ^o^

I wrote this up in Taking Browser Tabs Seriously. There’s a ton that could and should be done to ease the multiple tab experience - notifying, searching, grouping, sorting, etc etc etc. This is HUGE! ‘Nuff said, see the link for details.

Retain all viewed content (or at least index it for searching)

Come on, you know you want it! Instead of bookmarks, it’s time to finally bite the bullet and let users search through every page they’ve visited (if not forever, then at least for the last X days). How many times do you have to search for something on Google that you’ve already seen before. The browser could take care of it. Just as Google search slams Yahoo taxonomies, so too does full-page history search obliterates bookmarking. I don’t know how much capacity you’d need to do it, but I bet if you asked in 1994, “will we be able to do that in 2010, with 16 years = 1000x (ie 2^(16/1.5)) improvement in hardware stuff”, you would answer a resounding yes. With the right optimisations, I’m fairly certain this is possible. BTW I realise you can effectively achieve all this now with Google, but there are a lot of reasons why it should be built into the browser (intranet and hidden content; privacy and corporate security concerns; possibility of offline browsing old content).

Improve history

With or without the previous enhancement, Firefox history has always been a bit wanting. For starters, search needs improvement - you should be able to search history the same way you search through a page. ie instead of filtering, it should just highlight matches and scroll to them. This way, you could see which other pages you were looking at around the same time. Sometimes, I’m searching for one term, but only because I know I was looking at another page at the same time, and I can’t locate the other page via search. (Although I could do so if the previous enhancement was available!)

Search should also work without having to hit Enter - it should be updated on each keystroke just like page search.

Furthermore, it’s excruciating trying to work out what time and date you viewed something. You get the “View by Date” which sorts by date, but doesn’t let you search in this mode. And in any event, it still doesn’t show you time of day. And if you view in other ways (by site or last/most visited), you don’t get any time or date at all.

That Sidebar

I never get why there’s only one sidebar when there are several possible contents (bookmarks, history, add-on-specific features). And it’s also odd that you can only control that sidebar from the menu rather than the sidebar itself. You should be able to have multiple sidebars, or at least one accordian style sidebar containing all possible contents. Similar to the All-In-One Sidebar add-on.

I could imagine Firefox achieving flexible UI with an elastic docked window style UI like Eclipse (or indeed Firebug in some respects), but I think that would be too complicated for mainstream users.

Render Non-HTML Content

I realise a browser’s fundamental job is to show web pages and it’s rarely good when applications over-step their boundary, but in this case, I feel there is a case for putting in at least some effort on non-core activities. I’m talking about how Firefox (and every other mainstream browser) deals with non-HTML content. For example, this came up on Twitter today, where my colleague mentioned the problems we have viewing JSON in the browser. It requires a download basically and then manually opening and viewing in your editor, and that’s not very satisfactory. Even with XML, which is rendered by FF, how about some intelligent interpretation. If it’s RSS or Atom, for example, provide a suitable default stylesheet explaining what it’s good for and how to subscribe.

Better File System Navigation

When you visit a local directory using file://path-to-directory, what you are greeted with is an interface that hasn’t changed since the mid ’90s. Arguably, it’s doing enough already, so why complicate it? But I think that exploring the filesystem is a rather useful feature for the browser, given its sovereign posture which means it could be used as the primary control centre for managing your local file system.

It’s also impossible to visit the root of your filesystem using file:/// on certain OS’s, at least on my Mac. This is obviously fundamental behaviour that’s missing, and the bug was logged in 1999! On my Mac, I also can’t open up /Volumes/Macintosh HD due to the space. When I click on it from /Volumes, it changes to file:///Volumes/Macintosh%20HD and that URL breaks. Doh!

Just like Explorer or Finder, files should be shown in various possible formats - tabular (with sorting), thumbnail, etc. And it should use thumbnails to show file types.

Better Keyboard Binding Support

I like those programs like IntelliJ Idea which let you control keyboard bindings for pretty much anything, and also let you save and load configurations. Firefox should do the same. A lot of the shortcuts are hidden anyway and should be available somewhere in the interface, e.g. a keyboard preferences area.

Mouse Gestures

This was one of the awesome things about Opera when I first started using it and it made browsing fun and more subconscious, one step closer to the haptically brilliant Minority Report hand control scenario. I used it all the time, but on Firefox, the main add-on didn’t work so well and I never got into it. Gestures should be built in so they are available to all users and available to all add-on developers. It would be even better if they were available to web apps as well, using some kind of whitelist if necessary (as I explained earlier wrt Growl).

Comment on the Coop

Incidentally, there is also The Coop. This was intended to ship with FF3, but dropped at some point. It’s an attempt to embed social networking and stumbleupon-type functionality. To me, this is exactly the sort of thing that really is too far removed and should remain an optional add-on. It’s actually tied to a single server run by Mozilla, which doesn’t seem right for an open-source product. It might make more sense if it was generic platform/API, with the communication protocol published and the server software also open sourced.

Fin

Firefox is by far the best browser today, already has some great UI improvements from 2.0 days, and I am pleased that v3.0 is now more stable and performant than ever. Now I’m looking forward to usability improvements and looking forward to seeing other people’s wishlists!

I’ve been thinking about what language and framework to use for my next hobby projects. There has always been a frustrating trade-off between language quality and ease of hosting. The languages I know best are: Ruby (+Rails), PHP, Java, and Javascript. So these will be my options.

What I’m looking for is a hosting environment which will scale well as I add new sites, each with their own domain and database. I want to deploy new sites almost with a single click, but realistically in an hour or less. And I don’t want to pay for each new site I deploy - in virtual hosting and cloud style, I want to be paying for overall consumption, not for the number of applications.

(Update: A couple of people suggested I should add Python, so it’s now here for completeness.)

Ruby

I say Ruby, but really I mean Ruby on Rails. The only other feasible frameworks for Ruby are Camping and Merb. The first is too simple and the second is too immature and lacking web host support.

Hosting for Rails has been a tragedy for me. I developed a site a year ago and tried to outsource deployment to Amazon EC2, and both suppliers gave up on the job. This was after I deployed it myself on SliceHost, requiring much effort, and then found it couldn’t handle the load. There are big Rails-specific hosts, EngineYard and RailsMachine, but they are very expensive and not practical for hobby projects. When I was working on Quizr, I also spent ages getting it deployed on RailsMachine; it was not turnkey as some people expect it to be. All I wanted was a server, a backup server, and a DB - so a fairly standard config. That’s not a criticism of RailsMachine, but don’t expect it to be trivial if you want more than one server (this was 18 months ago, it might be different now).

Rails is a super coding environment and would be a no-brainer for all of my projects, if only hosting was simple and reasonably priced. With a virtual environment like SliceHost, I found it takes several hours to set up a new site, and you also end up with a lot of config files to maintain (ie mucking about with apache modules and load balancing). You can also use a grid solution like Heroku. This is a fantastic way to keep things lightweight. The downside, though, is there’s always some little tasks you need that are outside the sandbox. e.g. setting up a cron job or adding some custom C library. Right now, Heroku doesn’t have a commercial model and apparently doesn’t let you use your own domain name. This rules it out for anything too serious.

I think the ideal for me would be a virtual Linux environment like SliceHost, with some VERY high level scripts that deploy a Rails app in a single click. These would have to be client-side scripts. I found Deprec is close to this and might get there at some stage.

PHP

PHP isn’t the best language, but it’s certainly easy to get something up and running. I find for small projects, it’s quite maintainable too, even without using PHP5’s OO. However, I’d like to use one of the Rails-inspired frameworks, e.g. PHP on Trax, in order to speed things up and ensure it’s easily maintained. This is where things get more tricky, because there are so many clones to choose from, and the community for each suffers as a result. As many have pointed out, Rails benefits from being the only serious contender in Ruby space, and the same cannot be said for any other server-side language. With PHP, you don’t need any framework at all, so in fact each framework is probably only used by less than 1% of PHP developers. In contrast, Rails is probably used by > 99% of Ruby programmers. Less community means less documentation, support forums, books, and information about hosting and capabilities.

Java

Well, I mostly mention this for the sake of completeness. Java’s an okay language for web development, especially if you choose your framework right, but hosting has mostly been restricted to enterprises. I find it funny, given Moore’s Law, how much Java hosting still costs in 2008, but it’s still pretty silly and not easy to just keep adding domains and applications without expense. Furthermore, I find it funny that Sun missed the boat on grid hosting for Java. So while there are consumer-focused solutions for grid hosting Rails and Javascript, no such thing for Java. On the whole, consumer-focused Java hosting is a cottage industry. Given that the language itself isn’t so brilliant, not worth the effort.

Where Java might get interesting is the VM. You could use it to host Groovy or Rails projects via JRuby. However, until it’s really, really, mature, I feel like JRuby is going to be one of those open-source things that sounds good on paper, but feels less enticing at 4am and with 26 open browser tabs pointing at pertinent blogs and support forums. I will reserve judgement for now.

Python

Python has a number of modern web frameworks, with Django being the most prominent. The fragmentation didn’t help, as compared with Ruby with its one gigantically popular framework, so it helped when Guido blessed Django. I’m sure Django (and others like TurboGears) is a good framework, maybe even better than Rails from what I hear. However, as I already know Ruby, I don’t have that much interest in taking the time to learn it - the languages and frameworks are similar enough that it wouldn’t be worth the effort. Moreover, I like the whole Ruby and Rails community and all the web presence and docs and books around it. It’s more fun and focused on the things I care about, like usability, aesthetics, and being opinionated. Also, there is simply a much greater volume of resources.

That said, Python has gained a lot of momentum due to Google App Engine, which exclusively hosts Python. But Google loves Java and I’m sure it will expand to at least support Java at some stage, if not more generic stuff like EC2. Providing a Java runtime would allow for stuff like Scala, JRuby, and Rhino.

Javascript

I consider server-side Javascript the holy grail over the next few years. I’ve explained why before - in summary, it’s the fact that you’re using the same language on both sides and the synergy that results. It’s not just that it’s simpler for you, the human programmer, who only has to speak in a single tongue. It’s more about the code you end up with. You can have a single module representing a web page, which contains both client and server code. It says “show the user two buttons. If they clicked button 1, do this; If they clicked button 2, do that”. So the code is very much combining client and server sides; and unlike high-level web-app compilation services, the code is actually what gets executed on both sides.

The main problem with server-side Javascript is immaturity. It will get interesting when we get Rails like models, the kind of stuff Steve Yegge has talked about. However, it shouldn’t be too similar to Rails; otherwise, you’d miss out on the synergetic effect described above. For now, though, server-side Javascript does suffer a lot in comparison with Rails, due to lack of a cohesive MVC model.

The main options for hosting are AppJet and Jaxer. AppJet is a nice, easy, grid solution, but doesn’t yet have enough info on commercial hosting and custom domain names. Jaxer has a lot of promise, but the Linux module will need to come out, be not too processor-intensive, and be supported by web hosts. There is also the upcoming Aptana Cloud solution, but this is similar to AppJet. For now, can’t host your own domain. These grid solutions are very handy, but I want to be able to release new apps and domains willy-nilly, and I suspect the grids, when they do become fully commercial, will impose a charge for each new app/domain.

A problem with the current opensocial gadget spec is that there's no relative path support. This means you end up hard-coding any references to Javascripts, CSS stylesheets, images, and services which are distributed along with your gadget.

This is not good. For example, you may have a "prod" setup and a "dev" setup. While developing, you will have to hard-code <script> tags and so on to hit the "dev" location, but then for production, you will want them pointing at your scripts in the production location. Likewise, you might want to ship the bundle to someone else for them to host on their own network.

You can deal with it using a pre-compiler tool - you know, using sed-like token substitution to piff in the right location. But a pre-compilation step is a messy compromise - I just want to distribute the code and be done with it. Another solution would be to output the gadget spec from a script, which would let you solve the problem in one of two ways: (a) inline common Javascripts and stylesheets so that you don't even need to link to them; (b) piff in the right location on the fly. Again, this is not ideal. It introduces a new server-side language and demands the spec server supports it. My aim here is simple distribution of gadgets - as long as you can host a tree of static files, you can serve my gadgets.

The optimal solution right now is far from ideal, but is the best way I know to do it. You use On-Demand Javascript (and on-demand CSS and on-demand <insert resource type here>). First, you work out the gadget spec location, then you manipulate the URL to find the required Javascript, and then you pull it down using DOM manipulation.

I'm using a compressed version of something like this:

It's partly based on a mailing list comment from Arne Roomann-Kurrik. Massaging further, I end up with the following boilerplate block which must be cut-and-paste into each gadget to bootstrap reuse:

After dropping this at the bottom of my gadget, I know that (a) ../core.js will be pulled down and executed - it is relative path from wherever the gadget came; (b) initialise() will be called once core.js has been loaded. Each of my gadget's has an initialise() function which does gadget-specific initialisation. core.js is a common library and also happens to include loadScript() in case I want to pull down further Javascripts, and also loadStylesheet() so I can grab a stylesheet with relative path.

All this is not at all ideal, for several reasons:

1. For all the benefit of neatly using a separate Javascript file, you have to cut-and-paste boilerplate code into every page! But at least the boilerplate won't have to be changed often.
2. Performance will suffer as the script is loaded later than it should be.
3. It relies on the "url" being passed into the gadget; to my knowledge, this is not something you can definitely assume will happen, i.e. it's not mandated by the opensocial standard that the gadget receives a parameter called "url" identifying where the gadget came from.

I have requested on the opensocial spec mailing list to provide direct relative path support and the response has been positive, hopefully we'll see it soon, thus rendering large parts of this post obsolete. The solution looks to be rather neat - injecting a <base> tag to establish a base URL which all relative paths work against.

Writing a multi-timezone clock gadget, I found myself needing to convert timezones in Javascript. This is one uncharted area in Ajax/Javascript - unfortunately, there is no timezone library to my knowledge. Unfortunately, there's no public JSON service to tell you what time it is right now in Tokyo either. I could create one without too much difficulty, but I'm aiming to build a self-contained gadget, which means I want it all in Javascript.

Back to first principles, ZoneInfo is incredibly thorough, but that makes it difficult to use for a single gadget. There is a wikipedia list too, but not easily parsed. (If web 3.0 is all about the semantic web, then wikipedia will hopefully become a better home for structured data too.) I then found "City Time Zones" which is a user-generated, wiki-style, approach. The problem is it doesn't handle daylight savings, but it will do for now - the user will simply get a checkbox to say if it's daylight savings or not. Yeah it's a really ordinary solution, but the only one I can manage if I'm to build the gadget in a day or so.

I extracted from the ZoneInfo a simple list you might find useful. The first iteration is sorted by timezone and I've junked geo and other info, so you just have the timezone names, the offset, and major locations.

Tonga Standard Time,+13:00 Nuku'alofa
Kamchatka Standard Time,+12:00 Kamchatka
Fiji Standard Time,+12:00 Fiji, Marshall Is.
New Zealand Standard Time,+12:00 Auckland, Wellington
Norfolk Island Standard Time,+11:30 Norfolk Island
Central Pacific Standard Time,+11:00 Solomon Is., New Caledonia
Magadan Standard Time,+11:00 Magadan
Vladivostok Standard Time,+10:00 Vladivostok
Tasmania Standard Time,+10:00 Hobart
West Pacific Standard Time,+10:00 Guam, Port Moresby
AUS Eastern Standard Time,+10:00 Canberra, Melbourne, Sydney
E. Australia Standard Time,+10:00 Brisbane
AUS Central Standard Time,+09:30 Darwin
Cen. Australia Standard Time,+09:30 Adelaide
Yakutsk Standard Time,+09:00 Yakutsk
Korea Standard Time,+09:00 Seoul
Tokyo Standard Time,+09:00 Osaka, Sapporo, Tokyo
Ulaanbaatar Standard Time,+08:00 Ulaanbaatar
Taipei Standard Time,+08:00 Taipei
W. Australia Standard Time,+08:00 Perth
Singapore Standard Time,+08:00 Kuala Lumpur, Singapore
North Asia East Standard Time,+08:00 Irkutsk
China Standard Time,+08:00 Beijing, Chongqing, Hong Kong, Urumqi
North Asia Standard Time,+07:00 Krasnoyarsk
SE Asia Standard Time,+07:00 Bangkok, Hanoi, Jakarta
Myanmar Standard Time,+06:30 Rangoon
Sri Lanka Standard Time,+06:00 Sri Jayawardenepura
Central Asia Standard Time,+06:00 Astana, Dhaka
N. Central Asia Standard Time,+06:00 Almaty, Novosibirsk
Nepal Standard Time,+05:45 Kathmandu
India Standard Time,+05:30 Chennai, Kolkata, Mumbai, New Delhi
West Asia Standard Time,+05:00 Islamabad, Karachi, Tashkent
Ekaterinburg Standard Time,+05:00 Ekaterinburg
Afghanistan Standard Time,+04:30 Kabul
Caucasus Standard Time,+04:00 Yerevan
Tbilisi Standard Time,+04:00 Tbilisi
Baku Standard Time,+04:00 Baku
Arabian Standard Time,+04:00 Abu Dhabi, Muscat
Iran Standard Time,+03:30 Tehran
E. Africa Standard Time,+03:00 Nairobi
Russian Standard Time,+03:00 Moscow, St. Petersburg, Volgograd
Arab Standard Time,+03:00 Kuwait, Riyadh
Arabic Standard Time,+03:00 Baghdad
Turkey Standard Time,+02:00 Turkey
Israel Standard Time,+02:00 Tel Aviv, Jerusalem
Syria Standard Time,+02:00 Syria
Jordan Standard Time,+02:00 Jordan
FLE Standard Time,+02:00 Helsinki, Kyiv, Riga, Sofia, Tallinn, Vilnius
South Africa Standard Time,+02:00 Harare, Pretoria
Egypt Standard Time,+02:00 Cairo
E. Europe Standard Time,+02:00 Bucharest
Lebanon Standard Time,+02:00 Beirut
GTB Standard Time,+02:00 Athens, Beirut, Istanbul, Minsk
W. Central Africa Standard Time,+01:00 West Central Africa
Central European Standard Time,+01:00 Sarajevo, Skopje, Warsaw, Zagreb
Namibia Standard Time,+01:00 Namibia
Romance Standard Time,+01:00 Brussels, Copenhagen, Madrid, Paris
Central Europe Standard Time,+01:00 Belgrade, Bratislava, Budapest, Ljubljana, Prague
W. Europe Standard Time,+01:00 Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna
GMT Standard Time,+0:00 Dublin, Edinburgh, Lisbon, London
Greenwich Standard Time,+0:00 Casablanca, Monrovia
Azores Standard Time,-01:00 Azores
Cape Verde Standard Time,-01:00 Cape Verde Is.
Mid-Atlantic Standard Time,-02:00 Mid-Atlantic
Argentina Standard Time,-03:00 Argentina
E. South America Standard Time,-03:00 Brasilia
SA Eastern Standard Time,-03:00 Georgetown, South America Eastern Time
Greenland Standard Time,-03:00 Greenland
St Pierre and Miquelon Standard Time,-03:00 St Pierre and Miquelon
Uruguay Standard Time,-03:00 Uruguay
Newfoundland Standard Time,-03:30 Newfoundland
Atlantic Standard Time,-04:00 Atlantic Time (Canada)
Falkland Islands Standard Time,-04:00 Falkland Islands
SA Western Standard Time,-04:00 La Paz
Paraguay Standard Time,-04:00 Paraguay
Pacific SA Standard Time,-04:00 Santiago
Venezuela Standard Time,-04:30 Venezuela
SA Pacific Standard Time,-05:00 Bogota, Lima, Quito
Cuba Standard Time,-05:00 Cuba Standard Time
Eastern Standard Time,-05:00 Eastern Time (US & Canada)
US Eastern Standard Time,-05:00 Indiana (East)
Central America Standard Time,-06:00 Central America
Central Standard Time,-06:00 Central Time (US & Canada)
Mexico Standard Time,-06:00 Guadalajara, Mexico City, Monterrey
Guatemala Standard Time,-06:00 Guatemala
Honduras Standard Time,-06:00 Honduras
Canada Central Standard Time,-06:00 Saskatchewan
US Mountain Standard Time,-07:00 Arizona
Mexico Standard Time 2,-07:00 Chihuahua, La Paz, Mazatlan
Mountain Standard Time,-07:00 Mountain Time (US & Canada)
Pacific Standard Time,-08:00 Pacific Time (US & Canada); Tijuana
Alaskan Standard Time,-09:00 Alaska
Hawaii-Aleutian Standard Time,-10:00 Adak
Hawaiian Standard Time,-10:00 Hawaii
Samoa Standard Time,-11:00 Midway Island, Samoa
Dateline Standard Time,-12:00 International Date Line West

Yes, Tonga is 13 hours of UTC and 25 hours ahead of those little-known places that are UTC-12. In fact, Kiribati Istland (not shown here) is +14 so a day and 2 hours ahead of the others. This wasn't meant to be easy.

Anyway, I simplified the list further to:

+13:00 Tonga
+12:00 Auckland, Wellington, Fiji, Marshall Is., Kamchatka
+11:30 Norfolk Island
+11:00 Solomon Is., New Caledonia, Magadan
+10:00 Canberra, Melbourne, Sydney, Brisbane, Hobart, Guam, Port Moresby
+09:30 Darwin, Adelaide
+09:00 Seoul, Yakutsk, Osaka, Sapporo, Tokyo
+08:00 Taipei, Ulaanbaatar, Perth, Kuala Lumpur, Singapore, Irkutsk, Beijing, Chongqing, Hong Kong, Urumqi
+07:00 Bangkok, Hanoi, Jakarta, Krasnoyarsk
+06:30 Rangoon
+06:00 Sri Jayawardenepura, Astana, Dhaka, Almaty, Novosibirsk
+05:45 Kathmandu
+05:30 Chennai, Kolkata, Mumbai, New Delhi
+05:00 Islamabad, Karachi, Tashkent, Ekaterinburg
+04:30 Kabul
+04:00 Tbilisi, Yerevan, Baku, Abu Dhabi, Muscat
+03:30 Tehran
+03:00 Moscow, St. Petersburg, Volgograd, Nairobi, Kuwait, Riyadh, Baghdad
+02:00 Turkey, Tel Aviv, Jerusalem, Syria, Jordan, Helsinki, Kyiv, Riga, Sofia, Tallinn, Vilnius, Harare, Cairo, Bucharest, Beirut, Athens, Beirut, Istanbul, Minsk

+01:00 Sarajevo, Skopje, Warsaw, Zagreb, Brussels, Copenhagen, Madrid, Paris, Belgrade, Bratislava, Budapest, Ljubljana, Prague, Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna, West Central Africa, Namibia
+0:00 Dublin, Edinburgh, Lisbon, London, Casablanca, Monrovia
-01:00 Azores, Cape Verde Is.
-02:00 Mid-Atlantic
-03:00 Argentina, Brasilia, Georgetown, South America Eastern Time, Greenland, Uruguay
-03:30 Newfoundland
-04:00 Atlantic Time (Canada), Falkland Islands, La Paz, Paraguay, Santiago
-04:30 Venezuela
-05:00 Eastern Time (US & Canada), Bogota, Lima, Quito, Havana
-06:00 Central Time (US & Canada), Central America, Guadalajara, Mexico City, Monterrey, Guatemala, Honduras, Saskatchewan
-07:00 Mountain Time (US & Canada), Arizona, Chihuahua, La Paz, Mazatlan
-08:00 Pacific Time (US & Canada), Tijuana
-09:00 Alaska
-10:00 Hawaii, Adak
-11:00 Midway Island, Samoa
-12:00 International Date Line West

Also available in cut-and-paste form

Just as I was writing this post on ignorance of open source models, there was news of a major publisher making a print edition of Wikipedia. This led to backlash reactions which illustrate exactly the kind of misunderstanding that arises when people don't get open-source licenses (via ReadWriteWeb.

The whole point of Wikipedia is to provide a voluntary online source of information. That is why people voluntarily gave up some of their time to write the articles for free in the first place. It is a noble project, the writer believed in the project and so they participated in it. Had they known that Wikipedia would then use their work in a commercial printed venture, I’m sure they would have had second thoughts about writing those articles. At the very least they would have demanded a contract and perhaps a guarantee of financial compensation later.

Now the author might well have a point - maybe people don't know what they're doing when they're adding to wikipedia and maybe wikipedia should make it more clear. Still, I find it worrying that they would consider some big ball of wrong has been cast upon them. A great place for them to start researching would be the GNU article on wikipedia, the one article whose authors knew they would be victims right from the start ;).

There are several disturbing things about this:

• First, if this ignorance does exist, that's unfortunate. If anyone is a major enough contributor to wikipedia to care about this, they really ought to familiarise themselves with the basics of the GNU license they're operating under. (And for the record, I think it's a shame wikipedia uses GNU instead of a more permissive license.)
• It's scarcity mentality. Boo-hoo, the publisher is making money and I'm not, so they must be ripping me off. Yeah right! If you were willing to contribute to wikipedia before, your content remains just as available online as before. What's changed? Nothing, except someone else is going to make your content more digestible in certain contexts and accessible to more people. And yes, they will make money from it. Maybe they'll make lots of money. Whatever. That's what happens in a modern knowledge economy - people like Larry and Sergey make money without denying it from someone else. Far from removing others' assets, they actually add value to everyone else. Likewise, the existence of a print wikipedia will also add, in its own way, to society's wealth. (Although not the forests. That's a separate, more legitimate, argument.)
• There's an old psychology result that says people care more about their spending power in relative terms than absolute terms. ie you ask them "Would you rather earn $50,000 a year while other people make $25,000, or would you rather earn $100,000 a year while other people get $250,000? Assume price of service and goods would remain the same." People would rather earn half as much, if it means they are richer rather than poorer than others. That result says a lot for this kind of irrational backlash against anyone who might cash in on their "volunteer" work.
• It's hardly the first time this has happened. Answers.com and hundreds of fairly useless SEO sites have been profiting from wikipedia content for years. I'm surprised it took this long for someone to do it. I could imagine countless opportunities for books on niche content - bizarre trivia, country almanacs, movie guides, etc etc.

That a company can do something like this without having to formally negotiate with the wikipedia foundation and a million contributors...I call that a cause for celebration, not outrage. Personally, I've contributed various boring tidbits to wikipedia and I hereby invite any publisher to print what they want from my contributions, even if there's only limited demand for hard copy about snowclones.

Controversy arose in several open source projects last week: Ext JS, SpringSource App Platform, and the SWF/FLV specs. Dion has a good summary and concludes with a reminder that most people could care less:

Finally, I know that 99% of the developers out there may not even care, let alone users. There are open source wonks who like to argue about licensing and methodologies. As I watched the John Adams HBO series, I felt a little like those fine chaps arguing over the finer details of things. Many of the people didn’t know what was going on there, or why a particular Article was written the way it was. But, they had drastic implications for the people. I think that the same can hold here for some of the projects.

Now this ignorance of open source models in my experience and an interesting phenomenon. It took open source until around the middle of this decade for even the basic definition of open source to be understood, and to some extent, accepted, by mainstream IT and business communities. That's about five decades after the practice began, two decades after Stallman's GNU Manifest, and almost a decade after Raymond et al declared "Open Source" teh nu k00lness.

The basic definition of open source is understood, but as Dion notes, most people have no idea how the models vary, or what the implications are. Which is unfortunate, because the commercial implications are indeed vast.

I consider myself an expert on this topic ;P because I once picked up a copy of an O'Reilly book on open source models at a heavily reduced price and flicked through it intensely for at least two hours. Okay, so not quite an expert (!), more an armchair ignoramus, but certainly a stakeholder as both a consumer and publisher of open-source. And I'm hardly the Richard Stallman of the Ajax world either, but whenever I publish software that could potentially be used or adapted by someone, I make the tiny effort of including a license notice. Sometimes it almost seems arrogant to do so when the software is so trivial, but I'd rather do that than leave the license unstated, in which case it would come under the silly default, i.e. standard restrictive copyright. (Even if you agree with standard copyright, doesn't it seem silly that rights to anything anyone creates belongs to them by default, and for the greater part of a century?)

Well, since I include a notice, I've had to think about which license to use.

In my dim-witted view of models, there are three key categories. All allow people to use it free of charge and generally remove any liability from the provider. They vary in what happens when people change it:

• Copyleft/Viral (e.g. GPL, Creative Commons Share-Alike) - Anyone who modifies it must release the modified version.
• Permissive (e.g. MIT, BSD, Apache) - You don't have to release your modifications, though you still have some restrictions such as retaining the original license and author details.
• Public Domain - Anything goes.

There are other distinctions too, e.g. around charging and attribution to the original creation; but the above distinction is the most important in most cases. And this is the sad thing: most IT and business people simply don't understand these distinctions and the implications they have.

You can argue about ethics until you're blue in the face, but the reality is, there's no absolutely "right" or "wrong" model. It's just horses for courses and being aware of the implications. Personally, I do have a hard time justifying copyleft-style licenses and I sometimes wonder how many people release stuff as GNU just because they associate it with open-source, without realising the full implications. I've worked in many commercial situations where GPL is unacceptable and I hate the idea that developers will be in the same situation as me, i.e. gleefully eyeing something that would scratch their itch, but not being able to use it. Incidentally, Ajax Patterns was initially released with the more permissive Creative Commons license, but following the book deal, O'Reilly asked me to change it to share-alike as it discourages other publishers from publishing their own version of the same book, which has apparently happened in the past. This is similar to the reason some companies release dual licenses - GPL and standard copyright. It means anyone can use it for free under GPL, but other companies can't update and re-distribute for a fee. I can see that. It creates an incentive to release as open-source.

It's convenient for me to have a default license choice, so I don't have to think too hard whenever I release something. I have chosen The MIT License because it is extremely concise, and is basically the smallest (in words) mainstream license that satisfies my main aims: To allow people to work with it freely and free of the restrictive burden GNU creates; to receive credit for my work; to be free of liability. The three paragraph license can easily be embedded in any source file, which is great for releasing one-file libraries without having to include a separate README or LICENSE file. Best of all, it's easy to grok. If developers should be able to consume open source as easily as pulling out books from a shelf, you really don't want to release software under a license that only a lawyer could love. Don't let MIT's concise nature fool you into thinking it's only used for toy projects: MIT is used in Rails, Mono, and the system for which it was originally written: X-Windows.

The MIT License looks like this:

Copyright (c)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

I discovered at last minute a developer event on Yahoo! FireEagle at Covent Garden tonight and decided to rush down there. FireEagle is pretty intriguing as the first serious attempt at an OAuth API (though Google Contacts now qualifies too). For me, that was the main draw; but the actual service it offers is also compelling.

What's FireEagle?

Location based services have been hyped for years. By now, you should be walking around and receiving reviews of local restaurants on your phone, browsing nearby tourist attractions, and seeing which of your friends is in your vicinity. However, it hasn't caught on. The main reason is stovepipes and walled gardens. Your mobile provider might have an API available, but probably restricted access to a very limited set of developers. A mix of privacy and commercial concerns have rendered this whole area practically useless.

Enter the eagle. FireEagle completely opens up this area, by offering a model that is flexible and open, but without compromising user privacy. It's an API that essentially tracks one thing: Where in the world are all FireEagle users?

In other words, the FireEagle API allows any client to update a user's location and any client to retrieve a user's location. Of course, the user must consent to all this activity, and that's where OAuth comes in - as a way for users to say, for example, "I trust app abc to update my location and app xyz to read my location". So FireEagle detaches location providers from location consumers, opening up an entire ecosystem.

What kind of clients act as location updaters? You might think it's all automated stuff, like mobile devices and IP numbers, but actually many updaters are manual. Here's a sample:

• Mobile phone app. Anyone with the right access details could write a mobile phone app to use cell tower information, built-in GPS, or any other cues, to update location as the user moves around. This is the no-brainer example of an automatic mobile updater.
• Twitter. When you tweet with "L: london", a twitter monitor app could notice that and notify FireEagle. (As with all cases below, you must have authorised the monitor app to do that.) This example illustrates that users aren't always passive lemmings walking around with a mobile updater app in their pocket. It may be that location data is only ever updated when a user proactively tweets their location.
• A client that watches you adding geotagged photos to Flickr, and assumes you are in the last location you uploaded.
• Specialised tracker device. Devices based on GPS (e.g. SPOT) could easily be made to update FireEagle with their location.
• Car app. Based on GPS, a car's location could be tracked using FireEagle. (The car, not the driver.)
• Travel card. (My example.) Theoretically, your travel card could update FireEagle location as you move around. Same for transponder devices in cars.
• Many, many, more examples.

What kind of clients act as location consumers?

• Find nearby friends, tourist attractions, pubs, etc.
• Find currency rate (since you know what country they're in)
• Phrases in the local lingo or an online dictionary
• Location based games
• Many, many, more examples

Some clients may act as both updater and consumer.

Privacy

The other thing FireEagle gets right is great concern for privacy - users can give out as much or as little data as they like, and they can stop the service at any time. The first way this happens is with OAuth, which lets you manage which services can perform what actions. You can start and stop this at any time. In the future, there will be a simple client to let you log in from your mobile and control all this. In particular, you will probably be able to suspend all tracking at any time.

Additionally, you can apparently set granularity, so you could expose just a general area, e.g. whiiich city or country you're in, instead of a particular co-ordinate.

In the future, you will probably be able to authorise a client just for a certain time, e.g. during a conference, after which it can no longer access your data.

At some point, historical data may become available. The team said if this happens, they will allow users to delete and edit their past data.

In summary, the team has been very careful to ensure users have complete control over their own data,

Very Raw Notes from Presentation

Tom Coates is talking about FireEagle.

FireEagle is the old CS classic: a layer of intermediation - between location identification systems and location consumer systems. As a user, I can tell FireEagle where I am and any permissioned app can make use of that data.

London is about the most frequent location for fireeagle

What would your existing/past apps look like if they had location services available?

e.g. Navizon

e.g. ZoneTag

e.g. Firebot - Make it your twitter friend and direct message your location to it.

e.g. BrightKite - cf dodgeball

e.g. Rummble

e.g. Plazes - determines your location via wifi. Then shows where you are on map, people around you, etc etc. And updates FireEagle

WikiNear - ~1M geo-tagged articles in wikipedia. Wikinear exploits that data - as you're walking around, shows you the closest POIs that are in wikipedia.

Lightpole

Outside.In

Fireball - where your friends are

Fire Widgets - weather where you are, nearby Flickr photos

Moveable Type

Facebook "friends on fire" app - shows where your friends are and updates your Facebook status

Could be great...

Spot - specialised comms/phone device for backpackers, aid workers, etc. ~100pound for device, 100pound a year and keeps updating your location every� ten minutes. Doesn't integrate yet but great example if it did

Ambient Orb - e.g. changes colour as you move away

Nabaztag

Geotagging all user generated content - cinema listings, local traffic, local TV stations, nearby friends, weather forecast, local exchange rates, public holidays, windspeed. A lot of this is on wikipedia thanks to geotagging.

Friends and family widgets. e.g. where they are in the world, what time it is, weather, etc. (A gadget for each person.)

Last.FM - On cracked iphone, can get mobile scrobbler on iphone. What if it recorded where you were, then you could see which songs people play in particular areas. Cool!

Pacmanhattan - Lots and lots of game ideas (idea: scavenger hunt! reminds me of geocaching)

(later mention: geocoding animals, e.g. track migration)

Later on, expecting it to track historical data. (and since it will be user-modifiable which means you could back-track your entire life! (or someone elses eg fake shaekspeare))

http://tinyurl.com/5gtj92

http://tinyurl.com/3uklhl

Building an APP

(http://fireeagle.yahoo.net/developer/documentation/getting_started)

1 - Get API key 2 - User authorises your app 3 - Make API calls to Fire Eagle Outside.In -�

Who's within an area

Each consumer key and secret identifies an application using Fire Eagle.

User Authorises ....

There are three models and the difference is purely to do with differences in (app triggering web page) and (web page triggering app). Web: Can trigger in both directions Mobile: Can't trigger in either direction (maybe, but can't assume it) Desktop: App can trigger web page, web page can't trigger app

Web App model: Request token

I asked which model was used for widgets. Answer: Desktop. (Makes sense, with current technology. Web model would cause redirection from container. Later on, Opensocial will have oauth built in.)

Core Concepts

Note: social graph (user and friends) is beyond scope

location - point or bounding box location hierarchy - set of locations

Exposed RESTfully:

• user() - duh
• lookup() - provide location string and choose from list of resolutions (which "london"??)
• update() - duh. call it and it "moves you"
• within() - ?
• recent() - "map of soho and everyone in there" OR "last 100 updates from my users"

walking through http://github.com/mojodna/fireeagle-tutorial/tree/master/ruby

Will probably support xmpp/Jabber too - more appropriate than HTTP for this