Software As She’s Developed

Wolf Logan’s talking about paleosocial patterns (TOOLS Web 2.0 Patterns workshop). Very cool.

Marking - making permanent, public, marks. Dog marking its territory, “I was here” graffiti. e.g. blog comments, forums, guestbooks. Being obnoxious gets the comment noticed more.

Pointing - indicating resources to others, can be with opinions. Reputation will dictate how much people care. e.g. Reviewing/rating sites, Digg.

Competing - comparing self to others, ranking is dynamic Some people approach competition for their own value; others for their social value. Special case of marking - marking yourself - but with an element of objective measurement e.g. online gaming sites, xbox live, popularity contests.

Storytelling - describing sequence of events, often about yourself Different stories for different social groups e.g. blogs, journals, youtube

Grooming - small, ritualised, social interactions - can happen quickly and without much planning, Culturally bounded. Repeated frequently depending on relationshps. “Purring” - e.g. “how are you?” - don’t care, but it’s the purring that happens that makes it a grooming behaviour. The info transfer doesn’t matter, just that the event happened. e.g. friending, twitter, majority of IM content

Hunting - tracking, locating, acquiring resources. Serendipity plays a big part. Successful hunting leads to pointing - tell people what you found. e.g. web search, link following.

Wolf points out hunting and grooming are always the two critical patterns that are noted.

We’ve done a little group exercise at the TOOLS web 2.0 patterns workshop. Designing a video website, with heavy emphasis on mashing up, e.g. show videos from youtube, grab recommendations from netflix, grab friend lists from Facebook, etc.

Here’s the full feature list (click for full image):

And the design:

We eventually identified features, at back end and front end, and listed similar sites where on might get inspiration for each feature. Interestingly, when you do this with web 2.0 sites, you end up realising you could actually implement the feature by actually using this service.

Dragos has transcribed the list we came up with (partly visible in the photo above):

Day 1: Design Exercise

Problems + Solutions (1) [MM - these were back-end features]
-Key-value store: Amazon Dynamo/S3, BigTable (Google)
-Streaming: Hulu, YouTube streaming (not Flash)
-Client time synchro service: NTP, MMORPG
-Social network integration services: Oauth, OpenID/Passport
-Mash-up engine: PopFly, Yahoo! pipes
-Monetization (social cash -> $, analytics, instrumentation): AdSense, witkey.com, SiteMeter, Elance

Problems + Solutions (2) [MM - these were front-end features]
-Video: Vimeo, Nicovideo.jp, Jaman.com
-Mood: Musicovery, Yahoo! IM, Nabaztag.com
-Friend states: Twitter
IM: Facebook/MySpace, Twitter, Shoutbox
-Recommendations: Tuneglue, Netflix, Facebook

Some Emerging Themes
-Taxonomy of social networks
-Social profile aggregation: InfoCard
-Integration of services transparent from user
(Tim O'Reilly’s innovation by assembly)
-Privacy (who sees what)
-Biggest pile wins
-Cooler than thou 

In under 5 minutes, a few quick notes on questions I was asked and general feedback on the talk.

Someone asked me if I’m involved in the project (ie contributor etc). I think the talk gave that impression too much, like I’m an evangelist :). Doesn’t help that I had a mail gadget open in the demo and it’s full of opensocial mailing list stuff because I once subscribed there from my gmail account, which I basically don’t use!

• Bertrand commented on REST and asked me how it works with XML and JSON. I explained limitations wrt graceful degradation ie the way forms don’t produce XML, and the way every 3rd party JSON call is GET.

• Question on how this works with mobiles (limited bandwidth, sometimes offline)

• Dragos asked about other standards/frameworks. I agree, need to look more at others.

• Question about how it works with reverse ajax (ie comet / http-streaming)

• Query about “write once, run many” - is it realistic?

• OAuth - difficult on mobiles

It’s belatedly dawned on me how OpenSocial makes a great server-less Ajax platform. When you create an OpenSocial gadget, you’re building a lil Ajax app that performs much coolness that would normally require a server, but doesn’t. Effectively, you’re delegating the duties of the gadget’s host environment. All you have to do is write a single XML file and host it somewhere. In fact Google Gadget Editor gives you an editor to write it in and a place to host it.

In fact, this is so good that if I was teaching someone Ajax in 60 minutes or less, I would consider getting them to write a gadget. It’s that simple. Previously, I might have got them to write a static HTML file, with embedded Javascript and CSS, but now they can do the same thing and benefit from server-side features too, features they never had to install. This is like getting the usual benefits of the DOM and the Javascript API, but with much more. You’re getting both a set of APIs, just like when you use JQuery or Prototype, but you’re also getting a server which many of those APIs rely on. So it’s a true platform.

By the way, I’m assuming you’re using XML content type, not URL. XML content type is the way forward, especially as we move to a Caja world. XML gadgets will get you the richest functionality. URL gadgets are only suitable for legacy concerns, and in some cases, accessibility.

So you’re using XML content type, which means the entirety of your Ajax app lives in one file. What does an OpenSocial platform provide to your little XML gadget spec?

• Persistence. The most important thing is you can save application data. You can serialise any state you care to persist, and then store it as a hidden user preference. This is really neat. It means you can write a TODO gadget, for example, and ensure all the TODO tasks will be saved, but without having to store them on your own database and without having to build a persistence layer around it. And of course, the platform will (theoretically) provide the necessary security to ensure users only see their own tasks.
• Preferences. You can easily let users customise your app via the preference API. For free, you get form fields to let users state their preferences, with drop-downs for enumerated types. And of course, the preferences are persisted without any effort on your part.
• Content Fetching. It gets even better. On-Demand Javascript is often bigged up as the way to get data from third parties. But thanks to OpenSocial’s remote fetching APIs, it’s just as easy to grab content via the containers Cross-Domain Proxy. This means you can get any internet content and call any API, not just those specifically designed for cross-domain browser calls. There are also value-adds here, e.g. caching, scheduling periodic updates, converting feed data of any format into a consistent JSON structure.
• Identity and Social Networking. Ah yes, the almighty “social” in OpenSocial. On the right platform, your server-less Ajax app can get info about the page owner and the page viewer, and their friends. Friends and social stuff and knowing whose page it is, that’s cool and all. But what’s really exciting here is much simpler than that - it’s simply IDENTITY. What matters most is: With a single API call, you get an ID for the guy viewing your gadget!!! You didn’t have to handle registration, password reminder links, etc. etc., and go through all the associated security questions around it. All the usual stuff you have to do if you want to know who’s using your application. It’s all *free* to you, gadget developer. (Downside is your company doesn’t “own the user”, so your gadget will never get you pina coladas on the beach at noon and an environmentally-conscious hybrid vehicle to sip frappachino grandès in.) With this user handle, you can also save a hashmap of information about the user, persisted by our friend the server.
• Many other services. Each platform is free to provide any number of features beyond the required API; this is how platforms compete against each other in the OpenSocial world. “Feature” is actually the term used for a particular set of browser and server services, e.g. “dynamic height” - the ability for a gadget to update its height - is a standard OpenSocial feature. Notifications - the ability for gadgets to add little notes to the top which the user must manually clear - well, that’s a custom feature iGoogle provides but others don’t. So anyway, there are lots of these features and your gadgets get them for free. In most cases, they are just libraries which you could also get for free by using Scriptaculous or whatever…but still, with OpenSocial, it’s really, really, easy! No download or installation. (Although it must be said that with library CDNs like Google’s new effort, you can also use many Ajax libraries without installation too. The difference here is that at least some of these features are tied to server-side services as well.)

So, these days, OpenSocial may well be the easiest way for a newbie to learn Ajax, and for any developer to get an Ajax app up and running. I actually think we need more tools and services to allow standard Ajax apps - not gadgets - to get up and running fast too. After all, the UI in OpenSocial is based on a widget/gadget model, and this is a mini web app running on a bigger page. Nowadays, we have canvas gadgets, which are bigger, but still not so big and run inside another web page. What if you took the same cloudish platform principles and applied it to a big old Ajax app, occupying the entire page and with its own URL (though probably coming from the platform’s domain). You can sort of do that already by looking at the content of the gadget iframe, but it’s not designed for that purpose…and if it was, you could do some interesting stuff.

For instance, I envisage a simple-to-use cloud storage system based around principles of Persevere and CouchDB, and for limited use, it could even allow anonymous access. I’ll perhaps demo all this at some stage.

I have the honour of delivering a keynote at the Tools Europe conference in Zurich next week, as well as at the “Mining Web 2.0 Patterns” workshop, which, as you can tell from its title, I’m looking forward to participating in. The workshop chairs are Dragos Manolescu and Joe Yoder.

The keynote is on OpenSocial, Gadgets, and Design Patterns; and I’ll have more to say about it later. I believe it will be recorded too. For now, I’m using my blog to post my response for the pre-workshop preparation.

Sites and materials relevant to my presentation

• What is OpenSocial?
• iGoogle
• Ning OpenSocial gadgets directory
• BBC weather gadget
• TODO gadget

Questions:

• Which features make the gadgets easy or hard to use, and how might the gadgets be improved?
• At UI level, which best practice patterns are apparent?
• What changes to the platform would improve user experience?

Sites and materials relevant to other workshop tracks

Site Navigation:

• SkyScanner Kayak Not sure if it qualifies as navigation, but the live filtering mechanisms are well designed.
• DZone The search results are an example of Virtual Workspace, aka infinite scrolling as opposed to pagination.
• Delicious The quintessential demo of tags and folksonomy.
• Mainly for the way it re-thought mail and how it’s used. Early example of navigation via Ajax calls; “star this”.

Services:

• OAuth Very important for service design in the future.
• Heroku If we’re including cloud hosting …
• Twitter Good example of an easy-to-use REST-ish API that’s led to a proliferation of apps.

I just realised there’s no good executive summary for OpenSocial. Every resource fluffs about the “social” aspect and forgets about the UI aspect. The UI aspect, essentially a gadget specification, is just as important as the social aspect, notwithstanding its absence from the “OpenSocial” brand name. I’m not a fan of the name “OpenSocial” for this reason, though I acknowledge it does have more attention-grabbing bling than “open social widget framework 1.0″. In any event, it’s regrettable that most commentators gobble it up hook, line, and sinker; as evidenced by their almost exclusive focus on the social angle.

With no further ado, here’s my attempt at a one-paragraph summary of OpenSocial …

OpenSocial Executive Summary

OpenSocial is a standard for “mini” web apps, called “gadgets”, which are designed to be embedded other websites. The standard informs developers who wish to write portable gadgets, i.e. those which work in any OpenSocial environment; and likewise, it informs developers who are creating such environments. It defines an XML format for applications, typically containing the user interface (HTML/CSS/Javascript) as well as metadata, e.g. author, gadget size, and preferences. In addition, OpenSocial defines a number of APIs, so that gadgets can invoke services such as changing the gadget height and fetching remote content. One particularly important class of APIs are those of a social nature. These enable a gadget to access data and run services related to the user who is viewing the gadget, or the user whose web page the gadget is embedded in. For example, the gadget can - subject to security settings - find a person’s name and location, traverse their social network, and invite them to install an application. Access to social data is not limited to gadgets, however, as OpenSocial also provides a RESTful variant of the social API, suitable for any kind of net-enabled client. OpenSocial was launched by Google in November, 2007, and is to be managed by the non-profit OpenSocial Foundation.

I’m a redundancy fanboy. In visualisation, different formats suit different personalities and different tasks. With version control, the usual format is just a text log. This is good if you’re scanning for specific terms, but pretty ordinary for other activities - e.g. to get a feel for general trends that have arisen, the pace of change, or the rise and fall of specific contributors.

It’s encouraging, then, to see demos like the following, which shows the evolution of the Python language project (via Dion’s tweet).

code_swarm - Python from Michael Ogawa on Vimeo.

It reminds me of one of the first screencasts by Jon Udell, a fascinating walkthrough of the evolution of a wikipedia page over a year or so. The page he chose for this demo is as memorable as the message of the video itself.

These visualisations are cool as tasters for what might be, but they are “here’s one we made earlier”. Where are the tools to automate all this? I have no doubt such tools have been created in academic research projects, but let’s see them in action. I’d love to see the source code hosts - sourceforge, google code, github, et al - integrate this technology to produce visualisations on the fly.

Firefox 3.0 ticker tape parade

Today is Firefox 3.0 landing day. Maybe tomorrow as the servers have been down for many hours now. Funny how Twitter is up, but Mozilla is down…it feels like today is the Juneth of 18. Anyway, this is great news as you could wile away a few lazy hours tweeting and plurking about the irony and how you’re chomping at the bit for FF 3.0, and watching other people say the same thing but with different yet equally charming emoticons. But anyway, FF 3.0 is pretty much here. Yay!

Firefox 3.0 is faster, more stable, and standards-based, and that’s no mean technical feat, but the upgrade is not so obvious from a consumer’s perspective. And my own ADD mindset craves more visible changes that will impact on usability and utility.

The modern browser is the application billions of users spend the most time in, many hours a day, and the trend is set to continue as desktop migrates to web. And Firefox is used by 18% of users (Net Applications via wikipedia, Q2 2008) - that’s 18% of some billions of users. A very big number indeed, and even more remarkable in certain markets, where Firefox dominates share, and even in the entire population of certain countries, where it appears to be neck-and-neck with IE. Furthermore, as a major player, the features of Firefox are clearly going to influence those in competing browsers. What’s in Firefox matters a lot. So, by deludedly illogical inference, the present article must also mean a lot to the future of our green planet. Here I will outline UI changes in FF3, then present my wishlist for FF4.

But first, what’s in 3.0 then?

This is a wishlist for the next version of Firefox, FF 4.0. But first, let’s review Firefox 3.0, ignoring those important but not too tasty “-ility” updates, and instead focusing on what users will actually see. I had to think about this. After several months of using both versions alongside each other (on different boxes), I really don’t notice much difference at all. as noted by wikipedia, are pretty trivial. (I noticed a better list here, though there are no other major features mentioned.)

• More native look and feel. Nice, but a small change.
• Redesigned download manager, can search for downloads Same.
• Redesigned add-on managerIt’s integrated with the main distro site This is by far the most important new feature. I’ll elaborate later, but FF is all about plugins and the plugin process has been pretty silly up to now. In the book, Art and Science of Javascript, my chapter on Firebug included a section on installing the Firebug extension. And I can tell you, looking at it step by step, you can see how complicated the whole thing is.
• Microformats are supported. Cool, we like microformats and it leads the way to the so-called “Web 3.0″!
• Introduced Places for bookmarks. Yes, this is the most obvious new feature, but the thing is, years of rudimentary bookmarking support, combined with Google’s almightiness and Delicious et al, has weaned me, and I suspect many others, off bookmarking in general. I don’t know, how many people are really going to tag their bookmarks? What I do like is the new smarter address bar with history auto-completion and better display. It has leap-frogged IE here, which has searched by title for some time. Handy little feature.
• OSX version supports Growl, spell check, and Aqua. Well, Growl is nice and it’s available to Add-on developers…but it should also be available to all websites, perhaps using a whitelist setup. (Similar to the Add-On trust system now - a notification panel that says “this application wishes to notify you of critical events via Growl. Do you agree?”.)
• Default icons change Cool but fairly basic again.

These are all changes you might expect in a point release - it would be hard to justify going 3.0 in the absence of all the other improvements to stability, etc. That’s why I’m not trying to take anything away from the overall FF3 effort … there’s a lot of changes outside the UI. But when you look at these UI changes, they’re very basic. I’m hoping FF4 is more focused on seriously evolving the browser, and that means taking some chances and making some bold improvements to the shiny Fox machine.

And now … The Firefox 4.0 Wishlist!

Shipping with Add-Ons

As I’ve said before, Firefox needs to ship with a proper suite of pre-installed extensions. The “thin kernel model with optional extensions” is architecturally sound, and of course many applications now incorporate a plugin style architecture. The highly successful Linux and Apache web server projects are based on this model too. However, Mozilla misses a trick by shipping without key extensions. The attitude seems to be “if users want it, they can ^@!#$ install it themselves! The Mozilla people are in general extremely concerned with usability and the human factor, but in this case, the attitude reeks of Comic Book Guy geekery. Leaving out basic tab functionality, for example, yields unecessary advantages to Opera and Safari. Yes, it’s available in TabMix if you care to download it, but if it’s that important, then ship FF with TabMix!!!

There should be a standard distro with 5 or 6 popular plugins, all chosen for utility, simplicity, security, and compatibility with each other. This should be the one you get when you click the big, fat, super-easy-to-use, Download button (which I have always admired and has since been copied by many other websites). A bit deeper in the recesses of the Firefox distro site, you could also download the minimal edition as well as specialised editions. Alternatively, you might select your distro the first time you run a new Firefox profile. I could imagine a wizard that (a) asks you which distro (mapped to a set of add-ons), and then (b) lets you customise the list. This is much better than the current situation for creating a new profile, where you have to go and (re-)add all your typical add-ons manually, then restart.

It would be cool to see specialised editions for developers, students of different ages, gamers.

Apart from usability, this would be a boon for security, which is arguably Mozilla’s primary concern and greatest source of pride when comparing Firefox to IE. Relieving most users from the need to install add-ons reduces the chance of a screw-up involving installation of a malicious plugin, or some unanticipated interaction between plugins that creates a hole. The default stack would be designed and tested for security, and any alerts involving an add-on in that stack will be highly visible.

Make it easier, much easier, to write Add-Ons

This was supposed to be an enhancement for FF3 iirc, but may have been dropped (?). Anyway, writing extensions in Firefox is a complicated business. You have to know XUL and deal with the whole packaging structure. It should be just as simple as writing Greasemonkey scripts IMO. I’d like to be able to say something like $(”addressBar”).style.backgroundColor = “red” for example. These things could be made possible with enhancements to Greasemonkey, but of course Greasemonkey isn’t built in. What I’m looking for here must work with the raw version of Firefox. This will lead to a proliferation of cool new add-ons. It will also increase the risk of security violations as there will be more plugins around and it will be harder for Mozilla to vet them all, but relying on a difficult API to slow down add-on … that’s not the way to ensure security.

Installation of Add-Ons, Themes, Search

Installing search modules should work the same way as the new way to install add-ons and themes, ie all inside the client instead of bumping you to a website. Installing themes is half integrated but still requires you to go to a website. Even Add-Ons requires you to go to a website to explore fully. I think this should be unnecessary.

There is also confusion of terminology, with references to both “Add-Ons” and “Extensions”. On the official add-on website, “Search” and “Themes” are combined with 10 other category of Add-On (Appearance, Bookmarks, etc.), whereas they are in fact special types of add-on. So in some cases, they are kept distinct, and in other cases, they are lumped together.

Improve Profile Management

I like the idea of profiles - as a developer, it would be nice to switch between a developer profile and a user profile. And also between multiple dummy user profiles while testing. But switching profiles requires a restart. Not only a restart, but you have to run Firefox with the “-p” flag, and the installer doesn’t come with a launcher for profile-switch mode. For that reason, I bet most users probably don’t even know this feature exists.

The simplest improvement would be for the installer to add a launcher for profile-switch mode to the Start menu (in windows at least), just like there is one for Safe Mode. Moreover, just integrate profile switching into the actual interface…just include a dropdown in the Advanced Preferences or add a new menu item somewhere which expands to show each profile ID. Easy! True, you’d still need a FF reboot. I don’t see that constraint changing anytime soon …

Tab-Mania! Sugoi yo ^o^

I wrote this up in Taking Browser Tabs Seriously. There’s a ton that could and should be done to ease the multiple tab experience - notifying, searching, grouping, sorting, etc etc etc. This is HUGE! ‘Nuff said, see the link for details.

Retain all viewed content (or at least index it for searching)

Come on, you know you want it! Instead of bookmarks, it’s time to finally bite the bullet and let users search through every page they’ve visited (if not forever, then at least for the last X days). How many times do you have to search for something on Google that you’ve already seen before. The browser could take care of it. Just as Google search slams Yahoo taxonomies, so too does full-page history search obliterates bookmarking. I don’t know how much capacity you’d need to do it, but I bet if you asked in 1994, “will we be able to do that in 2010, with 16 years = 1000x (ie 2^(16/1.5)) improvement in hardware stuff”, you would answer a resounding yes. With the right optimisations, I’m fairly certain this is possible. BTW I realise you can effectively achieve all this now with Google, but there are a lot of reasons why it should be built into the browser (intranet and hidden content; privacy and corporate security concerns; possibility of offline browsing old content).

Improve history

With or without the previous enhancement, Firefox history has always been a bit wanting. For starters, search needs improvement - you should be able to search history the same way you search through a page. ie instead of filtering, it should just highlight matches and scroll to them. This way, you could see which other pages you were looking at around the same time. Sometimes, I’m searching for one term, but only because I know I was looking at another page at the same time, and I can’t locate the other page via search. (Although I could do so if the previous enhancement was available!)

Search should also work without having to hit Enter - it should be updated on each keystroke just like page search.

Furthermore, it’s excruciating trying to work out what time and date you viewed something. You get the “View by Date” which sorts by date, but doesn’t let you search in this mode. And in any event, it still doesn’t show you time of day. And if you view in other ways (by site or last/most visited), you don’t get any time or date at all.

That Sidebar

I never get why there’s only one sidebar when there are several possible contents (bookmarks, history, add-on-specific features). And it’s also odd that you can only control that sidebar from the menu rather than the sidebar itself. You should be able to have multiple sidebars, or at least one accordian style sidebar containing all possible contents. Similar to the All-In-One Sidebar add-on.

I could imagine Firefox achieving flexible UI with an elastic docked window style UI like Eclipse (or indeed Firebug in some respects), but I think that would be too complicated for mainstream users.

Render Non-HTML Content

I realise a browser’s fundamental job is to show web pages and it’s rarely good when applications over-step their boundary, but in this case, I feel there is a case for putting in at least some effort on non-core activities. I’m talking about how Firefox (and every other mainstream browser) deals with non-HTML content. For example, this came up on Twitter today, where my colleague mentioned the problems we have viewing JSON in the browser. It requires a download basically and then manually opening and viewing in your editor, and that’s not very satisfactory. Even with XML, which is rendered by FF, how about some intelligent interpretation. If it’s RSS or Atom, for example, provide a suitable default stylesheet explaining what it’s good for and how to subscribe.

Better File System Navigation

When you visit a local directory using file://path-to-directory, what you are greeted with is an interface that hasn’t changed since the mid ’90s. Arguably, it’s doing enough already, so why complicate it? But I think that exploring the filesystem is a rather useful feature for the browser, given its sovereign posture which means it could be used as the primary control centre for managing your local file system.

It’s also impossible to visit the root of your filesystem using file:/// on certain OS’s, at least on my Mac. This is obviously fundamental behaviour that’s missing, and the bug was logged in 1999! On my Mac, I also can’t open up /Volumes/Macintosh HD due to the space. When I click on it from /Volumes, it changes to file:///Volumes/Macintosh%20HD and that URL breaks. Doh!

Just like Explorer or Finder, files should be shown in various possible formats - tabular (with sorting), thumbnail, etc. And it should use thumbnails to show file types.

Better Keyboard Binding Support

I like those programs like IntelliJ Idea which let you control keyboard bindings for pretty much anything, and also let you save and load configurations. Firefox should do the same. A lot of the shortcuts are hidden anyway and should be available somewhere in the interface, e.g. a keyboard preferences area.

Mouse Gestures

This was one of the awesome things about Opera when I first started using it and it made browsing fun and more subconscious, one step closer to the haptically brilliant Minority Report hand control scenario. I used it all the time, but on Firefox, the main add-on didn’t work so well and I never got into it. Gestures should be built in so they are available to all users and available to all add-on developers. It would be even better if they were available to web apps as well, using some kind of whitelist if necessary (as I explained earlier wrt Growl).

Comment on the Coop

Incidentally, there is also The Coop. This was intended to ship with FF3, but dropped at some point. It’s an attempt to embed social networking and stumbleupon-type functionality. To me, this is exactly the sort of thing that really is too far removed and should remain an optional add-on. It’s actually tied to a single server run by Mozilla, which doesn’t seem right for an open-source product. It might make more sense if it was generic platform/API, with the communication protocol published and the server software also open sourced.

Fin

Firefox is by far the best browser today, already has some great UI improvements from 2.0 days, and I am pleased that v3.0 is now more stable and performant than ever. Now I’m looking forward to usability improvements and looking forward to seeing other people’s wishlists!

I’ve been thinking about what language and framework to use for my next hobby projects. There has always been a frustrating trade-off between language quality and ease of hosting. The languages I know best are: Ruby (+Rails), PHP, Java, and Javascript. So these will be my options.

What I’m looking for is a hosting environment which will scale well as I add new sites, each with their own domain and database. I want to deploy new sites almost with a single click, but realistically in an hour or less. And I don’t want to pay for each new site I deploy - in virtual hosting and cloud style, I want to be paying for overall consumption, not for the number of applications.

(Update: A couple of people suggested I should add Python, so it’s now here for completeness.)

Ruby

I say Ruby, but really I mean Ruby on Rails. The only other feasible frameworks for Ruby are Camping and Merb. The first is too simple and the second is too immature and lacking web host support.

Hosting for Rails has been a tragedy for me. I developed a site a year ago and tried to outsource deployment to Amazon EC2, and both suppliers gave up on the job. This was after I deployed it myself on SliceHost, requiring much effort, and then found it couldn’t handle the load. There are big Rails-specific hosts, EngineYard and RailsMachine, but they are very expensive and not practical for hobby projects. When I was working on Quizr, I also spent ages getting it deployed on RailsMachine; it was not turnkey as some people expect it to be. All I wanted was a server, a backup server, and a DB - so a fairly standard config. That’s not a criticism of RailsMachine, but don’t expect it to be trivial if you want more than one server (this was 18 months ago, it might be different now).

Rails is a super coding environment and would be a no-brainer for all of my projects, if only hosting was simple and reasonably priced. With a virtual environment like SliceHost, I found it takes several hours to set up a new site, and you also end up with a lot of config files to maintain (ie mucking about with apache modules and load balancing). You can also use a grid solution like Heroku. This is a fantastic way to keep things lightweight. The downside, though, is there’s always some little tasks you need that are outside the sandbox. e.g. setting up a cron job or adding some custom C library. Right now, Heroku doesn’t have a commercial model and apparently doesn’t let you use your own domain name. This rules it out for anything too serious.

I think the ideal for me would be a virtual Linux environment like SliceHost, with some VERY high level scripts that deploy a Rails app in a single click. These would have to be client-side scripts. I found Deprec is close to this and might get there at some stage.

PHP

PHP isn’t the best language, but it’s certainly easy to get something up and running. I find for small projects, it’s quite maintainable too, even without using PHP5’s OO. However, I’d like to use one of the Rails-inspired frameworks, e.g. PHP on Trax, in order to speed things up and ensure it’s easily maintained. This is where things get more tricky, because there are so many clones to choose from, and the community for each suffers as a result. As many have pointed out, Rails benefits from being the only serious contender in Ruby space, and the same cannot be said for any other server-side language. With PHP, you don’t need any framework at all, so in fact each framework is probably only used by less than 1% of PHP developers. In contrast, Rails is probably used by > 99% of Ruby programmers. Less community means less documentation, support forums, books, and information about hosting and capabilities.

Java

Well, I mostly mention this for the sake of completeness. Java’s an okay language for web development, especially if you choose your framework right, but hosting has mostly been restricted to enterprises. I find it funny, given Moore’s Law, how much Java hosting still costs in 2008, but it’s still pretty silly and not easy to just keep adding domains and applications without expense. Furthermore, I find it funny that Sun missed the boat on grid hosting for Java. So while there are consumer-focused solutions for grid hosting Rails and Javascript, no such thing for Java. On the whole, consumer-focused Java hosting is a cottage industry. Given that the language itself isn’t so brilliant, not worth the effort.

Where Java might get interesting is the VM. You could use it to host Groovy or Rails projects via JRuby. However, until it’s really, really, mature, I feel like JRuby is going to be one of those open-source things that sounds good on paper, but feels less enticing at 4am and with 26 open browser tabs pointing at pertinent blogs and support forums. I will reserve judgement for now.

Python

Python has a number of modern web frameworks, with Django being the most prominent. The fragmentation didn’t help, as compared with Ruby with its one gigantically popular framework, so it helped when Guido blessed Django. I’m sure Django (and others like TurboGears) is a good framework, maybe even better than Rails from what I hear. However, as I already know Ruby, I don’t have that much interest in taking the time to learn it - the languages and frameworks are similar enough that it wouldn’t be worth the effort. Moreover, I like the whole Ruby and Rails community and all the web presence and docs and books around it. It’s more fun and focused on the things I care about, like usability, aesthetics, and being opinionated. Also, there is simply a much greater volume of resources.

That said, Python has gained a lot of momentum due to Google App Engine, which exclusively hosts Python. But Google loves Java and I’m sure it will expand to at least support Java at some stage, if not more generic stuff like EC2. Providing a Java runtime would allow for stuff like Scala, JRuby, and Rhino.

Javascript

I consider server-side Javascript the holy grail over the next few years. I’ve explained why before - in summary, it’s the fact that you’re using the same language on both sides and the synergy that results. It’s not just that it’s simpler for you, the human programmer, who only has to speak in a single tongue. It’s more about the code you end up with. You can have a single module representing a web page, which contains both client and server code. It says “show the user two buttons. If they clicked button 1, do this; If they clicked button 2, do that”. So the code is very much combining client and server sides; and unlike high-level web-app compilation services, the code is actually what gets executed on both sides.

The main problem with server-side Javascript is immaturity. It will get interesting when we get Rails like models, the kind of stuff Steve Yegge has talked about. However, it shouldn’t be too similar to Rails; otherwise, you’d miss out on the synergetic effect described above. For now, though, server-side Javascript does suffer a lot in comparison with Rails, due to lack of a cohesive MVC model.

The main options for hosting are AppJet and Jaxer. AppJet is a nice, easy, grid solution, but doesn’t yet have enough info on commercial hosting and custom domain names. Jaxer has a lot of promise, but the Linux module will need to come out, be not too processor-intensive, and be supported by web hosts. There is also the upcoming Aptana Cloud solution, but this is similar to AppJet. For now, can’t host your own domain. These grid solutions are very handy, but I want to be able to release new apps and domains willy-nilly, and I suspect the grids, when they do become fully commercial, will impose a charge for each new app/domain.

A problem with the current opensocial gadget spec is that there's no relative path support. This means you end up hard-coding any references to Javascripts, CSS stylesheets, images, and services which are distributed along with your gadget.

This is not good. For example, you may have a "prod" setup and a "dev" setup. While developing, you will have to hard-code <script> tags and so on to hit the "dev" location, but then for production, you will want them pointing at your scripts in the production location. Likewise, you might want to ship the bundle to someone else for them to host on their own network.

You can deal with it using a pre-compiler tool - you know, using sed-like token substitution to piff in the right location. But a pre-compilation step is a messy compromise - I just want to distribute the code and be done with it. Another solution would be to output the gadget spec from a script, which would let you solve the problem in one of two ways: (a) inline common Javascripts and stylesheets so that you don't even need to link to them; (b) piff in the right location on the fly. Again, this is not ideal. It introduces a new server-side language and demands the spec server supports it. My aim here is simple distribution of gadgets - as long as you can host a tree of static files, you can serve my gadgets.

The optimal solution right now is far from ideal, but is the best way I know to do it. You use On-Demand Javascript (and on-demand CSS and on-demand <insert resource type here>). First, you work out the gadget spec location, then you manipulate the URL to find the required Javascript, and then you pull it down using DOM manipulation.

I'm using a compressed version of something like this:

It's partly based on a mailing list comment from Arne Roomann-Kurrik. Massaging further, I end up with the following boilerplate block which must be cut-and-paste into each gadget to bootstrap reuse:

After dropping this at the bottom of my gadget, I know that (a) ../core.js will be pulled down and executed - it is relative path from wherever the gadget came; (b) initialise() will be called once core.js has been loaded. Each of my gadget's has an initialise() function which does gadget-specific initialisation. core.js is a common library and also happens to include loadScript() in case I want to pull down further Javascripts, and also loadStylesheet() so I can grab a stylesheet with relative path.

All this is not at all ideal, for several reasons:

1. For all the benefit of neatly using a separate Javascript file, you have to cut-and-paste boilerplate code into every page! But at least the boilerplate won't have to be changed often.
2. Performance will suffer as the script is loaded later than it should be.
3. It relies on the "url" being passed into the gadget; to my knowledge, this is not something you can definitely assume will happen, i.e. it's not mandated by the opensocial standard that the gadget receives a parameter called "url" identifying where the gadget came from.

I have requested on the opensocial spec mailing list to provide direct relative path support and the response has been positive, hopefully we'll see it soon, thus rendering large parts of this post obsolete. The solution looks to be rather neat - injecting a <base> tag to establish a base URL which all relative paths work against.