What is OpenSocial?

I just realised there’s no good executive summary for OpenSocial. Every resource fluffs about the “social” aspect and forgets about the UI aspect. The UI aspect, essentially a gadget specification, is just as important as the social aspect, notwithstanding its absence from the “OpenSocial” brand name. I’m not a fan of the name “OpenSocial” for this reason, though I acknowledge it does have more attention-grabbing bling than “open social widget framework 1.0”. In any event, it’s regrettable that most commentators gobble it up hook, line, and sinker; as evidenced by their almost exclusive focus on the social angle.

With no further ado, here’s my attempt at a one-paragraph summary of OpenSocial …

OpenSocial Executive Summary

OpenSocial is a standard for “mini” web apps, called “gadgets”, which are designed to be embedded other websites. The standard informs developers who wish to write portable gadgets, i.e. those which work in any OpenSocial environment; and likewise, it informs developers who are creating such environments. It defines an XML format for applications, typically containing the user interface (HTML/CSS/Javascript) as well as metadata, e.g. author, gadget size, and preferences. In addition, OpenSocial defines a number of APIs, so that gadgets can invoke services such as changing the gadget height and fetching remote content. One particularly important class of APIs are those of a social nature. These enable a gadget to access data and run services related to the user who is viewing the gadget, or the user whose web page the gadget is embedded in. For example, the gadget can – subject to security settings – find a person’s name and location, traverse their social network, and invite them to install an application. Access to social data is not limited to gadgets, however, as OpenSocial also provides a RESTful variant of the social API, suitable for any kind of net-enabled client. OpenSocial was launched by Google in November, 2007, and is to be managed by the non-profit OpenSocial Foundation.

Where Do Widgets Come From? A Look at Widget/Gadget Content Types

Ajax, AjaxPatterns, Gadgets, Google, Web, Web 2.0, Widgets

Background

A while back, I walked through a Google Gadget I made called Digg Roundup, which simply shows Digg headlines and can be customised on topic and popularity. In my quest for an uber-simple tutorial, one thing I skipped on was content type, the subject of the present muttering. There are several content types possible, each with distinct implications for the page architecture and where the gadget sits within it. Below I’ll explain the options and help you understand how to decide between them.

A gadget is always expressed as some XML sitting at a URL somewhere on the net. When a developer uploads a gadget to Google, all they do is indicate their server location where the gadget is hosted. So a user’s iGoogle homepage is really just a bag of URLs (with layout and preferences for each). Anyways, the gadget is really a mini web page, and inside the gadget XML is a content tag that describes the HTML, CSS, and Javascript that makes up this page. The tag has a type attribute (i.e. <content type="xxx">) and there are three types available…

Three Content Types

html (<content type=”html”>)

All the page content is included in the XML file. It’s just like your standard web page HTML; it contains the (initial) HTML body from top to tail (sans enclosing <body> </body> tags), and CSS and Javascript, which can be inlined or linked, just like a normal web page. The content will be served in an iframe and Google will wrap its own html content around it. In particular, Google will look at other info in your XML file and output html code at the top, such as pulling in libraries and setting up preference data. Incidentally, Google won’t add any visual wrap around the widget at this stage; that is the responsibility of the container, which could be iGoogle, Ning, or any standalone web page that chooses to render the gadget.

Although the XML is held on your own server, the iframe will ultimately be served from a Google subdomain as its source. e.g. gmodules.com/blahblah. If you look at iGoogle, you’ll see the source is actually something like 50.gmodules.com/blahblah. The point of the “50” is that each gadget on a page comes from a different subdomain, thus isolating gadgets on the same page from each other for security reasons (there are ways around that restriction for consenting gadgets).

The main differences between a <htm> gadget and a normal web page is that your content must be static – the XML file cannot be generated on the fly by your server, because Google will cache it. Think of this kind of gadget as a single file that describes your gadget in isolation, which you could mail around or stick on a USB key. The dynamic behaviour comes from the fact that you can link to external CSS and Javascript (which could be generated on the fly) and, moreover, you will typically be making remote calls once the application is running (either on startup, due to a timer, or in response to system events).

url (<content type=”url” href=”myserver.example.com/…”>)

url. This could alternatively be called “external” or “hosted”. You simply provide a URL on your own web server and deliver up the gadget content from there. Specify “example.com” and the user will see an iframe pointing to “example.com”. The content for url gadgets must be dynamically generated, the exact opposite of html gadgets, which are always static. This is due to the way these gadgets use

html-inline.

This is no longer supported, but it’s worth knowing because (a) the contrasting model helps you understand the other models; (b) older gadgets, as well as Google’s own gadgets, are still able to use this model; (c) Google says they might allow special cases through in the future; (d) other containers do support this model (NetVibes anyway). In contrast to the previous types, inlined gadgets are not contained in an iframe; they are served as plain old HTML embedded into the fabric of the page. Just some content in a div.

A Note on Preferences

Persistent preferences are a key feature of widgets. Just a quick note that preferences work slightly differently on html vs url widgets (and inline, but I didn’t look into it).

With html widgets, the preferences are set as part of the HTML wrap around your widget’s HTML. ie some Javascript variables are set up.

With url widgets, the iframe source URL includes some CGI parameters expressing the preferences (e.g. up_storyAmount=5).

Either way, though, Google’s preference library abstracts this detail. As long as you declare a dependency on the library, it will be included and will let you access preferences the same way.

Decisions, Decisions: Which content type to use?

So that’s the definitions. Let’s look at the capabilities of each and understand how we might make decisions on which model to use.

Inline or IFrame (content-type=html/url)?

Firstly, inlined (html-inline) versus iframed (html and url), assuming inlined is an option for you. Living in the same DOM means that inlined gadgets can talk to each other; dead simple cross-gadget communication is a key benefit of the inlined model. Furthermore, gadgets and the main portal page can talk to each other. You can change the Google logo, for example, or put up a lightbox effect across the entire page.

Not too surprising that raw inlined gadgets were banned; the security risks are high. e.g. An inlined gadget that asks for your gmail credentials would end up storing username and password in global page space, which means a second gadget could simply read that data and upload it somewhere. You could also screw with iGoogle’s branding, which Google PR probably didn’t appreciate and could lead to phishing attacks. Google could rely on code reviews to minimise the risk of inlined gadgets, but that’s a very manual, unscaleable, approach, and users can in fact add gadgets from any URL (although they made it a lot harder to do that a few months ago; AFAICT you need to use the developer gadget in order to add a gadget that’s not in the catalogue). Maybe inlined gadgets will come back if a product like Caja proves itself worthy of automatically sanitising web apps. It’s a hard problem, but there are some good benefits to be had if widgets can live safely together on the same page and domain.

In summary, choose inlined only if you need the extra functionality available and only if you have assessed the risks of tampering from third-party widgets.

IFrame from widget server (content-type=html) vs IFrame from your own server (content-type=url)

Second and really the key decision you have to make with Google Gadgets today, html versus url. Reason to choose “html” content type over “url” content type:

  • Javascript API. You can use the full Gadget Javascript API with no effort – you don’t even have to include a script tag because iGoogle injects it for you. With the “url” type, using the Gadget Javascript API is a hassle – you have to dynamically generate the script tag, which is not only a bit of scripting effort, but forces you to generate your iframe from a script. Furthermore, cross-domain restriction means that even if you do pull in the API, you can’t use the remoting support (the _IG_Fetch* functions for proxying and caching).
  • Cross-gadget communication. You can run cross-gadget communication using the Pub-Sub framework.
  • Resources. Google serves it for you, which saves bandwidth and maintenance costs. In fact, you will have zero bandwidth costs if your widget functionality is either self-contained (e.g. a calculator) or only hits third-party services (as opposed to your own servers’ services).
  • Popularity. Pure speculation, but you’re probably more likely to be featured in Google’s gadget gallery as they can inspect everything about your application’s workings (except for any remote calls back to your server).
  • Inline-like. If inlined widgets do make a comback, you’ll have an easier migration path.

Reasons to choose “url” content type over “html” content type:

  • Full page refreshes. Because it’s coming from an external iframe, you can do traditional, non-Ajax, programming. ie. User clicks on link to see a new page. User submits a form. Page auto-refreshes. With this type of widget, you could write a perfectly working gadget without knowing a button of Javascript.
  • Personalisation and Security. You can make XHR calls from the widget directly to your own server and make use of any cookies that have been set up. e.g. If the user has already logged into your main web app, then you widgets will be able to present personalised data. With “html” widgets, you would only be able to make use of cookies using cross-domain JSON, and cross-domain JSON is unsafe. You could possibly do it more safely using the cross-domain iframe fragment identifier hackFacebook’s new JS lib works that way – but I haven’t investigated it.
  • You can easily host the widget as a standalone web page (though it’s largely pointless and will look silly unless you pull some CSS madness).

Want to Learn More?

Google Gadget API for Ajax Developers

I just made my first Google Gadget – a Digg roundup tool. Here are some quick notes on making a Google Gadget for Ajax developers. I assume you know Ajax and also that you’ve played around with iGoogle as a user.

What’s a Gadget?

  • The gadget is an XML file sitting on your server. In my case, http://ajaxify.com/run/widgets/google/diggroundup.xml. It will get cached, so effectively it must be a static file.
  • The user adds your gadget to their igoogle portal, or codes it into their own website, by specifying this URL (it may be done indirectly – via the gadget registry. You’ll appear in the registry if you’ve submitted your gadget to igoogle.)
  • The gadget is rendered as an iframe, so you have all the usual security constraints which stop you borking the portal and other gadgets. This also means you can’t communicate with other Gadgets other than via than remote calls to a common third-party server (or has anyone tried hooking them together using the iframe-fragment identifier hack? ).

The Gadget Content

  • The XML file contains three main sections: module settings, user preferences, and the iframe content.
  • Module settings (<ModulePrefs>) are the usual demographic stuff – module name, your email, blood type, etc. Mostly optional.
  • User preferences (<UserPref>) are things users can edit by hitting the Edit button. Note that these aren’t dynamic app variables like a user’s search query – they are longer-term preferences like background colour and number of results to show. Each time the user changes then, the entire gadget is reloaded, so you can assume in initialisation that the preferences have already been established, and you don’t need to bother checking if they’re changed. The nice thing about preferences is they’re declarative, so you don’t have to manage input and persistence. You just say “I need a number called resultAmount from the user” (<UserPref name="resultAmount" display_name="No. of Results" default_value="5" datatype="string" ></UserPref>). Later on, you’ll be able to interrogate a prefs object to get back the data.
  • The content. The content is pretty much the “body” part of an IFrame. ie. your html and … you’ll never expect this … the javascript. You probably expected the JS to be in a separate section of the XML file, but you simply include it using an inline script tag (<script tag="text/javascript">the script...</script>, wherever you fancy. The same goes for CSS – just include a <style type="text/css"> section at the start of your script or inline it. So much for unobtrusive Javascript/CSS, but you can and should still separate things out within your page content.

User Prefs example

This is what the user prefs looks like:

It comes from this definition:

  1. <UserPref name="bgColor" display_name="Background Color:" default_value="White" datatype="enum" >
  2.         <EnumValue value="White" />
  3.         <EnumValue value="Pink" />
  4.         <EnumValue value="Aqua" />
  5.         <EnumValue value="Lime" />
  6.         <EnumValue value="Yellow" />·
  7.         <EnumValue value="Orange" />
  8.         <EnumValue value="Black" />
  9.     </UserPref>
  10.    <UserPref name="container" display_name="Story Type:" default_value="" datatype="enum" >
  11.         <EnumValue value="" display_value="All" />
  12.         <EnumValue value="technology" display_value="Technology" />
  13.         <EnumValue value="science" display_value="Science" />
  14.         <EnumValue value="world_business" display_value="World Business" />
  15.         <EnumValue value="sports" display_value="Sports" />
  16.         <EnumValue value="entertainment" display_value="Entertainment" />·
  17.         <EnumValue value="gaming" display_value="Gaming" />
  18.     </UserPref>
  19.    <UserPref name="storyAmount" display_name="No. of Stories:" default_value="5" datatype="string" >
  20.    </UserPref>
  21.    <UserPref name="popularOnly" display_name="Popular Only?" default_value="true" datatype="bool" >
  22.    </UserPref>

and is accessed like this:

javascript

  1. var prefs = new _IG_Prefs(__MODULE_ID__);
  2. message.style.backgroundColor = prefs.getString("bgColor");

Remote Content

  • Your gadget probably takes remote content. Remember it’s a static XML file, so you can’t output such content on the fly – you have to use some kind of web remoting. Because of cross-domain restrictions, you will want to use a Cross-Domain Proxy. (I experimented with direct On-Demand Javascript, but no cigar.) Fortunately, Google provides a nice API for proxying, which also caches the content you’re grabbing. The main call is _IG_FetchContent(url, callbackFunc). The callback function is given the contents of URL. As mentioned in my previous post, this might cause some problems if you need to refresh the cache more than once an hour.

Development Tips

  • igoogle can give some pretty poor error messages when you’re just starting up. If there’s nothing at the URL you specify, you will get a weird XML parsing error rather than a good old 404 warning. I also found if you have certain errors such as a typo in your XML file, igoogle says the gadget doesn’t exist at that URL, even though in this case, it does! For those reasons, be sure to start with Google’s Hello World and work from there!
  • Develop on your server directly. (If the gadget is live, you might make a test copy of it for debugging.) You don’t want to be developing on your local box and continuously uploading/checking it in, because igoogle needs to fetch it from a URL each time you want to test it.
  • The secret sauce is this: The Developer Gadget. This is a special gadget provided by igoogle that, if you include it on your portal, will let you suppress caching of your app, which you absolutely have to do. It also makes it a bit easier to add a new gadget and view the gadget source code.
  • Speaking of gadget source code, you can view the entire gadget XML for any gadget you come across, so take advantage of that and avoid reinventing the wheel.

The Developer Gadget looks like this:

Looks Good, Tastes Good

The Search Engine Experiment – a blind test where users rate relevance of results – reveals that Google is better, but not that much better. The methodology is reasonable – the only serious flaw might be if people are assuming Google is always relevant, then trying to pick the Google results. Or if people go for Google because they’re used to it, so the results are the most comfortable. For example, when I tried the test, I jumped straight for the results that included wikipedia, partly because it just felt more pure and Googlish. It turned out to be a Yahoo! result.

Anyway, taking the results at face value, how to explain MSN and Yahoo! being more relevant than the grand-daddy of search 60% of the time? Seth has a good theory:

Google is better because it feels better and quicker and leaner and easier to use. The story we tell ourselves about Google is very different, and we use it differently as a result … Music sounds better through an iPod because we think it does.

cf. Nicholas Negroponte in “Being Digital”explains he always puts on his glasses to eat steak – it tastes better that way. (BTW “Being Digital” is the greatest tech book never to make it on Joel’s MBA reading list. A real mind-opener, like Philip and Alex’s Guide to Web Design).

So Google is a cognitive dissonance machine that actually has no clothes on? Hard to believe, but bring on more of these mashup experiments.

Google Base = Flickr + Odeo + Typepad + …

Google Base: It goes beyond classifieds. The popular view today seems to be that Google Base is all about classifieds, but as mentioned earlier, it’s much more than that. (Incidentally, still smarting I missed the “All Your Bases” happy fun humorous moment opportunity.)

If Google Base lets you tag entries, you could easily create a podcast feed by just uploading MP3s and giving each the same tag. Likewise, you could create a blog by uploading text entries with the same tag, and a vodcast with videos. Google could easily offer a wiki interface to let you manage all your content, and a feed to go with it.

From a programming perspective, what’s nice about all this is the prospect of a common API for all sorts of content.

So while Craigslist ought to be watching closely, so should many others. This is Google’s answer to Flickr and just about everyone else who’s hosting microcontent.

Google’s Microcontent Engine

Google’s apparently releasing base.google.com. While some see it as a Craigslist play, it’s a lot more than that: Google has seen the future, and the future is one big mashup of microcontent. “A” microcontent is a small chunk of information, typically with its own URL, but not actually a web page. It’s not a web page because it’s raw content, without being wrapped in navigation, disclaimers, etc. As some of the Ajax portals show, there’s a confluence here: Ajax allows for easy manipulation of microcontent. Tiddlywiki is based around the idea of microcontent, and allows you to conjure up some microcontent by clicking on a Microlink.

Classified advertising is the most obvious path to revenue, so the screenshot emphasises commercial applications. But a classified ads is just one special case of Microcontent. As they say: “Google Base is Google’s database into which you can add all types of content”. Sounds a lot like a repository of general-purpose microcontent.

Ladies and Gentlemen, get your mashup on! I’m betting on a Maps-Base mashup about 90 seconds after Base goes live. Google probably won’t bother integrating Maps themselves – they may as well sit back and see what works before buying one or developing their own solution.

In other Google-Ajax news, Jonathan Boutelle points out that Google’s foray into mobile is full-on motivated by Ajax: being offline is a big objection to migrating desktop apps, so Google wants to render that a non-issue.

Google: Edgy Minimalist or Choice-Deficient Simplist?

Don Norman questions the conventional wisdom on Google:

Anybody can make a simple-looking interface if the system only does one thing. If you want to do one of the many other things Google is able to do, oops, first you have to figure out how to find it, then you have to figure out which of the many offerings to use, then you have to figure out how to use it.

My first reaction was, “But the user’s always right!” If users think it’s clean, then it’s clean by definition. And it is users who sing Google’s praises, not just the usabilerati.

But when you think about all of Google’s other services, how many people really use Google for anything other than search? I bet people use a lot more of Yahoo’s services.

Take another careful look at Google’s front page. Want a map? You have to click once to be offered the choice, then a second additional time to get to the map page. Want to use Google Scholar to check references? Um, well, is that “Advanced Search” or “more.” What about their newly announced blog search? Why is Google maps separate from Google Earth? (Oh, those were purchased from different companies. Yes, but why should I, the user, care about the history of Google’s acquisitions?)

All of these things require you to click on “more” which gets you to the options page where there are 29 alternatives, plus links to “About Google,” “Help Center” (if Google is really so simple, why does one need help?), “Downloads” and then a special section on “web search features,” which has another 24 links of web features, a book search toolbar, and then another 23 sections of text — not links, text descriptions and an entire meta-language you can learn to improve the searches.

This made me think, why isn’t Google of all people playing the URL as command-line game with their own products? Sample searches:

  • “groups” – Yahoo Groups first, Google Groups 2nd.
  • “map” – MapQuest first, Google Maps 7th (Google Moon 90-something).
  • “news” – BBC first, Google News 5th.
  • “images” – YES! Google finally made it to no. 1! (Quick, get the SEO lawyers, Google’s cracked the Google algorithm!)

Now, this is all very nice and integral of Google to provide honest results rather than sprinkle in their own services. But what gets me is that they often provide no relevant link to their own service. I wouldn’t expect Google to show all of their services on the homepage, but I would expect it to point me to Google Maps when I search for “map”. And when I search for “Paris Map”, it’s nice that Google offers some images in the results, but I can think of something more relevant.