Rolled My Own Ajax Search

I just added an Ajax Searchroll with Rollyo, the new search engine that lets you “roll your own” (get it?) search. For example, click on the following link to search for “xmlhttprequest” on my selection of Ajax sites:

[http://rollyo.com/search.html?q=xmlhttprequest&sid=2220](http://rollyo.com/search.html?q=xmlhttprequest&sid=2220 )

(The 2220 ID corresponds to “Michael Mahemoff’s Ajax Search”.)

Rollyo lets anyone add up to 25 sites, and then constrains search results to those sites. Best thing is the site contains a link that lets you easily add a particular search to the Firefox search bar.

The Ajax search consists of links from the publicly-editable (nudge nudge) Ajax Links page as well as a few others worth searching. Please add any further suggestions here or on the links page. Given the constraint of 25 or under, I’ve limited it to Ajax content sites and very-Ajax blogs. Due to rollyo restrictions, the site must also have its own (sub-)domain rather than being qualified by a pathname (so I couldn’t, for example, add Scott Isaacs at http://spaces.msn.com/members/siteexperts/).

Rollyo wishlist:

  • Combine SearchRolls – e.g. there’s already a couple of other Ajax searches on there, it would be nice to search all at once. You could even give more relevance to sites listed in both. Maybe exclude a searchroll as well.
  • More than 25 URLs! Fair go, ref!
  • Sync URLs to some other source, e.g. a delicious tag, or in my case, the AjaxPatterns Links page.
  • Integrate as A9 columns, start.com gadgets, etc. They already have plans for Dashboard widgets, so they’re probably on to it.

It will also be interesting to see how Yubnub integrates with it. Also of note is the number of “high rollers” high-profile bloggers and celebs who’ve established their own Searchrolls (given that, based on my uid of 1882, there are under 2000 users). Looks like a nice example of guerilla marketing.

Google: Edgy Minimalist or Choice-Deficient Simplist?

Don Norman questions the conventional wisdom on Google:

Anybody can make a simple-looking interface if the system only does one thing. If you want to do one of the many other things Google is able to do, oops, first you have to figure out how to find it, then you have to figure out which of the many offerings to use, then you have to figure out how to use it.

My first reaction was, “But the user’s always right!” If users think it’s clean, then it’s clean by definition. And it is users who sing Google’s praises, not just the usabilerati.

But when you think about all of Google’s other services, how many people really use Google for anything other than search? I bet people use a lot more of Yahoo’s services.

Take another careful look at Google’s front page. Want a map? You have to click once to be offered the choice, then a second additional time to get to the map page. Want to use Google Scholar to check references? Um, well, is that “Advanced Search” or “more.” What about their newly announced blog search? Why is Google maps separate from Google Earth? (Oh, those were purchased from different companies. Yes, but why should I, the user, care about the history of Google’s acquisitions?)

All of these things require you to click on “more” which gets you to the options page where there are 29 alternatives, plus links to “About Google,” “Help Center” (if Google is really so simple, why does one need help?), “Downloads” and then a special section on “web search features,” which has another 24 links of web features, a book search toolbar, and then another 23 sections of text — not links, text descriptions and an entire meta-language you can learn to improve the searches.

This made me think, why isn’t Google of all people playing the URL as command-line game with their own products? Sample searches:

  • “groups” – Yahoo Groups first, Google Groups 2nd.
  • “map” – MapQuest first, Google Maps 7th (Google Moon 90-something).
  • “news” – BBC first, Google News 5th.
  • “images” – YES! Google finally made it to no. 1! (Quick, get the SEO lawyers, Google’s cracked the Google algorithm!)

Now, this is all very nice and integral of Google to provide honest results rather than sprinkle in their own services. But what gets me is that they often provide no relevant link to their own service. I wouldn’t expect Google to show all of their services on the homepage, but I would expect it to point me to Google Maps when I search for “map”. And when I search for “Paris Map”, it’s nice that Google offers some images in the results, but I can think of something more relevant.

Ajax + Multi-User = Chat

Ajax, AjaxPatterns, Chat, CSCW, Forums, IM, Patterns, Web, Web2.0, Wiki

Meta – new Ajaxian forum software – has been released (via Ajaxian). Like wikis, forums will look and feel very different in a couple of years.

Multi-user systems like wikis and forums and Groups/Clubs have traditionally communicated on a longer-term, asynchronous, timeframe. If you write something and someone replies immediately, that’s a surprise.

Ajaxify any system that takes contributions from its users, and you have the potential to get people communicating in real time. For forum software, imagine how the following would make it more like a chat system:

So it’s easy to see how a forum could easily turn into a chat among online users. It could equally apply to E-Commerce sites that lets people leave reviews or comments in blog entries.

Is this a good thing? Slashdot, for example, prohibits posting more than once every two minutes. Firstly, it’s to cut down on abuse. But secondly, it’s presumably to cut down on plain old noise. I certainly wouldn’t advocate holding back the technology just to force people to think longer, but designers will need to think carefully about how they help readers delineate – at a glance – between a conversation spanning several days and containing lots of insightful reflection, versus a “MS Sucks”-”No, Linux sucks more”-”I love Apple” flamefest that lasted all of 5 minutes.

New Ajax Demos

Further to the previous post on new Ajax programming patterns, it’s also worth noting there’s a new bunch of corresponding online Ajax demos too. Being programming patterns, they tend to be “pure refactorings” rather than enhancements, thus the user experience is mostly the same – you have to look under the covers to see what’s changed. The demos are mentioned in individual patterns, but here’s a sampling:

  • The wiki has a bunch of timeout-related refactorings. These refactorings are user-visible. (They’re not actually part of the programming patterns, but they were recently added.)
  • 2 other wiki refactorings involve On-Demand Javascript . (No UI change.)
  • A JSON playground.
  • The Portal Drilldown Demo has been refactored to render with XSLT and browser templating tools. (No UI change.)
  • A couple of refactorings added to demonstrate REST (broken right now, probably due to a PHP config issue) and RPC on the Ajax Shop demo. (No UI change.)

As with all the demos, they’re tested on FF and IE6, please let me know if anything breaks.

18 New Ajax Programming Patterns

I’ve uploaded full text for 18 new Ajax patterns, completing a first-cut draft for all Programming Patterns content, which will be one part of the book. This section bridges the gap between the very basics of Ajax – XMLHttpRequest, DOM, etc – and the high-level stuff like widgets and visual effects. For instance, how do you do design the services accessed by XMLHttpRequest? Or how do you deal with a lots of Javascript? The focus here is on traditional tech concerns such as maintainability and understandability; and less about usability.

Breaking it down:

  • Web Services patterns are about overall browser-server architecture – what kinds of services and what kinds of messages? It’s relevant to the conversation of Jon Tiersen and others about what sort of responses are possible – XML, JS, HTML, JSON. It’s also relevant to another recent conversation, on [http://www.theserverside.com/news/thread.tss?thread_id=35979 Ajax and webservices], as it covers REST and RPC.

  • Browser-Server Dialogue covers a mixed bag of patterns about XMLHttpRequest handling and what you can do with it, such as loading JS on demand.

  • DOM Population is about transforming incoming XML into DOM content, e.g. XSLT.

  • Performance Optimisation covers a variety of performance patterns such as caching and pre-fetching.

  • Breakout (can you think of a better name) is about going beyond standard Ajax constraints, namely using a server-side mediator to access external domains and introducing a plugin to go where no Ajax app is allowed to go.

  • Code Generation and Reuse is strongly related to some of the emerging frameworks like Echo2 and SAJAX.

This is all early days, so any feedback on the structure, names, or content will be taken into account and acknowledged too (really!). OK, here’s the lot …

Programming Patterns

Web Services

  • RESTful Service Expose web services according to RESTful principles.
  • RPC Service Expose web services as Remote Procedural Calls (RPCs).
  • HTML Response Have the server generate HTML snippets to be displayed in the browser.
  • Semantic Response Have the server respond with abstract, semantic, data.
  • Plain-Text Message Pass simple messages between server and browser in plain-text format.
  • XML Message Pass messages between server and browser in XML format.
  • JSON Message Pass messages between server and browser in Javascript Object Notation (JSON) format.

Browser-Server Dialogue

  • Call Tracking Accommodate busy user behaviour by allocating a new XMLHtpRequest object for each request. See Richard Schwartz’s blog entry (http://smokey.rhs.com/web/blog/poweroftheschwartz.nsf/d6plinks/RSCZ-6CEQAR).Note: Pending some rewrite to take into account request-locking etc.
  • Distributed Events Keep objects synchronised with an event mechanism.
  • On-Demand Javascript Download Javascript as and when required, instead of downloading it all on page load.

DOM Population

Performance Optimisation

  • Fat Client Create a rich, browser-based, client by performing remote calls only when there is no way to achieve the same effect in the browser.
  • Browser-Side Cache Maintain a local cache of information.
  • Guesstimate Instead of grabbing real data from the server, make a guesstimate that’s good enough for most user’s needs.
  • Submission Throttling Instead of submitting upon each Javascript event, retain the data in a local buffer and upload it periodically.
  • Explicit Submission Instead of submitting upon each Javascript event, require the user to explicitly request it, e.g. submit upon clicking a button.
  • Multi-Stage Download Quickly download the page structure with a standard request, then populate it with further requests.
  • Predictive Fetch Anticipate likely user actions and pre-load the required data.

Breakout

  • Cross-Domain Mediator Allow the browser to communicate with other domains by server-based mediation.
  • Richer Plugin Make your application “more Ajax than Ajax” with a Richer Plugin.

Code Generation and Reuse

  • Ajax Stub Use an “Ajax Stub” framework which allows browser scripts to directly invoke server-side operations, without having to worry about the details of XMLHttpRequest and HTTP transfer.
  • Server-Side Code Generation Automatically generate HTML and Javascript from server-side code.
  • Cross-Browser Component Create cross-browser components, allowing programmers to reuse them without regard for browser compatibility.

Aside I was interviewed yesterday for a Japanese magazine about how I’m using the wiki. So maybe some people will be interested to know that I always write the patterns offline because I am more creative in Vim and also to avoid textarea hell. (So much for Writely, will anyone create Vimly, there’s gotta be more money it than cloning MS-Word online :-D). I also use the Mozex or ViewSourceWith extensions to make partial edits using Vim.

This time, I decided not to upload once or twice at a time; but instead all 18 at once. There’s serious overhead of introducing each new pattern to the wiki (from Vim, the sequence is: 2mins spell-check, 5-10mins markup massaging, 2 mins fixing the link and description on the homepage; sometimes exacerbated by server lag.) Uploading all at once at least allowed me to focus fully on the task and also made some aspects more efficient, particularly updating the homepage.)

Must Have 5 Years “Ruby on Rails” Experience

If there’s one certainty in this industry, it’s the ongoing hilarity of job ads. David Heinemeier Hansson found a job ad that mentions “Rails” in the same breath as “enterprise” and “whitepaper”, and “struts” in the same breath as “latest and greatest”:

Should have ideally 5 years experience with all of the latest and greatest tools out there. The more the better. J2EE, struts, Ruby on Rails, Websphere, etc. Should have experience constructing Enterprise wide Java solutions from whitepapers.

“Why yes, I just pushed the Rational Rose Activationate button and Rose generated the Enterprise wide Java solution for me, along with the whitepaper. I didn’t even have to tell it anything first. Does that count?”

David’s reaction:

Rails has truly broken through the awareness barrier when its included as part of a job application that’s the antithesis of everything Rails is about. Clueless recruiters of the world, I salute you!

The “Must have 5 years Java” requirement, at a time when Java was barely out of the Sun Labs, have become legendary. There can be no finer way to attract dishonest candidates than to construct an infinitely high hurdle. I’ll donate a Pragmatic Rails text to whoever spots the first serious (as in seriously misguided) Rails equivalent.

Will Ajax be a Household Word?

Alright, it already is a household word, but how about our Ajax? Will it too become a household word? I’m seeing a trend that it will be.

Some terms should never really have escaped the tech world, but they somehow did. For example, Java and Flash. Okay, maybe not quite household, but well beyond the realm of techies and beyond that of gadget-obsessed teens and buzzword-obsessed managers. If a phone is “Java-enabled”, that means it probably has some games with graphics you’d be proud of in 1985. If a website has Flash, it’s got some fancy graphics and animation.

Monitoring the blogosphere, there’s an increasing trend of Ajax, Ajax, Everywhere. Non-techies are beginning to pick up on the term. And whereas Web 2.0 was previously a vague mixture of feeds and sharing and folksonomies and long tails and podcasts, a lot of people are homing in on the most salient aspect of Web 2.0: the <drum roll please> … Web. While I think all those other aspects of Web 2.0 are also vital, they’re quite frankly more abstract. Podcasts may be the most concrete thing about Web 2.0 – if you present it as “Ham Radio for the Y2K+” or some such – but you still have a lot of explaining to do about RSS and portable player integration and why you don’t have to have an IPod.

Okay, so post-techie early adopters are starting to wake up to the fact there’s an Ajax revolution going on. What does that mean?

I’m not sure. What do you think it means? A few ideas …

  • Once people begin to get conscious about this, the effect will snowball. You’ll see Ajax in business mags and journalists will begin to write about how companies can benefit from internal Ajax websites, which will create demand on intranets. The effects of Ajax are far-reaching, because Ajax is the converging point of the two biggest genres: web apps and desktop apps. Most of the apps in both genres are Ajax-bound.
  • If you’re a programmer, learn Ajax now. I’ve been encouraging a few students and grads to learn JS and DOM backwards, because clients are getting fat. Pure UI in JS – the ultimate presentation-model separation. Learn it now and walk into all the Ajax jobs out there in 12 months. Or hold off and compete against those who spent the past 12 months working it, as well as the 500 others who have flicked through “Advanced Ajax in 60 seconds”. (And learn Ruby too, but that’s a mostly unrelated story.)
  • If you’re a public website, better think long and hard about Ajax. GMail versus Hotmail? Google Maps versus MapBlast etc? Leapfrogging anyone?
  • If you’re the leader in desktop apps, think even harder. The MS perspective is interesting. One thing MS has never been is complacent, as they showed last time round when Marc Andreeson told reporters how the web would make Office obsolete and they’re showing with Ajax too. We’re starting to see where Ajax fits on Microsoft’s Atlas (sorry) and it looks like they’re embracing the other browsers, but perhaps the server needs to be MS. If that’s the case, I don’t think there’s much for people to complain about, but with Ajax apps getting more and more desktop-like each day, there’s still room for things toget nasty. Regarding Office vs Writely and others, MS’s best bet is to fully embrace Ajax and at the same time exploit the full power of the desktop for the standard version. That’s costly maintaining both versions, but if they don’t follow a dual strategy, they really have no case over a decent Ajax offering.
  • If you have an idea, now’s the time.

Cross-Domain Portlets with Start.com Gadgets/Widgets/Startlets

Scott Isaacs on a new Start.com feature (quoted on Ajaxian):

DHTML-based Gadgets: Start.com consumes DHTML-based components called Gadgets. These Gadgets can be created by any developer, hosted on any site, and consumed into the Start.com experience. The model is completely distributed. You can develop components derived from other components on the web.

Remote portlets take the Ajax portals to another level. In fact, the whole concept is one of the biggest developments in the history of the web. A9 pre-empted it with their “Open Search” capability, which allows developers to hook into the A9 interface with live search results. Now MS is going a step further, with Konfabulator-style interaction directly on the web. It’s likely Yahoo has similar plans, given that it’s now the owner of Konfabulator. What Google does is anyone’s guess. Strategically, MS has a big edge here because of its experience working with Developers, Developers, Developers! (YES!)

The idea combines [Cross-Domain Mediator](http://ajaxpatterns.org/Cross-Domain Mediator) with Portlet. Those patterns have subsumed a now-deleted pattern that previous covered this kind of thing:

Cross-Domain Portlet: Introduce Interactive Portlets to facilitate a conversation between the user and a third-party, e.g. for services or “advertising as conversation” (http://radar.oreilly.com/archives/2005/05/remaking_advert.html)

Ajax makes it possible because there’s no page refresh. Conceptually, each portlet is autonomous and capable of conducting a conversation with another domain in parallel with other portlets. When a result comes back to one portal, the other portlets are unaffected. (You can obviously yoke portals together if you want to.) In practice, there are a couple of finer implementation points:

  • You can’t make an XMLHttpRequest Call to an external domain. Hence, the portal server must mediate.
  • Conducting parallel conversations, you’ll need some form of Call Tracking. Hopefully, you’re using a library that abstracts those details from you.

Now that a happening, here are a few things I’d like to see:

  • An open standard for conducting cross-domain conversations, one that protects the end-user and the portal host, but provides sufficient functionality to the
  • Useful ads (the advertising as conversation) concept. There’s definitely a business model here for opt-in advertising. That is, users would be willing to add advertising portlets that offer genuinely useful information. e.g. a travel portlet that lets you check flight availability.
  • RSS aggregator integration The nature of a portal can change, right? So RSS aggregators should be able to include remote portlets.
  • End-user-created portlets. Make it easy for end-users to construct their own gadgets. This idea is inspired by Charlene Li’s comments on Konfabulator: “The problem is I already know what the ultimate widget would be me … There’s only one small problem — I don’t know JavaScript! So I am at the mercy of app developers …”

Sometimes the world is ready …

Researching JSON-RPC for the “JSON Message” pattern, I came across this interesting slashdot posting from January 24, 2005, a few weeks before “Ajax” was coined

Seen those funky remote scripting techniques employed by Orkut, Gmail and Google Suggests that avoid that oh so 80′s page reloading (think IBM 3270 only slower) … Now is the turning point. Forget that horid wait while 100K of HTML downloads when the application just wanted to update one field on the page. The XMLHttpRequest object has made it’s way into all the main browsers with it’s recent introduction into Opera and Konqueror (sans the Konqueror bug). This new form of web development now works on Internet Explorer 5, 5.5, 6, Mozilla, Firefox, Safari 1.2, Opera 8 Beta and Konqueror 3.3 (with a much needed patch). (Emphasis Mine.)

Ben and Dion @ Ajaxian also mentioned in their recent podcast they had to stay up late in Las Vegas reworking their TSS presentation because Ajax had just been coined and matched their content.

Sometimes the world is just ready for something.

A recent podtech podcast had a VC mentioning how he’ll often hear a great proposal for the first time, then hear the same idea from five different startups within two weeks. It reminded me of this ridiclous-but-true dotcom-days story from 2000, where a journalist wrote an April Fool’s story about a startup offering free cars with advertising on them. Several real companies meanwhile were planning to do just that, and legend has it that they sweated upon encountering the article. But here’s the punchline:

FreeCar’s Mr. Butler purchased the FreeWheelz joke Web site for $25,000 from Esquire and Mr. Fishman, who split the proceeds. “They got a lot of hits to their Web site,” Mr. Butler reasons. “I have access to their database and prevented anyone else from buying it.”

Aussie Geek Humour: Qantas Server and Kangaroos Who Fight Back

A little Aussie geek humour. Apparently Joel Spolsky named a server “Qantas” in honour of the airline’s immaculate safety record – the server that never crashed. As Jeff Attwood alludes to in that article, Rain Man is what made Qantas’s record famous.

Further Aussie geek humour. I was recently reminded of this classic software tale. Short summary (the way I heard it): military contractor codes a simulation of battle in Aussie outback, decides to add animated kangaroos at the last minute, implements them using solider objects, but forgets to disable soldier behaviour. Once the overhead chopper begins firing, the kangaroos dive to the floor and promptly retaliate with a torrent of missiles, hilarity ensues.

I’ve heard that story from a lot of sources. I can’t give it anything more than “urban legend” status, though a browse through the Risk Forum suggests stranger things have happened.