Thinking in Web Apps

Thinking In Web Apps is a short list of design principles for Chrome Web apps, published a couple of weeks ago by several of us in Chrome Developer Relations.

Many people think Developer Relations means blogging and speaking. That’s part of it, but it’s also important to be spending time with developers and understanding the challenges they face, as well as supporting them. In the case of Chrome Web Store, I’ve been working with several partners who are building apps in time for the store’s launch. Explaining how certain technologies work and taking questions back to the core Chrome development team. One of the things I’ve discovered in Developer Relations is the way patterns/principles emerge:

  • A new capability is announced, e.g. a new programming language, an upgrade to a hosting service, a new API. In this case, it’s the Chrome Web Store and the concept of installable web apps.
  • We make some guesses about how to use it and share them with developers. You have to see it as educated guesses; it’s the law of unanticipated consequences that says you can never be sure how people will use a capability, even if you’re the one who designed it.
  • Developers start to build use the new capability.
  • By aggregating across all of the pioneering developers, and talking to other developer relations people working with other pioneering developers, we gain a new appreciation of what works and what doesn’t.
  • We have more general info to share with partners. Wash, rinse, repeat.

The important thing is to be documenting what we’re learning as we go along, hence you can expect to see more articles like Thinking in Web Apps. Whether you want to call them patterns, principles, or whatever.

See also Task-Artifact Cycle.

It starts with a Yellow Fade: The need for a more comprehensive understanding of visual effects on the web

Around the time Ajax got coined, one of the already-known patterns was 37Signals’ Yellow Fade Effect. As techniques were shared and visual effects libraries emerged, we began to see visual effects become commonplace on the web. I documented four of them in Ajax Design Patterns: One-Second Spotlight, One-Second Mutation, One-Second Motion, Highlight. (I wish I called them something else, e.g. “Spotlight Effect”.) And all of these are around today, and even in the jQuery core which makes t hem pretty darn accessible to any developer. It’s as simple as $(“message”).fadeIn().

Where we are now, most developers are stuck with trial-and-error. In an ideal world, you’d have a SWAT team of UX experts to analyse each visual effect, test with users, and maybe they’d get it right, but most UI design in the real world and developers often just have to play around a bit and then move on to something else. So I think we could do with some simple guidelines, ideally based on empirical data, but even just based on pattern mining the web would be handy.

I’ll give some examples of the kinds of problems I’ve faced as a developer building in visual effects:

  • Choosing Visual Effect. I want to update some content that’s just been received from the server. e.g. the football score changed. Do I show a yellow highlight, make it blink, make it vibrate, do nothing?
  • Choosing Effect Parameters. I’ve decided to use a Slide Down effect. How longshould it last, i.e. the duration of time from not shown to show? Should it slide down at constant pace or accelerate and bounce at the bottom?
  • Composing Effects I have a block of content with the user can edit. When switching it between edit and view modes, there are two simultaneous effects taking place – one thing is being hidden while the other is being shown. How do I handle those? Do I slide one up and when that’s done, slide down the other? Do I fade one out and simultaneously fade one in? Do I immediately close one and transition in the other, or vice-versa? Many options come into play.

It should go without saying that there’s no right answer for any of these problems. But instead of developers always working from first principles, I think there are some very useful guidelines and discussions that could be had around the design patterns involved. Maybe even leading to a high level effects library supporting those patterns.

Paleosocial Patterns

Wolf Logan’s talking about paleosocial patterns (TOOLS Web 2.0 Patterns workshop). Very cool.

Marking – making permanent, public, marks. Dog marking its territory, “I was here” graffiti. e.g. blog comments, forums, guestbooks. Being obnoxious gets the comment noticed more.

Pointing – indicating resources to others, can be with opinions. Reputation will dictate how much people care. e.g. Reviewing/rating sites, Digg.

Competing – comparing self to others, ranking is dynamic Some people approach competition for their own value; others for their social value. Special case of marking – marking yourself – but with an element of objective measurement e.g. online gaming sites, xbox live, popularity contests.

Storytelling – describing sequence of events, often about yourself Different stories for different social groups e.g. blogs, journals, youtube

Grooming – small, ritualised, social interactions – can happen quickly and without much planning, Culturally bounded. Repeated frequently depending on relationshps. “Purring” – e.g. “how are you?” – don’t care, but it’s the purring that happens that makes it a grooming behaviour. The info transfer doesn’t matter, just that the event happened. e.g. friending, twitter, majority of IM content

Hunting – tracking, locating, acquiring resources. Serendipity plays a big part. Successful hunting leads to pointing – tell people what you found. e.g. web search, link following.

Wolf points out hunting and grooming are always the two critical patterns that are noted.

A Video Sharing Website based on Web 2.0 Patterns

We’ve done a little group exercise at the TOOLS web 2.0 patterns workshop. Designing a video website, with heavy emphasis on mashing up, e.g. show videos from youtube, grab recommendations from netflix, grab friend lists from Facebook, etc.

Here’s the full feature list (click for full image):

And the design:

We eventually identified features, at back end and front end, and listed similar sites where on might get inspiration for each feature. Interestingly, when you do this with web 2.0 sites, you end up realising you could actually implement the feature by actually using this service.

Dragos has transcribed the list we came up with (partly visible in the photo above):

Day 1: Design Exercise

Problems + Solutions (1) [MM - these were back-end features]
-Key-value store: Amazon Dynamo/S3, BigTable (Google)
-Streaming: Hulu, YouTube streaming (not Flash)
-Client time synchro service: NTP, MMORPG
-Social network integration services: Oauth, OpenID/Passport
-Mash-up engine: PopFly, Yahoo! pipes
-Monetization (social cash -> $, analytics, instrumentation): AdSense, witkey.com, SiteMeter, Elance

Problems + Solutions (2) [MM - these were front-end features]
-Video: Vimeo, Nicovideo.jp, Jaman.com
-Mood: Musicovery, Yahoo! IM, Nabaztag.com
-Friend states: Twitter
IM: Facebook/MySpace, Twitter, Shoutbox
-Recommendations: Tuneglue, Netflix, Facebook

Some Emerging Themes
-Taxonomy of social networks
-Social profile aggregation: InfoCard
-Integration of services transparent from user
(Tim O'Reilly’s innovation by assembly)
-Privacy (who sees what)
-Biggest pile wins
-Cooler than thou 

Tools Europe – Pre-Workshop Preparation

I have the honour of delivering a keynote at the Tools Europe conference in Zurich next week, as well as at the “Mining Web 2.0 Patterns” workshop, which, as you can tell from its title, I’m looking forward to participating in. The workshop chairs are Dragos Manolescu and Joe Yoder.

The keynote is on OpenSocial, Gadgets, and Design Patterns; and I’ll have more to say about it later. I believe it will be recorded too. For now, I’m using my blog to post my response for the pre-workshop preparation.

Sites and materials relevant to my presentation

Questions:

  • Which features make the gadgets easy or hard to use, and how might the gadgets be improved?
  • At UI level, which best practice patterns are apparent?
  • What changes to the platform would improve user experience?

Sites and materials relevant to other workshop tracks

Site Navigation:

Services:

Designing Like a Pollyanna: Have your Cake and Eat it Too

“The novel’s success brought the term “pollyanna” (along with the adjective “pollyannaish” and the noun “Pollyannaism”) into the language to describe someone who is cheerfully optimistic and who always maintains a generous attitude toward the motives of other people. It also became, by extension – and contrary to the spirit of the book – a derogatory term for a naïve optimist who always expects people to act decently, despite strong evidence to the contrary.”

– Wikipedia entry for Pollyanna

A little while ago, I was talking to some colleagues about an authentication problem. I suggested we need to improve its usability and my colleague’s immediate response was to argue there’s a security-usability trade-off, i.e. by improving usability, we must resign ourselves to decreasing security. Push one lever down, the other goes up. I’ve never bought into this argument. While certain forces do often conflict with each other, they don’t always, and it’s counter-productive to constrain your solution space with immediate presumptions like that.

Clarke Ching triggered this post with his posts on Non-Zero Sumness and having your cake and eating it. The latter is an example of a twerpy category of fallacious sayings and assumptions in this world which are based on scarcity mentality. For example, the notion that you have to “divide the pie” so that a gain for A is a loss for B. The alternative (as a certain presidential candidate was ridiculed for pointing out) is to make the pie higher. Whoever invented multi-storey buildings figured this out a long time ago.

(BTW “You can’t have your cake and eat it too” is a ridulous premise; what else are you supposed to do with your cake? I see from the wikipedia article that the original phrasing “wolde you bothe eate your cake, and have your cake?” is the reverse and makes more sense – you can’t eat your cake and still have it.)

Fortunately, we have sayings and phrases which remind us that it doesn’t have to be a trade-off. Unfortunately, these terms have been over-used by the proverbial “MBA suit guy” that we’re not allowed to use them without offering an apology. e.g. “win-win” and “synergy”. (Both terms I use without apology. I will stop short of “gestalt” though.)

All the above is theoretical rambling – here are some real-world examples.

  • Security-Usability Biometric authentication allows people to enter places without typing a password or carrying a card, and is, generally (and theoretically) speaking a more accurate indication of a person’s identity. Neither security nor usability suffers from this technology; both have been enhanced. (Privacy advocates will argue there is still a trade-off at stake.)
  • Safety-Usability As with security, safety is often assumed to be at odds with usability. In our work on safety-usability patterns, we sought synergies which would ideally improve both at the same time. For example, the pattern called “Behaviour Constraint”, sometimes referred to as a “forcing function”. In some buildings, when there’s a fire, a barrier goes up on the stairs leading to the basement, to ensure people don’t keep running down there (example by Don Norman). The wheels of a plane can’t be raised if the plane isn’t moving (to prevent a pilot from actually raising them while on the tarmac). These are examples which make life easier for users, by reducing the number of decisions they have to make, and at the same time, improve safety by blocking the transition to dangerous states.
  • Ajax-Accessibility When Ajax emerged, some people had a knee-jerk response – “Ooooh! Javascript! Bad for accessibility!”. In some cases, it was true. In many other cases, though, Ajax actually improves accessibility. e.g. Ajax makes it much easier to let people choose font size and colour scheme.

Knowing that certain forces trade off against each other is still useful as a kind of broad-brush stereotype. At a meta level, it’s a kind of general software engineering pattern to say “security and usability trade off against each other”. (It’s a pattern in the sense that if you looked at hundreds of software projects, you’d see people frequently dealing with this trade-off.) I would like to explain such a pattern to a first-year software engineering student, so they can more easily reason about how these things work. It’s also useful in scrutinising a design; I would be quite comfortable ending up with a design with “high” security and “low” usability, or vice-versa, after exploring all the options, because I know these principles do usually trade off. And for that reason, I’d consider it somewhat unrealistic for a client to expect a feature to have both “high” security and “high” usability, and would only engage on such a quest after explaining to said client that the effort has a high chance of failing to meet these criteria.

So these trade-offs are useful and are my defence against the second part of the Pollyanna definition above – “a derogatory term for a naïve optimist who always expects people to act decently, despite strong evidence to the contrary.” I’m not ignoring those trade-offs, I’m just saying they shouldn’t be an automatic assumption. It plays to a higher-level rule about design and creativity: Ignore constraints at first. In too many cases, people are afraid of going down certain paths because they can immediately detect an obstacle down the road. I once tried producing a novel design with a domain specialist who would immediately shoot down most any idea I had on the basis that users would never work with it. Once I bypassed him and got to the users themselves, the situation was quite different; the users embraced the concept and gave further suggestions for improvement. The domain specialist, who had long since working as an active user, was applying constraints too early and preventing us from proving the concept (“proving” == “bashing into shape”). In many other cases, the barrier is technical – you come up with an idea and it’s immediately blocked on the basis that it can’t be done with current technology. (The sort of limiting belief you would have heard if you proposed Google Maps in 2004). In the same way, when we automatically assume “improving X will diminish Y”, what we’re doing is limiting our options prematurely. Better to ask first “is there a way I can improve both”, or, failing that, “is there a way I can deliver the critical attribute X without diminishing Y too much?”.

The only thing wrong with GoF Design Patterns is …

Jeff Attwood recently pointed out the difference between Gamma et al’s Design Patterns and Alexanders’ equivalent and outlined a critique of the former which characterises it as “replacing actual thought and insight with a plodding, mindless, cut-and-paste code generation template mentality”.

First, I want to note that the critique above surely defies belief to anyone who has used the patterns in practice. Patterns like the GoF design patterns are grammar for software architecture. If you search the technorati archives from when grammar emerged many millennia ago, you will probably find one or two bloggers was bemoaning the loss of creativity that this concept of sentence structure will cause. Fortunately, sense prevailed and now writers can be creative about the things that matter and not have to micromanage those that don’t. The progression of any discipline involves a cycle experimentation followed by optimisation followed by explicit statements of best practice (patterns), spiralling upward in level of abstraction. If I was so inclined, I’d go make a T-Shirt on CafePress that says “Patterns Take You Higher” with a picture of Bob Marley refactoring to Template Method.

So the GoF patterns are perfectly choice by me. The main problem with them is not the content per se, but the fact that most software developers think that pattern==GoF pattern and pattern language==the 23 patterns by Gamma et al. (1995).. If there’s ever a sequel, it would double the number of patterns in the universe according to these folks! In fact, there are thousands and thousands of documented design patterns out there, in the realm of software alone. (Admittedly, the most popular realm for design patterns, though there are still plenty of patterns for other disciplines too.)

This perception is silly.

Why do software patterns != GoF patterns? We can expand out from the GoF patterns in several dimensions (even staying within the realm of software):

  • Architectural Paradigm Even with patterns of high-level software structure, like GoF, we can model different architectural paradigms. GoF is all about general-purpose OO architecture; if you wanted to work with functional or logical languages such as Haskall or Prolog, you would need different kinds of patterns.
  • Application Type GoF patterns apply to the general structure of any medium+ scale architecture. There is still a wealth of advice to be gained from studying existing applications of the same type as you’re working on. This could be web server design, in-browser script design (like some of the Ajax patterns), mobile phone design, etc. For instance, the various J2EE patterns around, which were probably the second best known category of patterns in the industry (trailing GoF by a long margin though).
  • Perspective People see software from different perspectives; GoF patterns see it from that of a technical architect working close to the ground. You can also see patterns from the perspective of a user experience designer (usability patterns), project manager (software development process patterns), business analyst (business process patterns), programmer (coding idioms), etc. In this paper, we explained how patterns for usability alone could have all sorts of representations.
  • Domain The GoF patterns apply whether you’re working in banking, gaming, or web design. But each of those fields have their own idiosyncracies and fortunately there are pattern languages which capture them.
  • Opinion Even if you aim for the same problem space as Gamma and colleagues looked at, you will have your own views on the correct solutions. Patterns are opinionated, not objective, so your patterns will differ from theirs. As time goes on, any static set of patterns will naturally become less valid – it’s a testimony to Gamma et al that the patterns remain so very useful 12 years on (that’s a 256-fold increase in processing power according to Moore’s Law, and industry transition from C++ through Java and .Net through to Javascript, Ruby, and Python). Still, they’re not perfect; Singleton anyone?

One of the reasons the Gamma patterns are so successful is the fact that they apply so universally. However, this also leaves room for other patterns to come in and provide more specialised support.

Ajax Patterns has certainly been subject to this misconception; one review liked the content but prefers not to see it as a real pattern book – “Anybody who has read a real design pattern book and then read this book will soon feel the artificiality. ” Another refers to Gamma et al as “I read the original ‘Design Patterns’ book” and states “Those that read the original book will understand that packaging these topics as design patterns is insulting to those that know the value of actual design patterns. “. After many years of explaining usability patterns, I had anticipated these concerns with an appendix explaining the difference, but moreover explaining that it’s the content that matters most. There is also a perception that “Patterns” in the title sells more copies; all I can say is “I wish” as that buzzword went out the window around 2002. It’s called “Ajax Design Patterns” because they are, well, Ajax Design Patterns.

Bill Sanders gets it and recently discussed this phenomenon:

Buried in Appendix C (pg 605), Mahemoff distinguishes his use of design patterns from that of GoF, but it is not the kind of place that most developers are going to look when they pick up a book about Design Patterns. Likewise, Mahemoff uses the inside front and back covers to list the Ajax design patterns. On the one hand, anyone who reads those pattern names is going to know that this book is not about the kinds of design patterns GoF examine. On the other hand, the fact that the patterns are listed there, just like the GoF book, is a sincere form of flattery in the form of imitation.

Bill makes a good point about the Appendix C content being buried. Actually, relating the second point back to the first, I had in mind the similar statement by Gamma et al about Alexander’s patterns and their relevance, which also appears at the end. Still, in a pattern book, I’m not sure where to put something meta that won’t be buried; I’m sure the preface and intro are read about as much as the appendices. And I’ve certainly discussed the issue online before.

As for the covers, I liked the idea from GoF as well as Fowler’s UML Distilled book, and was particularly insistent on this because O’Reilly couldn’t do page cross-referencing.

Mahemoff is quite right in noting that the “original” design pattern book is Christopher Alexander et al’s town planning and architecture patterns book from the 1970s.(In fact in the Foreword to GoF’s DP book, Grady Booch mentions Christopher Alexander’s work.) However, common usage of design patterns in the context of computer programming has come to mean the kinds of patterns GoF talks about. Head First Design Patterns is a good example. No one was disappointed because they were surprised by the meaning of design patterns. Common usages in a given context come to have a specific meaning. In the world of quilt-making, design patterns mean something entirely different than in programming, and so no one is surprised when they get a book called Quilt Design Patterns and find out that there’s nothing about the Creational, Structural, Behavioral Design Patterns. However, in the world of programming, that is exactly what can (and did) happen in the case of Ajax Design Patterns, despite the fact that no less than Ralph Johnson reviewed parts of it.

(Emphasis mine) Yep, if you published a book of music patterns, there’d be no confusion. Still, I think our industry misses out because of this confusion. In an ideal world, there would have been 2 or 3 equally big hits at the same time, to get people’s mind around the fact that patterns are everywhere, not confined to the one (albeit excellent) text (I was going to call it “bible” but the one time I met John Vlissides, he told me he didn’t like it when his colleagues called it that :)). As it is, there’s the big ‘un, a few prominent others – typically in similar “high-level architecture” style to that one (e.g. the J2EE books, the POSA series and some of Fowler’s work), a bunch of books which used “pattern” for the hype value, and thousands of other books and one-time papers and blog entries. At one stage, Ward Cunningham was going to work with Microsoft to build PatternShare, a kind of meta-repo for all things pattern, which may have changed things, but apparently it never happened and the site is now down.

Quite apart from the book’s title, Mahemoff, does take up the invitation to contribute to design patterns offered up by GoF and he documents a number found in Ajax. While on a different level and with a different specific meaning than that found in GoF book, Mahemoff provides a wider view of design patterns and has what may be the best Ajax book on the market. It would be a shame if his work is diminished by the title, but among some, especially on the higher end, that may happen. Fortunately, the bulk of the reviews are readers who were looking for a good Ajax book and got exactly what they wanted.

This is where things get even more confusing, because the Ajax patterns (as with just about any other form of software pattern) actually relate to he GoF patterns. They’re different, but not entirely orthogonal. For instance, the daddy of all UI patterns, Observer, is present in patterns such as Virtual Workspace. My attitude is that patterns are there to be referenced and so I will happily reference other patterns from other languages if need be. In my mind, it’s all one big network of patterns (a PatternShare wiki, if you will).

Podcast: Thoughts on Patterns

Stream-of-consciousness thoughts on patterns – where we’re at in 2007. Not a tutorial. Definitely not a tutorial.

  • Where did patterns come from? Not GoF, not Alexander.
  • Alexander, patterns, and architecture.
  • Patterns in the organisation – how can a pattern language pervade an organisation and help workers carry out its mission statement and operating principles? Different high level patterns (e.g. “People first” for one company, “Technology first” for another) mean different lower-level patterns (crude example: staff pattern in people-oriented firm – “managers come from HR” versus staff pattern in tech-oriented firm – “Managers become tech leadership”)
  • Wikis were literally built for patterns
  • Promoting and deploying patterns

“Ajax Design Patterns” – Book of the Month

Ajax Design Patterns is Book of the Month in this month’s .Net mag (p.23, Issue 155, October, 2006). Incidentally, the mag is about the ‘Net, not specifically MS .Net (which it pre-dates).

<

p>The review says:

So AJAX might be the hottest thing in programming since, er, ordinary Javascript, but it’s no good just learning how to implement it – you need design inspiration too. Ajax Design Patterns fits the literary void that exists in AJAX design by using real examples of best practice to enhance your apps.

 

<

p>I’m glad they emphasise use of real examples, because we can debate ad infinitum about whether everything in the book is a pattern or not, but the more important thing is that the examples are real, concrete, and as accessible as typing a URL into your browser.

Thankfully, Ajax Design Patterns is one of the most organised books on any programming subject. It’s a massive book, but you won’t get lost as the chapters are sensibly divided up and the sound layout means there’s nothing whatsoever to fear.

<

p>I’ve had a lot to say about presentation of patterns in the past The fairly unusual presentation of the patterns is the reason it’s not an O’Reilly animal book, and it’s good to see it helped.

Rails, Selfishness, and Opinionated Patterns

When people talk about their favourite benefit of Rails over framework X in language Y, they’ll usually mention ActiveRecord, Ajax support, etc. But at a deeper level, the thing that really stands out is that Rails is opinonated software. This is where Rails derives its power and agility.

It’s pretty well-understood in software that consistency trumps piecemeal optimisation. Consistency comes from opionated software, piecemeal optimisation comes from design-by-committee. By piecemeal optimisation, I’m talking about something that’s full of good little chunks of local functionality, but falls apart when you use it because everything works differently.

Stated in another way, opinionated software is task-driven, whereas design-by-committee tends to be technology-driven. When DHH says the sky is now red, he’s selfishly refactoring things in such a way that he can work more effectively. I was about to slap quotes around “selfishly”, but the truth is, it really is a selfish act. And therein lies the benefit – it’s often better for me, as an end-developer, to work with a framework that one man (being DHH) has produced for himself than to work with one produced by a committee of agents with a variety of potentially conflicting interests. This, of course, assumes that DHH is talented and that he works on similar things as me.

Caveat: The above is a very crude approximation of the truth which ignores the contributions of many talented core developers, contributors, and feedbackers. And of course, I don’t mean selfish in a malicious sense, I mean it in an “enlightened selfishness” sense.

What’s particularly interesting about opinionated software is that it mirrors the subjectivity of patterns that has been discussed for years, by Richard Gabriel, Jim Coplien, and others (see Patterns of Software). While patterns are sometimes thought of as the endpoint of rational, wiki-ideal-like, trial-and-error, they are actually highly subjective. Indeed subjectivity is a key distinguishing factor between pattern languages and pattern collections.

To appreciate the importance of a pattern language, it is necessary to comprehend the subjective basis of pattern languages. Far from being the objective and exhaustive catalogue of ideas they may initially seem, patterns are based heavily on an underlying set of values. They explain how forces are identified and resolved according to certain principles; in doing so, they are encapsulating a particular approach. Alexander identified, valued, and discarded patterns in a process which embodied his own architectural philosophy (Kerth, 1997).