Alistair Cockburn’s Keynote at SPA 2005

Alistair Cockburn gave a 75 minute keynote on the game-playing model of software development.

Software is like Calvinball – you never play the same game with the same rules twice. (Calvinball is apparently based on the Calvin and Hobbes cartoon, where the characters improvise on the rules as they play the game. This was actually a question, Alistair agreed it’s a good analogy.)

Buzz-phrase: Project Management Declaration of Inter-Dependence (2005). Similar style to agile manifesto, but a bit more formal.

Developing software is making ideas concrete in an economic context. And it’s difficult: * The problem keeps changing * The solution keeps changing * The language is nothing like a human language, not very expressive etc. * The interpreter is unforgiving

Cooperative game manifesto

Different types of games. Infinite vs. finite. Competitive vs. co-operative. Goal-directed vs. No-fixed-end. Software is finite, goal-directed, co-operative.

Software project: Always preparing for the next game. So there are actually two conflicting games in one. Explains tension about documenting, for example. It benefits the next game, but not the current game.

People are the players in the game. Weak on: Consistency, discipline, following instructions, changing habits. Strong on: Communicating, looking around, copying/modifying. Can use this model, e.g. for documentation:it should get readers mind in a frame where they can look around, but usually don’t need to go beyond that – they can work it out themselves from the code as long as they know what code to look at.

Methodology as swimsuit: People and methodologies come in different shapes and sizes. But methodologies, somehow are often used as if they’re for everyone. Methodologies are just the conventions you operate by, and it makes sense those conventions should change as time goes on.

Flow of memes. Interventions often focus on ways to get memes flowing faster. Simple way is to put people in the same room. Can see the flow as a convection current. It’s like perfume! The closer you go, the more info flows. Also, morale flows in the same way – morale can go down quickly if people in the same room and some people start to feel down.

BNP Reconciliation Notes: Romilly Cocking at BNP Paribas

Romilly Cocking gave a 75-minute presentation about his work at BNP Paribas, using agile techniques to implement a reconciliation engine. Unusually for a bank, it’s fine with this information being public and Romilly said it would be fine to write about it here.

The Reconciliation Problem

Reconciliation: Comparing and contrasting two sets of records that should tell the same story. To detect fraud or error. e.g. A well-organised [ie [rare/fictitious] person would compare bank statement against chequebook. Note that we’re not talking about matching (subtle differences).

Steps in reconciliation:

  • Read
  • Select
  • Transform
  • Link
  • Compare
  • Report

SmartRec Development

Wrote SmartRec as a proof-of-concept. Thus, was able to choose the tools and techniques. Used: Java, Eclipse, JUnit, JMock, XMLUnit, XStream, Jaxen, SmartArrays. Methodology was agile. Basically, XP, but with one developer. Used stories; TDD; automated functional tests – easy in this domain because no UI, inputs and outputs are XML files; close interaction with user; weekly reviews.

Technical Issues

Used SmartArrays because it’s based on APL, which Romilly has a lot of experience with, and was confident it would provide a neat solution. Did encounter some impedance mismatch: Switch between two mindsets: thinking in arrays versus thinking in objects. Ended up exposing arrays more than required. So code too closely coupled to arrays. Realised mistake, but pairing would probably have caught it straightaway.

Specifying reconciliation definition in XML was ugly, so spent a few days and created domain-specific language for specifying a reconciliation. Worked well, although creates some impedance – must change translator each time the requirements change. However, it actually had the major benefit of helping to refine requirements.

Technical Issues

Users’ immediate reaction was they want a GUI, not the domain-specific language Ideal thing might have been Eclipse plugin with syntax highlighting, etc., but would take too long to develop. So the compromise was a wizard – quick to write – that generated the DSL code. Some discussion about the potential of using Lisp or Smalltalk to create the DSL as part of the language itself. The danger here is that users could do other things they weren’t expected to do.

Coaching Workshop Notes – Michael Feathers at SPA 2005

Michael Feathers of ObjectMentor led a three-hour workshop, “Coaching Software Development Teams”.

If you’re after a quick fix, jump straight to Michael’s patterns below.

The Nature of Coaching

Discussion prompted by definition of coaching as causing change. Coaching in the business community is more about fostering change, like counselling – don’t have to have the business knowledge. In IT, tends to be more about leading, provoking on design, etc. Is the coach part of team? In XP, yes; a bit like a sport coach, usually a player as well.

Coaching and Human Behaviour/Psychology

Coaching must build on model of human behaviour. “Agile about people, but people and human behaviour not often talked about”. Also, we rarely talk about whether people want to be working – many people treat it as “just a job”, don’t want to devote too much mental effort. Age and lifestyle effects too.

Stereotyping

Stereotyping and pigeonholing people was a major theme of this workshop.

Important to avoid stereotyping, try understanding reasons for behaviour. Two problems with stereotyping: * Warp the team’s perspective on someone – an individual becomes “the guy who can’t write test code”, etc. * Can be a self-fulfilling prophecy…people will see themselves according to the label and act accordingly. [Cognitive dissonance?]

Role-playing stereotypes exercise.

Being aware of emotions

Some research indicates men tend to be less capable of reading emotions, and this is probably even more true for software developers. Due to selection bias or reinforcement or both? Don’t know. But if that’s the case for a coach, then the coach needs to be aware of blind spots.

Seeing the group as an individual

Groups tend to act as a whole. Every class has its own personality. So it can be interesting to anthropomorphise the entire team as an individual, although Michael admits that this is a bit funky, not something that will resonate with everyone. A participant mentioned that when people leave, others ends up filling the same role. Which is interesting if true, because certain ways of doing things are systemic. This indicates firing someone for not being part of the team may not solve a problem because someone else might solve the problem.

Understanding motivations

Listed motivations for working in software – about 15 of them (achievement, money, learning, etc.)

How can such a model be used?

  • Understand impact on interventions. Point about agile is how it topples some of those, e.g. people’s sense of prestige and status can be destroyed if they feel like they’re just another coder again.

  • Hiring strategies. Michael observed that greater companies devote much greater care to hiring people, to ensure they fit in. They don’t just focus on the technical side, look more at collaboration skills. A participant noted this can be a problem for companies in growth phase – companies hiring too quickly often end up with different types of people.

Learning and Tension

See Ward’s Cunningham’s “Episodes” pattern language (especially the graph, apparently not online). Tension can be a tool. Ideally, let people learn the lesson themselves by resolving tension. So creating that tension, in the right way, can be good. Coaches should learn to feel the tension in the room, helps them find the right level.

Ethical Questions on Coaching

Should manipulations be conducted? Above board? One view, to paraphrase Kent
Beck, is that manipulation is when you wouldn't get the same effect if you
were upfront about the intervention. Or is there an issue about the coach
seeing themselves as more experienced, qualified, etc. Should they use that
knowledge to encourage someone to do something they wouldn't normally do,
because they see the genuine benefit? And the person may see the benefit in
hindsight, but not upfront. (Here I'm thinking the Karate Kid snatching flies
with chopsticks.) Ultimately, the distinction may come down to respect: a
care for the individuals and the outcome.

One view: Sometimes, it’s simply not possible to explain why we’re doing everything in advance. People can only learn some things by doing.

Patterns

Michael presented these patterns towards the end. Note: I missed a couple of these patterns!

**Go Sideways:** Look at another problem completely. Usually people are still
working on it in their background.

**Go Home:** Just stop working on it - go get some water, etc. We all have
experiences of the answer coming to us in the shower, while jogging, etc. So
can be good if team members are able to suggest taking

**Antennae Up:** Develop a sensitivity for what's happening. Focusing on the
human side. ** Gerald Weinberg: when called in for what's perceived as a technical
problem, it's usually a people problem. Participant: Can be useful to
keep a diary to order your thoughts, and even better to discuss it with
others. Writing is good for looking at things analytically.

**Pair Coaching:** Discuss with someone who knows the team. Can check
observations before acting.

**Ask the Room:** Encourage team to think hard when they feel like they need
to break a rule that's been adopted.

**Active Listening:** With minimal judgment. When it's not recognised,
*resistance* has been identified. Participants mentioned the virtues of
paraphrasing what's been said, and this can overcome listening based on the
stereotype. Also, it's about checking your own understanding. Sometimes,
coaches can have a great technical theory in the head, but the developer has
a real code that doesn't fit the textbook.

**Advance/Retreat:** Judiciously retreating from helping someone with a
problem and helping them take control of the problem themselves;

**Tending 'Pat':** If this team was a person, what would the person be right
now? Is Pat tired? Scared, relaxed?

**Personal Encouragement/Discouragement:** Most coaching is one-on-one.
"Know the person, know the feeling, feel it first, address." Will often be
wrong, but better preparation. Someone also mentioned getting beyond the
technical issues.

**The 'Flounce':** Identifying the elephant in the room (i.e. a big issue
no-one wants to talk about). Pointed questions, soliciting comments, ending
with silence ... and suddenly a stark, emotional, assessment of the problem.
Then "leave" the conversation (probably stay there, but quieten down and
withdraw in body language, etc.) Again, quite idiosyncratic (Used by a
former colleague of Michael. Sounds very dramatic!) The intention is to open
the floodgates and let people discuss it openly among themselves.

**Team Surgery:** Perform surgery on 'Pat'. Can be difficult due to politics,
etc. Adding a person forces the greatest change.

**Push in the Water:**  Sometimes have to ask people to confront their fears
and go beyond their limits.

**Self-Disintermediation:** Works well with Tension-Release. Encourage
people to help each other rather than go to the leader. Someone noted it's
difficult for consultants, because no-one wants to pay them anymore. (Maybe
that's why it doesn't happen so often - people like to see big
interventions.)

**Cheerleading:** Not necessarily overt, but identifying what's gone well,
which means relating back to goals.

**Cultivate Respect:** Sometimes people develop intimacy by complaining
about others. Coach's reaction makes a big difference.

Java Annotations Tutorial Notes – Benedict Heal at SPA 2005

Benedict overviewed the basics of Java annotations in a 75-minute tutorial. I was looking forward to this as I attended his Tiger session last year and it was a spot-on summary of all the main points. This tutorial likewise.

Many different types of comments

Including:

  • Historical info
  • State of code (e.g. TODOs, bug notes)
  • Behavioural expectations (assumptions)
  • Environment requirements
  • Algorithm descriptions.

Annotations are a big step beyond Javadoc tags

Java already has some structured metadata ([email protected] tag), but annotations let anyone create such structures. Exactly how they’ll be used is an open question, ideas are emerging. Some examples: say where help files are, what classes are required, input assumptions. The sort of thing you might have previously stored in XML files or simply written in prose comments and hoped someone would see them.

Annotations categorised according to associated info

Marker annotations

We might have a “financial” annotation, which would be used to log all methods:

@Financial public class Invoice

Or we might have a “test” annotation instead of JUnit’s naming convention (See TestNG):

@TestMethod void testSomething();

Beyond marker annotations

These are annotations with a parameter, e.g.:

@Help(url=”http://example.com/help”)

Don’t have to name the parameters – can use an array too. @Annotation(a,b,c)

Useful to use enums here – the value of a parameter can be an enum.

Java 1.5 offers three built-in annotations

@Override public String toSTRing(){} //Will complain because it’s not overriding.

@Deprecated public void oldMethod(){}

@SuppressWarnings(value={“unchecked”}) class DubiousClass

How to write an annotation?

Looks like an interface …

public @interface Security { String access() default “all”; }

Note: Because its an interface, you can’t have behaviour on it (unlike C#). But an interface is no good without some related code; otherwise it may as well just be a comment. Which leads to the question: where does the related behaviour go? There was some discussion here … A useful idiom is to sneak the class into the annotation as a default value. Or another trick would be to declare a constant class member.

Annotation Targets: What type of thing does the annotation apply to? What’s being annotated?

Annotations can declare what they apply to: packages, classes, methods, attributes etc. These are called “Annotation Targets”. And can use @inherited to ensure the annotation is inherited.

Meta-Annotations: Annotations about Annotations

Meta-annotations are annotations about annotations. For instance, @Documented marks whether to add the annotation to Javadoc. [Sounds overly complex for the fun of it, but apparently proving quite useful.]

Source, Classloader, Runtime: When and How is the Annotation Used?

Can transform source, can intercept classloading, can manipulate at runtime.

  • Source Code: Could write your own processor. But you can also use the Annotation Processing Tool (APT) ships with Java 5. For an annotation, implement an AnnotationProcessor which walks through source code, picks out annotation types, and outputs the code accordingly. Annoyingly, APT doesn’t let you read the method code, so you can’t do any manipulation there. Apparently, that’s a sticking point and will hopefully be resolved soon.

  • Classloader: Could write your own classloader, but BCEL or Javassist are tools that let you easily change classloading behaviour without doing so. To manipulate the Loader, you implement a Translater. Can then add the code. Apparently easier than you’d think (a toy example is actually very small).

  • Runtime: Can ask a class/method/etc about its annotations. Could be used in any way, as with regular reflection.

Open Issues

How we use annotations is still something of a mystery, and we’ll be learning a lot as they start to be used. For instance, what’s the point of a marker annotation such as @TestMethod. Isn’t the name alone enough to tell us it’s a test method. When you see a call to a method, you see it’s name but not the annotation.

AOP will benefit from a standard notation – the AOP/annotation combination may well be a good combo.

Some discussion here about how much they really offer, and is it enough given the great drawback that there’s a lot of potential extra complexity added.

JSR 250 is specifying common annotations and RAPT – https://rapt.dev.java.net/ – is building a library of annotations.

SPA 2005 Notes

I attended SPA 2005 earlier this week and tapped in some session notes while I was there. I’ll be posting each one in its own entry, coming up. As promised to workshop attendees, I’ll also be posting the slides for my workshop on writing self-documenting software.

SPA is a BCS group which describes its concerns as “good and efficient design, the positioning of new technologies, and the promotion of reflective, inclusive and balanced processes.”. It was previously the OOPS, but renamed to de-emphasise the OO. One nice thing is that the conference is quite small, 120 people, and many of them are presenting or running booths. I’d say most people are hands-on software architects and leads, and there was a good mix of freelancers, IT companies, corporates.

Amazon Needs to Make Better Use of Reviewers

Amazon has a couldabeen-great feature that lets you focus on a particular reviewer and see all their reviews. So you could, for instance, see what the alpha reviewers – those in the top 10 are saying.

However, it’s a bit pointless because there’s no support for wading through the hundreds of reviews each of these guys post. Check out the UK uber-dude, Jason Parkes, at 898 reviews and counting. I get ten per page, and there’s no way to sort them, filter them, or categorise them in any way. And when I say categorise, I’d settle for something as simple as separating DVDs from books.

In an ideal world, I’d like several features: * Filter on a category such as books or DVDs. This issue came up because I use Amazon’s DVD rental and was mining for recommendations. * Sort, just like in the usual site, on criteria such as popularity and price. * Sort or filter on the ratings themselves. Unless the guy’s got a really warped correlation with my own taste, I’m usually looking – not surprisingly – for higher ratings.

The Generator Pattern for Unit Tests: Math.random() Takes a Beating

Chris Stevenson discusses some pitfalls in testing with random numbers.

Here’s my advice on unit testing with random numbers: best avoid it and certainly don’t tie yourself to it. I’ll explain the flexibility part below. But First, let’s be clear on randomness. As Chris alludes to, random numbers in most programming environments are actually pseudo-random. Math.random() says: ” When this method is first called, it creates a single new pseudorandom-number generator”. That’s scary and confusing to many people, who assume random() means random. In fact, it’s just a sequence of number generated by a formula. Like the digits of Pi, for instance. You can create a much more random sequence by taking inputs from the physical world, like current, mouse and keyboard movements, or images of a lava lamp in motion. But would you want to?

random(), then, is simply a sequence of numbers. I’d rather have a bit more flexibility on that sequence. So instead of:

    nextInput = Math.random();

I’d rather say:

    nextInput = generator.generate()

which implies some kind of Generator interface:

     interface Generator {  // Not RocketScienceFolks
       Object generate()
    }

Then, you can have any generation strategy you like. Math.random() becomes a special case. But, by wiring up the unit test with a different Generator implementation, you can have any sequence of numbers you care to name. More to the point, a single unit test can contain many tests which vary only on the generator. A generator like this can be configured with a pre-defined sequence of numbers, or use some strategy such as an arithmetic sequence. You then have more control and understanding over how you’re systems being tested.

Here are four ways to generate test data:

  • “Truly” random numbers – Not replicable. Might work on someone’s desktop, then crash on a server. Painful. However, it does serve a purpose for some types of testing, such as stress and load testing. If you’re abso-positive-ly sure you’re systems working based on standard unit tests with replicable data, what’s the harm in trying out completely random numbers before taking it live? If the code structure is in place, it’s a cheap source of extra data to agitate your system with.

  • Pseudo-random numbers – Can’t see the use, unless it’s just an implementation of a Generator, and even then, I’d generally prefer an understandable sequence. If I really want random to mess with the system, I’d rather use true random, because otherwise I’m just using the same numbers again and again, which defeats the purpose. (For those developers who make the mistake of performing all their unit tests with random numbers, pseudo-random is at least a better approach due to replicability. Which leads to the happy double-negative that many people who actually want to random inadvertently use random() and are saved by the fact that random() isn’t random.)

  • Understandable sequence (such as an arithmetic or geometric sequence, modulo’d) – Very useful. I can reason about how long the cycle is before repetition take place, I can perform calculations on the sequence to determine expected answers. I know a lot more about what’s going on than some obscure random() formula that’s probably documented somewhere on the sun site. And in other languages, random() may well be unspecified and inconsistent across platforms. Also, let’s say I’m using the sequence “2,4,6,8” and I see a “6” in the stack trace. Great, that tells me straightaway when the problem occurred. If I’d used a random sequence, I’d have to go and log all the values to fish out when the “6” occurred.

  • Explicit sequence of literals (e.g. “1, 5, 6”) – Also very useful. This is, of course, a generalisation of the standard unit-testing scenario … specify an explicit input variable and verify the result. Sometimes, you just want to use a whole lot of numbers, which is where the sequence approach comes in handy. So explicit literals works well in tandem with an automated sequence generator.

IDE Wishlist: Library Installation for the Google Generation

Let’s say I’m coding in exploration (or “spike”) mode. Just tinkering a bit, playing around with some ideas. Or maybe experimenting with some new technologies. So it’s quite likely I’ll be downloading and installing some libraries. And I don’t want it to distract me … it should be as transparent as possible.

Even savvy IDEs offer limited support for this sort of thing. Maven can do some magic, but I don’t think it’s very well integrated into the IDEs (I can’t be sure though?). A few wishlist items:

  • Online import intentions When I add a class that’s not in the classpath, give me a list of all known matches in the world. Like what I’d get if I performed a Jarhoo search. ** A “library” database** Similar to the Idea’s plugin manager, which is immensely helpful as it automatically installs plugins, it would be possible to integrate a browser and searcher similar to Jarhoo.
  • Automatically download libraries Triggered by an intention or direct command by the developer of the library.
  • Drop ‘N’ Go Directory As with plugins and deployable components in many containers, let me specify a directory where all JARs are automatically included in the project. That way. I don’t have to go through the configuration hassles.

ITunes Bloopers: Duplicate and Missing Songs

ITunes has two serious usability bloopers in its handling of the music database.

Duplicate songs

It’s very possible you’ll end up with the same song twice. Perhaps you forgot you’d already imported it. I’ve been having problems with IPodder pushing songs into ITunes , so I’ve been manually dragging results of a Recent Files query, leading to some duplicates in the overlapping dates.

Until recently, ITunes did nothing. Now, I’ve just installed 4.7 after being seduced by Apple’s promotion of its “Duplicate Songs” feature:

Eliminate Duplicate Tracks Easily Ever import a “Greatest Hits” album only to find that you already have one or more of the songs in your iTunes music library? Now we offer a handy way to resolve that little complication. iTunes 4.7 makes it easy to find multiple copies of the same song. Just choose “Find duplicates” from the Edit menu, and iTunes will display all the duplicate songs in your music library, making it very easy for you to decide which of the duplicates — if any — you’d like to delete.

Easily??? Nope. Find Duplicates really means what it says: it shows all the versions of the same file. So you’re left with all the twins an triplets in the same list. As in:

  • DSC 1 – Adam Curry
  • DSC 1 – Adam Curry
  • ITC 1 – Gillmor Gang
  • ITC 1 – Gillmor Gang

You get the idea. How do you “eliminate” those tracks??? The “handy way” Apple offers is to take a shovel and get to work, because you’re going to be deleting them one-by-one. According to this IPodLounge, that’s a feature and not a bug:

Apple’s feature was designed more to prevent mistakes than to annoy you. It’s much safer to delete songs one by one, and really be sure you want to delete them, rather than delete a whole bunch of songs and regret it later. Other programs may work better or worse, depending on your needs.

That’s only speculation, but if it’s really true, that’s just wrong: the computer is much better at performing a tedious operation like that than the human. Design it right, test it thoroughly, and it will work fine. It’s a trivial algorithm after all. In contrast, ask a user to manually delete duplicates and there will be boredom at best, tears at worst. Remember there’s not always two items; sometimes there will be three or four. So you can’t just go into a trance of “delete, yes, down, down”. You have to watch the whole thing. BORING!

More to the point, duplicates should never be allowed in the first place. The best designs prevent errors, rather than worrying how to recover from them. That’s the logic behind disabling buttons that just don’t make sense at that time. When you import a file, it should prevent it from being included if the location already exists. Not rocket science.

Missing Files

A related problem occurs when files go missing. After some unspecified time, ITunes notices and offers “?” icons. Thanks, better than nothing, but you know these are meant to be some pretty big databases … could you let me filter on those anomalous files, so I can work out what to do with them. Just treat “underlying file exists” as another attribute like Date Added, and then it can be incorporated easily into the View Options.

“But You’re Trampling on its Model of Simplicity”

… I hear you cry. Actually, duplicate files can happen to even the most basic of users, and there’s no threat to ITunes’ simple model by preventing them in the first place. Missing files is a slightly more “power user” function, since it implies you’ve been playing with the file system. Still, ITunes works by synchronising with the underlying system, so it’s still in order to provide some support for dealing with this problem. A simple filter would do nicely.

Around the Amazon in 80 Milliseconds

If you shop at Amazon.co.uk, you’re often out of luck when it comes to reader comments. So I often find myself editing the URL, switching back and forth between .co.uk and .com. Luckily, this transatlantic adventure usually works out, as the crazy Amazon IDs match.

Still, I can’t help thinking Amazon ought to drop in comments from the various Amazons, or at least from those that speak the same language. It’s a lot of extra data for free. And they could happily point out the edition it came from, just like their “this refers to the hardcover edition” disclaimers.