Mocks, Stubs, Dependency Injection, and … XMLHttpRequest

“A Mock Is Not So Stupid After All!”

Dave Crane’s been talking about Mocking the Server-Side:

A Mock Object is a stand-in for the real thing. Few modern programs are really standalone, and enterprise apps require a very complex context in order to operate; containers, databases, directories, web services, etc. This can make testing difficult, because to set up the test and run it, one needs to provide all the necessary context. Mock objects can be thought of as really dumb implementations of a contextual element, one that would never be useful in production, but provides the predictability and ease of setup required for a test. A mock database might always return the same four rows of data, whatever the WHERE clause. A mock web service might always serve up the same XML document.

So the idea is to create a simulation XML document (or plain-text document) to be retrieved with XMLHttpRequest.

Dave’s followed up with some good tips to that end. All good timing as I’ve recently been working on a demo for the final lot of patterns – the process patterns – and one of the refactorings uses a canned plain-text document to illustrate The Pattern Currently Known As “Server-Side Simulation Stubs” (who said alliteration had no place in software writing?).

Worth noting the terminology used, and here I’ll be pedantic. Dave refers to a Mock what I call a Simulation Stub. In the past couple of years, the terms “Mock” and “Stub” have often been used interchangeably, but I think the distinction is worth keeping, because otherwise the original meaning of Mocks is lost. I don’t really care what things are called, but when two concepts become one name, at least one of those concepts is going to get hurt. It’s a testament to the creators of Mocks that they managed to produce a name so evocative that it’s come to consume a much older concept. (“Stub” and “Shunt” have always been pretty vague terms anyway.)

Under the original definition, a Mock is an object that verifies how it’s being used by a tested object, and by inference, how things like it will be used by the class being tested. Yes, it might spit out simulation data, and it might even have a fancy interface that lets you tell it how to respond. However, those functions are only there to help verify how it’s used. It’s what goes into the mock that counts, not what the mock pumps out. Corollary: a Mock is not stupid. It might not win a Nobel Prize anytime soon, but it’s smart enough to know what to expect and how to check if that expectation’s been met. The mixing of terms has carried over to some parts of the Ruby/Rails community, as Aslak recently observed during a RubyConf presentation. (Podcast on mocks, Fowler’s Mocks Aren’t Stubs.)

In a web context, what might the distinction mean?

  • A Simulation Web Service (or a “Stub Web Service”, but I prefer “Simulation”) is typically some XML or plain-text document that’s been written manually. It could also be a bit more sophisticated, offering a means for the server-side developer to tweak its output.
  • A Mock Web Service is a server-side script that’s been configured to expect a certain sequence of XMLHttpRequest messages, and will carp as soon as the sequence is broken.

As Dave’s articles explain, a Stub Web Service is certainly a useful thing, and I suspect quite a few developers follow that practice. (Please tell me about it if you do!)So how about Mock Web Services? That’s pure speculation!! I could see the benefit, but I’ve not done it, haven’t heard of anyone doing it, and I seriously doubt there are any frameworks that facilitate it.

A variation on Simulations and Mocks would be a completely browser-side approach, in which you adopt an XMLHttpRequest wrapper, and make it output a dummy document while in “testing” mode. This raises further questions:

  • How are mocks handled in Javascript? Well, William Taysom recently noted that you can approximate method_missing, and a Mock library like the awesomely useful JMock library is clearly feasible. There are a few JUnit equivalents out there, so why not a mock lib? (Would you use such a thing?) If you say that would be over-engineering, then the same charge could be made about JS testing in general, since mocks are really a natural consequence of unit testing and TDD … any time you find yourself testing an event mechanism, a mock might be a good idea.
  • How to switch context in Javascript? Okay, so mocks are a big reason for the rise of Dependency Injection and “lightweight” ((http://www.almaer.com/blog/archives/000846.html)) containers in Java/J2EE/JEEWhiz. Well then, doesn’t Javascript need the same thing? Again, I know, “you’re over-engineering it, Mahemoff”. So maybe I am. It’s been said that Dependency Injection doesn’t do much for Ruby, and maybe the same’s true for JS. So how then, do you make your browser app switch between a mock web service and a production service? The answer is not, “change the source code”, because to achieve continuous integration, you want to keep an automated test around that calls the mock service. Maybe the best thing to do is rely on DI – or some other context-switch mechanism – in the server, so that the Javascript that it spits out will differ according to the environment.

Ajax, not “AJAX”: A User-Centered Definition

Many equate Ajax to “Is it using XMLHttpRequest?”, which I think is taking the acronym too literally. There’s a reason why I’ve learned to say “Ajax” rather than “AJAX”: the term is user-centric, not techno-centric, and best defined in terms of what it gives users rather than how you deliver it. And what it gives users is a rich, continuous, experience to rival the desktop, with standards-based technologies.

Jesse James-Garret, who coined the term, doesn’t define Ajax in terms of specific technologies. Here’s what he said in the recent Ajaxian.com podcast interview: “In my opinion, Ajax refers simply to using browser-native technologies, open standard technologies, in ways that depart from the traditional interaction model of the web – the kind of call-and-response interaction model where every user action is tied to some kind of server communication, and while that server communication is going on, no user actions can take place. Any time you’re decoupling the flow of user interaction with the application from the flow of server communication, and you’re doing that with browser-native standard technologies, I think that’s Ajax.”

If you can pull off a rich browser client using HTML 1.0, you got Ajax! In reality, you’ll end up using a mix of technologies suggested by the acronym (“Asynchronous Javascript and XML”) and some other things too (“DOM/DHTML/CSS”), but not necessarily all of those things. Having peeked under the covers of various high-profile Ajax sites, it’s clear that many are using custom-format messages, rather than XML for transfer. Similarly, XMLHttpRequest is not the only valid means of transfer. Google Maps uses IFrames, I’ve come across various enterprise systems that do the same, and I expect other Ajax apps do it too (and still on the lookout for examples BTW).

New Patterns: XMLHttpRequest Call and IFrame Call

Full drafts are now online for the two big Web Remoting patterns: XMLHttpRequest Call and IFrame Call. There are also a couple of new demos to illustrate GETting and POSTing with both call types: XMLHttpRequest Example, IFrame Example.

A few extracts follow, but first, let me ask you: Do you know of any public IFrame remoting examples? Google Maps is the only public example I know. All the other examples here are tutorials and framework demos.

The XMLHttpRequest alternatives was the most fun thing here (headings shown below). I went through the various techniques I’ve seen, read about, and tried out, to list as many ways as possible to programmatically call the server. As you might expect for these “Foundational Technology” patterns, there’s a lot more code in the Solutions than in later patterns, where the focus shifts more towards the -ilities, particularly usability and maintainability.

As usual, feedback much appreciated. (Mainly on the content, rather than woeful spelling and outright grammatical disasters at this stage.)

XMLHttpRequest Call

Solution: Use XMLHttpRequest objects for browser-server communication. XMLHttpRequest is a Javascript class capable of calling the server and capturing its response. Just like a browser or a command-line web client, an XMLHttpRequest issues standard HTTP requests and grabs responses. Note: The code shown throughout this demo is closely based on an online companion demo (http://ajaxify.com/run/xmlHttpRequestCall/).

The above example hopefully illustrates that the fundamental technology is pretty simple. However, be aware that it’s a very basic usage, not yet fit for production. Fundamental questions remain:

* How can XMLHttpRequests be created?
* How does an asynchronous call work?
* How do our code detect errors?
* What if the service requires a "POST" or "PUT" request rather than a "GET"?
* Which URLs can be accessed?
* How can you deal with XML responses?
* What's the API? 

The following sections address these questions and show how the code above needs to be modified. …

Decisions

6.1 What kind of content will web services provide?
6.2 How will caching be controlled?
6.3 How will you deal with errors?

Alternatives:

9.1 IFrame Call
9.2 Persistent Connection
9.3 Richer Plugin
9.4 On-Demand Javascript
9.5 Import XML Document
9.6 Image-Cookie Call
9.7 Stylesheet Call

IFrame Call

Solution: Use IFrames for browser-server communication. IFrames are page-like elements that can be embedded in other pages. They have their own source URL, distinct from their parent page, and it can change dynamically. IFrame Calls work by making the IFrame point to a URL we’re interested in, then reading the IFrame’s contents once the new content has loaded.

A Few Ajax Gotchas At Jalecode

Andrew Sutherland offers a few Ajax Gotchas/Tips. I’ll add some comments.

  • Escape content with encodeURIComponent() which is superior to escape.

  • XMLHttpRequest’s readyState tells you how far the request has progressed. MM: If you’re confused about readyState‘s transition from 0 to 4, you have good reason to be. Read the recent posting and comments on David Flanagan’s blog, and you’ll learn that 2 and 3 are ambiguous to the point of being unusable. Essentially, you want to wait for either 4 or timeout, and probably ignore everything else.

  • Permission Denied” for XMLHttpRequest is usually due to trying to call another domain. MM: The standard security policy is that requests can only be sent to the originating server, just like the traditional policy for Java applets. To get to another domain, you can set up a Cross-Domain Mediator. This security issue has become interesting with the growing popularity of Single Page Applications (SPA). What can an HTML page sitting on you hard drive access? All domains or no domains? It would certainly be convenient if it could access the web at large. I don’t think it can access any domains on standard browsers, but it’s still possible if the user wants it to happen. Here’s what Steve Yen (TrimPath) says on this issue: “I’m shooting for now to have explicity user-driven synchronization working, which my experiments lead me to believe is workable.”

  • MM: Finally, I’ll add another gotcha-inspired tip to Andrew’s collection: Set content type to XML (in the case where you want to treat the response as XML), e.g. in PHP, header("Content-Type: text/xml");.

Spying on Users With XMLHttpRequest

Earl Castledine points out the allegedly dark side of XMLHttpRequest: spying on users (thanks Ajaxian). It seems that this capability will cause it to “likely fall from grace”! (I mentioned the issue a little while ago regarding the Heartbeat pattern).

Two examples are given in the new article:

you write an e-mail to Apple support that says: “I just bought a brand new iPod. I dropped it down a set of stairs. It stopped working.” You then decide to delete the second sentence to help your cause. TOO LATE! If the site uses AJAX, your response may already have been zapped to the complaint desk in the sky!

I agree this could actually happen. The live chat/wiki demo, for example, is based on a Fat Client model, where the browser essentially runs a full app and periodically synchronises with the server, during which time intermediate info is logged. HOWEVER, even if the Apple support guy had access to the data, and felt like poring through every partial text at 5-second intervals, and the user was unlucky enough to have submitted the guilty message, I doubt it would hold up because you can sync all you like, but the user can’t be held responsible for what they wrote until they’ve acknowledged the message is done. This legal motivation is one reason why you’ll always see Explicit Submission in some Ajax apps, even though Submission Throttling renders it unnecessary from a technical perspective.

The second example:

Or — a little more malevolently — consider this: most people have one or two username/password combinations that they use for all of their “unimportant” sites such as news sites, blogs, and forums. They probably also have a few reserved for use on more sensitive sites — banking, Web mail and work accounts. It’s a very common and easy mistake to begin typing incorrect login details for a given page. Force of habit is responsible, but people usually realize what they’ve done before hitting the submit button.

I suppose it’s possible once again, but since there’s a concrete example to go by, I’m going to take this one literally too. The problem here could just as easily occur in a traditional app – I’m sure users frequently submit the wrong user/password too. That would actually be more dangerous than just periodically capturing what the user’s typing, because how do you know when they’re finished. Even when someone does capture such information, they still have to start guessing where it applies. Also, if users only have one or two combinations, they’re pretty likely to type in the same password anyway, ne? So what’s the devious webmaster getting at that they couldn’t get at before?

The thing is, you could have captured all this information before. Here’s how you do it the new way, with XMLHttpRequest:

  • Every few seconds, post the current browser state to the server.

Here’s how you do it with standard forms:

  • Every few seconds, push the state onto an array.
  • When the user eventually submits a form, upload a string representation the array. Note both of the examples above involve submitting a form eventually, and note that even an external hyperlink can easily be scripted to force a form submission. It’s a little more work, but it was always feasible.

Admittedly, one difference is that XMLHttpRequest provides some cover for malicious activity, because periodic submission is standard practice, whereas periodically submitting history is just a little suspect.

While this sort of stuff is mostly FUD, Ajax does bring out the common tension between usability, security, and privacy. The Heartbeat pattern, for instance, involves monitoring page events to see if the user’s active. Definite security benefit, definite benefit for resource conservation, but also a definite increase of personal monitoring.

In any event, users need to be aware the security model of the web is that whatever basic actions they perform on the page are open to the operator of that site. XMLHttpRequest may play a small part, but it won’t be falling from grace anytime soon.