The Missing Man Page

If only man pages were as good as the first Google result, I idly tweeted while trying to coerce curl into post a form [1]. The ensuing conversation led me to think about exactly what is missing in man pages.

When I say “the first Google result”, I’m not talking about StackOverflow in this case. We can all agree Stack has mystic levels of ability to answer the question your man page can’t, or not in under 15 minutes anyway. I’m thinking more about reference material; what exactly is wrong with man pages? And why do the newer alternatives, such as tldr pages, fall short of replacing them?

My primary thesis is that man pages are an artifact of a long-bygone era, where the only electonics docs available were those that shipped with the tool itself. There was no web and no convenient way to collaborate at scale.

With man pages being tethered to the tool itself, this means that the docs are maintained by the tool maintainer, who is not always best equipped to write documentation. This is probably better nowadays, for tools using Github/Gitlab etc, where others can come in and improve docs, however it was suggested on Twitter that some maintainers may block valuable contributions. There’s also the well-known lack of funding for generic tools, which means even if project maintainers wanted someone to create better docs, there’s no funding for it.

Compare that to the incentive to create a gret web reference, where a company like W3CSchools can rake in $millions as the top Google result for just about any search. It’s a competitive marketplace.

In fact, this is a problem with all official docs. Even tools which do have viable commercial models still have poor online documentation in many cases. Take a look at MySQL’s documentation on creating a new user. User management is one of the main pain point of friction in the system, and this is their official documentation for it. You land from Google and you’re greeted by this giant complicated regex, wrapped in a jungle of navigation links! Is it any wonder DigitalOcean pays a writer to make a sane third-party reference guide in an SEO play that could end up outranking the official doc?

This illustrates that the people creating the tool aren’t necessarily the people best placed to document it? That doesn’t have to be the case of course. These days, companies understand the value of developer experience and some, like Stripe, produce outstanding docs. However, that doesn’t translate to your run-of-the-mill Unix staple.

This is where tools like tldr and cheat can make a difference. They are separate from the tool itself, anyone can contribute. However, their explicit goal is to be concise, which means they are not an apples-to-apples alternative because man pages are designed to be exhaustive.

My biggest gripe with the format of man pages is that they try to act too much like textbooks, which is entirely inappropriate when you are trying to get some quick info to solve a problem. Again, it’s an artifact of a time before the web, where the most convenient way to ship around an essay about the tool was ship it with the tool. Here, for example, is what I see with man find.

You’d never see this level of caveats and hand-waving in a Digital Ocean sponsored post on how to find files on your file system. At the least, it would be buried at the bottom of the doc, with the important stuff, like a simple annotated example first.

So what I’d like to see is something exhaustive, but with options grouped logically, not alphabetically, and each one of them illustrated by examples. Man pages don’t do this and nor do tldr or cheat.

Making better documentation is a lot of hard work, which is why StackExchange abandoned its Documentation experiment. So I don’t see the problem being solved anytime soon, and it’s why I’ll continue using a web search for now.


  1. I’d normally be using the more inuitive HTTPie for this, but sometimes you need to share it with people who are used to curl.

Google to embrace Swift?!!

In other news, Microsoft to embrace Linux. Hahahaha ha ah oh wait.

NextWeb reports Google’s considering Swift as a first class Android language. I’m not surprised companies like Facebook and Uber are embracing Swift as it’s sufficiently open and highly attractive in an environment where iOS is king.

But no, Google won’t embrace it. Just because something is open source doesn’t mean everyone has equal influence. There’s still control at the top of the repo and this is why Google forked Webkit, an effort to control its own destiny instead of relying on the very same company that controls Swift.

Furthermore, Google and Android is still engaged in a tired, ongoing, battle with Oracle over Java, another language that is – “for varying definitions of” – open. And Oracle isn’t even a direct competitor.

It’s true that there’s a lot to be said for a more dynamic, scripty, language on Android. While Android Studio has done a lot to improve the developer experience for Android devs, a lot of work done is exploratory UI, something that a language like Swift can help with.

If Google were to embrace a dynamic language, and assuming they don’t start from scratch, there are really only three contenders: Go and Dart, since Google completely control them (more likely Dart as it’s more suited to UI). And JavaScript, since it’s immensely popular and under no one company’s control (and as a bonus, Dart is designed to compile nicely to JS, so Google can still support it as a higher level alternative). Just as Swift has made iOS a much more approachable platform for casual developers, embracing the world’s most popular language could be a nice boost for developer traction in the Android world.

Progressive Web Apps have leapfrogged the native install model … but challenges remain

I visited Google’s Progressive Web Apps shindig the other day and I’m pleased to see the progress browsers have made towards appiness in the past 2 years. Much of what I wished for in frankly the web’s darkest days is now available in 3 major browsers and counting.

The install model has truly gone from being a non-starter to something that more closely maps the needs of users than any native platform has achieved. Here I will reflect on the current state of the web as a platform for apps, identify some remaining concerns, and propose where the biggest wins will come from.

The web can do apps

While the HTML5 era (2008-2012, say) introduced many conventional app components, they emerged in parallel with the mobile revolution. I don’t say “so-called revolution”. It was an actual revolution in every way, changing forever how apps are designed, developed, and distributed. HTML5 was already ushering in a flood of new tech, it was never going to be possible to make it all mobile-savvy at the same time, and this led to a world where people went crazy for apps.

The progressive web movement plugs the gap in several ways:

  • Push notifications. These are lifeblood for many apps. Critical to the functionality of messaging apps like Skype and Slack, and – in a world of fickle users and high churn rates – vital to retention for all apps.
  • Background processing. Doing stuff when the user doesn’t have your app open is also vital for a modern app’s functionality, performance and offline capability. This is about the app acting as your digital assistant, not just something you interact with for the time it’s on the screen in front of you.
  • Low-level APIs. As part of the extensible web manifesto, developers now get access to the low-level innards of the web stack. This not only helps the standards to evolve, but lets developers deliver useful functionality unanticipated by committee-driven standards processes.

Furthermore, it comes at a time when browser performance is strong and web debugging tools are built into all the major browsers and have become stupendously useful. All of which means, it’s now possible to replicate the functionality and interface of many popular native apps.

But how will users install web apps?

Until progressive web movement came along, websites never had a chance on mobiles. Browser bookmarking features were about the same as in 1995 – just a flat list. Users didn’t know how to install to homescreen and even with libraries prompting it, it gave no confidence the site would work offline. Offline tech itself, and the website couldn’t do anything in the background as mentioned above above.

Now – with progressive web apps – two things have changed. Firstly – at least on Android – web apps have been elevated in their presence. The task switcher presents each recent website alongside each recent native app; they are all equivalent. The traditional 38-step “add to homescreen” process has been replaced by a simple menu item in Chrome. And most importantly, the browser will proactively prompt an install.

The progressive web’s install model rocks

A common argument for native apps has been the importance of app stores (insert $legally_acceptable_synonym for app store on your platform of choice). The web’s counter-argument has heretofore been either “Ah but the web has search engines” or “Ah but there are x billion apps on the app store and only the top 10 get any installs”. Neither of these arguments hold much water for me.

SEO is a real thing and search rankings are as much or as little a meritocracy as app rankings are. For every startup blowing $5K a day on Facebook app install ads, there’s another startup paying for fake forum posts in the hope of Google juice. The “long tail” argument also applies just as much to the web.

So how does “progressive web” improve things? By letting users progress from a fly-by visit to a fully installed app, on their own terms.

The conversion funnel from “non-user” to “user” for a traditional app looks like this:

  1. User discovers app.
  2. User installs app. Waits a minute or two for download. App is installed.

Getting users from (1) to (2) is extremely hard for developers. Most users don’t want to clutter up their phone with hundreds of apps, don’t like to go through the hassle of downloading the app, and don’t want to feel the remorse of installing a lemon (even a free lemon). Pulling off a large install base relies on a difficult-to-achieve store ranking, a viral loop that is fleetingly rare in practice, or several dollars of ad spend per paid install.

The progressive web install looks like this:

  1. User discovers website.
  2. User kicks tyres. Grants one or two permissions.
  3. Re-visit or re-stumble on website a few times.
  4. Say yes when browser prompts homescreen install (or explicitly install it).

This is a much more logical transition. Instead of making the install decision based on the store listing, the user makes the decision based on actually interacting with the app. At no point are they obligated to install it, but as they gain confidence in it, they can decide to do so. From the developer’s point of view, it’s easier to win long-term users if you have a product that’s compelling (and if not, why do you care about installs? You will lose users anyway if you don’t have good retention).

Admittedly, going from (3) to (4) is still hard. If the user hasn’t yet installed the app, how confident can you be that they will re-visit your site often enough to be prompted? Part of the answer comes from background notifications, which means the user can still be engaged even without that install. As well, if the user’s social contacts keep recommending the same app, it will likely lead to an install prompt. Compare that to a native share, which would often lead to nothing unless the content was amazingly compelling.

Indeed, it’s likely developers will care less about installs in general, as long as they still have users who are engaged via notifications and spontaneous interactions they wouldn’t see at the moment. Spontaneous usage is particularly compelling when you consider the physical web. There’s no way I’m going to install an app for the restaurant I’m in or the airport I’m flying to … but I’ll gladly open a rich web app from a prompt that shows up on my phone.

Remaining concerns

So that’s it huh? The web came back in the third act and triumphed. No. Not even. There are still many challenges ahead.

Challenge: Apple (“There’s a 586 billion dollar elephant in the room and it’s not happy Jan”)

Here are some facts about Apple, which – when combined – lead to skepticism about any efforts to progress the web:

  • Apple and iOS are hugely influential. It’s the platform companies care about most when it comes to development efforts.
  • Browser innovation on iOS is controlled by Apple. Google, Firefox, and Opera may produce their own iOS browsers, but they will still run on Apple’s engine and any progressive web tech on iOS therefore relies entirely on Apple’s whims.
  • Apple’s incentives for browser innovation are “mixed” at best. While it wants a great user experience – including web interaction – it also has a lot to fear from a platform beyond its control that lets developers “write once, run many”.
  • Apple is sticky. Most mainstream users don’t care about detailed OS features like homescreen widgets and notification interfaces. Even the tiny iPhone 4 screens were not enough for most users to look elsewhere. That was at least a very clear and visible advantage for the competition. So how much will users care about a better web experience if they can still get the same apps natively? It’s a moderate OS benefit and may help Android (or other platform users) with retention, potentially clawing back market share in the long term, but it’s unlikely to sway many users away from Apple. It’s too difficult a concept to even explain, let alone for anyone to really care about it if they aren’t already using it.

Don’t hold your breath for an iOS which supports native-level video calling, gaming, and podcasting. Some features will make their way over time, but by then, there will be even more features on both native and – on other platforms – web. The only question worth asking is, how much does it matter?

For me, the answer is “not as much as you might think”. People are still making web apps anyway. The whole thing about progressive web apps is they are progressive, not binary. So your web app can still work quite nicely on iOS, but do even more on other platforms. This won’t apply to all genres, e.g. it’s quite useless to make a voice calling app without push notifications.

Furthermore, there is a whole class of developers with iOS apps but lacking apps on Android, Windows, and other platforms. Enhancing their existing web presence is an increasingly bright alternative to hiring dedicated native developers, considering they usually have a web app and developers already (even more so if they are one of many companies now running Node/JavaScript on the server side). It may not give as perfect an experience as a native app, but it’s infinitely better than doing nothing on those platforms and may well be as good as they would produce anyway outside of iOS.

Challenge: Discoverability (“Websites on a plane”)

It’s still difficult to find good, installable, web apps. There are some hints when you’re already using one – e.g. the prompts to install on home screen or receive push notifications, and the color scheme supported by the web manifest protocol. However, if I want to find an installable app to do X, where do I go? On native, I can just search in the store.

On web, I can search in … Google? Nope. I’ll usually get a pile of ugly and ad-ridden sites that happen to be old enough to have reached high rankings. Thankfully, Google does care more about performance and mobile-friendliness now, but it still doesn’t come close to the app browsing experience of a native app store.

This is exactly what Chrome Web Store should be doing in 2016. I hope Google is working to finally bring the web store to native. (Or, amid much controversy, integrate web apps into Google Play.) And other browsers are similarly working on this problem.

Until then, there are curated showcases.

Challenge: Native keeps moving forward (“2010 called.”)

Native remains a fast-moving target. The web may have caught up on many features of a modern smartphone, but native has moved on to power virtual reality, cars, home appliances. Additionally, there are still many basic functions that aren’t yet possible on the web, though they are being debated and worked on. e.g. I still can’t make a full-fledged offline podcast app because of SSL and cross-domain restrictions. Bluetooth, USB, background audio … these APIs are all being worked on, but aren’t there yet.

Challenge: Payments (“Shut up and take anyone’s money”)

Frictionless payments obviously drive a huge amount of activity in the app world and this is a realm where the web really hasn’t changed since the introduction of smartphones. In-web payments is a complicated 4-way problem – there are users, browsers, operating systems, and payment providers. Add security, UX, and privacy to the mix, and now you see why there are casual games earning millions each day on native platform but nothing on the web.

If this can be cracked in the context of progressive web apps, game-changer.

Challenge: Native app streaming is also progressive

The progressive engagement model is no longer exclusive to the web. Never afraid to boil the ocean, Google has now begun previewing native apps by streaming them from the cloud. Yes, it relegates your device to a “dumb” display unit and runs the app on Google’s servers, at least until you decide to install it. It’s a very different type of progressive engagement, but it may steal some of the web’s progressive thunder nonetheless, especially if Apple was to follow suit.


Progressive web technologies are making it possible to go beyond just rich websites to “real deal” digital assistants like people have become accustomed to with native apps. The install model mirrors the way an app or service builds trust over time, and for this reason, it goes beyond the binary “installed or not” situation for regular native apps. While many challenges remain, the good news is … it’s progressive. Developers can already see the benefits by sprinkling in these technologies to their existing websites and proceed to build on them as browsers and operating systems increase support.

The Class1-Class2 Naming Antipattern for Associations

Join classes (aka join tables) are entities whose main purpose is to associate one object (aka record) with another in a NxN relationship.

A common and decades-old pattern, which is almost always wrong, is to name these classes after both association classes.

Examples of NxN relationships:

  • PersonStock maps owners to their stock.
  • UserFeed maps users to feeds they are subscribed to.
  • StudentCourse maps students to their courses.

What’s wrong with these names? First, they are awkward to say and cumbersome to deal with in code (as a general rule, multi-word entities are best avoided, because it becomes confusing and ambiguous when they are combined with other words). Second, they are redundant to anyone who is looking at the class’s foreign keys (admittedly, some redundancy is okay if it makes the code more understandable, but one should always be weary of a naming scheme which could be auto-generated by a trivial script). Third, and the biggest complaint: it’s wholly unnatural to anyone versed in the domain, therefore not a good model of reality. Only programmers and DBA use terminology like this; domain specialists do not.

The fundamental problem is it frames these associations as being entirely about the things they associate, instead of treating the association as a first-class citizen, which is inevitably how they are treated by a practitioner in the field you’re modelling. Once you start seeing the association as a model in its own right, you can start to enrich it with meaningful properties and behaviours. And this is typically true in the real world – associations are more than just dumb pairings of item A and item B.

More than just modelling these associations and finding an appropriate name, it can also prompt you to talk with domain specialists about what actually are the NxN join concepts in this domain.

Revisiting these examples:

  • PersonStock is better modelled as Ownership. Now that we have a concept of “ownership”, we can think about things like when was it created (ownership.created_at) and what kinds of conditions must be required to create an “ownership”. You could do this kind of reasoning with a “OwnerStock” thingy, but it’s more mental gymnastics and takes you a step away from domain specialists.

  • UserFeed is better modelled as Subscription. Now we can attach properties of the subscription, e.g. a ranking/rating indicating how much the user loves any particular feed. This data may then be used to determine how the user is notified of updates and perhaps how the “river of news” is sorted. Or maybe a visibility attribute indicating who can see the subscription, ie is it public that a given user is subscribed to a given feed.

  • StudentCourse is better modelled as Enrolment. Now we can record a “passed” or “grade” attribute against the enrolment and consider pre-conditions for creating an Enrolment, such as looking at the user’s past Enrolments.


p>Not all associations have a natural word to describe them, but even when they don’t, it’s worth thinking really hard about coming up with a new term. The Class1-Class2 name is almost always the road to pain.

Installing Android Marshmallow – quick gotcha notes

What the various guides often omit.

  • Run android-sdk/tools/android and update platform-tools
  • The oem unlock setting on phone kept reverting. But “fastboot oem unlock” fixed it.
  • Ensure “adb devices” shows the device
  • Ensure “Device State” on phone shows as UNLOCKED
  • “cannot load ‘system.img'” may happen because it requires several GB of free memory. So try chunking it: android-sdk/platform-tools/fastboot flash -S 512M system system.img

Power Chrome

Here are some random handy tips for Chrome power users. [Alternative Buzzworthy title: “Each of these shortcuts could save your life one day”.] These are specifically not devtools-related; just features developers (and others) will benefit from.

about:about As you know, Chrome has a lot of nice diagnosis and config screens, but who can remember them all? Good news, you don’t have to enter “chrome memory” “chrome dns” etc into Google every time. Just remember one URL – about:about – and you’ll always have the full list at your fingertips.

File Menu > Warn before quitting Come on, how many times did your finger veer a bee’s hair from cmd-w to cmd-q. You thought you were shutting down Hacker News and instead you blasted 60 tabs. The implementation of this quit warning is smart too – you just have to keep pressing cmd-q. There’s no annoying “Did you really mean to …” dialog.

Multiple profiles Incognito mode is already a developer’s best friend – it allows you to check how your site looks to a new user, free of extension interference, and cancelling out any logins. Multiple user profile extends this to let you jump between profiles all day long. It’s vital if you have to manage multiple Google accounts, Twitter accounts etc and even more so if you login to other sites with those. (Chrome recently botched the new UI for this, but for now at least you can keep the original interface by setting chrome://flags#enable-new-profile-management to disabled.)

Control Freak If you have a need to tweak pages to your convenience, you can use the control freak extension as it’s super-fast to add CSS or JS rules to any page (much quicker for quick tweaks than Greasemonkey imo). Disclaimer – I originally wrote this, I’ve since passed it on as I couldn’t maintain it, but still find it useful.

Pin tabs Get into the habit of pinning tabs for reference material you’re frequently coming back to and sites you’re testing (e.g. your localhost).

Open email in your browser Make sure you’ve configured Chrome so that Gmail et al may request to act as protocol handlers.

Server-side rendering of Single Page Apps

In the simplest case, a Single Page App is merely an empty HTML body with JavaScript and template elements used to bring the page to life.

Web developers have begun to re-consider this starting point for SPAs. Even if an empty body tag is digestible by Googlebot and acceptable to screen-readers, there’s a performance problem. The quintessential case study is Twitter, who found it’s not such a good idea to send and run a megabyte of scripts just to view 140 characters. They returned to server-side rendering in 2012 to improve their “Time to first tweet” metric.

Server-side rendering

One approach is what AirBNB famously calls the Holy Grail: running the same NodeJS client on both ends. along those lines, EmberJS is working on FastBoot, a way to render on the server, and Tom Dale has written about it.

But what if you don’t have, or don’t want to have, your server-side code base in JavaScript? You could still separate out a web tier microservice (it’s the future!) in JavaScript. If you don’t want to do that, you could pre-render every page using a headless browser and build it as a static HTML file. That has the advantage of being super-fast, but requiring a bunch of infrastructure.

An alternative approach I’m exploring

Wanting to keep my solution lightweight, and not have to run Node on the server or pre-render millions of pages, my plan for the Player FM website is a variant of the old “Dynamic Placeholder” approach where the initial page is served with “holes” in it and the client subsequently makes requests to populate the holes. Instead of serving pages with holes, we could serve the entire page and have the client refresh dynamic content blocks in a way that is as unobtrusive as possible.

It goes like this:

  • Serve pages as static assets cached for an hour or so (the length will perhaps depend on the anticipated update frequency).
  • Dynamic sections in the page will use a data tag to keep track of timestamps for dynamic content.
  • A script tag (at the top of the page) will request the latest timestamp for each dynamic unit.
  • If any dynamic block has changed, its new content will be requested. This request will include a timestamp property in the URL, so that the block may be long-cached and should then return quickly.
  • To avoid a Flash Of Unwanted Text (FOUT), the page content won’t be rendered until the initial freshness check has returned, up to a timeout of a few hundred milliseconds has passed, in which case it will indeed be rendered along with a progress indicator until we get the freshness response and can deal with it.

It’s a little convoluted, but should mostly be out of the way once the framework is established. As the site already uses a PJAX approach to loading new pages (ie HTML output from server, but only the changed content), subsequent pages could optionally be served even faster by building on this technique, (i.e. in parallel to requesting the next page’s HTML, the framework can also request the relevant timestamps. This assumes we are willing to imbue the framework with upfront details of each dynamic page’s structure, an increase in complexity for a further performance boost.)

What happened to Web Intents?

Paul Kinlan:

The UX .. killed it especially on desktop, we hit a huge number of problems and the choices we made designing the API that meant we couldn’t solve the UX issues without really changing the way we built the underlying API and by the time we realized this it was too late.

I mentioned web intents would be one of the most profound technologies for the web a few years ago. And then, it disappeared. I still think it will be exactly that when it’s revived, taking into account the lessons Paul outlines here.

In fact with the coming improvements slated for the browser, intents (aka iOS extensibility) actually stands out as one of the biggest native-web gaps of the short-term future web.

Don’t API All The Things … The Downside of Public APIs

Paul kicked off an interesting conversation about pros and cons of a public API [1]. The benefits of APIs have been well-cited, certainly valid, and I would also argue in favour of public APIs in many situations. Yet startups like Uber and WhatsApp managed to grow to stratospheric heights without any API. Which begs the question often ignored in some API-gushing circles: In the ROI equation, what are the downsides of publishing an API?

This is a deliberately one-sided post as I wanted to make a reference of all the downsides in one place. Following are the potential cons that should be considered in any pros-versus-cons API feasibility analysis:

  • Friction and lock-in – APIs are notoriously difficult to upgrade as clients don’t want to keep updating their code. So once you publish a production-ready API, you have a long-term commitment to maintain it even as you grow and shift the underlying functionality.

  • Server load and quality of service – Of course, your servers are going to be doing more work when 50 other clients are connected, or 50x clients if the API clients’ end-users are communicating dirctly with your API. (OTOH they might be unofficially scraped in the absence of an API, in ways that may be even worse because you can’t fully consider cacheability and anticipate what will be called.)

  • Need a higher-quality API – Before your API is public, you can “fake it till your API makes it”. Meaning that you can put logic in your clients which should really be on the server, but because of other priorities or implementation differences, it’s easier to do a quick job in the client(s) [2]. (Which is fine as long as security-critical functionality like input validation still happens on the server.)

  • Developer Experience cost – There’s no point putting out a public API unless it’s going to be adopted by developers and continuously improved in response to real-world experience. That requires good API design, documentation, developer engagement, and issue tracking. Extra work.

  • Conflict of Interest – An API invites competing apps cough Twitter cough or even apps that combine several services and thus turn you into a commodity [3]. If you’re a pure “API company”, e.g. Stripe or Twilio, that’s par for the course. If you are more of a UX/app company, it may be an issue depending on your business model. If you’re heavily ad-based and not interested in charging developers, like Twitter, then yes it’s an issue that someone can make a good app without ads. If it’s based on a charge for server-side services, e.g. Dropbox, then no it’s not much of an issue that there are alternative Dropbox apps around [4]. Guardian’s API is an interesting example of a hybrid model which allows the provider to have similar incentives to the developer, at the expense of extra complexity.

  • Distraction – As well as the extra effort required, the API and surrounding developer relations is adding another department to the organisation, and therefore a new source of management distraction.


p>Most companies can and should deliver public APIs at some point and for some functionality. None of the points above argue against that, but should give pause to anyone blindly following the 2010-era doctrine of “API All The Things”.


  1. “Public” APIs because I take it as a given that any cloud service has a private API. That’s just a truism. Whether companies frame it as an API and benefit from principles of public API design, that’s not always the case, but they have some API either way.

  2. For example, Player FM’s Android app does some search filtering to remove junk results because until recently, the server didn’t provide that.

  3. When Uber did release its first very limited API, it added an exclusivity clause, meaning apps can’t also use Lyft etc APIs, so it avoids direct price comparisons. TweetDeck was also on this path, with its Facebook integration, as was FriendDeck from none other than the man who inspired this post, Paul Kinlan.

  4. It’s still an issue as some of those apps could turn around and provide a one-click conversion to Box, GDrive, etc. But far less concerning that for Twitter, the worry of a stunning Twitter app that doesn’t show ads