Power Chrome

Here are some random handy tips for Chrome power users. [Alternative Buzzworthy title: “Each of these shortcuts could save your life one day”.] These are specifically not devtools-related; just features developers (and others) will benefit from.

about:about As you know, Chrome has a lot of nice diagnosis and config screens, but who can remember them all? Good news, you don’t have to enter “chrome memory” “chrome dns” etc into Google every time. Just remember one URL – about:about – and you’ll always have the full list at your fingertips.

File Menu > Warn before quitting Come on, how many times did your finger veer a bee’s hair from cmd-w to cmd-q. You thought you were shutting down Hacker News and instead you blasted 60 tabs. The implementation of this quit warning is smart too – you just have to keep pressing cmd-q. There’s no annoying “Did you really mean to …” dialog.

Multiple profiles Incognito mode is already a developer’s best friend – it allows you to check how your site looks to a new user, free of extension interference, and cancelling out any logins. Multiple user profile extends this to let you jump between profiles all day long. It’s vital if you have to manage multiple Google accounts, Twitter accounts etc and even more so if you login to other sites with those. (Chrome recently botched the new UI for this, but for now at least you can keep the original interface by setting chrome://flags#enable-new-profile-management to disabled.)

Control Freak If you have a need to tweak pages to your convenience, you can use the control freak extension as it’s super-fast to add CSS or JS rules to any page (much quicker for quick tweaks than Greasemonkey imo). Disclaimer – I originally wrote this, I’ve since passed it on as I couldn’t maintain it, but still find it useful.

Pin tabs Get into the habit of pinning tabs for reference material you’re frequently coming back to and sites you’re testing (e.g. your localhost).

Open email in your browser Make sure you’ve configured Chrome so that Gmail et al may request to act as protocol handlers.

Server-side rendering of Single Page Apps

In the simplest case, a Single Page App is merely an empty HTML body with JavaScript and template elements used to bring the page to life.

Web developers have begun to re-consider this starting point for SPAs. Even if an empty body tag is digestible by Googlebot and acceptable to screen-readers, there’s a performance problem. The quintessential case study is Twitter, who found it’s not such a good idea to send and run a megabyte of scripts just to view 140 characters. They returned to server-side rendering in 2012 to improve their “Time to first tweet” metric.

Server-side rendering

One approach is what AirBNB famously calls the Holy Grail: running the same NodeJS client on both ends. along those lines, EmberJS is working on FastBoot, a way to render on the server, and Tom Dale has written about it.

But what if you don’t have, or don’t want to have, your server-side code base in JavaScript? You could still separate out a web tier microservice (it’s the future!) in JavaScript. If you don’t want to do that, you could pre-render every page using a headless browser and build it as a static HTML file. That has the advantage of being super-fast, but requiring a bunch of infrastructure.

An alternative approach I’m exploring

Wanting to keep my solution lightweight, and not have to run Node on the server or pre-render millions of pages, my plan for the Player FM website is a variant of the old “Dynamic Placeholder” approach where the initial page is served with “holes” in it and the client subsequently makes requests to populate the holes. Instead of serving pages with holes, we could serve the entire page and have the client refresh dynamic content blocks in a way that is as unobtrusive as possible.

It goes like this:

  • Serve pages as static assets cached for an hour or so (the length will perhaps depend on the anticipated update frequency).
  • Dynamic sections in the page will use a data tag to keep track of timestamps for dynamic content.
  • A script tag (at the top of the page) will request the latest timestamp for each dynamic unit.
  • If any dynamic block has changed, its new content will be requested. This request will include a timestamp property in the URL, so that the block may be long-cached and should then return quickly.
  • To avoid a Flash Of Unwanted Text (FOUT), the page content won’t be rendered until the initial freshness check has returned, up to a timeout of a few hundred milliseconds has passed, in which case it will indeed be rendered along with a progress indicator until we get the freshness response and can deal with it.

It’s a little convoluted, but should mostly be out of the way once the framework is established. As the site already uses a PJAX approach to loading new pages (ie HTML output from server, but only the changed content), subsequent pages could optionally be served even faster by building on this technique, (i.e. in parallel to requesting the next page’s HTML, the framework can also request the relevant timestamps. This assumes we are willing to imbue the framework with upfront details of each dynamic page’s structure, an increase in complexity for a further performance boost.)

What happened to Web Intents?

Paul Kinlan:

The UX .. killed it especially on desktop, we hit a huge number of problems and the choices we made designing the API that meant we couldn’t solve the UX issues without really changing the way we built the underlying API and by the time we realized this it was too late.

I mentioned web intents would be one of the most profound technologies for the web a few years ago. And then, it disappeared. I still think it will be exactly that when it’s revived, taking into account the lessons Paul outlines here.

In fact with the coming improvements slated for the browser, intents (aka iOS extensibility) actually stands out as one of the biggest native-web gaps of the short-term future web.

Don’t API All The Things … The Downside of Public APIs

Paul kicked off an interesting conversation about pros and cons of a public API [1]. The benefits of APIs have been well-cited, certainly valid, and I would also argue in favour of public APIs in many situations. Yet startups like Uber and WhatsApp managed to grow to stratospheric heights without any API. Which begs the question often ignored in some API-gushing circles: In the ROI equation, what are the downsides of publishing an API?

This is a deliberately one-sided post as I wanted to make a reference of all the downsides in one place. Following are the potential cons that should be considered in any pros-versus-cons API feasibility analysis:

  • Friction and lock-in – APIs are notoriously difficult to upgrade as clients don’t want to keep updating their code. So once you publish a production-ready API, you have a long-term commitment to maintain it even as you grow and shift the underlying functionality.

  • Server load and quality of service – Of course, your servers are going to be doing more work when 50 other clients are connected, or 50x clients if the API clients’ end-users are communicating dirctly with your API. (OTOH they might be unofficially scraped in the absence of an API, in ways that may be even worse because you can’t fully consider cacheability and anticipate what will be called.)

  • Need a higher-quality API – Before your API is public, you can “fake it till your API makes it”. Meaning that you can put logic in your clients which should really be on the server, but because of other priorities or implementation differences, it’s easier to do a quick job in the client(s) [2]. (Which is fine as long as security-critical functionality like input validation still happens on the server.)

  • Developer Experience cost – There’s no point putting out a public API unless it’s going to be adopted by developers and continuously improved in response to real-world experience. That requires good API design, documentation, developer engagement, and issue tracking. Extra work.

  • Conflict of Interest – An API invites competing apps cough Twitter cough or even apps that combine several services and thus turn you into a commodity [3]. If you’re a pure “API company”, e.g. Stripe or Twilio, that’s par for the course. If you are more of a UX/app company, it may be an issue depending on your business model. If you’re heavily ad-based and not interested in charging developers, like Twitter, then yes it’s an issue that someone can make a good app without ads. If it’s based on a charge for server-side services, e.g. Dropbox, then no it’s not much of an issue that there are alternative Dropbox apps around [4]. Guardian’s API is an interesting example of a hybrid model which allows the provider to have similar incentives to the developer, at the expense of extra complexity.

  • Distraction – As well as the extra effort required, the API and surrounding developer relations is adding another department to the organisation, and therefore a new source of management distraction.

<

p>Most companies can and should deliver public APIs at some point and for some functionality. None of the points above argue against that, but should give pause to anyone blindly following the 2010-era doctrine of “API All The Things”.

Notes

  1. “Public” APIs because I take it as a given that any cloud service has a private API. That’s just a truism. Whether companies frame it as an API and benefit from principles of public API design, that’s not always the case, but they have some API either way.

  2. For example, Player FM’s Android app does some search filtering to remove junk results because until recently, the server didn’t provide that.

  3. When Uber did release its first very limited API, it added an exclusivity clause, meaning apps can’t also use Lyft etc APIs, so it avoids direct price comparisons. TweetDeck was also on this path, with its Facebook integration, as was FriendDeck from none other than the man who inspired this post, Paul Kinlan.

  4. It’s still an issue as some of those apps could turn around and provide a one-click conversion to Box, GDrive, etc. But far less concerning that for Twitter, the worry of a stunning Twitter app that doesn’t show ads

Migrating WordPress

After another WordPress migration [1], here’s my 2p on WordPress migrations: Don’t even think about following WordPress’s standard backup-and-restore instructions. The Duplicator plugin captures everything in one go, including comments, themes, etc as it dumps both the database and the file structure in a single archive. Also, you can easily scp the dump onto the new machine; whereas the usual procedure requires rebuilding WordPress, installing a plugin, and then worrying about the dump being too big when you try to upload it over HTTP (as there’s still no obvious way to scp and restore WXR files, really!)

Duplicator’s restore phase isn’t too clear from the docs, but I found the answer here. You don’t have to manually install WordPress. You simply create a new temp folder somewhere under web servers public/ folder, open your browser, and navigate to the install.php file there in this folder. It will do the rest for you!

  1. Now that Linode has dropped its minimum box to $10/month (and a decent spec at 1GB RAM with SSD storage), I decided to consolidate and move the utility box on Linode instead of DigitalOcean.

How to use different favicons for development, staging, and production

It’s useful to have different favicons for each environment during development, as Pamela recently pointed out. Here’s how I do it.

First, generate the images. Most graphics editor have some kind of Colorize tool and if you do it often, use ImageMagick to change colors programmatically. If you’re lucky enough to have a green brand logo, a cool combo would be traffic lights. Justsaying.

Now, set the favicons according to your environment. Should you just use /favicon.ico location or point to the location in head? The answer is both. /favicon.ico is useful for third-party plugins whose HTML you don’t control, as well as quick hack pages you can’t be bothered configuring with proper metadata. A link tag in html head is useful for revving the icon when it changes, to bust any cached version. So use both.

To set favicon in HEAD, ensure the path is dynamically generated instead of hard-coded:

  1. %link(rel="icon" href="#{favicon_path}" type="image/x-icon")

The path is generated like such as:

  1. def env_suffix
  2.   Rails.env.production? ? '' : "-#{Rails.env}"
  3. end
  4.  
  5. def favicon_path
  6.   asset_path "favicon#{env_suffix}.ico"
  7. end

It’s arguably bad form to use “production” strings in production, so the env_suffix nonsense above will hide it. If you don’t care about that, just call your production icon favicon-production.ico and save yourself a little hassle.

As mentioned earlier, we also want just /favicon.ico to exist. A quick way to do that is copy it when the apps starts or define /favicon.ico as a route and serve the file as favicon_path, defined above, with sufficient cache expiry time (e.g. 1 day). e.g.:

  1. def favicon
  2. path = ActionController::Base.helpers.asset_path "favicon#{env_suffix}.ico"
  3. send_file path, type:"image/x-icon", disposition:"inline"
  4. end

For bonus points, you might want to use similar techniques to provide overriding stylesheets for development and staging. Then you can introduce a text label or change background color, etc.

Rails quirk: Adding to DateTime

Just came across a weird one with DateTime. Adding an int value will increment by a day, not a second as you may expect:

  1. $ x=1
  2. 1
  3. $ x.class
  4. Fixnum
  5. $ DateTime.new(2000, 1, 1, 0, 0, 0)+x
  6. Sun, 02 Jan 2000 00:00:00 +0000

But adding by a second is possible using explicitly 1.second. The strange thing is both inherit from FixNum and essentially act as the same number. So if you want it to mean seconds, one way to achieve it is use “x.seconds”.

  1. $ x=1.second
  2. 1 second
  3. $ x.class
  4. Fixnum
  5. $ DateTime.new(2000, 1, 1, 0, 0, 0)+x
  6. Sat, 01 Jan 2000 00:00:01 +0000

How HTTP client libraries should support redirects

I don’t think I’ve ever come across an HTTP library (and I’m including XMLHttpRequest here) that covers all bases on redirects. In the hope of pushing the field forward, here is my wishlist. I’m limiting this to just GETs as it gets even more complicated with the mutating actions. (Several browsers do XMLHttpRequest wrong in the case of POST redirects.)

  • Follow redirects by default (it’s the most typical need after all), but provide an option to turn that off.
  • Allow a limit on redirect-following recursion, to prevent too much redirecting [1].
  • Prevent infinite redirect cycles. Which arise due to misconfiguration or malicious denial-of-service efforts [1].
  • Report back the chain of redirection, with corresponding status codes (ie was it permanent?).

It’s really the last point that’s universally missing. For services like Player FM which need to retain a record of permanent redirects, it’s necessary. The lack of this features means having to write custom code to follows redirects. Which isn’t a giant burden, but it’s error-prone, a pain to test, and has to be done for every kind of resource.

  1. The recursion limit requirement has the side benefit of preventing infinite redirect cycles, so that alone may be adequate for a library. But it’s worth treating a separate requirement because: (a) it can be even more optimal if handled separately (ie it can terminate even before the limit is reached); (b) if the library does rely on some limit to prevent infinite recurison, the library should nevertheless enforce some maximum, e.g. 100, and also set some sane default, e.g. 5.

Photo Credit: Smabs Sputzer via Compfight cc

Key-based cache expiry: A developer’s primer

Key-based cache expiry is a powerful pattern for efficient and reliable caching. I’ve been using it on for some time now after reading DHH’s original post and it’s worked well. This post explains the more conventional approaches to explain how key-based caching arises as a fruitful alternative.

So then, how does caching normally work, and what’s so bad about that anyway?

Clear after some time. Sure, you can say “this stock price table expires in 5 minutes” and then re-render it every 5 minutes (or longer if no-one immediately requests it). The problem is, you’re often making a big compromise on both ends. On the one hand, you’ll end up with stale results when the stock price changes during this cache window, e.g. if it changes 2 minutes after you serve it, you’re sending wrong data for another 3 minutes. And on the other hand, what if it only changes once a day? Most of the time you’re needlessly re-retrieving data, re-calculating and re-rendering it. Wasteful.

Clear manually. Seeing the problems of time-based expiry, you could be tempted to just keep the cache up to date manually. So when you’re notified of a price change, you explicitly update the value in the cache. This will certainly be more efficient, but the logic gets pretty complex as you scale up. You end up with NxM code complexity as all N bits of code needs to be aware of which M cache items could be affected by changes.

So one technique is easy but inefficient and inaccurate; the other is efficient and accurate, but hard. Let’s see if we can find a sweet spot which is easy AND efficient AND accurate.

Don’t clear. With key-based cache expiry, everything’s put there forever and never cleared. How is that possible? Because it takes advantage of the cache’s built-in automatic expiry mechanism. We must use a cache, such as Memcached or Redis, which supports some kind of expiry based on least-recently-used (LRU) or similar selection. In that sense, we have reached our application’s sweet spot by offloading complexity to the cache framework.

How this works is the keys must reflect a version or timestamp of the object being cached, e.g. a key might be “article-123-201404070123401″, generalised as “type-id-timestamp”. Normally clients won’t request the object by version, so you’ll need to do a quick lookup to find the object’s latest version or timestamp [1]. Then you retrieve it from the cache, or write through to the cache if it’s not already present. And the important thing is you write to it with infinite expiry.

The technique can be used at many levels – HTTP caching, memcached, persistent databases. I first asked about it here and I’ve since used it effectively in production on Player FM’s website and API. It’s certainly how various frameworks handle asset serving (ie compiling CSS with a timestamp and so on), and it’s also an official part of Rails 4, and I expect other frameworks in the future. So it’s a pattern programmers should be familiar with in an era where performance is held in high esteem, and rightly so.

  1. Looking up timestamp or version is work you don’t have to do with manual expiry, so it’s again a trade-off that makes this slightly less efficient, but a lot easier. Furthermore, if you arrange things right, you can actually have clients request the latest version/timestamp for all but the original resource (when they are requesting several resources in succession).

Photo Credit: Paolo Margari via Compfight cc