Abstraction Begets Fragmentation

Christian Heilmann raises the ugly issue of offline storage. LocalStorage’s synchronous nature makes it slow, WebSQL is deprecated, and IndexedDB’s API induces headaches as well as being unsupported on many browsers. He asks whether the LocalStorage standard should be improved.

One of the predictable responses is “just use a wrapper library”. We could use a fancy wrapper library that gives us the key-value simplicity of LocalStorage, but with an asynchronous API, and backed by those more beastly SQL-based solutions if they should exist on the device.

Now “a wrapper library” has all the usual concerns of abstractions. Does the common-denominator stance remove useful functionality from the underlying APIs? Does it separate the programmer so much from the bare metal that they can’t get their head around performance issues? And so on. All true. But on the web, where open-source is so heavily used, wrapper libraries have an additional cost: Fragmentation.

So we end up with a dozen storage libraries on GitHub. BackStorage, SpineStorage, OrthopedicStorage, you name it. That’s great, now we can use a nice API that works everywhere. Fast forward, and every programmer and their canine is writing offline single-page apps, so now we need libraries to cache, spool, and throttle messaging back to the server. Those libraries rely on underlying storage. Do they talk straight to the raw storage APIs? But then they’ll spend all their time worrying about those APIs. So maybe they reuse one of those storage libraries. One of them uses BackStorage, another one uses SpineStorage, and a couple more use OrthopedicStorage. Yes, a whole plugin ecosystem gathers around each storage solution.

So which are you? Do you consider yourself a BackStorage man/woman? Card-carrying SpineStorage-head? Certified OrthopedicStorage practitioner? You’ll need to pick one, because each is its own community with its own set of plugins and conferences. Not to mention the learning curve involved in adapting to something new. And the conflicts that arise if you try to use two wrappers in the same app. Wait, did SpineStorage.Login just stomp on that username I was trying to retain with BackStorage.Cache?

Choice is good and evolving faster than standards is good too. Which is why we do benefit enormously from wrapper libraries. But the cost of fragmentation is high. It can be justified if you’re reaping the benefits of magic conferred by a highly opinionated framework. But if it’s just there to cover up for a standard with an unnecessarily confusing API, I’d rather the browsers work on simplifying said API. Because I don’t want to identify myself as a developer skilled in the arts of BackStorage/SpineStorage/OrthopedicStorage.

I’d rather just be a “web programmer”.

6 thoughts on Abstraction Begets Fragmentation

  1. These two projects are the most robust IDB abstractions available https://github.com/axemclion/jquery-indexeddb https://github.com/jensarps/IDBWrapper

    I totally disagree that we should skip over these wrappers so we’re closer to the bare metal. I personally don’t want to deal with IDB perf unless I have to. Also it’s likely the wrapper/abstraction author will be a better person to improve the performance than me (but if not, I’ll file a pull request).

    We don’t want every developer repeating the same steps — just like every developer doesnt want to deal with the DOM and its problems, so we use jQuery, Zepto, whatever.

    When it comes to fragmentation I agree. This is a problem. In fact, lately I’ve been asking two OSS projects to merge every week or so. (diveintohtml5 mirrors, cache manifest validators, sass/less most recently).

    But JS developers are scared as shit from dependencies. This is incredibly frustrating. I chalk it up to our lack of either proper modules or package management, both which would make the ecosystem a lot more friendly to dependencies (node is a great example here).

    So yeah. Developers need to stop reinventing and start reusing. And we need to push the accelerator on efforts that leads to better use of dependencies. You don’t want to be the maintainer for your entire stack.

  2. Paul, for sure, I certainly wouldn’t advocate any developer write to the bare metal of IndexedDB/WebSQL/FS. They all need wrappers in order to be usable, realistically.

    Any time you have a layered architecture, each layer adds a cost which needs to be justified in terms of value-add. Functionality like intelligent caching is definitely in the category of a value-adding layer, whereas covering up for shortcomings in the layer below is unfortunate. A quick-fix until the layer below can be fixed (or not so quick if the layer below is an industry-wide fix, but still necessary).

  3. Pingback: Mozilla Hacks Weekly, March 8th 2012 ✩ Mozilla Hacks – the Web developer blog

Leave a Reply