Wanted: Massive Local Storage

Local storage – beyond 2KB cookies – is now a step closer with the latest Firefox effort. You get a local storage API like this:

javascript

  1. sessionStorage.setItem(..)
  2. globalStorage.namedItem(domain).setItem(..)

The fantastic thing is Brad Neuberg’s Dojo work means we can code independently of the local storage mechanism. Since IE also has local storage, as well as Flash, most bases are covered, or soon will be: Anyone with EITHER IE or Firefox or Flash will have local storage. (Incidentally, we had a discussion in the comments on Ajaxian about the possibility of S3 and other remote bindings as well, which I’m guessing Brad will implement at some point.)

But what I’d like to know, what I’d really really like to know, are the limits for these various techniques (I think Flash is 10MB per domain, right?)

As I’ve said before, there are valuable applications of massive storage – hundreds of gigs, e.g. media players that store data server-side, but cache content locally to cut down on the need for streaming from the server. Hopefully, these storage technologies won’t try to second-guess the imagination of web developers by setting a dumb arbitrary limit that “no-one would ever need more than”, like 100MB or 1GB or whatever.

If there’s room on my hard drive, and I trust the domain, let me store as much as I please.

10 thoughts on Wanted: Massive Local Storage

  1. in Flash, the settings are: 10K, 100K (default), 1MB, 100MB and unlimited, based on user consent (beyond the default).

    You can see it at: http://www.macromedia.com/support/documentation/en/flashplayer/help/settings_manager03.html

    So I don’t see the size as the major roadblock (yet). I’d rather see better and richer APIs, and using a shared store and APIs between browsers.

    Also, even if you stored some media (say, some cached mp3) in the client-side storage, there is no good way to address it to play it in the browser. In Firefox, you could use “data:” urls, but it’s kinda hacky and it’s not supported in IE.

  2. Julien,

    “but it’s kinda hacky and it’s not supported in IE.”

    kinda hacky = What’s new :-) not supported in IE = Ideally it would work in IE, but at least in my example, it’s only the lack of caching that IE users would suffer.

  3. Pingback: Outside the Box » Blog Archive » Storage and Javascript

  4. ‘640k should be enough for anyone’ – billy goat

    Should that be 100k?

    I dont see any problem with taking a custom approach for each browser. Firefox 2 will have local storage (which I would prefer to use over the flash storage) and IE has 60K.

    If you are able to, and you’re conservative about it, you should be able to just store the diff, or the changes between the original document and the edited one in local storage. This can make 60k seem like a lot of space. For a lot of use cases (like working on a web app from an airplane etc) this should be fine for a couple hours of work. You can always abstract the storage stuff out of the javascript and use the 100k from flash in addition to the 60k local storage.. lots of ways to approach it.

  5. In practice, the max storage in IE using the userData behaviour is not limited to the amount they state on MSDN which is:

    Security Zone – Doc Limit (KB) – Domain Limit (KB) Local Machine128__1024 Intranet________512_____10240 Trusted Sites128__1024 Internet________128_____1024 Restricted______64______640

    We have stored a very large amount of data using userData with no sign of a limit – that was IE 5.5 I think though too. It would seem that Brad has found otherwise.

  6. Thanks for the IE info…I agree these sizes are okay for basic Tiwy-style wiki usage (though not brilliant even for that), but at the same time, it’s still utterly stupid that these limits get set. There should be some sensible defaults, and beyond that, it should be up to the user whether they want to store 100K or 10GB. And in this day and age (ie post-1985), max capacities of 128K-1024K are frankly absurd. Many of us carry 10,000-100,000 times that much in our front pocket. Just what are they trying to protect users from, with limits like that?

    Anyway, the point is not to pick the right value, but to leave it flexible long term. I hope Firefox makes the right choice on that, as the time is right for all this stuff, and an arbitrary capacity will lend FF a big edge.

  7. Dave, that is very interesting; I remember testing and finding something similar and getting excited, then I remembered I was loading from a file:// and localhost URL. Have you found the same results from full http:// urls from remote hosts that aren’t trusted? I seem to recall that my data was chopped off after about 60K.

  8. Pingback: Storage and Javascript | BenCrowder.net

Leave a Reply