by Nik Cubrilovic on May 29, 2008

The next-gen web is starting to gather pace, as this week MySpace integrated Google Gears, Yahoo! announced their new BrowserPlus product and Google launched a browser-based edition of their 3D Earth product. Technologies and formats such as AIR,Silverlight, JavaFX, Gears, XUL, Web Applications 1.0 (DOM5, HTML5 etc.) allow developers to accelerate beyond AJAX and towards a new generation of web applications with better performance, more functionality and tighter desktop integration.

Developers and users are now presented with more web technology choice then ever before; "DLL hell" has been superseded by "plug-in hell", as a variety of companies present their versions of what the next-gen web will look like. But on the web, such choice can come at a cost to both users and developers. More than a decade has passed since the first battle over web formats, back then it was Microsoft, Netscape, Apple, AOL and others laying different foundations in the form of browsers, scripting languages, web servers and more. The legacy of that battle is still being felt today, as Javascript developers rely on whole libraries to assist them in developing cross-browser code and CSS developers depend on a catalog of hacks so that their sites can look consistent across different browsers.

With the new rich web application technologies still in the development phase, there is an opportunity to not repeat the mistakes of the past and instead take a standards-based approach. Thankfully during the course of the previous decade companies such as Microsoft became more receptive to open standards, data portability and cross-platform support. Having broad support for open standards simplifies technology for both users and developers, but it is obvious that not all of the currently announced technologies, such as those listed above, will

During the course of a series of posts here on Techcrunch we will look at various elements that make up the next-gen web and evaluate the options available, current support as well as how well standards are being adopted. In light of the recent announcement from MySpace that they are using Google Gears in their application, in this first post we will evaluate browser-based local storage cache's.

Browser-based Local Storage

As web applications became more popular there was a general demand for an ability to run web-based applications offline. The first such solutions that could work without requiring a browser plugin or separate application were those that relied on the caching headers within HTTP to store objects within the browsers cache. Javascript libraries such as Dojo implemented support for offline web applications using the same principals, but applications were very limited in scope as there was no easy way to store structured data on the browser (Dojo now also abstracts a variety of other storage engines including Gears- tip: Dylan) .

In May of 2007, Google launched Google Gears, a browser plugin that allows web applications to synchronize data into a local data store and then allow web applications to function offline. At the launch of Gears, Google Reader was adapted to support it, and the emphasis of the pitch for Gears was about offline application access. What was less known is that Gears is a lot more than just offline access, as it provides three primary functions:

• Caching of resources (HTML pages, images etc.)
• Structured data storage in a database
• Asynchronous background worker threads

The part of this we will focus on here is the local object and structured data storage. Gears provides these functions via a Javascript API, which can be accessed by any web application. The structured storage is provided by Sqlite, a popular lightweight RDBMS. With the local database, the developer can not only perform queries and inserts to record new data, but also more complex SQL like joining between tables etc. Although you can have multiple applications using Gears, each app runs in a sanboxed environment with a domain-based security model (similar to cookies and Ajax requests). Sqlite has been built into Firefox since version 2.0, but its API is only accessible from an add-on or a core Firefox component. The Gears plugin bridges that gap and makes it available within the client scripting environment.

Before Gears was launched, the Web Hypertext Application Technology Workgroup(WHATWG) had begun work on its Web Applications 1.0 draft spec, which included structured data storage as part of HTML5. The current draft spec from the working group includes definitions for a Database object for accessing and querying a local data store. The details of the implementation are left up to the vendor, but the API is detailed in the spec. Firefox will be implementing parts of the same storage API from the WHATW spec in version 3.0 of the browser, which is currently available as a preview release. The key components of the WHATWG spec are:

• ApplicationCache - for storing objects in the local browser cache (and checking them)
• navigator.onLine - check if the browser is online or not (and use cache plus local data store if required)
• Storage interface and events - used for storing name and value pairs via thesessionStorage DOM attribute.
• Database interface - used for connecting to the local database. Supports SQL (or a subset thereof, depending on the server used), versioning, error events via callback
• Threading and Callbacks - so that multiple requests can be sent to the local data store asynchronously.

Implementing local storage, caching and offline access are relatively easy. The application can first check to see if these functions are supported, and then setup the local cache by synchronizing the users data in background processes. While a thread is running, either uploading or downloading, you can query it to check on its status to provide the user with feedback (eg. a progress bar). Once the data is local, by running database queries on the local machine developers are able to drastically improve performance. Currently many web applications use the browser as only a presentation layer, for eg. a spreadsheet application may do a round-trip back to the server to work out even elementary calculations such as=1+1. By utilizing the local data store and client-side code, the developer is able to offload processing and storage to the client and provide a much smoother, desktop-like experience at the same time.

Current And Future Support

The issue is that the majority of the WHATW specification was written after Gears was released, so the Database and LocalServer objects used in Gears are not compatible with WHATW - for now. The good news is that Google have come out and fully backed the storage portions of the WHATWG HTML5 spec, so developers with apps running on Firefox 3 with Gears installed will have a choice to use either the native implementation, or the Google implementation. Google go on to say that they will likely offer extra features as an incentive for developers to continue to target Gears over-and-above the HTML5 implementation (features such as desktop shortcuts, etc.).

Other alternatives for local data storage, such as Flash local storage, are completely incompatible with the WHATW specification. The developers at WebKit were very quick to announce that they have started implementing the storage portions of the HTML5 spec also. It is currently available in nightly builds, so in the near future we will see support in both Konquror and Safari. Opera have also announced similar plans and they are actually leading everybody when it comes to implementing HTML5 and Web Forms. Yahoo! BrowserPlus was only announced yesterday, and it is currently unclear wether their local storage support is compatible with the specification as laid out by the working group.

Local storage is a major new feature of the new web API, and developers will not only have consistent support across browsers but will also have the option of Google Gears (which is already available) as well as Yahoo! BrowserPlus (depending on how it works). There is just one browser maker missing in this discussion so far and that is Microsoft. Microsoft have released an early preview of IE8 and announced a raft of new features, a lot of which are based on open standards such as better CSS and Javascript support (with a more standardized object model). The big question is wether we will see consistent local storage support from IE8 following the same spec as the other browser vendors. The IE team have announced that IE8 will support DOM Storage, but that is only part of the overall local storage spec (ie the Storage object described above only).

Overview of Current And Future Support

Notes: Once details of BrowserPlus are known the table will be filled in. Gears has pledged to adapt the standard. IE8 has only announced partial support. WebKit have most functionality available in their nightly builds. Both Flash and Silverlight have some form of local storage but not the HTML5 standard API.

It is very rare that a new technology such as local browser storage is so broadly advocated and supported mostly under a single spec. While Microsoft may not have yet announced full support, there is no doubt that they are heading in the right direction. It is also encouraging that implementations from Google Gears and Firefox 3 will follow the working group spec for HTML5. While the new versions of these browsers will not be available broadly for a while longer, Google Gears is available today and since all vendors are targeting the same API, developers can today confidently target and write for the Gears storage API with confidence - something that was almost impossible to do not so long ago.

So with local browser storage and caching, so far the open standard is the winner. The alternate solutions are likely to fall by the wayside or will adapt to implementing the same API.