Kaizou » cache

HTML 5 offline web applications

David Corvoysier — Sat, 04 Apr 2009 17:21:21 +0000

HTML 5 provides a new caching feature to support offline web applications.

Basically, you can specify in a specific file called the “cache manifest” the caching policy to apply to a particular web page.

To activate the HTML 5 caching feature for a specific document, you only need to specify the URI of the cache manifest file in the manifest attribute of the element.

...

The file that references the cache manifest file is called the master entry.

Please note that to be properly parsed by the browser, the manifest file must have a text/cache-manifest MIME type (which means that you will need either to add a custom file type extension binding to your web server or to specify manually the mime-type, for instance using PHP header directive).

A cache manifest file typically looks like this:

CACHE MANIFEST
index.html
style.css
images/logo.png

NETWORK:
search.php

This file specifies several files to cache (the cache explicit entries), and then specifies that search.php should never be cached (it is said to belong to the online white list), so that any attempt to access that file will bypass the cache.

Whenever the user browses to the page corresponding to a document to which a cache manifest is associated, the browser must check on the server if the latter has been modified:

If it hasn’t, then the cached content is used,
If a new version of the cache manifest exists, then the cached content is considered as staled and reloaded from the server.

The cache manifest for a document may also contain fallback entries that can be used if the browser can’t retrieve the original content. Below is an example where a static image is used when the retrieval of a dynamically generated image fails:

CACHE MANIFEST
FALLBACK:
images/mirror-effect.php server-busy.jpg

HTML 5 also defines an application cache API to manually trigger cache updates:

update(): updates the cache for the current document in the background,
swapCache(): reflow the HTML document using updated cached content,
onupdateready(): event triggered when the cache content has been updated,
onobsolete(): event triggered when the cache content has been marked as obsolete,
…

Browser HTTP 1.1 caching compliancy test

David Corvoysier — Sun, 22 Mar 2009 15:20:21 +0000

Following a previous post about HTTP 1.1 caching mechanisms, I have been asked whether common browsers were actually compliant or not, so I wrote a little test to verify if at least basic features were supported.

The results for your browser will appear in the frame below.

Update: This is a new version with cache headers completely handled by PHP instead of using apache mod_expires. I had to do this because I could’nt deactivate chunk transfer encoding of pages generated by php on my host, and this prevents HTTP 1.1 caching headers to be inserted by apache … the test seems to work now, but it is still in beta

Sample Apache cache configuration

David Corvoysier — Thu, 26 Feb 2009 12:33:46 +0000

The Apache 2 web server can be configured to define a custom cache policy, allowing a fine-grained control over your web site content lifecycle in browser caches. This article provides a step-by-step simple configuration sample. The reader is invited to read first this other article presenting the HTTP 1.1 cache mechanisms.

Step 1: Activate expiration module

By default, Apache 2 doesn’t insert any expiration header in served documents. This feature is provided by a specific module, mod_expires.

Two operations are required to activate the feature:
• add mod_expires to the list of enabled modules,

a2enmod expires

• add the ExpiresActive On directive to the site configuration.

NameVirtualHost *

...
   
      ExpiresActive On
   
...

Step 2: Define a default expiration policy

A default expiration policy that applies to all documents can be specified using the ExpiresDefault directive.

I would recommend to define the default expiration period to a very short value, in order to avoid client browsers keeping forever in cache documents for which you forgot to provide an explicit expiration period (see next paragraphs).

NameVirtualHost *

...
   
      ExpiresActive On
      ExpiresDefault "access plus 10 minutes"
   
...

Step 3: Define an expiration date per document type

A specific expiration policy can be defined for each document type using the ExpiresByType directive.

It is for instance possible to define an explicit expiration policy for HTML pages, javascript, images and json text files in order to control their lifetime in the browser cache, and avoid unnecessary validity checks from the client.

NameVirtualHost *

...
   
      ExpiresActive On
      ExpiresDefault "access plus 10 minutes"

      ExpiresByType text/html "access plus 1 day"
      ExpiresByType text/javascript "access plus 1 day"
      ExpiresByType text/x-json "access plus 1 day"

      ExpiresByType image/gif "access plus 1 month"
      ExpiresByType image/png "access plus 1 month"
      ExpiresByType image/jpg "access plus 1 month"

   
...

Note: The apache 2 web server uses the MIME-type to categorize documents. In order to have the proper directive applied to them, JSON text files must be sent as text/x-json documents.

Step 4: Mark session-specific documents as non-cacheable

Even if a large part of a specific site might be cacheable without disturbing the service, some specific documents are purely dynamic (like the results of a search query for instance) and must not be cached.

Fortunately enough, it is allowed to define exceptions to the general expiration policy defined for a given file type, using file pattern matching with the FilesMatch directive.

Thus, it is possible to specify that a Cache-Control header corresponding to “no-cache” be inserted for all files matching a given criteria, that can be based on a regular expression.

The Header directive is used to insert specific Cache-control values. It is provided by the mod_header module, which is activated by default.

NameVirtualHost *

...
   
      ExpiresActive On
      ExpiresDefault "access plus 10 minutes"
      ExpiresByType text/html "access plus 1 day"
      ExpiresByType text/javascript "access plus 1 day"
      ExpiresByType text/x-json "access plus 1 day"

      ExpiresByType image/gif "access plus 1 month"
      ExpiresByType image/png "access plus 1 month"
      ExpiresByType image/jpg "access plus 1 month"


      
            Header set Cache-control "no-cache"
      

   
...

HTTP caching explained

David Corvoysier — Wed, 11 Feb 2009 13:08:44 +0000

Web developers can take advantage of HTTP caching mechanisms to improve significantly the performance and the robustness of their site.

HTTP caching mechanisms are however not well-known and often not-well understood. This article aims at clarifying things without going into too much details.

For a more complete description of these mechanisms, please refer to the HTTP/1.1 RFC.

The HTTP/1.1 protocol includes two basic “caching” mechanisms to:

eliminate the need to send requests in many cases, using an “expiration” mechanism,
eliminate the need to send full responses in many other cases, using a “validation” mechanism.

The HTTP/1.1 Expiration model

The HTTP/1.1 expiration model, as defined in [RFC2616 §13.2], is designed to avoid the client making requests to the origin server until the corresponding content has expired.

The HTTP/1.1 expiration model does not mandate that the origin server explicitly specifies expiration dates for contents. For that reason, each client needs to define its own caching algorithm.

Note: A simple algorithm is provided in [RFC2616 §13.2], based on the last modification date of a document. If that information is provided, then the expiration date equals ten percent of the difference between the reception date and the modification date.

However, deterministic client behaviour is often desirable to avoid server congestion, and two headers can be used to explicitly specify the expiration date of a particular content:
• Expires (see [RFC2616 §14.21]),
• Cache-control, max-age directive (see [RFC2616 §14.9]).

Note: The Cache-control header always takes precedence over the Expires header

Example:

A page that expires in ten minutes (see how both headers are here redundant)

HTTP/1.1 200 OK
Date: Mon 06 Oct 2008 12:33:48 GMT
Server: Apache/2.2.8
Cache-Control: max-age=600
Expires: Mon 06 Oct 2008 12:43:48 GMT

More, the Cache-control header can also be used to further control the cache handling on the client side, by using a set of directives that directly control the client caching algorithm.

Example:

Never cache this page (Expires header will be ignored):

HTTP/1.1 200 OK
Date: Mon 06 Oct 2008 12:33:48 GMT
Server: Apache/2.2.8
Cache-Control: no-cache
Expires: Mon 06 Oct 2008 12:43:48 GMT

Always revalidate this page (Expires header will be ignored):

HTTP/1.1 200 OK
Date: Mon 06 Oct 2008 12:33:48 GMT
Server: Apache/2.2.8
Cache-Control: must-revalidate
Expires: Mon 06 Oct 2008 12:43:48 GMT

The HTTP/1.1 Validation model

The HTTP/1.1 validation model, as defined in [RFC2616 §13.3], is designed to request the client to verify first if a content has changed before fetching it from the server.

To verify if a stale cache entry is still valid, the client simply inserts specific headers in the request it sends to the origin server, passing back a specific tag it has received in the initial request. If the origin server recognizes that the tag corresponds to a content that hasn’t changed, it responds with a “304 Not Modified”. Otherwise, it sends back the updated document.

Note: Hints that this mechanism is at work can thus be found by looking for “304 Not Modified” responses in the web server logs.

The following two headers are typically used to tag a particular content:
• Last-Modified (see [RFC2616 §14.29]),
• Etag (see [RFC2616 §14.19]).

Caching documents retrieved through HTTPS

By default, most browsers do not cache documents retrieved through HTTPS for security reasons.

However, the origin server can use the Cache-Control header to specify that a specific document is public, and can thus be cached.

Example:

HTTP/1.1 200 OK
Date: Mon 06 Oct 2008 12:33:48 GMT
Server: Apache/2.2.8
Cache-Control: public
Expires: Mon 06 Oct 2008 12:43:48 GMT