Caches can be used to improve the performance of web applications, that is reduce their response times.

The general idea of a cache is to save the results of an expensive operation for quick retrieval without having to perform the expensive operation. The gives rise to the issue of potentially stale data that will need to be invalidated.

To demonstrate where caches can be used to improve performance we will look at a familiar shop catalogue page.

Example

A typical pipeline for getting this page to a client:

1. A client (e.g. web browser) makes a request for the page
2. Server retrieves product data (e.g. stock, price) from a source (e.g. database)
3. Server inserts the data into a HTML template with references to static files like JS, CSS and images
4. Server sends the resulting files back to the client

Service Cache

Retrieving data from a database or other service can be slow and burdensome, so a cache may be a good option.

Cache invalidation

• If we manage the gets and updates to a piece of data, we can cache the gets and simply invalidate the cache after we perform an update.
  • e.g. we get and cache the current stock at 5, someone buys the product so we update the stock to 4 and
• If the data is not expected to change often, we can opt to set a static Time to Live (TTL) that matches the cadence at which our data changes
  • e.g. Price changes happen once a day, so we can set a TTL of 1 day

Since we have control of this cache, in the event we need to invalidate it, we can do it manually.

Implementation

There is no specific technology to use, but you will likely see the following:

• Local cache
  • Any key-value store that is local to each server instance, so no sharing
  • Simple to setup as most languages offer the concept of a key-value store
• Shared cache
  • e.g. redis or memcached
  • Flexible, as it just lifts the key-value store to be used by all instances of our server
• Service specific
  • Database like ElasticSearch and Postgres have their own proprietary caching mechanisms

Content Delivery Network

Our server is located in Australia, but we have clients in the US, Europe, etc. A Content Delivery Network (CDN) can greatly improve delivery speed of files to these distant clients by caching our files on servers closer to those clients.

When using a CDN, typically 1 client in a region will incur the cost of making a trip all the way to our server for the files. At this point the CDN has cached our request, and all subsequent requests by any client in the region will now try to get the files from the CDN.

Cache Invalidation

CDNs typically allow us to configure a TTL for certain files.

A common strategy is:

• Do not cache .html files, that is a TTL of 0
  • .html are inherently tied to how a user accesses our page at a given URL, e.g. /products will try to retrieve /products.html
• Aggressively cache everything else with a large TTL (e.g. 1 year)
  • HTML is the endpoint that contains references to these other files, e.g. .css, .js, .jpg, etc. So as long as we can change the HTML, we can update which resources we are fetching.
  • Cache-busting is a technique where we attach a unique hash to the file name that changes when the underlying file changes
    1. An image image.jpg has a unique hash of 123abc
    2. The image will be referenced in the HTML at image.123abc.jpg
    3. Clients will fetch image.123abc.jpg and the CDN will cache that file so future requests hit the CDN
    4. We change the size of the image, and regenerate the hash to def456
    5. The image will now be referenced in the HTML at image.def456.jpg
    6. Clients will fetch image.def456.jpg, missing the cache as the file name does not exist

Since we control the CDN to an extent, in the event we need to invalidate its cache, we can do it manually.

Implementation

• AWS and Cloudflare operate some of the largest CDN, that is they have a lot of servers all over the world for every region

Client Cache

Certain clients, like web browsers, can cache files from the server on the local machine. It is often convenient to think of these like ultra-local CDN, where each node in the network is a single user.

This is a cost-effective way of adding caching and reducing the load/cost on a CDN and our server.

Cache Invalidation

This is where things get a bit scary. Since this all happens on the client, mistakes are hard to rectify as we don’t have access to invalidate every client’s cache. A misonconfigured TTL can result in a client silently never getting the newest code updates.

A similar strategy to CDN can be used, where we do not cache the .html entrypoint, but cache everything else - as long as we have a cache-busting mechanism.

Implementation

• HTTP Cache-Control header
  • max-age is behaves much like a TTL
• Service workers acts as a proxy between the browser and the network. It can do cool things like caching, background fetching, etc - but a misconfiguration can be detrimental.