Services: Caching

The World Wide Web has changed the face of networking, making it possible for almost anyone to access on-line information, and to publish information for others to read. The technicalities have been concealed to the extent that multimedia information is available world-wide to users who may be unaware of its location or the demands on the intervening network infrastructure of its retrieval. The consequent enormous growth of traffic demand on local, national and international network backbones is far exceeding the rate at which additional bandwidth can reasonably be provided, even allowing for the falling costs which technology and deregulation are delivering. Although users expect and demand a high quality of service with modest response times, they are increasingly experiencing the World Wide Wait.

One remedy is to install a Web caching service. A cache contains copies of popular on-line documents and is located close to the users who most frequently request them. Browsers already use a local cache of their own to store recently-accessed documents and images, meaning that those documents do not have to be repeatedly transferred across the network if the users want to look at them again. However, if ten users want the same document, it still has to be fetched ten times even if subsequent accesses are fulfilled from the users' local caches. By setting up a cache which is shared by a larger group of users, it is possible to arrange for every document to be retrieved only once whether it is requested by one user in the group or by all of them. Users get faster response times, network managers see less traffic and Web servers see lower request rates. Based on information provided with web documents, content is stored and rechecked to ensure consistent and updated documents.

Local caching services are already in widespread use. DESIRE is taking the idea one step further and building a network of interconnected caches which serve local, regional, national and international users, with the ultimate aim of being able to provide a co-ordinated service across the research networks of Europe.

Already, both UNINETT (the national research network of Norway) and SURFnet (the national research network of the Netherlands) have set up national caching hierarchies which are interconnected with each other and with those of other countries (Picture of the UNINETT caching mesh, Picture of the SURFnet caching mesh).

A report on costs and benefits of building a web cache system was written to analyse the web cache system. Web traffic is reduced by 30-50% by sending it through a web cache system (one or more servers), depending on cache tuning and user homogeneity. Latency is reduced noticeably, depending on network load and connectivity, more reduction for bad networks than for sane ones. The analysis shows that on every level of the mesh (institutional cache, top level cache, whole mesh) the benefits of caching exceed the costs. Furthermore, a significant reduction in latency is achieved.

New Internet protocols, such as HTTP 1.1 and ICP are being integrated into the project, which is now also working closely with TERENA (the Trans-European Research and Education Networking Association) to improve the quality and scope of international caching services.