Varnish-Cache for OpenSim Grid Services

11-Nov-2010

My job is working as an IT consultant and my current contract is all about Varnish-Cache and how to accelerate some of the biggest web portals run by the Deutsche Telekom AG in Darmstadt, Germany. Suddenly the idea was born to accelerate my OpenSim grid services for the Annuna Grid. In theory the R.O.B.U.S.T. server is a web server, too. Because my own web site (where this page is hosted) is also using a varnish, the only task would be to add a few lines to the current VCL configuration in order to pass all grid service request to a new backend.

In the varnish configuration file the following lines were added:

backend AnnunaGrid {
   .host = "grid.annuna.net";
   .port = "8003";
}

sub vcl_recv {

   [...]

   if (req.http.host ~ "grid.annuna.net") {
      set req.backend = AnnunaGrid;
   }

   [...]
}

After reloading varnish was prepared to serve requests for the Annuna Grid service. Next was reconfiguring the regions for the new connection using the varnish. Only one single file needs to be updated: bin/config-include/GridCommon.ini. Simple change is just removing the port number from the server URIs because I changed the grid service port from 8003 to 80 by putting the varnish in between.

Example:

[...]
; before:
; AssetServerURI = "http://grid.annuna.net:8003"
; after:
AssetServerURI = "http://grid.annuna.net"
[...]

Of course this change must be done for every service URI.

After restarting the regions they connect flawlessly using the new connection with the varnish cache inserted before the grid service. Next on my list is to examine the behaviour and do some more fine tuning using varnish’s VCL.

Guess I can say that Annuna Grid is the world’s first grid service provider using Varnish-Cache for acceleration purposes.

More about:

Using Keepalive probing for OpenSim Varnish-Cache

Normally a keepalive mechanism is used for monitoring backend status. On Apache this is usually a simply HTML page delivering a status code 200 when being fetched by the varnish. For OpenSim we don’t have the opportunity to put some simply HTML page on the webserver, but we can easily request one of the smaller default assets instead. Most important is that it delivers a status code 200 – and it does.

# backend definition for OpenSim grid services
backend AnnunaGrid {
   .host = "grid.annuna.net";
   .port = "8003";
   .probe = {
      .url = "/assets/00000000-0000-0000-0000-000000000000";
      .interval = 5 s;
      .timeout = 3 s;
      .window = 5;
      .threshold = 3;
   }
}