Apache mod_rewrite cache
Wednesday, July 22nd, 2009Lets pretend you have a stock ticker application that is getting hammered. A lot of people are interested in one particular stock for some reason.
The poor little app server is running some lumbering hulking piece of code written in a legacy language (java). It can’t keep up with all the requests for this same stock symbol over and over.
Furthermore, the developers haven’t been able to put their own caching code into the application just yet.
How do we fix this, from a system administrators perspective?
Perhaps the better way would be to use mod_cache.
Unfortunately, this option did not exist on our web servers and we needed something PDQ to take the load off the app servers. We went with mod_rewrite instead and a small script to do the caching.
Cron runs a script every 5 minutes, which calls the app server, caches the results in a file. Then mod_rewrite rules tell Apache to use that instead of going to the app server (via existing rewrite rules)
In the apache virtual host entry :
RewriteCond %{REQUEST_URI} ^/ticker/s=AAPL$
RewriteCond /app/ticker/cache/AAPL -f
RewriteRule ^(.*) /app/ticker/cache/AAPL [L]
A script to maintain the cache :
#!/bin/bash
export PATH=/usr/bin
CACHEFILE=/app/ticker/cache/AAPL
TMPFILE=${CACHEFILE}.$$
trap "/bin/rm ${TMPFILE}" 0 1 15
curl -D - -H "host: www.example.com" \
"http://app01/ticker/s=AAPL" > ${TMPFILE} \
&& mv ${TMPFILE} ${CACHEFILE}
The cron entry to make it go :
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /app/ticker/bin/update-ticker-cache >/dev/null 2>&1
Now when the outside url http://www.example.com/ticker/s=AAPL is hit, apache looks for the file /app/ticker/cache/AAPL and delivers its contents instead of passing the request through to the app server (via other rewrite rules).
Cron and the script keep this cache updated every 5 minutes.
The graph speaks for itself.
This is based on a real world example. Details have been changed to protect the guilty parties.
