A Place for my head

Since, we started to scale to multi machine clustering, our rails application was showing one weird problem. We were generating Stock Market Technical charts and caching them in memcache (with 1 hour ttl). During testing we found that, if requests gets routed to mongrel cluster running on other machine, charts aren’t appearing, which was weird because we had memcache cluster configured properly in “production.rb” file:

1
2
3
4
5
6
7
8
9
10

memcache_options = {
  :c_threshold => 10_000,
  :compression => true,
  :debug => false,
  :namespace => 'foobar'
  :readonly => false,
  :urlencode => false
}
CACHE = MemCache.new(memcache_options)
CACHE.servers = SIMPLE_CONFIG_FILE["memcache_servers"]

Where SIMPLE_CONFIG_FILE['memcache_servers'] contains list of memcache clusters participating in clustering. After, debugging for few hours(*gasp*) and turning on verbose logging on all participating memcache clusters, I found that, trusty CacheFu , was replacing the CACHE constant with following code:

1
2
3
4
5
6
7

silence_warnings do
  Object.const_set :CACHE, memcache_klass.new(config)
  Object.const_set :SESSION_CACHE, memcache_klass.new(config) if config[:session_servers]
end
 
CACHE.servers = Array(config.delete(:servers))
SESSION_CACHE.servers = Array(config[:session_servers]) if config[:session_servers]

Now this deal is real (and sucks). After couple of minutes of hacking, I took out cache_fu config file out of svn,wrote code to generate it on the fly during deployment. Now, pigs can fly.