Subsequent unicorn requests are really slow - ruby-on-rails

Question

I'm running a rails 2 app on unicorn and experiencing weird behavior. When I first start the server and make the first request it's pretty fast (< 3 sec for entire page load). Every subsequent request takes upwards of 30-45 seconds. When I tail the logs nothing is actually being logged for the first 30+ seconds. Then all of a sudden the page just loads and the logging shows up. It's as if the request just hangs for over 30 seconds.
When I run the same code using script/server or thin it works fast and as expected.
Currently we are running unicorn on Heroku without these problems. I'm in the process of migrating to AWS and it's where I'm running in to the problems. I'm also running in to the same issue on my local VM when running unicorn.
Here is my configuration.
bundle exec unicorn -c ./config/unicorn.rb
worker_processes Integer(ENV["WEB_CONCURRENCY"] || 2)
timeout 90
preload_app true
listen (ENV['PORT'] || 8000), :backlog => Integer(ENV['UNICORN_BACKLOG'] || 20)
before_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
Process.kill 'QUIT', Process.pid
end
# AR
ActiveRecord::Base.connection.disconnect!
# Redis
begin
Timeout.timeout(2) { REDIS.quit }
rescue Exception
# This likely means the redis service is down
end
end
after_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to send QUIT'
end
# AR
ActiveRecord::Base.establish_connection
# Redis
uri = URI.parse(ENV["REDIS_URL"])
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
# Dalli
Rails.cache.reset
# Mongoid
Mongoid::Setup.run
end
UPDATE
I've narrowed it down to something preventing the worker from completing. I have a timeout of 30 seconds and even if the request only takes 1 sec to complete it will not accept any more requests until the timeout occurs. Are there any good ways to figure out what is preventing the worker from becoming available?