Rails server latency is very slow - ruby-on-rails

Question

I have a Rails app on Passenger on Nginx on EC2 micro and the server time on an empty page is 3 seconds. Empty page means there's no layout, no JS, no CSS, HTML only consists of header and body tags with a "TEST" text. The controller does nothing.
Here is the server wait time from Chrome console for the empty page:
I also ran ruby-prof on the home page of my app and the total time is 0.2 seconds:
Total Time: 0.2830359935760498
Curiously, I created a new Rails app and deployed that instead, and the server wait time shown on Chrome is about ~1.8 seconds, so 1 second faster.
Anyone have any idea what I can do or try? I am using an EC micro instance. Should I move to a box with better network? I feel like this shouldn't be the issue. I've also tried similar testing with Apache instead of Nginx and had similar results. I don't think I have anything weird in Rails.
This is in my Gemfile:
source 'https://rubygems.org'
gem 'rails', '4.1.8'
gem 'sass-rails', '~> 5.0.1'
gem 'uglifier', '>= 1.3.0'
gem 'bower-rails'
gem 'angular-rails-templates'
gem 'therubyracer', platforms: :ruby
gem 'jquery-rails'
gem 'jquery-ui-rails'
gem 'jbuilder', '~> 2.0'
gem 'sdoc', '~> 0.4.0', group: :doc
gem 'spring', group: :development
gem 'rake'
group :development, :test do
gem 'rspec-rails', '~> 3.5'
end
group :development do
gem 'pry'
gem 'better_errors'
gem 'binding_of_caller'
gem 'byebug'
end
gem 'oauth2'
gem 'stripe'
gem 'aws-sdk', '~> 2'
gem 'passenger'
gem 'google-adwords-api'
gem 'google-ads-common'
gem 'google-api-client'
gem 'httparty'
gem 'angularjs-rails'
gem 'dotenv-rails', :groups => [:development, :test, :production]
gem 'mailgun-ruby', '~>1.1.2'
gem 'font-awesome-rails'
gem 'ultrahook'
gem 'rack-reverse-proxy', :require => 'rack/reverse_proxy'
gem 'omniauth-google-oauth2'
gem 'city-state'
gem 'readmorejs-rails'
gem 'nokogiri'
gem 'rollbar'
gem 'ruby-prof'