Heroku: see params and sql activity in logs? - ruby-on-rails

When i view my heroku logs on the server (with heroku logs --tail --app myapp) i see something like this:
2011-06-21T14:09:25+00:00 app[web.1]: Started PUT "/reports/19.xml" for 77.89.149.137 at 2011-06-21 07:09:25 -0700
2011-06-21T14:09:25+00:00 heroku[router]: PUT myapp.heroku.com/reports/19.xml dyno=web.1 queue=0 wait=0ms service=7ms status=401 bytes=28
2011-06-21T14:09:26+00:00 heroku[nginx]: PUT /reports/19.xml HTTP/1.1 | 77.89.149.137 | 656 | http | 401
While in my local log i'd see something like this:
Started PUT "/reports/19" for 127.0.0.1 at 2011-06-21 15:27:01 +0100
Processing by ReportsController#update as XML
Parameters: {"report"=>{"workflow_status"=>"3"}, "id"=>"19"}
Person Load (0.9ms) SELECT "people".* FROM "people" WHERE "people"."email" = 'madeupemai#lkklj.com' LIMIT 1
Report Load (0.4ms) SELECT "reports".* FROM "reports" WHERE "reports"."id" = 19 LIMIT 1
DEPRECATION WARNING: Object#returning has been deprecated in favor of Object#tap. (called from update_report at /home/max/work/rails_apps/flamingo_container/flamingo/vendor/plugins/resource_this/lib/resource_this.rb:135)
Creating scope :open. Overwriting existing method Task.open.
Task Load (2.0ms) SELECT "tasks".* FROM "tasks" WHERE "tasks"."id" = 14 LIMIT 1
Completed 200 OK in 1648ms (Views: 568.2ms | ActiveRecord: 3.2ms)
Ie with a lot more information, particularly the params, info from the router, generated sql, any templates rendered etc etc.
Does anyone know how i can get my heroku log to be as verbose as my development one? I've done the following already:
1) Set the log level in the relevant (rails 3) environment file:
config.log_level = :debug
2) Told heroku to use a different logger level, as described on http://devcenter.heroku.com/articles/logging
heroku config:add LOG_LEVEL=DEBUG --app myapp #from CLI
Neither has made any difference...any ideas anyone?
thanks, max

You're essentially wanting to show the SQL / params output in the Heroku logs. You can do this by adding the line shown below to the config block within your production.rb file:
MyAppNameHere::Application.configure do
# add this line
config.logger = Logger.new(STDOUT)
end
By the way, setting the log level to debug just means that Rails.logger.debug will output to the logs when you're on Heroku

In your production.rb add config.log_level = :debug and redeploy. That will give you the same logs as development mode :)

The detailed log you want is generated by the function start_processing in log_subscriber.rb.
action_controller/log_subscriber.rb:
def start_processing(event)
payload = event.payload
params = payload[:params].except(*INTERNAL_PARAMS)
info " Processing by #{payload[:controller]}##{payload[:action]} as #{payload[:formats].first.to_s.upcase}"
info " Parameters: #{params.inspect}" unless params.empty?
end
I checked with rails 3.0.4 in development and production environment. In both environments, we have the detailed logs.
This is an info level log. That's why the debug log level is not changing the output.
I installed the plugin used by heroku rails_log_stdout (Heroku logging) but I still have the desired output.
Right now, I can't test with heroku to find out why you don't have all the logs.
In the heroku example (see Heroku logging, section log retrieval), we don't see the "Processing" and "Parameters" lines. I think that either this method is not called when the app is running on heroku (it is somehow disabled) or heroku skips logs starting with whitespaces. Could you try to log messages starting with whitespaces and see if heroku is showing them?

Rails doesn't generate those logs when in production mode. http://groups.google.com/group/heroku/browse_thread/thread/d778fafedc9a378a

The real problem is actually due to the way Heroku works in conjunction with rails. The right way to solve this is to add: gem 'rails_12factor', group: :production
See https://devcenter.heroku.com/articles/rails-integration-gems for more info on the matter

Looks like the puma server doesn't play well with Heroku. In my project, I tried everything, but it still wouldn't log. I then replaced Puma with Unicorn, and bam, full logs are being shown.

Related

Google API Services - Could not parse PKey: no start line in production (heroku) only

ANSWER: The two environment variables were mixed up in Heroku. CERT pointed to the private key and PRIVATE_KEY pointed to cert. Fixed that. Authorization now works.
Will mark this question answered when I can.
I am hooking into the Google Analytics Reporting API with a services account from my Rails 4 application. I have set up the authorization process and got it working in development. Upon deployment, I realized that using the .p12 key file was a problem - I either had to commit it to my repository or find a way around it. So I found a way around it, by breaking the key file into two .PEM files and storing the contents of those files as environment variables.
Works great in development (am using dotenv-rails to store environment variables there). But something must be happening to the variables on Heroku, because when I try to use my authorization method in production, I get the error ArgumentError: Could not parse PKey: no start line.
I've searched around for some clue about what's going on here - I've seen that related errors can happen due to version problems (someone downgraded from Ruby 2.2 to 1.9.8 and fixed a similar problem, although theirs just flat out didn't work anywhere, and mine works in development). Downgrading my Ruby version is not really an option. :/
Here's the related code:
def authorize_with_services
begin
self.client = Google::APIClient.new(application_name: 'Playground', application_version: '1')
p12 = OpenSSL::PKCS12.create('notasecret', 'descriptor',
problem spot --> OpenSSL::PKey.read(ENV['PRIVATE_KEY']),
OpenSSL::X509::Certificate.new(ENV['CERT']))
p12_binary = p12.to_der
self.client.authorization = Signet::OAuth2::Client.new(
token_credential_uri: 'https://accounts.google.com/o/oauth2/token',
audience: 'https://accounts.google.com/o/oauth2/token',
scope: 'https://www.googleapis.com/auth/analytics.readonly',
issuer: GA_EMAIL,
signing_key: Google::APIClient::PKCS12.load_key(p12_binary, 'notasecret')
).tap { |auth| auth.fetch_access_token! }
rescue Signet::AuthorizationError
self.client = nil
end
self.client
end
And then the ENV variables as stored in .env:
PRIVATE_KEY: "Bag Attributes\n friendlyName: privatekey\n localKeyID: 54 69 6D 65 20 31 34 33 35 36 38 34 30 33 31 30 32 39\nKey Attributes: <No Attributes>\n-----BEGIN RSA PRIVATE KEY-----\nKEY\n-----END RSA PRIVATE KEY-----\n"
CERT: "Bag Attributes\nfriendlyName: privatekey\nlocalKeyID: 54 69 6D 65 20 31 34 33 35 36 38 34 30 33 31 30 32 39\nsubject=/CN=636085506886-096q9j1uotf9kp1tv4evhn2crip6dec8.apps.googleusercontent.com\nissuer=/CN=636085506886-096q9j1uotf9kp1tv4evhn2crip6dec8.apps.googleusercontent.com\n-----BEGIN CERTIFICATE-----\nCERT\n-----END CERTIFICATE-----\n"
And I've set them exactly the same in heroku config:set PRIVATE_KEY and heroku config:set CERT.
Edit
Here's the session in the Heroku console which leads to the error:
# 11:15:36 (ruby-2.1.5#yt-ga-playground) ~/projects/confreaks/yt-ga-playground (master)$ heroku run console
Running `console` attached to terminal... up, run.8357
Loading production environment (Rails 4.2.0)
irb(main):001:0> querier = AnalyticsQuerier.new
=> #<AnalyticsQuerier:0x007f81fc6f5358 #options={"ids"=>"ga:104082366", "start-date"=>"2015-01-01", "end-date"=>"2015-07-04", "metrics"=>"ga:totalEvents"}, #ids="ga:104082366", #start_date="2015-01-01", #end_date="2015-07-04", #metrics="ga:totalEvents">
irb(main):002:0> querier.authorize_with_services
ArgumentError: Could not parse PKey: no start line
from /app/app/services/analytics_querier.rb:56:in `read'
from /app/app/services/analytics_querier.rb:56:in `authorize_with_services'
from (irb):2
from /app/vendor/bundle/ruby/2.1.0/gems/railties-4.2.0/lib/rails/commands/console.rb:110:in `start'
from /app/vendor/bundle/ruby/2.1.0/gems/railties-4.2.0/lib/rails/commands/console.rb:9:in `start'
from /app/vendor/bundle/ruby/2.1.0/gems/railties-4.2.0/lib/rails/commands/commands_tasks.rb:68:in `console'
from /app/vendor/bundle/ruby/2.1.0/gems/railties-4.2.0/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
from /app/vendor/bundle/ruby/2.1.0/gems/railties-4.2.0/lib/rails/commands.rb:17:in `<top (required)>'
from bin/rails:8:in `require'
from bin/rails:8:in `<main>'
And the equivalent session in my development console:
# 11:17:47 (ruby-2.1.5#yt-ga-playground) ~/projects/confreaks/yt-ga-playground (master)$ be rails c
Loading development environment (Rails 4.2.0)
2.1.5 :001 > querier = AnalyticsQuerier.new
=> #<AnalyticsQuerier:0x007fe97d380928 #options={"ids"=>"ga:104082366", "start-date"=>"2015-01-01", "end-date"=>"2015-07-04", "metrics"=>"ga:totalEvents"}, #ids="ga:104082366", #start_date="2015-01-01", #end_date="2015-07-04", #metrics="ga:totalEvents">
2.1.5 :002 > querier.authorize_with_services
=> #<Google::APIClient:0x007fe97d36bc80 #host="www.googleapis.com", #port=443, #discovery_path="/discovery/v1", #user_agent="Playground/1 google-api-ruby-client/0.8.2 Mac OS X/10.10.3\n (gzip)", etc.>
And I have a controller method which uses the AnalyticsQuerier. I pushed it to production when I thought that it would work. Here's how that plays out in the logs:
[...] app[web.1]: Started POST "/events" for 104.231.12.227 at 2015-07-04 15:14:11 +0000
[...] app[web.1]: Completed 500 Internal Server Error in 16ms
[...] app[web.1]: Processing by EventsController#create as HTML
[...] app[web.1]: Parameters: {"authenticity_token"=>"1fvzDI1n+AlTssDybGFHAyiG4M+kir2UDLsA4dZGiAJvrWXdK6OnlAf1wK0V+/dI4QXz3lxonmRw15XbFjpOAQ=="}
[...] app[web.1]: (0.8ms) SELECT COUNT(*) FROM "analytics_calls"
[...] app[web.1]:
[...] app[web.1]: ArgumentError (Could not parse PKey: no start line):
[...] app[web.1]: app/services/analytics_querier.rb:56:in `read'
[...] app[web.1]: app/services/analytics_querier.rb:56:in `authorize_with_services'
[...] app[web.1]: app/controllers/events_controller.rb:10:in `create'
2015-07-04T15:14:11.542398+00:00 app[web.1]:
[...] app[web.1]: app/models/analytics_call.rb:19:in `get_batch'
[...] app[web.1]:
[...] heroku[router]: at=info method=POST path="/events" host=desolate-fortress-9280.herokuapp.com request_id=3822f668-6b8c-4cb2-9764-c171fc0dc67b fwd="104.231.12.227" dyno=web.1 connect=3ms service=33ms status=500 bytes=1754
Edit 2 - Possible lead in the right direction?
So, as pointed out by Val Asensio, there's a problem with the SSH setup provided by Heroku (this is more or less unfamiliar territory for me, but I'll try to word things as clearly as possible, given my understanding needs some fleshing out here).
I've found a gem called heroku-buildback-ssh, which definitely suggests that you need to do more than just load a private key into an env variable in order to use it properly. However, the readme specifies that the private cannot have a passphrase in order to be accessed. It also seems to assume you're using a file called something rsa, with no indication that this works for .pem files?
I found another discussion here about ssh tunneling from Heroku. I think this is the route to take. My understanding is that I need to be able to 'rebuild' the private key file using the env variable PRIVATE_KEY behind the scenes, and that I can write a script to do this when Heroku starts up.
However, my understanding here is so shaky, and the use-case described in the answer script is different enough that I'm not sure how to extrapolate how to go about building a solution using this kind of method.
What I want to know now is, can I build the .pem key file using a bash script that Heroku runs behind the scenes, and if so, how do I then make sure that that file is what is being read when my script runs OpenSSL::PKey.read? I guess I could also try to build the original .pk12 file itself instead?
ALSO - big question: am I going to screw anything up so terribly that I'm left in a tangled knot of confused code and have to trash the whole application (it's a code spike application, but I still want to know if anyone can provide an answer here) if I start poking around with writing scripts that create files and manage permissions on those files in my production environment?
For non-Heroku deployments
Although I've not used this particular Google API, this seems like an SSH key problem on the server- the PKey error tells us that. The error indicating "no start line" leads me to think that a key part of the SSH setup is missing on the server side.
App side
Make sure you have this in your deploy.rb
ssh_options[:forward_agent] = true
Server side
The server side SSH must be setup to access the Google API. Do you have an
~/.ssh/config file? Do you have the Google API SSH set there?
# Google API
Host hostservingAPIname.com
User yourusername
IdentityFile ~/.ssh/svn_id_rsa
# Your app
Host yourappsdomian.com
User yourrailsappusername
IdentityFile ~/.ssh/id_rsa
ControlMaster auto
I'm not sure this info will be helpful, but since it doesn't seem like anyone who has the definitive answer is going to chime in, I hope my suggestion will be helpful in moving you towards a solution.
Note: Someone may search for this question and their app is not hosted on Heroku, so they may find this answer useful, so I'll leave it here.
I had the two environment variables mixed up.
That's it.
That's all it was.

Ruby on Rails, image compilation and precompilation

I just deployed a rails app and I am having an error when I load a page saying:
We're sorry, but something went wrong.
In the production.log I see that message:
Started GET "/" for 128.179.252.130 at 2014-11-30 08:32:28 +0000
Processing by StaticPagesController#home as HTML
Rendered static_pages/home.html.erb within layouts/application (0.4ms)
Rendered layouts/_shim.html.erb (0.0ms)
Rendered layouts/_header.html.erb (2.4ms)
Completed 500 Internal Server Error in 4ms
ActionView::Template::Error (./icons/icon_nameplate.png isn't precompiled):
37:<ul class="dropdown-menu">
38:
39:<li>
40: <%= menu_dropdown('Profile', "./icons/icon_nameplate.png", "Nameplate Icon", current_user) %>
41:</li>
So I believe the error comes from this part of the message:
./icons/icon_nameplate.png isn't precompiled
I searched the internet, found three possible solution, none of which did work:
1. config.assets.compile = true (in production.rb file)
2. config.assets.precompile += %w[*.png *.jpg *.jpeg *.gif] (still in production.rb file)
3. bundle exec rake assets:precompile RAILS_ENV=production
All of this changed absolutely nothing, and the website still shows the same error. So I am turning for you to help.
Thank you very much in advance.
What finally did the trick for me is completely restart the rails app. This is described in the link below.
The recommended version did the trick for me which is:
Create (or update) a file called restart.txt in the tmp folder of your Rails application on the server.
After that no problem at all.
Cheers

Ruby on Rails unicorn throws errors

I setup a droplet with Ruby on Rails on digitalocean and I removed the rails directory (App) and replaced it with the one I had locally. It is working fine, if I connect with ssh and go to the IP with the port 3000. However whenever I try to go to port 80, then it throws me an error:
http://prntscr.com/2qoljb
I have checked the logs and it says the following:
Rails Error: Unable to access log file. Please ensure that /home/rails/log/production.log exists and is chmod 0666. The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
Rails Error: Unable to access log file. Please ensure that /home/rails/log/production.log exists and is chmod 0666. The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
I, [2014-02-08T18:40:59.192086 #735] INFO -- : worker=1 ready
I, [2014-02-08T18:40:59.198211 #731] INFO -- : worker=0 ready
Started GET "/" for 188.193.201.119 at 2014-02-08 18:41:00 +0000
Processing by QuestionsController#index as HTML
Completed 500 Internal Server Error in 43ms
ActiveRecord::StatementInvalid (Could not find table 'users'):
app/controllers/application_controller.rb:18:in `current_user'
app/controllers/application_controller.rb:10:in `require_login'
Started GET "/" for 188.193.201.119 at 2014-02-08 18:41:01 +0000
Processing by QuestionsController#index as HTML
Completed 500 Internal Server Error in 47ms
ActiveRecord::StatementInvalid (Could not find table 'users'):
app/controllers/application_controller.rb:18:in `current_user'
app/controllers/application_controller.rb:10:in `require_login'
Started GET "/" for 188.193.201.119 at 2014-02-08 18:41:02 +0000
Processing by QuestionsController#index as HTML
Completed 500 Internal Server Error in 2ms
ActiveRecord::StatementInvalid (Could not find table 'users'):
app/controllers/application_controller.rb:18:in `current_user'
app/controllers/application_controller.rb:10:in `require_login'
Started GET "/" for 188.193.201.119 at 2014-02-08 18:41:02 +0000
Processing by QuestionsController#index as HTML
Completed 500 Internal Server Error in 2ms
ActiveRecord::StatementInvalid (Could not find table 'users'):
app/controllers/application_controller.rb:18:in `current_user'
app/controllers/application_controller.rb:10:in `require_login'
The tables should exist, as everything is working completely fine, if I manually type in the IP with port 3000...
Thanks for your help in advance
EDIT
After setting permissions and the owner I am getting the following error instead:
/usr/local/rvm/gems/ruby-2.0.0-p353/gems/bundler-1.3.5/lib/bundler/runtime.rb:216: warning: Insecure world writable dir /home/rails in PATH, mode 040777
/usr/local/rvm/gems/ruby-2.0.0-p353/gems/bundler-1.3.5/lib/bundler/runtime.rb:216: warning: Insecure world writable dir /home/rails in PATH, mode 040777
I, [2014-02-08T19:27:30.231141 #1984] INFO -- : worker=1 ready
I, [2014-02-08T19:27:30.292714 #1981] INFO -- : worker=0 ready
First two lines explicitly talk about wrong file permissions but the Unicorn process shoud run under the same user as log file created by. I guess the capistrano and Unicorn run under different users.
If you are not sure what users should be and you have no problems with security just run:
$ chmod 0666 /home/rails/log/production.log
Once you restart the Unicorn the issue should gone.
I would not care about the problem with the log file in the moment. I do not think that is the root cause.
What catches my eye is Could not find table 'users'. The app does not find a table users in the database or does not find the database at all. Did check your credentials to the production database in config/database.yml? Did you run migrations on the database? Perhaps connect to the production database and check if the schema looks like you expect it to be.

Different output in server log

I have 2 Ruby on Rails projects. When I start a rails server for the first project and login, I see this in the console:
Started POST "/users/sign_in" for 127.0.0.1 at 2013-11-15 10:40:24 +0100
When I start the rails server for the second project and login, I see this in the console:
Started POST "/users/sign_in" for 127.0.0.1 at 2013-11-15 10:39:46 +0100
Processing by SessionsController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"s8eI2YYtuduMy1Hxa7kJJUeCJoLi5pjEb7FmhpOE5/c=", "user"=>{"email"=>"test#test.nl", "password"=>"[FILTERED]"}, "commit"=>"Inloggen"}
User Load (4.5ms) SELECT "users".* FROM "users" WHERE "users"."email" = 'ed#salves.nl' LIMIT 1
I don't see any differences between the setup of both projects, but how can I make sure the first project also shows the exetended information?
Rails Logger verbosity can be defined with config.log_level in your config/environments files.
You can set it to :debug, :info, :warn, :error, :fatal.
Make sure your first project has the same settings of your second one.
I figured out it had to do with the recent switch to unicorn. Unicorn by default outputs the log to a file instead of STDOUT. So log\development.log is filled with all log messages, but not the terminal screen. To fix this, add this to config/development.rb:
config.logger = Logger.new(STDOUT)
config.logger.level = Logger.const_get(
ENV['LOG_LEVEL'] ? ENV['LOG_LEVEL'].upcase : 'DEBUG'
)

Deployng ElasticSearch on Heroku with Tire & Bonsai.io

I'm trying to setup the new Bonsai ElasticSearch heroku add-on, but until now I can just run the search on my local developement machine, as soon as deployed on Heroku the same search fails silently, or better, Rails logger doesn't log anything while I get a 500 Internal Server Error. This are relevant Heroku logs during the failing query (no evident errors found): https://gist.github.com/2867221
I'm looking for some advices on how troubleshoot that. For example, I set Tire.configure { logger 'elasticsearch.log', :level => 'debug' } on Bonsai initializer config/initializers/bonsai.rb like the following:
Tire.configure { logger 'elasticsearch.log', :level => 'debug' }
if ENV['BONSAI_INDEX_URL']
Tire.configure do
url "http://index.bonsai.io"
end
BONSAI_INDEX_NAME = ENV['BONSAI_INDEX_URL'][/[^\/]+$/]
else
app_name = Rails.application.class.parent_name.underscore.dasherize
app_env = Rails.env
BONSAI_INDEX_NAME = "#{app_name}-#{app_env}"
end
but it logs on a file, which is good for local but is not accessible after deployed.
Bonsai ElasticSearch is a pritty new on-testing addon, so I wonder if someone of you guys has already had some succesfully experiences deploing that on Heroku and can give me a feedback and some advices.
UPDATE
Setting up logger logger $stdout, :level => 'debug' into Tire.configure finally heroku can log something while it's failing search :
2012-06-04T21:36:26+00:00 heroku[router]: GET gitwatcher.com/categories?utf8=%E2%9C%93&query=Asynchronous+Web+Frameworks dyno=web.1 queue=0 wait=0ms service=32ms status=500 bytes=728
2012-06-04T21:36:26+00:00 heroku[nginx]: 93.34.212.216 - - [04/Jun/2012:21:36:26 +0000] "GET /categories?utf8=%E2%9C%93&query=Asynchronous+Web+Frameworks HTTP/1.1" 500 728 "http://gitwatcher.com/categories" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:12.0) Gecko/20100101 Firefox/12.0" gitwatcher.com
2012-06-04T21:36:26+00:00 app[web.1]: # 2012-06-04 21:36:26:387 [_search] (["123456789"])
2012-06-04T21:36:26+00:00 app[web.1]: #
2012-06-04T21:36:26+00:00 app[web.1]: curl -X GET "http://index.bonsai.io/123456789/category/_search?load=true&pretty=true" -d '{"query":{"query_string":{"query":"Asynchronous Web Frameworks"}}}'
2012-06-04T21:36:26+00:00 app[web.1]:
2012-06-04T21:36:26+00:00 app[web.1]: # 2012-06-04 21:36:26:387 [200] (1 msec)
2012-06-04T21:36:26+00:00 app[web.1]:
... please see more on issue opened to https://github.com/karmi/tire/issues/365
Just for tracking purposes,
I'll report here (cat&past) the resolution I post on GitHub Tire issues:
DEFINETLY WORKS BY:
1) dropping out `to_indexed_json` from Category MongoID model ( at the opposit of README indication )
2) removing and re-adding Bonsai.io Heroku add-on
3) reindexing ElasticSearch by running `heroku run:detached rake environment tire:import CLASS='Category'`
... infact, if you just run rake environment tire:import CLASS='Category' FORCE=true it doesn't work on Heroku/Bonsai.io, becuse of Bonsai authorized operation :
2012-06-11T20:39:30+00:00 app[run.1]: curl -X DELETE http://index.bonsai.io/my-fake-index-11223344
2012-06-11T20:39:30+00:00 app[run.1]: # 2012-06-11 20:39:30:749 [401]
2012-06-11T20:39:30+00:00 app[run.1]: #
2012-06-11T20:39:30+00:00 app[run.1]: # "{\"error\": \"Not authorized: Some endpoints are admin-only, ask support#onemorecloud.com.\"}\n"
than if you want to reset/reindex your eventually dirty data, the only way is dropping the Bonsai add-on and recreate:
heroku addons:remove bonsai:test
heroku addons:add bonsai:test
That saied, I really had a dirty data indicated by :
BSON::InvalidObjectId (illegal ObjectId format: LAm-hNglS5mbrMzDlVVHCQ):
app/controllers/categories_controller.rb:25:in `index'
If you'll need the full gem stack to compare with other cases, here it is: https://gist.github.com/2912881

Resources