I have a Rails website which, for the last 24 hours has been running very slowly. The odd thing is that (see logs below) every request is taking 45 or 46 seconds, as though is is waiting for some kind of timeout. Static content is loading quickly and the server is otherwise performing normally.
The site is at http://plantality.com and is hosted by rail playground. It uses Apache + Passenger, Rails 2.1.1, attachment_fu, thinking sphinx, rmagick, bbruby.
Server log samples:
Processing WikiController#finder (for 66.249.67.103 at 2010-12-05 02:58:59) [GET]
Session ID: e9e947f58fb1f0f60eeff2fc62a2de72
Parameters: {"group"=>"1", "upright"=>"1", "houseplant"=>"1", "wi_6to12"=>"1", "bushy"=>"1", "action"=>"finder", "controller"=>"wiki", "pots_tubs"=>"1"}
Cookie set: referer=; path=/
Cookie set: geo_country=; path=/
Rendering template within layouts/global
Rendering wiki/finder
Completed in 45.06793 (0 reqs/sec) | Rendering: 0.03833 (0%) | DB: 0.00878 (0%) | 200 OK [http://plantality.com/finder?bushy=1&group=1&houseplant=1&pots_tubs=1&upright=1&wi_6to12=1]
Processing MainController#index (for 67.225.164.12 at 2010-12-05 02:59:44) [GET]
Session ID: 0389102261c509523911c65b28c7661b
Parameters: {"action"=>"index", "controller"=>"main"}
Cookie set: referer=; path=/
Cookie set: geo_country=; path=/
Rendering template within layouts/global
Rendering main/index
Completed in 45.42652 (0 reqs/sec) | Rendering: 0.23435 (0%) | DB: 0.04589 (0%) | 200 OK [http://plantality.com/]
Solution: I was using an IP to country geo-coding service. I coded it so that the result was cached in a cookie, but my site was getting about 100,000 daily hits from google. Google (and other spiders) weren't returning the cookie to me, so the IP service was throttling my usage of it.
Yah boo to all those who said this was a hosting issue and not a programming one.
Related
I have a rails server and this is a snip from the server log
Processing CustomController#my_action (for 10.0.2.2 at 2018-05-11 08:38:53) [GET]
Parameters: {"id"=>"12345", "action"=>"my_action", "controller"=>"custom"}
...
all the other stuff that gets logged
...
Completed in 251ms (View: 181, DB: 60) | 200 OK [https://myserver.com/custom/my_action/12345]
...
...
Processing CustomController#my_action (for 10.0.2.2 at 2018-05-11 08:38:55) [GET]
Parameters: {"id"=>"12345", "r"=>"876646", "h"=>"16", "m"=>"16", "s"=>"51", "send_image"=>"1", "gears"=>"0", "cookie"=>"1", "gt_ms"=>"1174", "uid"=>"dummyemail#myserver.com", "realp"=>"0", "java"=>"0", "qt"=>"0", "urlref"=>"https://myserver.com/reporting/report_builder/", "controller"=>"custom", "action_name"=>"", "idsite"=>"", "rec"=>"1", "_idts"=>"1525949211", "_idn"=>"1", "_refts"=>"0", "res"=>"1280x720", "_id"=>"f2fe432581e2e376", "wma"=>"0", "fla"=>"0", "_idvc"=>"1", "_viewts"=>"1525949211", "dir"=>"0", "action"=>"my_action", "url"=>"https://myserver.com/custom/my_action/12345", "pdf"=>"1", "ag"=>"0"}
...
all other stuff
Completed in 227ms (View: 177, DB: 39) | 200 OK [https://myserver.com/custom/my_action/12345?action_name=&idsite=&rec=1&r=705226&h=16&m=16&s=52&url=https%3A%2F%2Fmyserver.com%2Fcustom%2Fmy_action%2F2408&urlref=https%3A%2F%2Fmyserver.com%2Freporting%2Freport_builder%2F&uid=ctl-support%40myserver.com&_id=f2fe432581e2e376&_idts=1525949212&_idvc=0&_idn=1&_refts=0&_viewts=&send_image=1&pdf=1&qt=0&realp=0&wma=0&dir=0&fla=0&java=0&gears=0&ag=0&cookie=1&res=1280x720>_ms=757]
The problem with the above log is, I see duplicate server calls made for a single action.
I see this only when piwik tracking script is loaded. However, if I monitor the network tab in the browser, I could see only one call to the application server(myserver.com) and one call to the piwik server (mypiwikserver.com).
What might be the cause of this and how to fix this ?
Now that the error has been fixed by adding require 'tournament_system' here is my new error:
This is the terminal:
Rendering html template
Rendered html template (0.0ms)
Rendered tourneys/index.html.erb within layouts/application (10.8ms)
Completed 401 Unauthorized in 16ms (Views: 0.3ms | ActiveRecord: 1.1ms)
RuntimeError - Not Implemented:
app/controllers/tourneys_controller.rb:77:in `tourn'
app/views/tourneys/index.html.erb:37:in `_app_views_tourneys_index_html_erb__1482772099755741577_70314301318160'
Started POST "/__better_errors/deabb75178ea93ac/variables" for ::1 at 2018-04-17 00:55:03 -0700
This is on the webpage
Not Implemented
tournapp/controllers/tourneys_controller.rb
72
73
74
75
76
77
78
79
80
81
82
def tourn
driver = Driver.new
# Generate a round of a single elimination tournament
TournamentSystem::SingleElimination.generate driver
#Challonge::Tournament.find(:all)
##teamArray2 = render html: "<div>#{Challonge::Tournament.find(:all)}</div>".html_safe
end
>>
This is a live shell. Type in here.
Request info
Request parameters
{"controller"=>"tourneys", "action"=>"index"}
Rack session
(object too large. Modify ActionDispatch::Request::Session#inspect or increase BetterErrors.maximum_variable_inspect_size)
Local Variables
driver
#<Driver:0x00007fe6a4eba5e0>
I believe the error has to do with not calling the driver correctly. Also I created the controller Tourney using a scaffold.
Your Class Driver was inhereting from TournamentSystem::Driver but that module was not available as it was not explicitly required.
Adding require 'tournament_system' should take care of that.
Regarding the NotImplemented error, this is not related to your code. If you follow the source code of def generate in module SingleElimination (source code) and traverse:
def generate calls create_matches which calls create_match which finally calls build_match and that has a raise 'Not Implemented'.
The Developer of the project has not completed implementing all the required functions.
In my rails app I have an Item model which has many Versions. My controller has a versions action.
def versions
#item = Item.find(params[:id])
respond_to do |format|
format.json { render json: #item.versions }
end
end
Whenever I request versions, some items return them successfully and some don't. But when I go into the console, I can get the versions of all items with no problem.
Successful (returns JSON array of versions):
Started GET "/api/versions/4.json?"
Processing by ItemsController#versions as JSON
Parameters: {"id"=>"4"}
Completed 200 OK in 106ms (Views: 0.1ms | ActiveRecord: 19.8ms)
app[web.1]: cache: [GET /api/versions/4.json?] miss
heroku[router]: GET x.x.com/api/versions/4.json? dyno=web.1 queue=0 wait=0ms service=113ms status=200 bytes=831
Not successful (returns two responses, both status 200, no JSON data):
Started GET "/api/versions/1.json?"
Processing by ItemsController#versions as JSON
Parameters: {"id"=>"1"}
Completed 200 OK in 249ms (Views: 0.1ms | ActiveRecord: 162.9ms)
app[web.1]: cache: [GET /api/versions/1.json?] miss
heroku[router]: GET x.x.com/api/versions/1.json? dyno=web.1 queue=0 wait=0ms service=257ms status=200 bytes=3540
Any idea what could cause this? I don't know how to log SQL on Heroku so I'm out of ideas.
Re your SQL logging: Enable statement logging by issuing the SQL SET log_statement='all'; when you open each connection. Then examine the Heroku logs for Pg to see what's happening.
heroku logs --ps postgres
See:
Heroku's documentation on their PostgreSQL support
Heroku's logging documentation
a Google search for: how to log SQL on Heroku
This won't log query results, only query text, query parameters, and error/success.
To handle result logging you'll need to do it in your application; just print messages to stderr and Heroku will add them to the application logs so you can get them via heroku logs.
Your current code looks dodgy anyway. Looping over a result set and emitting json feels wrong:
It isn't explicitly clear that there can only be one result and thus only emit one json block; and
If there are no results, no json is emitted
Instead, save the result of the find - if any - to a variable. Then handle missing results properly: in a RESTful API you'd probably report HTTP status 404 Not Found; in a more RPC-style API you'd return an empty JSON document or a JSON document with a "not found" flag of some kind and report HTTP status 200 OK.
This was actually an error on the client side. On iOS I was handling the returned data in connection:didReceiveData:, which (when returning large chunks of data) is called numerous times with partial results. To fix it, I instead concatenated all the partial results in that callback and handled the data in connectionDidFinishLoading:. Stupid mistake, not related to Heroku or Rails.
Thanks for the help Craig.
Here's a snippet of my production.log:
Started GET "/product/514034/754240" for XX.XX.202.138 at 2012-06-21 11:52:28 -0700
Started GET "/product/614409/666897" for XX.XX.228.38 at 2012-06-21 11:52:28 -0700
Processing by ProductsController#show as HTML
Parameters: {"category_id"=>"514034", "product_id"=>"754240"}
Processing by ProductsController#show as HTML
Parameters: {"category_id"=>"614409", "product_id"=>"666897"}
Logged in 2940659 via auth cookie
Logged in 585210 via auth cookie
[e3e3fc56bb6bd137741b269ee397683c] [2940659] Read fragment views/global-caches/header (0.7ms)
[e3e3fc56bb6bd137741b269ee397683c] [2940659] Rendered shared/_email_form.html.haml (0.7ms)
[d81bb986be5acc0277c0c9e11b414249] [585210] Read fragment views/global-caches/sharebar-message (0.7ms)
[d81bb986be5acc0277c0c9e11b414249] [585210] Rendered shared/_email_form.html.haml (0.7ms)
...
As you can see, it's logging two concurrent sessions of two different users simultaneously to the same log file. This makes it impossible to parse my logs and determine, for example, the time it took to generate each kind of page, because the entries are not in the expected order of:
Started GET "/URL/BLAH" for IP at DATE
... stuff...
Completed 200 OK in XXms (ActiveRecord: YY.Yms)
Instead I get an unpredictable interleaved log like this:
Started GET "/URL/BLAH" for IP at DATE
Started GET "/URL/BLAH" for IP at DATE
... stuff...
Completed 200 OK in XXms (ActiveRecord: YY.Yms)
...stuff...
Completed 200 OK in XXms (ActiveRecord: YY.Yms)
So it's impossible to match the "completeds" with the "Started."
What I'd like is a way to have each child process write to its own log or something. Or if it's possible a way to write the each pageview's log atomically, but that might be impossible or difficult or hurt performance.
Rails 3.2 provides nice option config.log_tag
You can add to your production.rb:
config.log_tags = [ lambda { Time.now.to_i }]
So each line in your logs will be prepended by numbers. Example:
[1351867173] Started GET "/" for 127.0.0.1 at 2012-11-02 16:39:33 +0200
[1351867173] Processing by RecipesController#main as HTML
Logs are still shuffled, but now we can normalize, order them.
sort -f -s -k1.1,1.11 production.log | sed 's/^.............//' > sorted_production.log
(Sorter by first symbols (by timestamp) and remove timestamp by sed)
Now logs are easy to analyze.
In addition there is fix on related issue https://github.com/rails/rails/pull/7317 in rails 3.2.9
So keep this in mind.
Sorry for bad English... )
I'm running a Rails 3.0.7 app with nginx and Passenger. I have a custom 500 page that is properly displayed when the app encounters an 500 internal error, however the actual '500' status is not being output to the logs.
I'd like to be able to periodically grep the logs to find 500 errors, but I can't seem to figure out why the actual status is not being rendered. I've even looked through the Rails code, and everything looks fine. All other status codes are successfully logged.
Here is an error-free 200 response:
Completed 200 OK in 1265ms (Views: 1262.4ms | ActiveRecord: 69.6ms | Sphinx: 0.0ms)
Here is a 500 response:
Completed in 500ms
It appears that something is supposed to be there, but is not, so spaces are output instead.
Looks like this has been resolved in Rails master, but is not in the gem for 3.0.7 yet.
https://github.com/rails/rails/commit/7927fc2ff77543a0ab151ac1cb3d60318e2dfa68