I'm trying to figure out why my rails server is taking lots of extra time to serve requests now that I switch to thinking sphinx from solr.
I'm getting things like
Completed 200 OK in 1242ms (Views: 248.8ms | ActiveRecord: 89.3ms | Sphinx: 5.3ms)
and
Completed 200 OK in 881ms (Views: 4.7ms | ActiveRecord: 7.1ms | Sphinx: 29.4ms)
I'm trying to use Thinking Sphinx to serve up JSON for an autocomplete drop down. I'm trying to figure out where the bottle neck is. I've tried running lots of benchmarks, but I can't seem to find it. When I run stuff from the console it seems snappy. Any ideas about why I'm seeing these kinds of render times?
Before I switched the render times were effectively the sum of the Views and ActiveRecord so in the ballpark of <300ms.
I should also note that I'm only seeing the bottlenecks on pages that interact with use search, which is why I'm thinking that it may be due to Thinking Sphinx.
Related
When the Rails logs says something like
Completed 200 OK in 454.8ms (Views: 117.9ms | ActiveRecord: 199.7ms | Solr: 0.0ms)
What kind of time is being displayed? CPU time, wall time, or something else?
http://guides.rubyonrails.org/v3.2/performance_testing.html#request-logging mentions that time is measured, but not what kind of time. I haven't found any other documentation in the Rails Guides about logging, apart from how to generate messages in the Rails logger.
Wall-time.
Check the implementation of the Notification Instrumentation:
https://github.com/rails/rails/blob/2746a227fbb7e56bd51ab47fa97919f206972ab2/activesupport/lib/active_support/notifications/instrumenter.rb
and the implementation of the LogSubscriber:
https://github.com/rails/rails/blob/b5eb2423b6e431ba53e3836d58449e7e810096b4/actionpack/lib/action_controller/log_subscriber.rb
and this:
https://github.com/rails/rails/blob/7f18ea14c893cb5c9f04d4fda9661126758332b5/activesupport/lib/active_support/subscriber.rb
it is using Time.now, which is wall-time.
I'm a newbie in Rails, and I had a problem with my Rails project. I used Log4r for my logger. I also used config for Log4r at How to configure Log4r with Rails 3.0.x? . But when I send some request at the same time, the output of Log4r has error. It is unordered. :(
Example about error output log file:
Started GET "/task_results.json" for 172.29.73.182 at 2013-06-17 17:36:38 +0700
Processing by TaskResultsController#index as JSON
Started GET "/favicon.ico" for 172.29.73.182 at 2013-06-17 17:36:38 +0700
Processing by ApplicationController#missing_page as
Parameters: {"path"=>"favicon"}
Completed 404 Not Found in 1ms (Views: 0.2ms | ActiveRecord: 0.0ms)
[1m [36m (994.0ms) [0m [1mSELECT task_result_id,task_id,worker_id,product_id,passed,date_time,details FROM task_results ORDER BY task_result_id DESC; [0m
Completed 200 OK in 8656ms (Views: 0.2ms | ActiveRecord: 994.0ms)
And I wanna ask that how can I config Log4r to synchronize output? Or how can I fix my problem?
ruby is single-threaded, so I assume you are running your rails app on a server like unicorn?
The only solution I can think of is timestamps (with micro seconds) and not using the multi-line Rails log output which is not suitable for anything but single requests in dev environments. But even this does not guarantee "ordered" logs because they can be written when the request was served or at a random time afterwards when the buffer was flushed again.
This is an in-depth tutorial on how to use Log4r that also shows how to get custom timestamps into your logs and have all the important information on a single line. This prevents logs from other requests writing inside your main request and you can have a timestamp resolution of microseconds and order afterwards manually. However the order should not matter if you are running concurrent app-servers because the order is not guaranteed there anyway.
I'm looking at console of with working rails application.
Every request/response finished with next line
Completed 200 OK in 1169ms (Views: 74.1ms | ActiveRecord: 0.0ms)
The problem is that ActiveRecord always show 0.0 and it's wrong, how to fix this ?
the rails logging is not very precise. you might want to have a look at time_bandits for more detailed info: https://github.com/skaes/time_bandits
I have two rails applications (both now on Rails 3.1.1), and they work nicely. However, I have a dependence between the two. Application A uses data of application B by linking to it. These links are created automatically, but they have to be computed by doing a lookup to the data of application B. I'm working on Windows 7 with Ruby 1.9.2 and Thin as web server, and this will not be changed :-(
I have tried the following:
Use just a RESTful resource, so defined a controller, called its action (get_xml_obj with some params in it), read the needed values from the XML. Worked, but needs around 0.5s to 1s per call.
Replaced it by ActiveResource#find which worked as well, but with the same performance as the solution before.
I have installed nginx and configured it so, that the connection are keepalive, so that the connection handling should be much faster. But noticed no difference at all when calling B from A.
When I compare the time spent, these are typical examples (here with 4 references in one web page):
Application A:
Started GET "/tasks/search_task/1803" for 127.0.0.1 at 2011-11-02 14:11:04 +0100
Processing by TasksController#search_task as HTML
Parameters: {"id"=>"1803"}
Rendered tasks/_tooltip.html.haml (4529.5ms)
Completed 200 OK in 4532ms (Views: 4527.5ms | ActiveRecord: 2.0ms)
cache: [GET /tasks/search_task/1865] miss
Application B:
cache: [GET /service/get_xml_obj?key=notice&value=rails] miss
Started GET "/service/get_xml_obj?key=notice&value=rails" for 127.0.0.1 at 2011-
11-02 14:11:05 +0100
Processing by ServiceController#get_xml_obj as */*
Parameters: {"key"=>"notice", "value"=>"rails"}
Completed 200 OK in 6ms (Views: 3.0ms | ActiveRecord: 1.0ms)
and 3 other calls with a similar length (< 10ms).
So is there something I can do to tune the retrieval (without accessing the database directly)? Do you know of any good documentation how to measure and tune the web server and middleware? These are only personal applications, so there is no way of deploying them on a decent server. I use a cache for the retrieved information, so it gets better over time, but 1 second is too much to wait for. And there may be more than 1 or 2 links in a page I want to render.
Ok, I finally gave up and implemented the following:
Added file b.rb to my models directory in application A.
Included there all raw models, where the base models (used sti) are defined like that:
class Notice < ActiveRecord::Base
self.establish_connection(
:adapter => "sqlite3",
:database => "../b/db/dev.db"
)
end
...
I am now able to ask: Notice.where(:key => 'rails') which results in a real Rails model object.
The whole thing was implemented in around 20 minutes, and now there is no difference in including no link from application A to B to include 5 links.
At some point in time, I would like to know what is the slow part in using RESTful resources here ...
I have a problem with integration between CouchRest and Sunspot. When I search the book detail, the result from Sunspot is empty. I try to google it for a long time but no help.
Started GET "/books/search?utf8=%E2%9C%93&query=Book of Life&commit=Search%21" for 127.0.0.1 at 2011-09-08 11:27:41 +0700
Processing by BooksController#search as HTML
Parameters: {"utf8"=>"?", "query"=>"Book of Life", "commit"=>"Search!"}
Rendered books/index.html.erb within layouts/application (10.7ms)
Completed 200 OK in 145ms (Views: 20.6ms | ActiveRecord: 0.0ms)
[] <-- I got empty result
My System
Ruby 1.9.2p290
Rails 3.0.10
CouchDB 1.1.0
File structure ( https://gist.github.com/1164637/ )
Model (/app/models/book_detail.rb)
Controller (/app/controllers/books_controller.rb)
Sunspot Adapter for CouchRest (/config/initializers/couchdb.rb)
Sunspot Adapter Module (/config/initializers/sunspot_couch.rb)
NOTE: Sorry about code link. I always got "Please indent all code by 4 spaces using the code toolbar button". I try to remove all tab and follow SO code formatting guideline but it not work anymore.
Forgive me if I'm missing something, but I can't see how Sunspot is mapping "keywords" to the searchable fields on your CouchRest objects.
To debug first I'd visit Couch in the browser admin UI to make sure my that end is working. Then I'd double check that sunspot is getting anything. If sunspot contains your records then the bug is on the search side, if it is empty that maybe something is up with the object lifecycle management code it injects into your model class.
It's been ages since I did any serious Ruby, wish I could be more helpful. One option is to take advantage of some of the direct CouchDB full text offerings like CouchDB Lucene: https://github.com/rnewson/couchdb-lucene