There's an oft-called method in my Rails application that retrieves ~200 items from the database. Rather than do this again and again, I store the results using Rails.cache.write. However, when I retrieve the results using Rails.cache.read, it's still very slow: about 400ms. Is there any way to speed this up?
This is happening in a controller action, and I'd prefer users not have to wait so long to load the page.
FYI regarding Rails caching, from the Rails Guides, "...It’s important to note that query caches are created at the start of an action and destroyed at the end of that action and thus persist only for the duration of the action."
If you can share the method, I may be able to help more quickly. Otherwise, a couple performance best practices:
Use .includes to avoid N+1 queries. Define this in the model and
call it in the controller.
How are your indexes set-up (if any)?
Related
In a Rails 3.2 app, I have a number of queries defined in application_controller.rb. The data returned by the query will change very infrequently.
Looking at the logs, these queries appear to be run on every page load.
How can I cache these arrays, so that they are refreshed less frequently, helping to increase page load time.
Thanks
By using something like Memcached or Redis?
There are a few ways. Most simply, you can cache the queries in some instance variables across requests, with something like this:
#variable ||= Model.query
This will save them from being loaded on every request. However, if it's something that's very complex, you might consider throwing it in Redis (which is great for things like this).
Normally you wouldn't want to do something like this. You might want to reconsider the architecture of the app if you're doing something like this a lot.
So, For the sake of performance, I'm using database sessions. I figure that while the sessions are server side, I might as well store commonly accessed objects in the session. So, I'm storing serialized versions of the current_user, current_account, and the current_user's permissions.
The User model handels a lot of the permissions methods (things like user.can_do_whatever), but since i'm trying to be more efficient, and store commonly accessed things in the session (this allows for far fewer DB accesses), does it make sense / break any design standards to (upon each request) store the session in an instance variable in the current_user ?
As of right now, I can't think of any alternatives.
ROR application have by default a RESTful design. One rules of REST is stateless. that mean each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server.
If you have trouble with Database performance, use a cache system like memcached wich is already integrated in rails (Caching with Rails).
I found a couple of references warning against storing non-primitive data types in the session, but they were all just warnings, and boiled down to: Storing complex objects is "Expecially discouraged" [sic], but if you decide you need to... well, just be careful.
Anyway, I'm kinda taken by the idea of having the users table double as the sessions table, but serialization still seems a bit sketchy. If you're just trying to cut down the number of DB requests, what about storing IDs and using :joins when looking up your user (might require a bit of hackery to get that worked into the default session loading). That avoids synchronization problems and serialization sketchiness, and still only generates a single DB query. Just make sure to use :joins and not :include, as the latter generates a query for each table.
Hope that helps!
We're on Rails 3.0.6.
We maintain a list of numbers that changes only once a month, but nearly every page request requires access to this list.
We store the list in the database.
Instead of hitting the database on every request and grabbing the list, we would like to grab the data once and stash it in memory for efficient access.
If we store the list in each user session, we still need to hit the database for each session.
Is there a way to only hit the database once and let the values persist in memory across all users and all sessions? We need access to the list from the controller. Should we define a class variable in the controller?
Thanks!
I think Rails.cache is the answer to your problem here. It's a simple interface with multiple backends, the default stores the cache in memory, but if you're already using Memcached, Redis or similar in your app you can plug it into those instead.
Try throwing something similar to this in your ApplicationController
def list_of_numbers
#list_of_numbers ||= Rails.cache.fetch(:list_of_numbers, :expires_in => 24.hours) do
# Read from database
end
end
It will try to read from the cache, but if it doesn't find it, will do the intensive stuff and store it for next time
The pattern you're looking for is known as a singleton which is a simple way to cache stuff that doesn't change over time, for example, you'll often see something like this in application_controller.rb -- your code always calls the method
def current_user(user_id)
#current_user ||= User.find user_id
end
When it does, it checks the instance variable #current_user and returns it if not nil, otherwise it does the database lookup and assigns the result to the instance variable, which it returns.
Your problem is similar, but broader, since it applies to all instances.
One solution is with a class variable, which is documented here http://www.ruby-doc.org/docs/ProgrammingRuby/html/tut_classes.html#S3 -- a similar solution to the one above applies here.
This might be a good solution in your case, but has some issues. In specific, (assuming this is a web app) depending on your configuration, you may have multiple instances of Rails loaded in different processes, and class variables only apply to their specific instance. The popular Passenger module (for Apache and Nginx) can be configured to allow class variables to be accessible to all of it's instances ... which works great if you have only one server.
But when you have multiple servers, things get a little tricky. Sure, you could use a class variable and accept that you'll have to make one hit to the database for each server. This works great except for the when that the variable ... varies! You'll need some way of invalidating the variable across all servers. Depending on how critical the it is, this could create various very gnarly and difficult to track down errors (I learned the hard way :-).
Enter memcached. This is a wonderful tool that is a general purpose caching tool. It's very lightweight, and very, very smart. In particular, it can create distributed caches across a cluster of servers -- the value is only ever stored once (thus avoiding the synchronization problem noted above) and each server knows which server to look on to find any given cache key. It even handles when servers go down and all sorts of other unpleasantries.
Setup is remarkably easy, and Rails almost assumes you'll use it for your various caching needs, and the Rails gem just makes it as simple as pie.
On the assumption that there will be other opportunities to cache stuff that might not be as simple as a value you can store in a class variable, that's probably the first place to start.
Hey all, if you've ever posted on [craigslist], this question should make sense to you. Whenever you post a listing (to sell furniture or an apartment, for example), your listing is not immediately thrown up on the site. Rather, listings will appear in batches (numbers vary) about every 10-15 minutes. At first I was really over-thinking this behavior, trying to hold records and then do mass inserts, but I realized it was much simpler. After talking with some colleagues, it made sense that Craigslist is caching their pages and then emptying that cache every 10-15 minutes. This severely decreases the load on their database.
Now, to my question. How do I accomplish the same thing in Rails? I know how to implement caching - I've read the [caching with Rails guide]. I will be using action caching and fragment caching (because I can't cache the whole page). I still need to do validations and access controls, so I can't fully cache the page...
To accomplish timed page caching you can utilize the standard Rails' caching plus a little timed cleverness.
First you want to determine your level of caching. You've got three options:
Page Caching - caches the whole page but subsequent requests don't go to through Rails stack. So if this is a Craiglist-esque page that will be hit thousands of times a second this request will only go to your webserver (e.g. apache) not Rails or your db making it much faster. The trade off is that you lose authentication, session variables, etc that Rails provides.
Action Caching - caches the whole page but brings the request into Rails so that it can execute any filters associated with that action.
Fragment Caching - caches a segment of the page, essentially bypassing the need to execute the code with in the block (and any consequential calls to the DB).
Then you'll need pick the appropriate level of caching and implement it in your app (check out the links above for implementation examples).
Once you have implemented the caching you now have to figure out a way to expire the cache. I can think of two ways to do this, both come with benefits and drawbacks. For now let's assume you've chosen to use action caching.
Reliable but more involved - create an action within your controller that expires the cache and a cron job task that makes a request to that action. I've asked a similar question that addresses this 'built in' scheduled task. For security precautions, you may want to include a generated hash or something similar so that someone can't manually expire your cache by going to '/products/expire_cache'.
class ProductsController < ApplicationController
caches_action :index
def index
# implementation
end
def expire_cache
if params[:verification_hash] == 'sa89sf8sfsfehiwaf89yfea98fh'
expire_action :action => :index
end
end
end
Unreliable but easier - simply expire the cache in your action with an arbitrary conditional. This implementation assumes that there will be enough traffic to regularly ensure that someone will come to your site on the 0, 15, 30, and 45 minutes. You could decrease this interval in order ensure that the cache will be reset at a more probable interval.
class ProductsController < ApplicationController
caches_action :index
def index
# implementation
expire_action :action => :index if Time.now.min % 15 == 0
end
end
I think there are two approaches to what you want to accomplish:
Simply cache with fragments and actions so that on the first hit of the page the database is accessed and the page loads normally, but every subsequent hit is from the cached version. The major upside of this approach is that you don't need to deal with delayed jobs and rendering your pages outside of the regular flow of things.
Create a delayed job that renders your page or the individual fragments that get cached. During the rendering in the delayed job the page fragments will actually get cached as if a user were viewing them (provided you have implemented fragment and action caching normally). Once the delayed job is done, populate a column in your database that indicates that this record/page is ready for viewing.
Probably the easiest way for caching in Rails is using Memcached with :expires_in option.
You will probably need a VPS server to use it which could be expensive for smaller sites.
But you can do good time based caching even without memcached. This little SimpleCache snippet has worked wonders for my shared hosted sites.
Not sure how to ask it so sorry if I mess up my terminology. In using jruby on rails, how would I query two (or more database) to serve back to the view page?
I have seen where I set up my database connection in database.yml and it works fine but I am now wondering how I move beyond this to hitting many databases with jdbc and puttting them in one webpage view.
My title is worded the way it is because I am used to a non-orm non-mvc way of doing things. For example, need to query somethingn in classic asp? Just create the object and emit html and your dataset for each database. Cumbersome but at least I know how to do it. Need to hit five databases in asp.net? Just have a control and in the code-behind bind it to the table adapter or do a sql query. Here, I'm just lost on how to do this.
How do I do query several databases, different ones (oracle, sql server, etc.) in Jruby on Rails with jdbc and put them all in one page? And do this the "right" way so that I don't end up with my jror application looking like classic asp.
EDIT: I think something like this might be what I am looking for, but I am not sure how to put both sets of data on the same page.
Put the query logic in the appropriate models (activerecord models for your main db tables, non-activerecord models for other sources of data). These should have intention revealing method names.
In the controller action query for the information your web page needs. Load these data into instance variables.
In the view, interact simply with the instance variables to display them using ERB. If you find logic in your views, you're doing it wrong and should refactor.
For multiple DBs, this may be of help http://tomayko.com/writings/rails-multiple-connections