I'm trying to optimize my application and load certain things into the Rails cache (eventually memcached) at application start. Essentially I have a few tables in my database that only exist for normalization purposes and RARELY change. When they change, I can handle the logic to update the cache.
Where should I write the 'initializer' to read these various models and load them into the cache? I tried writing a traditional initializer, but it gets run BEFORE my models actually exist... I essentially need to load all my models and stuff, then create the cache, then run the application. How can I enter that middle state?
I would either put this code in a file in your lib directory and require the model files first, or directly into your model files so that the initialization runs after your model is defined:
class NormalizedTable
...
end
CacheInitializer.fill_cache_with :normalized_table
Where CacheInitializer#fill_cache_with is defined in your initializers or lib directory. I would also recommend that you run these functions in the background, for example in a delayed job. If you are running this code every time you load up your Rails app it will slow your boot process down and it's probably not necessary that the data is in your cache at all times - i.e you app can use the database while the cache is being populated.
Related
I'm interested in uploading a script (file) via my Rails application and then being able to execute this on demand.
I imagine the first step here is to handle the file upload such that I a Model which I can reference the file like Model.script. This makes sense to me.
And then from here, my plan is to expose a route/controller method which would execute this script, but I'm not sure how to handle the actual execution inside this method. For example if I had
class Model
def run_script
# logic to run self.script
end
end
How would I execute the associated file/code given that it is a Ruby file? Note that the script does not need to run in the context of the Rails application, it just needs to run.
You can use eval, which will run a Ruby script, but be very, very, careful!
For a detailed explanation see "Code is data, data is code".
Example:
eval(#model_instance.script)
eval is very dangerous. If you give a user the ability to upload and run an unchecked script, it may create a huge problem. For example, someone can upload a script like this:
User.delete_all
This will delete all users from the users table using a SQL query without invoking any Active Record callbacks.
You can use Ruby's taint method to add some additional safety, but it is not 100% foolproof.
For a detailed example, see "Locking Ruby in the Safe".
I am working on a feature to limit the number of pages a user can access in a day. My plan is to create a class variable in the ApplicationController which is instantiated on startup. One of the features I want though is for this value to be changed by an administrator without having to worry about changing the config file, hence the class variable.
How can I have rails call a function in the application controller when rails starts up?
You can't do it this way. You must operate on the presumption that your Rails application will consist of multiple independent processes with entirely arbitrary lifetimes, that is they may be spawned if needed and killed if they're idle at any time.
You're stuck having to persist this somewhere. A flat file can work if you're using a single server, but a database of some sort, SQL or otherwise, is also viable. For light loads, that is less than dozens of requests per second, SQL won't be a problem.
In a rails web app, if I write messages to a queue like rabbitmq, how will clients be notified when a producer sends a message to the queue?
I'm guessing I have to create a seperate process that runs in the background to respond to messages correct? i.e. this code is outside of the scope of a web application.
If this is the case, is it possible to re-use the models/libs that are in the rails application already? do I have to copy this code in 2 places then?
It looks like your application requires what's usually called a background or worker process. This is a fairly common requirement for any moderately complex web application.
I'm guessing I have to create a seperate process that runs in the background to respond to messages correct?
Yes - you're right about this. Whilst it's perfectly possible to use threads to handle the background tasks (in your case, reading and processing messages from RabbitMQ), the standard and recommended route for a Rails application is to run a separate background process.
If this is the case, is it possible to re-use the models/libs that are in the rails application already?
Absolutely. The simplest possible way to get this working is by using Rails' built in runner command.
Another option is to create a ruby script which loads up your Rails application. For example, you could create the file my_script.rb in the root of your project, which might look something like this:
# Load my application:
require File.join(File.dirname(__FILE__), 'config/environment.rb')
# Now you can access your Rails environment as normal:
MyModel.all.each { |x| x.do_something }
If your needs become more complex, or you find that you need to run more than one background process to keep up with the volume of data you need to process, you might want to look at one of the many available libraries and frameworks which can help with this.
Once you've created your background process, you'll need a way to run it continuously when you deploy it to your production server. Whilst it's possible to use libraries like daemons, as suggested by ctcherry, I would recommend using a dedicated tool like upstart (if deploying to ubuntu) or runit. A good summary of the most popular options is available here.
You are correct, you do need a background process. And you can keep the code for that process in the lib folder of the Rails project if you like, I have done that before without issue, and it keeps related code together which is nice.
I used this library to create my long running process, it was quite simple:
http://daemons.rubyforge.org/
In order to re-use models from your rails application you can run a require on the config/environment.rb file to get everything loaded. (Set RAILS_ENV as an environment variable first to select the correct envrionement) From that point the script behaves as though you are inside a rails console session.
So I'm creating a Rails app that will function as a directory for contact information in our organization. But it's not as simple as it sounds.
We have a large LDAP-enabled directory which contains information for all of the users in our organization of tens of thousands.
We have a smaller, separate LDAP-enabled directory which contains additional information for our department of several hundred, as well as some information duplicating or taking precedence over fields in the larger directory.
We will want to hand edit some of this data to override some of the fields in our local directory, which will be represented by a SQL table in the Rails app.
The remote directories will be periodically mirrored as SQL tables, and the 3 tables (Organization, Department, Local) will be compared to choose the correct value displayed in the app.
I know that sounds ridiculous, but nothing can be done for it. Our organization is very decentralized, and this is the most straightforward way to get what we want.
This is my second Rails app, so I'm comfortable with most of the design and plumbing, but I don't know the best way to periodically poll the data from the remote directories and import it into our local SQL tables.
Where in my app should I periodically import data into my tables from LDAP? Do I use Rails? Should I do this in straight Ruby and run it as a cron job?
If you want to have the sync functionality as part of your Rails app, then you can create that logic in a separate model class (let's call it LDAPSynchroniser).
Then you can reuse it from multiple places, including:
Rake task for manual syncronisation;
Cron job Running the Rake task;
Trigger the synchronisation from the web application (take into account the time it takes to run!)
The rake task would look like:
task :cron => :ldapsync do
puts "Sync-ing with LDAP..."
status = LDAPSynchroniser.new.run
puts "done: #{status.to_s}"
end
The web application trigger would be a regular controller:
def LDAPSyncController < ...
# probably authentication is needed...
def sync
status = LDAPSynchroniser.new.run # or run it in a separate thread-ish
# respond with status
end
end
Now to answer your questions:
Where in my app should I periodically import data into my tables from LDAP?
Use rake task + cron.
Do I use Rails?
You probably need to boot rails, but you don't need to run the rails web server for that.
Although you might want to trigger the task from the web application itself.
Should I do this in straight Ruby and run it as a cron job?
Doing it in Rails would be a little bit easier as you already have your model and all you need.
With Plain Ruby it might be possible as well but I don't think it is worth the effort.
The rails books and web pages I've been following have all stuck to very simple projects for the sake of providing complete examples. I'm moving away from the small project app and into a realm of non-browser clients and need to decide where to put code that is shared by all involved parties.
The non-browser client is a script that runs on any machine which can connect to the database. Browser clients write commands into the database, which the script examines and decides what to do. Upon completion, the script then writes its result back. The script is not started by the RoR server, but has access to its directory structure.
Where would be the best place for shared code to live, and how would the RoR loader handle it? The code in question doesn't really belong in a model, otherwise I'd drop it in there and be done with it.
I'd put the shared code in the Rails project's /lib directory and consider making it a custom Rake task.
It really depends on how much you use this shared code. If you use it everywhere, then throw it in the lib folder (as has already been stated here). If you are only using it in a few places, you might want to consider making a plugin out of it and loading it only in the places that use it. It's nice to only load what you need (one of the reasons I'm loving merb).