Note: note sure if this is the best title for the question / open to suggestions to edit for future value *
I have a multi-tenant rails application, which allows clients to use their own custom TLDs. So I can have:
www.clientA.com
www.clientB.com
www.clientC.com
etc....
For better or worse, my database (postgres) has a tenants table, which has approximately 60 columns with various settings & configurations for each tenant. Some are simply flags, and some are large text values.
In application_controller.rb I have some logic to parse the URL, query the tenants table based on the domain and instantiate a #current_tenant object. That #current_tenant object is available throughout the lifecycle of the page. This happens on every. single. page. of my app.
While I have some heavy caching in place, this design still feels very wrong to me, and I am sure it can be improved.
Is there a best practice for this? What is the best pattern to handle the #current_tenant object? I am concerned about memory management.
your current_tenant is no different from current_user that is ubiquitous in Rails apps.
stop microoptimizing. unless you benchmarked it and it shows to be a major bottleneck (which it is not, I can assure you).
any minuscule performance improvement (if at all) will be offset by the increased complexity of code, caching problems and what not.
do. not. do. that. ;)
one thing though, do NOT do a before_filter that assigns #current_tenant, instead add a current_tenant method to the application_controller (or one of its concerns) that will cache the result for the remainder of the request:
def current_tenant
#current_tenant ||= ....
end
You could also cache the #current_tenant object using Rails cache, ie:
def current_tenant( host = request.host )
Rails.cache.fetch(host, expires_in: 5.minutes) do
Tenant.find_by(tld: host)
end
end
(Note: I haven't touched Rails for 3 years, so take this answer with the appropriate fistful of salt.)
Is there a best practice for this? What is the best pattern to handle the #current_tenant object?
The best way I've seen this kind of stuff implemented is in PHP, rather than Ruby on Rails. More specifically, in the Symfony framework.
In a nutshell, a Symfony app's layout is like so:
/app <-- app-specific Kernel, Console, config, cache, logs, etc.
/src <-- app-specific source files
/vendor <-- vendored source files
/web <-- public web folder for the app
To run multiple apps from the same code base, you'd basically go:
/app
/app2
/...
/src
/vendor
/web
/web2
/...
... and then point each domain to a different /web folder.
As I understand, it's possible to re-organize a Rail project's directory structure to something more or less equivalent. See this related answer in particular:
https://stackoverflow.com/a/10480207/417194
From there, you could theoretically boot up one instance of your app per tenant, each with their separate resources, hard-coded defaults, etc., while continuing to use a shared code base.
The next best option is, imo, what you're currently doing. Use memcached or equivalent to quickly map specific domain names to tenants, and fall back to a database query as necessary. I imagine you're doing this already, since you've "some heavy caching in place".
As an aside, you might find this related question, and the ones linked within it, interesting:
Multiple applications using a single code base in ruby
(And FWIW, I ended up not sticking with PHP for multi-tenant apps. There was no magic bullet last I played with Ruby or Rails.)
With respect to memory usage, methinks don't worry too much about it: strictly speaking, you're looking up and creating the tenant object a single time per request. Even if doing so was dreadfully slow, cursory monitoring of where your app is actually spending time will reveal that it's negligible compared to innocuous looking pieces of code that get run a gazillion times per request.
I designed an application that is doing basically exactly what you have described. Since you are loading a single object (even if on every page request), just make sure the query only returns the one row (current tenant) and doesn't do a crazy amount of joins. A single row query with a LIMIT applied to it is not going to bring down your site, even if requested hundreds of times a second. And regardless, if you are getting that type of traffic, you will have to scale your server anyways.
One thing that can be done to help, is to make sure your search column is indexed in the database. If you are finding the current tenant by url, index the url column.
Here is an example of what I did. I globalized the variable so this information is available in all controllers/models/views.
In my application controller:
before_filter :allocate_site
private
def allocate_site
url = request.host_with_port
url.slice! "www."
# load the current site by URL
$current_site = Site.find_by({:url => url, :deleted => false})
# if the current site doesn't exist, we are going to create a placeholder
# for the URL hitting the server.
if $current_site.nil?
$current_site = Site.new
$current_site.name = 'New Site'
$current_site.url = url
$current_site.save
end
end
If you don't mind adding extra gems to your application, I would recommend using apartment. This was done exactly to serve your purpose : handle rails application with multiple tenants.
This would present two advantages to your problem:
handling #current_tenant (or whatever you wish to name it) is done through a middleware, so you won't need to set it in your ApplicationController. Take a look at the Switch on domain section of the README to see how it is done. (Note: apartment uses Apartment.current_tenant to refer to your #current_tenant.
The best part: in most case, you will not need #current_tenant anymore since apartment will scope all requests on the appropriate postgresql schema.
Related
I have an application in Ruby on Rails with mvc framework. As of now, I have API calls in the controller but don't think this is the right place for them. What kind of file should all my API calls go in? Thanks
def getDetails(id)
api_response = HTTParty.get(base_uri, :query => {:DID => id, :DeveloperKey => devKey})
#json_hash = api_response.parsed_response
return #json_hash
end
API calls to external services (3rd party) are not specific to your app, as their service is available to everyone (in theory). It is my understanding that these sorts of features go in the lib/ directory because they are not app specific. Ideally you could then pull out the code from your lib in your project, and drop it into someone else's lib/ in another project and it would still work just fine.
Put the call in the lib/. If you want, you can create the a model from the returned data in your controller.
It would look something like this:
app/controller/
class YourController < ApplicationController
def getDetails
# keep in mind, api call may fail so you may want surround this with a begin/rescue
api_response = YourApiCall.new.get_details(params[:id])
# perhaps create a model
#model = SomeModel.new(fname: api_response[:first_name], lname: api_response[:last_name])
# etc...
end
end
lib/
require 'HTTParty'
Class YourApiCall
def get_details(id)
HTTParty.get(base_uri, :query => {:DID => id, :DeveloperKey => devKey})
#json_hash = api_response.parsed_response
return #json_hash
end
end
Very late to this one, but thought I'd add my 2p/2c.
I like to try to keep my controllers clean of anything apart from controller code, which I loosely define as program flow code based on request type and parameters. For example, choosing the correct template for the request type or choosing the correct method to call based on whether a user is logged in or not.
When it comes to calculating the responses, I don't like to litter the controller with lots of code that manipulates models and sets instance parameters. This is hard to test and even harder to re-use. I prefer to defer to another object and return a single value object to the template.
Sometimes I can defer to a model: maybe it's a simple look-up and I'm just sending a single model to the template, or an array of models.
Maybe I've implemented a useful method in a model to return an appropriate value or value object.
However sometimes I'm doing something that doesn't use a model, or that uses a several models, or that doesn't feel like it should actually be cluttering up the model. In this case, neither the controller nor a model is an appropriate place for the code.
The lib directory doesn't feel right either. I tend to treat the lib directory as somewhere that contains code that I haven't been bothered to turn into gems yet. If the code I'm writing only makes sense in the context of the application, it doesn't sit well.
So I turn to service objects. Under the 'app' folder I have a 'services' folder, which contains small, functional classes that encapsulate single chunks of site behaviour. (Or sometimes, coordinate several other services to provide a simple interface for the controller.)
This allows me to slim down my controllers AND my models, and makes a perfect place to put code that needs to contact an API.
If you wanted to go one step further you could wrap the API itself in a wrapper class (or set of classes) and keep those in the lib directory (for conversion to a gem at a later date perhaps). Then the service object would perform the task of calling the API wrapper with the appropriate values (passed from the controller) and responding with something that a template can interrogate cleanly.
Of course, you can go further than this and add more layers. A presentation layer, for example, could sit between the service object (providing generic values) and format data for a specific view. (Maybe you want to provide both a web page and an RSS feed and they need different date formats for example.)
But you get the idea.
By my coding style (and understanding of MVC), external calls would be placed in a "tableless" model. RailsCasts 193 talks a bit about this concept, and a less clunky syntax is supported in Rails 4. If you need to have any manipulation of the code, the model seems like the logical place to place these. Moving those methods into the controller would work, but could create problems as your app grows.
Another consideration with external API calls is actually storing those in a database, which would should definitely be in a model at that point, so (to me) it becomes clearer that these really should be in the model.
Introduction
I have a (mostly) single-page application built with BackboneJS and a Rails backend.
Because most of the interaction happens on one page of the webapp, when the user first visits the page I basically have to pull a ton of information out of the database in one large deeply joined query.
This is causing me some rather extreme load times on this one page.
NewRelic appears to be telling me that most of my problems are because of 457 individual fast method calls.
Now I've done all the eager loading I can do (I checked with the Bullet gem) and I still have a problem.
These method calls are most likely ocurring in my Rabl serializer which I use to serialize a bunch of JSON to embed into the page for initializing Backbone. You don't need to understand all this but suffice to say it could add up to 457 method calls.
object #search
attributes :id, :name, :subscription_limit
# NOTE: Include a list of the members of this search.
child :searchers => :searchers do
attributes :id, :name, :gravatar_icon
end
# Each search has many concepts (there could be over 100 of them).
child :concepts do |search|
attributes :id, :title, :search_id, :created_at
# The person who suggested each concept.
child :suggester => :suggester do
attributes :id, :name, :gravatar_icon
end
# Each concept has many suggestions (approx. 4 each).
node :suggestions do |concept|
# Here I'm scoping suggestions to only ones which meet certain conditions.
partial "suggestions/show", object: concept.active_suggestions
end
# Add a boolean flag to tell if the concept is a favourite or not.
node :favourite_id do |concept|
# Another method call which occurs for each concept.
concept.favourite_id_for(current_user)
end
end
# Each search has subscriptions to certain services (approx. 4).
child :service_subscriptions do
# This contains a few attributes and 2 fairly innocuous method calls.
extends "service_subscriptions/show"
end
So it seems that I need to do something about this but I'm not sure what approach to take. Here is a list of potential ideas I have:
Performance Improvement Ideas
Dumb-Down the Interface
Maybe I can come up with ways to present information to the user which don't require the actual data to be present. I don't see why I should absolutely need to do this though, other single-page apps such as Trello have incredibly complicated interfaces.
Concept Pagination
If I paginate concepts it will reduct the amount of data being extracted from the database each time. Would product an inferior user interface though.
Caching
At the moment, refreshing the page just extracts the entire search out of the DB again. Perhaps I can cache parts of the app to reduce on DB hits. This seems messy though because not much of the data I'm dealing with is static.
Multiple Requests
It is technically bad to serve the page without embedding the JSON into the page but perhaps the user will feel like things are happening faster if I load the page unpopulated and then fetch the data.
Indexes
I should make sure that I have indexes on all my foreign keys. I should also try to think about places where it would help to have indexes (such as favourites?) and add them.
Move Method Calls into DB
Perhaps I can cache some of the results of the iteration I do in my view layer into the DB and just pull them out instead of computing them. Or I could sync things on write rather than on read.
Question
Does anyone have any suggestions as to what I should be spending my time on?
This is a hard question to answer without being able to see the actual user interface, but I would focus on loading exactly only as much data as is required to display the initial interface. For example, if the user has to drill down to see some of the data you're presenting, then you can load that data on demand, rather than loading it as part of the initial payload. You mention that a search can have as many as 100 "concepts," maybe you don't need to fetch all of those concepts initially?
Bottom line, it doesn't sound like your issue is really on the client side -- it sounds like your server-side code is slowing things down, so I'd explore what you can do to fetch less data, or to defer the complex queries until they are definitely required.
I'd recommend separating your JS code-base into modules that are dynamically loaded using an asset loader like RequireJS. This way you won't have so many XHRs firing at load time.
When a specific module is needed it can load and initialize at an appropriate time instead of every page load.
If you complicate your code a little, each module should be able to start and stop. So, if you have any polling occurring or complex code executing you can stop the module to increase performance and decrease the network load.
In my app there're objects, and they belong to countries, regions, cities, types, groups, companies and other sets. Every set is rather simple - it has id, name and sometimes some pointers to other sets, and it never changes. Some sets are small and I load them in before_filter like that:
#countries = Country.all
#regions = Region.all
But then I call, for example,
offer.country.name
or
region.country.name
and my app performs a separate db query-by-id, although I've already loaded them all. After that I perform query through :include, and this case ids, generated by eager loading, do not depend on either I've already loaded this data with another query-by-id or not.
So I want some cache. For example, I may generate hashes with keys as records-ids in my before_filter and then call #countries[offer.country_id].name. This case it seems I don't need eager loading and it's easy turn on Rails.cache here. But maybe there's some smart built-in rails solution that does not require to rewrite everything?
Caching lists of models like that won't cache individual instances of that exist in other model's associations.
The Rails team has worked on implementing Identity Maps in Rails 3.1 to solve this exact problem, but it is disabled by default for now. You can enable it and see if it works for your problem.
I am looking for a Ruby/Rails tool that will help me accomplish the following:
I would like to store the following string, and ones similar to it, in my database. When an object is created, updated, deleted, etc., I want to run through all the strings, check to see if the CRUD event matches the conditions of the string, and if so, run the actions specified.
When a new ticket is created and it's category=6 then notify user 1234 via email
I am planning to create an interface that builds these strings, so it doesn't need to be a human-readable string. If a JSONish structure is better, or a tool has an existing language, that would be fantastic. I'm kinda thinking something along the lines of:
{
object_types: ['ticket'],
events: ['created', 'updated'],
conditions:'ticket.category=6',
actions: 'notify user',
parameters: {
user:1234,
type:'email'
}
}
So basically, I need the following:
Monitor CRUD events - It would be nice if the tool had a way to do this, but Ican use Rails' ModelObservers here if the tool doesn't natively provide it
Find all matching "rules" - This is my major unknown...
Execute the requested method/parameters - Ideally, this would be defined in my Ruby code as classes/methods
Are there any existing tools that I should investigate?
Edit:
Thanks for the responses so far guys! I really appreciate you pointing me down the right paths.
The use case here is that we have many different clients, with many different business rules. For the rules that apply to all clients, I can easily create those in code (using something like Ruleby), but for all of the client-specific ones, I'd like to store them in the database. Ideally, the rule could be written once, stored either in the code, or in the DB, and then run (using something Resque for performance).
At this point, it looks like I'm going to have to roll my own, so any thoughts as to the best way to do that, or any tools I should investigate, would be greatly appreciated.
Thanks again!
I don't think it would be a major thing to write something yourself to do this, I don't know of any gems which would do this (but it would be good if someone wrote one!)
I would tackle the project in the following way, the way I am thinking is that you don't want to do the rule matching at the point the user saves as it may take a while and could interrupt the user experience and/or slow up the server, so...
Use observers to store a record each time a CRUD event happens, or to make things simpler use the Acts as Audited gem which does this for you.
1.5. Use a rake task, running from your crontab to run through the latest changes, perhaps every minute, or you could use Resque which does a good job of handling lots of jobs
Create a set of tables which define the possible rules a user could select from, perhaps something like
Table: Rule
Name
ForEvent (eg. CRUD)
TableInQuestion
FieldOneName
FieldOneCondition etc.
MethodToExecute
You can use a bit of metaprogramming to execute your method and since your method knows your table name and record id then this can be picked up.
Additional Notes
The best way to get going with this is to start simple then work upwards. To get the simple version working first I'd do the following ...
Install acts as audited
Add an additional field to the created audit table, :when_processed
Create yourself a module in your /lib folder called something like processrules which roughly does this
3.1 Grabs all unprocessed audit entries
3.2 Marks them as processed (perhaps make another small audit table at this point to record events happening)
Now create a rules table which simply has a name and condition statement, perhaps add a few sample ones to get going
Name: First | Rule Statement: 'SELECT 1 WHERE table.value = something'
Adapt your new processrules method to execute that sql for each changed entry (perhaps you want to restrict it to just the tables you are working with)
If the rule matched, add it to your log file.
From here you can extrapolate out the additional functionality you need and perhaps ask another question about the metaprogramaming side of dynamically calling methods as this question is quite broad, am more than happy to help further.
I tend to think the best way to go about task processing is to setup the process nicely first so it will work with any server load and situation then plug in the custom bits.
You could make this abstract enough so that you can specify arbitrary conditions and rules, but then you'd be developing a framework/engine as opposed to solving the specific problems of your app.
There's a good chance that using ActiveRecord::Observer will solve your needs, since you can hardcode all the different types of conditions you expect, and then only put the unknowns in the database. For example, say you know that you'll have people watching categories, then create an association like category_watchers, and use the following Observer:
class TicketObserver < ActiveRecord::Observer
# observe :ticket # not needed here, since it's inferred by the class name
def after_create(ticket)
ticket.category.watchers.each{ |user| notify_user(ticket, user) }
end
# def after_update ... (similar)
private
def notify_user(ticket, user)
# lookup the user's stored email preferences
# send an email if appropriate
end
end
If you want to store the email preference along with the fact that the user is watching the category, then use a join model with a flag indicating that.
If you then want to abstract it a step further, I'd suggest using something like treetop to generate the observers themselves, but I'm not convinced that this adds more value than abstracting similar observers in code.
There's a Ruby & Rules Engines SO post that might have some info that you might find useful. There's another Ruby-based rules engine that you may want to explore that as well - Ruleby.
Hope that this helps you start your investigation.
I am developing a Ruby on Rails website and I have an "architectural" question : my application needs some parameters and I'm wondering where to store them.
In concrete terms, my application receive some requests which are evaluated and then sent. So, the Request model must have attributes concerning these treatments : a validation status and a sending status. For instance, validation status can be "accepted", "rejected" or "waiting". Sending status can be "sent", "waiting", "error during sending" or stuff like that. I have to store those status codes parameters somewhere, but I don't know what is the best solution.
I could create a model for each one and store them in the database (and having an active record model ValidationStatus for instance) but : wouldn't it be a bite excessive to create a database/model for storing data like that?
I could also just use them in the code without "storing" them, I could store them in a YAML file...
So, a more simpler question: how do you deal with your application parameters in RoR?
There are lots of global configuration plugins, most of them revolve around the idea of loading a YAML file at some point. Check this page, this plugin and even this Railscast.
I put them in the database. I have a lot of these, and they are all pretty straightforward lists of strings. The tables are all the same - id, name, description.
I generate models for them rather than having an actual model file for each one. In app/models I have a file called active_record_enums.rb, which in your case would look something like this:
ACTIVE_RECORD_ENUMS = %w{
ValidationStatus
SendingStatus
}
ACTIVE_RECORD_ENUMS.each do |classname|
eval "class #{classname} < ActiveRecord::Base; end"
classname.constantsize.class_eval do
# Add useful methods - id_for(name) and value_for(id) are handy
end
end
This file has to be required in a config file somewhere; other than that it's pretty straightforward.
(Have since viewed that rails cast mentioned above [episode 85] - it looks like a bit more 'the rails way' than below)
Another approach is to build on the existing configuration mechanism in Rails.
Lets presume there are two types of configs:
App wide configs common to dev/test/prod environments
Configs specific to envrionments dev/test/prod
For the first scenario, items in "RAILS_ROOT + '/config/environment.rb'" work. Just see that the names are captialised so they are Ruby constants. A variation to this is have a reference to another file in that environment.rb file ...
require RAILS_ROOT + '/config/appConfigCommon.rb'
and place relevant config items in that file. This has the advantage of being able to be referenced independant of Rails.
For scenario 2, a similar approach can be taken. Place items for development in "RAILS_ROOT + '/config/environments/development.rb'" or something like
require RAILS_ROOT + '/config/environments/appConfigDev.rb'
and place environment specific items in that required file, making sure they start with caps. And follow the same pattern for test/prod (and others if need be).
The config items are directly accessible in views and controllers (not sure about models) by simply using the constant name.
I am not using Ruby but I will tell you that I started out (in ASP.NET) placing lots of settings in a Web.Config file (similar to a YAML). As time went on, though, the system evolved to the point where different instances needed different settings. So, almost all of them have migrated to the database. So...if you'll be deploying multiple instances of your site, I'd strongly recommend keeping settings in a table of your database (mine has just one record, with fields for various settings). If I had done this to start, I'd have saved a significant amount of time.
I tend to just make a string column for each, and use validates_inclusion_of to set what's acceptable for them.
class Request < ActiveRecord::Base
validates_inclusion_of :validation_status, :in => ['accepted','rejected','waiting']
validates_inclusion_of :sending_status, :in => ['sent','waiting','...']
end
If you need to have things happen (ie. emails sent) when the status changes, look into using the Acts As State Machine plugin to manage it.