I'm building a system which allows the user to modify the database.yml contents via an admin frontend interface.
That changes to database.yml obviously don't have any affect until the application is restarted. Rather than forcing the user (who may not have SSH access to the physical box) to restar the application manually, I'd like a way to force ActiveRecord to reload the config post startup.
Rather than requiring them to restart the server, is there a way to force ActiveRecord to re-initialize after initial startup?
Rationale
There are two use cases for this - a) initial setup wizard b) moving from sqlite evaluation database to production supported database.
Initial Setup Wizard
After installing our application and starting it for the first time the user will be presented with a setup wizard, which amongst other things, allows the user to choose between the built in evaluation database using sqlite or to specify a production supported database. They need to specify the database properties. Rather than asking users to edit yml files on the server we wish the present a frontend to do so during initial setup.
Moving from sqlite evaluation database to production supported database
If the user opted to go with the built in evaluation database, but alter wants to migrate to a production database they need to update the database settings to reflect this change. Same reasons as above, a front end rather than requiring the user to edit config files is desired, especially since we can validate connectivity, permissions etc from the application rather than the user finding out it didn't work when they try to start the application and they get an activerecord exception.
Restart your Rails stack on the next request just as you would if you had access to the server.
system("touch #{File.join(Rails.root,'tmp','restart.txt')")
Building on #wless1's answer in order to get ERB-like ENV vars (e.g. <%= ENV['DB_HOST'] %>) working try this:
YAML::load(ERB.new(File.read(File.join(Rails.root, "config/database.yml"))).result)
ActiveRecord::Base.establish_connection config[Rails.env]
Theoretically, you could achieve this with the following:
config = YAML::load File.read(File.join(Rails.root, "config/database.yml"))
ActiveRecord::Base.establish_connection config[Rails.env]
You'd have to execute the reconnection code in every rails process you're running for it to be effective. There's a bunch of dangerous things to worry about here though - race conditions when changing the database config, syntax problems in the new database.yml, etc.
Related
I am trying to reduce some logging noise I am getting from PostgreSQL on my Heroku/Rails application. Specifically, I am trying to configure the client_min_messages setting to warning instead of the default notice.
I followed the steps in this post and specified min_messages: warning in my database.yml file but that doesn't seem to have any effect on my Heroku PostgreSQL instance. I'm still seeing NOTICE messages in my logs and when I run SHOW client_min_messages on the database it still returns notice.
Here is a redacted example of the logs I'm seeing in Papertrail:
Nov 23 15:04:51 my-app-name-production app/postgres.123467 [COLOR] [1234-5] sql_error_code = 00000 log_line="5733" application_name="puma: cluster worker 0: 4 [app]" NOTICE: text-search query contains only stop words or doesn't contain lexemes, ignored
I can also confirm that the setting does seem to be in the Rails configuration - Rails.application.config.database_configuration[Rails.env] in a production console does show a hash containing "min_messages"=>"warning"
I also tried manually updating that via the PostgreSQL console - so SET client_min_messages TO WARNING; - but that setting doesn't 'stick'. It seems to be reset on the next session.
How do I configure client_min_messages to be warning on Heroku/Rails?
If all else fails and your log is flooded by the server logs you can't control or client logs you can't trace and switch off, Papertrail lets you filter out anything you don't want. The database/client still generate them, Heroku still passes them to Papertrail, but Papertrail discards them once they come in.
Shotgun method, PostgreSQL-specific
REVOKE SET ON PARAMETER client_min_messages,log_min_messages FROM your_app_user;
REVOKE GRANT OPTION FOR SET ON PARAMETER client_min_messages,log_min_messages FROM your_app_user;
ALTER SYSTEM SET client_min_messages=WARNING;
ALTER SYSTEM SET log_min_messages =WARNING;
ALTER DATABASE db_user_by_your_app SET client_min_messages=WARNING;
ALTER DATABASE db_user_by_your_app SET log_min_messages =WARNING;
ALTER ROLE your_app_user SET client_min_messages=WARNING;
ALTER ROLE your_app_user SET log_min_messages =WARNING;
And then you need to either wait, restart the app, force it to re-connect or restart the db/instance/server/cluster it connects to.
If your app opens and closes connections - just wait and with time old connections will be replaced by new ones using the new settings.
If it uses a pool, it'll keep re-using connections it already has, so you'll have to force it to re-open them for the change to propagate. You might need to restart the app, or they can be force-closed:
SELECT pg_terminate_backend(pid) from pg_stat_activity where pid<>pg_backend_pid();
The reason is that there's no way for you to alter session-level settings on the fly, from the outside - and all of the above only affects defaults for new sessions. The REVOKE will prevent the app user from changing the setting but it'll also throw an error if they actually try. I'm leaving this in for future reference, keeping in mind that at the moment Heroku supports PostgreSQL versions up to 14, and REVOKE SET ON PARAMETER was added in version 15.
To need all these at once, you'd have to be seeing logs from both ends of each connection in your Papertrail, connecting to multiple databases, using different users, who also can keep changing the settings. Check one by one to isolate the root cause.
Context
There's one log written to each client, one or more written on the server.
client_min_messages applies the client log, sent back in each connection.
log_min_messages applies to the server log(s).
Depending on what feeds the log into your Papertrail, you might need to change only one of these. Manipulating settings can always be tricky because of how and when they apply. You have multiple levels where parameters can be specified, then overriden:
system-level parameters, loaded from postgresql.conf, then postgresql -c/pg_ctl -o and postgresql.auto.conf, which reflects changes applied using ALTER SYSTEM SET ... or directly.
database overrides system. Applied with ALTER DATABASE db SET...
user/role overrides database. ALTER ROLE user SET...
session overrides user/role. Changed with SET... that clients also use upon connection init. If the value for client_min_messages set under min_messages is specified both in database.yml and ENV['DATABASE_URL'], Rails will use the env setting, overriding the one found in .yml with it DATABASE_URL=postgres://localhost/rails_event_store_active_record?min_messages=warning
transaction-level parameters are the most specific, overriding all else - they are basically session-level parameters that will change back to their initial setting at the end of transaction. SET LOCAL...
When a new session opens, it loads the system defaults, overrides them with the database-level, then role-level, after which clients typically issue their own SETs.
It might be a good idea to make sure you're using the settings in database.yml that you think you're using, since it's possible to have multiple sets of them. There can be some logic in your app that keeps altering the setting.
I think you want log_min_messages, not client_min_messages:
Controls which message levels are written to the server log. Valid values are DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1, INFO, NOTICE, WARNING, ERROR, LOG, FATAL, and PANIC. Each level includes all the levels that follow it. The later the level, the fewer messages are sent to the log. The default is WARNING. Note that LOG has a different rank here than in client_min_messages. Only superusers and users with the appropriate SET privilege can change this setting.
I'm not sure if your database user will be allowed to set it, but you can try doing so at the database level:
ALTER DATABASE your_database
SET log_min_messages TO 'warning';
If this doesn't work, and setting at the role or connection level doesn't work, and heroku pg:settings doesn't work (confirmed via other answers and comments), the answer might unfortunately be that this isn't possible on Heroku.
Heroku Postgres is a managed service, so the vendor makes certain decisions that aren't configurable. This might be one of those situations.
I am working on an app where I now need to add test mode or sandbox mode features like Stripe provides in the dashboard. In this feature, when the user turned the test mode on, the user can test the functionality and can create dummy data in the same login session/token.
I have tried to use the Rails 6 feature to use multiple databases but have a few questions:
is it good to switch the connection in production for test mode even we will not have many test requests?
will it be good to have a separate instance for test mode with the test subdomain? in this case how we should manage the login sessions? should we copy data to the test the database? will it be good and common practice?
I only can manage to implement this if I have user data in the test database so that when I switch database connection system will not send an unauthenticated response.
Note:
for login and user update, it will use the primary database all time and for other actions, it will use the test_mode database. I am doing this by skip_around_action in a specific controller.
We are doing this so that when the user turned off test mode, it will update the primary database and the next request will use the primary database as per around_action logic
Here is my current code in application_controller.rb:
around_action :setup_db_connection
def setup_db_connection
database_key = (user.test_mode?) ? :test_mode : :primary
ActiveRecord::Base.connected_to(database: database_key) do
yield
end
end
test_mode database key has test database configuration in database.yml and similarly for primary database key. Both are completely two different databases.
Can anyone please tell me if I am going in to correct direction? Any help or thought will be appriciated. Thank you in advance!!!!
You should just create an additional environment as this is the exact problem that they are designed to solve.
Rails just ships preconfigured with development, test and production as thats the bare minimum you can get by with. You can actually have an unlimeted number of environments.
So lets say you want to create an enviroment named staging which closely matches your production environment. You can start by copying the productions settings:
cp app/environments/production.rb app/environments/staging.rb
And then just setup an additional hash in your database.yml and credentials files.
staging:
<<: *default
database: my_app_staging
You can then set the environment by using the RAILS_ENV env var or the command line arguments when starting the Rails server. Your staging environment can be run for example in a separate Heroku application made available on a subdomain. If you're using Heroku or some other SAAS platform a hobby tier will often be sufficient for staging.
When it comes to data you can either work on a set of dummy data generated by wiping and seeding or regularily wipe the staging database with a backup from your production database.
I would not consider using a "test mode switch" a good idea as it makes far to easy to inadvertantly mess things up. Using a separate environment lets you use a completely different set of credentials as well so you won't accentially do something like create a real credit card charge or destroy data on third party services. Sandboxes should be isolated.
I am aware of multiple database support in Rails 6 but it seems to be on model level, like we tell for each model that from which database it belongs to.
A client wants two databases for production server with similar schema say
primary_live_database
secondary_test_database
idea is that app will send a header parameter to give a hint to its test app or live, and depending upon this we will switch database.
how can we achieve this?
You really just want to setup an additional environment. This is done by setting the RAILS_ENV environmental variable. Usually such an environment is called staging but its really up to you whatever you want to call it.
Setting RAILS_ENV = 'staging' will:
Cause Rails to load config/environments/staging.rb.
Rails will use the staging settings in config/database.yml.
The secrets for staging will be loaded from config/secrets.yml.enc if your Rails version supports per env secrets.
Rails.env == 'staging' and Rails.env.staging? == true.
This lets you run a near copy of the production environment but with different settings. For example you often want to limit access to the staging environment to collaborators or push changes to the staging environment to collaborate on features that may not be production ready.
If your clients reaction is Waaaaaaah, so much work! then you can use a callback in your ApplicationController to call ActiveRecord::Base.establish_connection. I don't recommend it though.
Currently, I'm working on the application which is developed in rails 3* and ruby 1.9.3. I have configured some email address in the initializers section as YML file for each environments.
But, the requirement is keep on changing (but it'll happened every month 1 or 2 times) that need to add/remove the email address from the configuration. Hence, I need to restart the server on every changes. Because, I configured those address in the initializers.
Is there any better way to handle this situation?
If you want to change the email without re-deploying / restarting the server, you can always create a Email model and persist it to the database. By adding a current field / column (boolean value) and a scope scope :current, -> { where(current: true) } you can access the email via Email.current.first.address, for instance. You might need to ensure that one and only one 'current' Email object ist present at any given time.
edit
creating a model does not mean you have to create UI for it. Just use the console to change the email if you have to.
the configuration and models are loaded at startup, and if you don't want to do any reloading in production, which is slow and not recommended, you have to use the DB for persistence.
if you really want to go down the "reload" route, set config.cache_classes = true in config/environments/production.rb and specify the email in a constant outside the config directory (in some model or controller) like this EMAIL = 'whatever#email.com'. You would have to change the production code on each production machine, without a server restart. Sounds very hacky. Look into zero-downtime deploys à la Github for a more elegant solution. Redeploying should be cheap and painless.
I want to be able to switch between different DB schemas in a Rails 4 app.
The plan is to add a new middleware in the very beginning of the stack that will do that for me.
The only way to do it is by setting ActiveRecord::Base.connection.schema_search_path = '"$user",my_schema'.
The problem I have with this is that this connection will go to the pool and all the following requests will use the schema that was set in the first one (basically leaking it through).
So the solution I see is to always reset the search path to what it was before and always set it on each request.
But I don't want to do this because:
99% of the requests will go to the default (public) schema, executing set search_path to '$user$,my_schema' would be additional query that could have been avoided
higher risk of leaking (other middleware may establish the connection earlier, or some changes to Rails or gems outside of my control)
All that especially applies to threaded servers, like Puma.
So are there any better alternatives to my solution with a middleware?
Thanks.
When you return connections to the pool, you must ensure the pool runs DISCARD ALL; to reset the connection state.
That will clear any SET ROLE, SET SESSION AUTHORIZATION, session variables, search_path setting, etc.