I'm using heroku and heroku postgresql. How do I set db command timeout so that I get an exception when a sql command takes longer than 10 seconds?
Configure your database.yml like this, the key bit being the variables hash:
defaults: &default
adapter: postgresql
encoding: unicode
pool: 5
min_messages: warning
variables:
statement_timeout: 5000
I found a solution here: Ruby on Rails: How to set a database timeout in application configuration?
Add
ActiveRecord::Base.connection.execute('set statement_timeout to 10000')
in the end of the environment.rb file.
Does anyone see a better way?
I don't know ruby, but in PostgreSQL you can simply run this:
SET statement_timeout = '10s'
and afterwards, all queries in this connection (session) will have strict limit on runtime.
You can also change it in postgresql.conf as global default, or even make it default in given database or for given user like:
ALTER DATABASE web SET statement_timeout = '10s';
ALTER USER bad_user SET statement_timeout = '10s';
Adding
ActiveRecord::Base.connection.execute('set statement_timeout to 10000')
to the end of the environment.rb file did not work for me.
What worked was adding
ActiveRecord::Base.connection.execute("SET statement_timeout = '10s'")
to an initialize file.
Related
Our database.yml is added to .gitignore so devs can customize local environments and we plan to use ENV['DATABASE_URL'] for production servers. For default setup, this works. However, we need to configure encoding and collation to utf8mb4.
encoding: utf8mb4
collation: utf8mb4_unicode_ci
I tried padding it to query parameters, like the ?pool=5 example in the docs, but it doesn't seem to work.
DATABASE_URL=mysql2://user:passwd#host:port/dbname?encoding=utf8mb4&collation=utf8mb4_unicode_ci
The tables created are still using the default encoding/collation so I assume the parameters doesn't work. Is there any way I could configure this by any other methods? Encoding and collation is the same for all environments.
Requirement is that dev environment can have a file with a config whereas prod should have no special file added, it should only use ENV variables. Maybe add this to one of the files inside config dir like application.rb or other files?
Thanks in advance.
PS: I'm new to Rails and Ruby (1 week) since I'm coming from PHP/Python.
You shouldn't specify any of the configuration options in the ENV variable. You should keep it clean and specify these options in database.yml file:
production:
encoding: utf8mb4
collation: utf8mb4_unicode_ci
pool: 5
The way it works is that rails takes what it can from the DATABASE_URL, and then applies options configured in database.yml for specified environment. If you want to learn more, follow this link: https://devcenter.heroku.com/articles/rails-database-connection-behavior#rails-4-1-and-beyond
Let's say I have s different database connections...
Right now for the purposes of this example let's only be concerned with development. When I run 'rake db:setup' it only creates development because from the rake task perspective the other connection is another environment. I could pass in the other connection as the RAILS_ENV to create the database. However, the problem comes in with how we've defined our database connections. I'd rather not undo this as it makes management and deployment much easier. What we've done is this...
In our database.yml we've added the following code.
databases_file = File.join(Rails.root.to_s, "config", "databases", "#{Rails.env.to_s}.yml")
IO.read(databases_file) if File.exist?(databases_file)
Then in a databases sub-folder under config we've created the different environment yml files.
So in development.yml we have...
development:
... details ...
logging:
... details ...
Now if I set RAILS_ENV to 'logging' it will never load the file since the file is called development.yml. Thus it will never find the logging connection.
If I consolidate all these settings back into the database.yml file then we're back to not checking in the database.yml into the git repo and having to manually create it on the development machines for new hires and such. We'd also have to change our deploy process to start putting the file in place on the lower environments. (Our production process already supports this)
You could use environment variables to manage these. You could have a single database.yml with things defined such as:
development:
database: ENV['DEVELOPMENT_DATABASE']
And then set the environment variables in your .bashrc/.zshrc.
We have multiple databases and we use branches heavily... so I came up with this solution to create multiple databases and tie their names to the branch being worked on...
# config/database.yml
<%=
databases_file = File.join(Rails.root.to_s, "config", "databases", "#{Rails.env.to_s}.yml")
if Rails.env.development? || Rails.env.test?
branch = ENV['BRANCH'] || `git branch --no-color 2> /dev/null`.chomp.split("\n").grep(/^[*]/).first[/(\S+)$/,1].sub("-", "_")
puts "Using databases for #{branch}"
IO.read(databases_file).gsub!("<branch>", branch) if File.exist?(databases_file)
else
IO.read(databases_file) if File.exist?(databases_file)
end
%>
And then you create a database file for each environment. In the development.yml and test.yml you use the token to let this script look for what it needs to in order to do replacement. So an example file would look like this.
# config/databases/development.yml
development:
adapter: mysql2
host: 127.0.0.1
database: <branch>_dev
username: user
password: pass
versions:
adapter: mysql2
host: 127.0.0.1
database: <branch>_versions
username: user
password: pass
reporting:
adapter: mysql2
host: 127.0.0.1
database: <branch>_reporting
username: user
password: pass
ods:
adapter: mysql2
host: 127.0.0.1
database: <branch>_ods
username: user
password: pass
I then extended the rake tasks to handle multiple databases
This is where most of the work went in. Therefore, I'm going to leave it out of the answer and make you work for it! No actually, it's just a big large mess that I'm not proud of and would love time to fix it, but haven't found any. I don't want to lead anyone down the wrong path but if you have questions just get me a message and I'll be happy to help.
Working with a rails app, having some manner of weird database / rake issues.
When I execute:
rake db:migrate
I am getting the following error:
Mysql2::Error: No database selected: SHOW TABLES
(See full trace by running task with --trace)
The trace isn't revealing much useful information. Can be seen here: http://pastebin.com/WdsguudC
The config file looks right, and the user is getting logged in, or I would have gotten some kind of access error. The database exists, the user has correct permission, and I can access and manipulate it manually. I have done a bunch of googling, and haven't found anything helpful. Not sure if there is any other code that needs provided, because this seems like fairly low level problem.
after all that it was a spacing issue in the yaml.
Note that ruby has exchanged its YAML parser in a recent 1.9.2 version.
This might also cause this problem.
In order to switch back to the old YAML parser syck, use this in boot.rb:
require 'yaml'
YAML::ENGINE.yamler= 'syck'
Well, it is a common issue for us beginners. This issue comes from the moment when you create your new project in rails. Let’s say to have an example
$ rails new toy –d mysql
After you do the bundle and start your server, most likely you will have an error. To correct it you need to go to your database.yml and modify the following:
Add a password in the password field as shown below, this is the password you use to secure mysql.
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: root
password: mypassword
socket: /tmp/mysql.sock
Also, comment out the database adding a hash tag (#)before the name as shown below
development:
: *default
database: #toy_development
Then restart your command line and go to the root of your application and type:
$ rails s
You have to see the Ruby on Rails welcome page..
After, you need to create a database.
Create a DATABASE.
The issue message is saying that not DATABASE is selected. It is because I didn’t create one. When you work with MySQL you have to create one, so:
Go to the root of my application and type:
$ mysql –u root –p
$ Passwor: mypassword (Enter your password, this is the one you entered to secure MySQL)
Note: This example works wit a project called toy and the user I wanted to grant privileges is mark and the password I’ll give is 45mark. Below you will see where I apply these elements. Remember to apply your own elements on each part of the statement.
Create and user for this project
Once you are in, you will see the pointer (mysql> ), so type after it:
mysql> GRANT ALL PRIVILEGES ON toy_development.* TO 'mark'#'localhost' IDENTIFIED BY '45mark';
Then type:
mysql> exit;
Check that it is working by typing:
$ mysql –u mark –p toy_development
Enter password: 45mark (You enter the one you gave)
Open database.yml file and configure what is needed and fix as required. In my case I will chance the username to mark and the password to 45mark
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: mark
password: 45mark
socket: /tmp/mysql.sock
- Also, REMOVE the hash tag (#) added before
development:
: *default
database: toy_development
Save it.
Go to the root of the application and type
$ rake db:schema:dump
Done!!
I hope this helps. Happy coding!!
Thanks
I had the same issue with ruby 1.9.2-p180 , upgraded to p290 and it works
Just restart the server; in the command line:
Press Ctrl + C
execute:
rails s
I had a similar error when i typed rake db:schema:dump and it turns out that I just have to comment out all the databases on my yaml file except my development one.
Give a try to this.
rake db:test:prepare
Install this to see if you have actually created a table or not. Open the "development.sqlite3" in db folder
http://sqlitebrowser.org/
Its a simple error checkout the entire database.yml file and see that where is default decription is given database name is given or not if not then look below it there will another development name is also given where configuration of database is use check that give your database name in it
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: root
password: 12345
host: localhost
development:
<<: *default
database: db_name
One potential cause is that there is a DATABASE_URL environment variable defined.
$ echo $DATABASE_URL
=> mysql2://root#localhost:3306
If you get a similar output to the above url (i.e., the database name is not present), then you might want to add the database name to the string or unset the env var.
$ export DATABASE_URL=mysql2://root#localhost:3306/my_rails_app_development
$ unset DATABASE_URL
If you unset the var, you probably want to specify the database details in database.yml instead.
I'd like to set my database to timeout requests that do not complete within a set amount of time, in order to prevent outlier requests from monopolizing the entire application.
Is there anything I can add to my Rails configuration files in order to set this?
I tried to add a line I saw often online of timeout: 5000 to my database.yml, but that didn't seem to have any effect.
I tried to make calls to ActiveRecord::Base.connection.execute('set statement_timeout to 5000') in the environment.rb but that causes Rails to error out.
I'm running a Postgres database on Heroku, where I do not have direct database access, hence I cannot do this with database configuration directly. Even if I remotely execute that command from the Heroku console, they can restart my application at any time, and if this isn't re-executed as the application starts, my change gets lost.
database.yml:
defaults: &default
adapter: postgresql
encoding: unicode
pool: 5
min_messages: warning
variables:
statement_timeout: 5000
Got this working, just needed to include the line in environment.rb at the very end, rather than in the beginning or in the block.
Try this syntax:
SET statement_timeout = 5000;
I'm new to Rails. I had created a Rails application earlier and also collected a few data records in the development database. Suppose, I create a new Rails application and I prefer to use the existing development database of the 1st Rails application in my newly created Rails application, how do I do that?
Just change the file config/database.yml and set the database name.
your database.yml should look like this:
development:
host: localhost
adapter: mysql
database: your_database_name [just the name, not the path]
username: your_username
password: your_password
test:
...
production:
...
You need to change the database name in the database.yml file.
The seconnd problem you run into is migrations.
I would copy the migrations form your previous application over so that you maintain migration integrity with version numbers and rolling back if that is needed.
Also, if you are seeing development.locs - that locs refers to the table_name, which in restful context is usually also coincidentally the name of the controller.