Hi im trying to follow this link to create a scraper and I am stuck trying to create a db
https://towardsdatascience.com/job-board-scraping-with-rails-872c432ed2c8
I get this error when typing rake db:create I've tried creating inside and outside the folder and I get this error I am not sure why, thank you very much. I have been closely following the link so I have all the same files and am sure everything was installed properly.
rake aborted!
ActiveRecord::DatabaseConfigurations::InvalidConfigurationError:
'{ default => }' is not a valid configuration. Expected '' to be a URL string or a Hash.
This is my database.yml file
# SQLite. Versions 3.8.0 and up are supported.
# gem install sqlite3
#
# Ensure the SQLite 3 gem is defined in your Gemfile
# gem 'sqlite3'
#
default: &default
adapter: postgresql
pool: 5
timeout: 5000
development:
<<: *default
database: scraper_development
test:
<<: *default
database: scraper_test
production:
<<: *default
url: <%= ENV['DATABASE_URL'] %>
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
Thanks in advance everyone!
Update
Thanks for the pointer I had [the database.yml file] indented but I though an error told me to remove tabs. new issue now:
could not connect to server: No such file or directory
Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
Couldn't create 'scraper_development' database.
Please check your configuration.
rails aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"
The file is in YAML format. The sub-items must be indented.
default: &default
adapter: postgresql
pool: 5
timeout: 5000
development:
<<: *default
database: scraper_development
You can add " gem 'pg', '>= 0.18', '< 2.0' " to the gem file and make the necessary settings in the YAML file.
I ran into the same problem and what I did was to fix the indentation, and then it worked.
The database.yml file is in YAML format. so make sure the sub-items are indented correctly. Please check the example below on how you are supposed to indent it.
This is how your database.yml file should look like
default: &default
adapter: postgresql
pool: 5
timeout: 5000
development:
<<: *default
database: scraper_development
test:
<<: *default
database: scraper_test
production:
<<: *default
url: <%= ENV['DATABASE_URL'] %>
I looked for hours on the web but none of my attempts to adapt my database.yml work out. I just started with Rails and the exercise is to use sqlite3 for development, and postgreSQL for production. I am using rvm 2.4.1 and rails 5.3.1.
I added this to my gemfile:
group :production do
gem 'pg'
end
And this is my database.yml
# SQLite version 3.x
# gem install sqlite3
#
# Ensure the SQLite 3 gem is defined in your Gemfile
# gem 'sqlite3'
#
default: &default
adapter: sqlite3
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
timeout: 5000
development:
<<: *default
database: db/development.sqlite3
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: db/test.sqlite3
production:
<<: *default
database: db/production.sqlite3
So that does not work out and I get the following error when I try to deploy to heroku:
Specified 'postgresql' for database adapter, but the gem is not loaded. Add
`gem 'pg'` to your Gemfile (and ensure its version is at the minimum required
by ActiveRecord).
I am assuming I need to somehow let the production part lead to postgreSQL but the manuals on the heruko website and other posts on stackOverflow and elsewhere are not helping me. How can I get it work? I can add :default to the groups, gem 'pg' is in within my Gemfile but then other things like scaffolding produce errors later on. I would really be thankful to get specific help on this as a spend about 12 hours on this single issue already..
first, since you informed in your production using postgresql then you must change setting in database.yml (see production part)
default: &default
adapter: sqlite3
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
timeout: 5000
development:
<<: *default
database: db/development.sqlite3
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: db/test.sqlite3
production:
adapter: postgresql
encoding: unicode
database: project_name_production
pool: 5
host: localhost
username: your_username
password: your_database_password
and then you can follow this rake command in heroku
second about your gem file, if the problem still persist then you can put out from production environtment and push to heroku, and then do bundle install to install the adapter
GEMFILE
gem 'pg'
here is good for your to read to solve your problem link information for heroku with postgresql
I have been been repeatedly getting this error despite trying to change many things.
I viewed other SO pages and changed indentation, names, etc. Really not sure what's going on. Any insight would be greatly appreciated.
# SQLite version 3.x
# gem install sqlite3
#
# Ensure the SQLite 3 gem is defined in your Gemfile
# gem 'sqlite3'
#
default: &default
adapter: postgresql
pool: 5
timeout: 5000
development:
<<: *default
database: lightpath_development
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: lightpath_test
production:
<<: *default
database: lightpath_production
Try switching back to the default database.yml file.
# SQLite version 3.x
# gem install sqlite3
#
# Ensure the SQLite 3 gem is defined in your Gemfile
# gem 'sqlite3'
#
default: &default
adapter: sqlite3
pool: 5
timeout: 5000
development:
<<: *default
database: db/development.sqlite3
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: db/test.sqlite3
production:
<<: *default
database: db/production.sqlite3
You're likely getting that error because you're trying to use Postgres database without properly configuring Postgres on your development machine/server.
For details on how to switch over to postgres you should watch this RailsCast episode.
I'm learning ruby. I've created some application and played with it.
i wanted to change the database to postgres. I modified gemfile with
gem 'pg'
[pg already installed in usr/bin/pg]
then update database.yml with
postgresql where sqllite is used.
pg connection bad:
FATAL: role "user" does not exist
i've created some postgres user already. what should i have to change to work with postgres?
development:
adapter: postgresql
database: db/development.postgresql
pool: 5
timeout: 5000
user:
adapter: postgresql
database: db/test.postgresql
pool: 5
timeout: 5000
production:
adapter: postgresql
database: db/production.postgresql
pool: 5
timeout: 5000
If you don't specify a username to connect to PostgreSQL as, libpq will connect using the login username of the current running user.
libpq is what the Pg gem used by Rails uses to connect to PostgreSQL.
So, in your database.yml, specify the PostgreSQL user to connect to the database as with a user: entry. Make sure this corresponds to a user that exists in PostgreSQL, and that pg_hba.conf permits this user to connect.
To learn more, see the client authentication chapter of the PostgreSQL docs.
I know its a two year old question, but I just ran into the same problem, and i fixed it by changing the syntax of config.yml, then I rake db:migrate
default: &default
adapter: postgresql
encoding: unicode
pool: 5
development:
<<: *default
database: yourappnamehere_development
test:
<<: *default
database: yourappnamehere_testdevelopment:
production:
<<: *default
database: YOURAPPNAME_production
username: YOURUSERNAME
password: <%= ENV['YOURAPPNAME_DATABASE_PASSWORD'] %>
I have a rails app that's databases are in SQLite (The dev and production). Since I am moving to heroku, I want to convert my database to PostgreSQL.
Anyways, I heard that the local, development, database does not need to be changed from SQLite, so I don't need to change that, however, how do I go about changing the production environment from SQLite to PostgreSQL?
Has anyone ever done this before and can help?
P.S. I'm not sure what exactly this process is called, but I've heard about migrating the database from SQLite to PostgreSQL, is that what needs to be done?
You can change your database.yml to this instead of using the out of the box sqlite one:
development:
adapter: postgresql
encoding: utf8
database: project_development
pool: 5
username:
password:
test: &TEST
adapter: postgresql
encoding: utf8
database: project_test
pool: 5
username:
password:
production:
adapter: postgresql
encoding: utf8
database: project_production
pool: 5
username:
password:
cucumber:
<<: *TEST
The steps below worked for me. It uses the taps gem, created by Heroku and mentioned in Ryan Bates's Railscast #342. There are a few steps but it worked perfectly (even dates were correctly migrated), and it was far easier than the Oracle -> DB2 or SQL Server -> Oracle migrations I have done in the past.
Note that SQLite does not have a user id or password, but the taps gem requires something. I just used the literals "user" and "password".
Create the Postgres database user for the new databases
$ createuser f3
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) y
Shall the new role be allowed to create more new roles? (y/n) y
EDIT - Updated command below - use this instead
$ createuser f3 -d -s
Create the required databases
$ createdb -Of3 -Eutf8 f3_development
$ createdb -Of3 -Eutf8 f3_test
Update the Gemfile
gem 'sqlite3'
gem 'pg'
gem 'taps'
$ bundle
Update database.yml
#development:
# adapter: sqlite3
# database: db/development.sqlite3
# pool: 5
# timeout: 5000
development:
adapter: postgresql
encoding: unicode
database: f3_development
pool: 5
username: f3
password:
#test:
# adapter: sqlite3
# database: db/test.sqlite3
# pool: 5
# timeout: 5000
test:
adapter: postgresql
encoding: unicode
database: f3_test
pool: 5
username: f3
password:
Start the taps server on the sqlite database
$ taps server sqlite://db/development.sqlite3 user password
Migrate the data
$ taps pull postgres://f3#localhost/f3_development http://user:password#localhost:5000
Restart the Rails webserver
$ rails s
Cleanup the Gemfile
#gem 'sqlite3'
gem 'pg'
#gem 'taps'
$ bundle
Now its become easy with the single command
bin/rails db:system:change --to=postgresql
Since you're moving to heroku, you can use taps to do this:
heroku db:push
This will push your local development sqlite data to production, and heroku will automagically convert to postgres for you.
This should also work to push a production sqlite db to heroku, but it's not tested.
RAILS_ENV=production heroku db:push
you will also need to add the line "gem 'pg'" to your gemfile, 'pg' being the current postgres gem for Rails.
Simply update the config/database.yml file:
default: &default
adapter: postgresql
encoding: unicode
pool: 5
development:
<<: *default
database: projectname_development
test:
<<: *default
database: projectname_test
production:
<<: *default
database: projectname_production
username:
password:
The above is what's generated when you run:
$ rails new projectname --database=postgresql --skip-test-unit
Also add this to your Gemfile:
gem 'pg'
Just Update you datatbase.yml
development: &development
adapter: postgresql
database: Your_database_name
username: user_name
password: password
host: localhost
schema_search_path: public
min_messages: warning
test:
<<: *development
database: test_database_name
production:
<<: *development
database: production_db_name
We are using rails and the basic standards should be follow like DRY, Convention over Configuration etc.. so in above code we are not repeating same code again and again.
It's been mentioned above me, but I don't have enough reputation as a lurker to be able to upvote it. In the hopes of drawing a little more attention for Rails newbies reading this answer:
you will also need to add the line "gem 'pg'" to your gemfile, 'pg' being the current postgres gem for Rails.
^^^ This is a key piece in addition to the database.yml file described in the selected answer to migrate your Rails app to Postgres.
After replacing gem 'sqlite3 with gem pg in the gemfile, I kept getting the sqlite3 error when pushing to Heroku master because I forgot to commit the updated gemfile. Simply doing the following solved this:
git add .
git commit -m 'heroku push'
heroku create
git push heroku master
This is how I have mine setup. If you are only using MRI and not Jruby you can skip the logic in the adapter settings.
defaults: &defaults
adapter: <%= RUBY_ENGINE == 'ruby' ? 'postgresql' : 'jdbcpostgresql' %>
encoding: unicode
pool: 5
timeout: 5000
development:
database: project_development
<<: *defaults
test:
database: project_test
<<: *defaults
production:
database: project_production
<<: *defaults
You can try following:
sqlite3 development.db .dump | psql dbname username
or try with sqlitetopgscript:
http://trac-hacks.org/browser/sqlitetopgscript/0.10/sqlite2pg
A possible solution (not for heroku) it's to use yaml.db from:
http://www.railslodge.com/plugins/830-yaml-db
Today I had the same issue. I'm working on Rails 4.2.8. The solution was specify the pg gem version, in my case, 0.18.4.