We have an existing Rails 3 app that has been copied and loaded on a separate server. We've setup the posgres DB for this server; and also configured the database.yaml, pg gems, etc to setup for the port.
However, only the database schema can be migrate...though all the data files has the correct content.
I've tried variations of the db migrate, dump, resets, load, etc. But I'm not success getting the actual data in the database. Again, the server migration is for identical hardware/software config. So, its Rails3.1/Postgres9/Ruby 1.92
I don't get any errors, the data doesn't populate. The ultimate goal is to have an identical app on the 2 servers.
Any ideas? I've already spent 4 days fighting. Many thanks!!
"...the actual data in the database"
If you have an existing database with transactional data - then I think you want to use postgres tools to move the database? maybe I am not understanding the question correctly?
on the source machine
pg_dump DATABASE_NAME > ~/DATABASE_NAME_dump.sql
copy the dump file to the target machine
on the target machine
bundle exec rake db:create
psql DATABASE_NAME < ~/DATABASE_NAME_dump.sql
lots of good information here - http://www.postgresql.org/docs/9.0/static/backup.html
Have you tried the taps gem?
It enables you to transfer schema and data from one instance to another.
Related
I have crm rails application deployed in heroku so my database is postgresql. I have an iphone app which is using mysql database. Now what i need to do is to synch my iphone database with my heroku database.
Can anyone please help me on this is it possible to sync two different databases in heroku
Thanks in Advance
The way we did this was to do it manually
Yep, it's a real pain, but at least you know it's done correctly
--
Process
For migrating from Postgres to MYSQL is as follows:
"Backup" the database using pg_backups
Download the backup locally
In MYSQL (we used PHPMyAdmin), create a series of SQL calls to insert the respective data into the correct tables
Migrating from MYSQL to Postgres is, as much as I can determine, a similar process:
Download the SQL dumps of your MYSQL DB (I only know how to do this in PHPMyAdmin)
Configure those files to be "PG" ready
Send the SQL to your Heroku postgres db's
--
Notes
For specifics, I only have the following:
If you dump from Heroku, you'll not get a traditional SQL file; you'll receive a .dump file. To change this, you'll need to use pg_restore in the following way:
$ cd c:/your/postgres/install/pg_restore/location
$ pg_restore path_to_your_dump.dump > path_to_your_sql.sql
This will give you the ability to "read" the file, uploading to your db & importing as you wish. That's when the fun begins :)
Importing MYSQL
The way to import MYSQL into a PG db will have to be done manually
Although we've never done this directly (we only migrated from PG), there are several things you can do. Firstly, you can use the PG2MYSQL converter to give you an idea as to the syntax differences between MYSQL & PG
Then I would perform a "dump" from MYSQL in SQL format. This will give me the data required for PG. This will allow me to insert into my PG database (you might have to do this locally to give you the ability to determine the syntax), from which you can then patch in the MYSQL data
I understand it's not the most elaborate explanation - I hope it gives you some ideas
I'm running the rails-composer script with
rails new myproject -m https://raw.github.com/RailsApps/rails-composer/master/compser.rb
And everything goes along smoothly until it asks me if I want to go ahead and drop the db in. I say yes. All drops fail, and all creations fail.
Everything else finished fine. And testing the site brings up a beautiful error site with lots of details on how the database password wasn't accepted. But of course it was never created...
How do I give the script permission to create the database without a password? I've tried preempting creating the username as the app name in postgres. I tried building the Rails project as the postgres user.
It should be a simple and straightforward solution since the rest is automated.
Did you specify something other than SQLite for the database?
From the README:
Choose “SQLite” for the easiest setup. If you choose PostgreSQL or
MySQL, the databases must be installed and running before you run
Rails Composer.
Rails composer does not install and setup your database server for you. It assumes there's a properly named database already present before you run this if you're using PostgreSQL or MySQL.
We have 2 Heroku apps - the first one is production and the second one is staging. I would like to pull data from one table from the production app (it's table users with all user's data) and push it to the staging database.
After a little research I found the addon called pgbackups - I have just a few concerns:
Does this addon also allow to get data only from one table, not from the whole database?
The second thing is this - let's say that on production are users with IDs from 1 to 300. On the staging version users with IDs from 1 to 10. How to put those 300 users from the production to the staging version the way, that these 300 users would be counted from the ID 11 (we would like to keep our staging users in the staging database as well).
Thank you
There are ways to do this in straight SQL. If you're comfortable with that, go for it. This way is for devs comfortable in Rails -- so we pull data out using JSON, and create users with a new ID in the new database from that JSON.
Since you're pulling only 1 table, AND you want to reset the IDs to the new database, I recommend:
bring a copy of the database locally with pbackups
File.open('yourfile.json', 'wb') {|file| file << User.all.to_json }
Connect to your new database, and move yourfile.json up there
then:
users_json = JSON.parse(File.read('yourfile.json'))
users_json.each do |json|
json.delete("id")
User.create(json)
end
This script pulls data from your Heroku database into a local postgres database. You need the pgbackups addon installed. Execute it from the root directory of your Heroku app.
#!/bin/bash -ex
# This script asks Heroku to backup the current production data, then downloads that dump and imports it into the local db.
# It is at least 3 times quicker than heroku db:pull, and we can only do it because we are dumping from postgres to postgres.
heroku pgbackups:capture --expire
curl -o tmp/latest.dump `heroku pgbackups:url`
pg_restore --verbose --clean --no-acl --no-owner -d your_db_name tmp/latest.dump
You can check my answer in this thread. You can use a library called forceps to do exactly what you are asking for.
I just deployed my rails app to a Linode VPS, and was wondering what would be the best way of adding records to the DB.
I have tables such as Categories, which I'd like to populate.
I thought of the Taps gem, using a csv, or an sql dump file.
I'd like to know if there are any tools out there for this?
Thanks
For this puporse there are the so called seed file which is default in:
db/seeds.rb
You can add entries here ( there is an example in the seed file ), which you can generate after deployment with a rake task:
rake db:seed
You probably are using bundler as well, so use:
bundle exec rake db:seed
In case of large number of seeds you can always create multiple files, see this blogpost about handling large seed files.
However, if you are in a state, where the already existing data in the app is crucial and you are changing servers or database drivers you wanna take a look on yaml_db gem which gives a nice method to abstract the existing data away from your actual db driver and export it into a .yaml file which you can import later back e.g.: after deploying on a new server.
See Railscast - #179 about seeding.
The rails way would be to use seed data in db/seeds.rb and then populate it by using rake db:seed.
You could also use a sql dump file and restore by issueing mysql -u <user> -p <database_name> < <mysql_dump_file>
The Easy Reference Data gem is similar to db:seed, but will update records if entries already exist. It also has easy integration with Capistrano.
Full disclosure: The company I work for developed the gem.
I have a production.sqlite3 file which I want to import its data to the current rails project, the database schema is match between the file and my current project. I did copy the content to the development.sqlite3 file but this does not work. The only way I know to insert data to the database is by loading some yml file or use the seed command. Is there any magic command or other ways to let the rails loading the data from .sqlite file? Because from what I disover, the behavior of rails is that it only creates the .sqlite3 file when you do rake db:create &migrate, but it will ignore what ever manual changes happen in that file. Please help!
It looks like renaming your .sqlite3 file should work. Make sure you have restarted your server, and that the server is running in the correct environment. If you move both the .sqlite3 files into a completely different folder, and restart the server, an error should occur. If it doesn't, your server didn't properly restart - kill all ruby processes and try again.
In answer to your question about migrating data, I had a similar problem recently, where I had to migrate from sqlite3 to mysql. Best solution I found was yaml_db. It seemed to do the job beautifully - add it to the project, then do
rake db:data:dump
Swap the database configurations (or migrate a new database file, or change your development environment, whatever it is you need to do to make an empty but structured database active), then:
rake db:data:load