I am trying to import data to a postgres database in heroku. I am able to upload the correct schema to heroku by using the command:
heroku pg:psql -a [app name] < db/structure.sql
so now when I run heroku pg:psql and connect to my database I can run:
\dt
and it lists all of the tables that I want. However, only the schema is uploaded, not the actual data. I have looked over a bunch of different ways to upload databases to heroku, but have been unable to get any to work. I was wondering if there is an easy way to import local csv data to heroku. Locally, I can run:
sqlite3 development.sqlite3
.mode csv
.import [table name] [filename]
And it imports the data that I want. Is there a similar way to do this for heroku databases? I know that heroku uses postgres and not sqlite3, but since I was able to load the schema easily, I thought there might be an easy way to upload data. This is my first ruby on rails project I've ever attempted to publish, so any guidance would be appreciated!
Thanks!
Upload your csv to heroku and run
COPY zip_codes FROM '/path/to/your_file.csv' DELIMITER ',' CSV;
in pg shell
Look https://gist.github.com/jboesch/5605747, Importing csv to heroku postgres how you can do it
Related
Can someone please help me how can i pull single table from heroku database.
I have tried the following way:
heroku db:pull --tables table_name
with no luck as heroku db:pull is deprecated now.
Thanks in Advance
Not sure if you can pull a single table from heroku but there are certainly some options to pull whole database.You can use Herokus PG Backups add-on to import or export your database. First you'll have to add it as add on in your app by
$ heroku addons:add pgbackups
$ heroku update
After that you can download your database by
$ heroku pgbackups:capture
$ curl -o latest.dump `heroku pgbackups:url`
You can also use pg:transfer Heroku CLI plugin to transfer the data in a single step.
From your app's directory, copy your production database locally (assuming a local PG db), by installing the plugin and execute the pg:transfer command.
$ heroku plugins:install https://github.com/ddollar/heroku-pg-transfer
$ heroku pg:transfer
Not sure if this is still relevant, but you can achieve what you want pulling a CSV dump of a single table (or any other format) and then import it. The steps would be as follows:
# 1. Connect to psql on Heroku
$ heroku pg:psql --app your_app_name DATABASE
Then you can download a table like so:
# 2. Pull table_you_want_to_copy
your_app_name::DATABASE=> \COPY table_you_want_to_copy TO '~/local/path/your_table.csv' WITH (FORMAT csv, DELIMITER ',', HEADER true);
A simple version of an importer could look like this:
# 3. Seed local table from fetched CSV
CSV.foreach('~/local/path/your_table.csv, headers: true) do |row|
YourModel.create(row.to_hash)
end
In this block you can ofc define whatever other logic you might require.
I have facing problems with migrating data to my heroku app which has Postgresql as database for my hosted site(Production). At my development site i have rails 3.2.13 with Sqlite3 as database. I have followed Ruby on rails Tutorial by Michael Hartl
i have used git push heroku to update my site at heroku. i also want to update database along with data. But heroku run rake db:migrate migrates schema not data. I tried db:push to push data to heroku but i get error
dependency.rb in 'to_specs' :Could not find sequel (~) 3.20.0
also i have searched and found that i should first my sqlite data to dump.sql and then run
heroku pg:psql HEROKU_POSTGRESQL_COLOR --app app_name < file.sql as answered in https://stackoverflow.com/questions/15371394/...
but it failed with
the local psql command could not be located
please tell me what i am doing wrong. or what is the right way to update heroku postgresql with my development sqlite3 data.
Thanks in Advance
It is not a good idea to fill the production database with the data that you have now in the developement database. Because, if you have problems with your production database in the future, and you need to refill it again, your development db may changed (e.g dropped), and you are not going to be able to do it again.
For this need, Rails provides seeds in db/seeds.rb file. You should create all the neccessary objects there.
Then when you push your code to Heroku, Heroku is going to prepare the database, create the schema, and seed it. If you need to seed the db manually, you can run bundle exec rake db:seed, if you want to run it in Heroku: heroku run bundle exec rake db:migrate
Is there way to dump heroku postgres database and import the data into my local SQLite database?
I tried using https://github.com/ludicast/yaml_db and heroku db:pull with no success. I am developing on Windows 7.
Please read through
https://devcenter.heroku.com/articles/heroku-postgres-import-export
and follow those to export.
Though the above link can be used to export alone, it does not have solution for the entire process. So you can look on the similar question and its solution
Copy a heroku postgres db to a local sqlite db
I have created the test, production and development databases on server manually but not able to figure out, how to transfer data from local myapp_development database to myapp_production database on the server ?
I searched and found this How to move Rails app + pgsql database from localhost to my server?, capistrano is giving me lot of errors and I am sure there must be a standard Rails way or postgres way to do this as exporting data as dump and again importing it though I may have to do it if I can't find anything better.
You can dump the database using pg_dump then scp the database dump up to the database and run it on the server using psql databse_name < database.dump
I use yaml_db gem. It can dump your data into a file and then load it back into the database. It's written using activerecord so it works with most databases.
URL: https://github.com/ludicast/yaml_db
rake db:data:dump -> Dump contents of Rails database to db/data.yml
rake db:data:load -> Load contents of db/data.yml into the database
Prefix the commands with RAILS_ENV=development or RAILS_ENV=production to choose target and destination scheme.
After deploying an app of Heroku I am having trouble getting the database populated with some of the data I need there. I ran heroku rake db:populate and it did create the initial admin user, but failed to put the rest of the data in.
I am populating the database with files from my local disk. I suspect the problem is caused because it sees nothing in the directory listed in the sample_data.rake file, as it in on my hdd and not the server. How can I get around this?
I figure that I have to either host all the files and change the directory to the servers directory, or find a way around this. I would guess there is an easy way to move the database from my computer to heroku? I'm obviously pretty new to this.
Thanks!
heroku commands have changed. now it should be:
heroku run rake db:push
heroku pg:reset DATABASE
heroku run rake db:seed
also data from local database can be uploaded to heroku by installing taps gem and pushing data from local PostgreSQL db.
gem install taps
heroku db:push
If you have a clean local database (i.e. just the good stuff, no silly testing data), then you could try heroku db:push:
db:push [<database_url>] # push a local database into the app's remote database
And please remember to develop on PostgreSQL if you're going to use the Heroku shared or dedicated databases, using the same database and version (version 8.3 for shared, 9.0 for dedicated) in both your development and production environments will save you much pain, suffering, and confusion.
References:
How can I import my existing data to Heroku?
Import: Push to Heroku
I setup the development db, and push it to Heroku.
rake db:reset
rake db:seed
heroku rake db:push
If you are using the default PostgreSQL, another option is heroku pg:reset
heroku rake db:seed
I use the following in the seed.rb file:
require 'pathname'
RailsRoot = Pathname.new(RAILS_ROOT).expand_path
print "Loading data..."
fileData = File.read (RailsRoot + "db/data-file.csv")