I have a Ruby on Rails application and I want to get the data from a Heroku server to a local machine.
I have tried the steps mentioned in
https://devcenter.heroku.com/articles/heroku-postgres-import-export, but it just copies the Data Definitions not the actual data on the server.
Is there any way to get the data from Heroku to local databases?
Try using the steps detailed here.
Basically, grab the information from your Heroku database using heroku config:get. Look for your Heroku Postgres url. It will look something like this: HEROKU_POSTGRESQL_RED_URL: postgres://user3123:passkja83kd8#ec2-117-21-174-214.compute-1.amazonaws.com:6212/db982398. Note that the string is configured as database://username:password#host:port/databasename. Thus, in this example, the username is user3123, the database name is db982398. You'll need this for the next part.
Then, you'll use the information in the above postgres string to make a local copy of your Heroku database using pg_dump. Enter the following into your terminal:
pg_dump --host=<host_name> --port=<port> --username=<username> --password --dbname=<dbname> > output.sql
In each place where <....> is in the above code, enter your specific information. After typing this, the terminal will ask you for your password. Enter it.
Finally, you'll load that dumpfile into your local database using psql.
psql -d <local_database_name> -f output.sql
Be sure to put the name of your local database.
Related
I am working with a Rails 4 app, I am using PSQL both in my development and production. Due to some reason I have to work with a new computer/laptop so I set up Rails environment in it and cloned my app into it, but what I really require is that, I need my existing databse with data in it, how to do it ?
To do this you need to create a dump on system 1 and restore it on system 2, here are the steps:
sudo -u postgres pg_dump <DB_NAME> > dump - creating a file dump
Copy this file via dropbox or whatever to another system.
sudo -u postgres psql <DB_NAME> < dump - copy the new created dump to new system.
Note:
You should have empty created database on your new system, or you can use dataonly dump passing the --data-only to pg_dump command.
Also you can read documentation for pg_dump to find any other options which you might need.
When I am developing locally, I'll just run a
rails c
and I can fire of queries at the console.
However, how can I query and access my postgres database that is up on heroku?
You can run heroku run rails c --app <appname>. I will also add, if you want to view your database, you can make use of PG Commander https://eggerapps.at/pgcommander/ which actually automatically parses all the credentials out for you.
If you want to do this, run heroku config --app <appname> and then copy the DATABASE_URL. After you've done this, create a new favourite in your PG Commander and you'll see that it has already automatically filled in all the credentials based on what was in your clipboard.
I've got a local .csv file with updated user data that I want to run a simple one-time script on in production (on Heroku, Rails 4 app). It's open source, so I can't just include the file in the repo without exposing the data. I'd like to be able to do it from the CLI but can't seem to figure it out.
There's probably a better solution, but I've been trying unsuccessfully to set the contents of the file (which isn't huge) to an environment variable on Heroku. My Bash skills are weak...
$ heroku config:set MY_CSV=<./my_local_csv_path.csv
didn't work -- it will not set anything. (running $ heroku config shows the env variable blank -- MY_CSV:).
Is there a better way to make this .csv file accessible to a script on Heroku? I suspect it's a similar problem to just accessing the local filesystem from the Heroku console.
I just did something similar to what you are doing where the user uploads a CSV for processing in my app. I am also hosting on heroku. Check out...
http://railscasts.com/episodes/396-importing-csv-and-excel
Basically let your model doing your processing for you that way you don't have to store any files on heroku.
I was able to do this with the heroku cli -e flag. It has to be a relatively small file. (all arguments combined are limited to 32kb)
It goes something like this:
# gzip and base64 encode the file
# (without the encoding, the gzip characters break this argument)
FILE_CONTENTS=$(cat /path/to/file.csv | gzip -fc | base64)
# Pass the variable from above to the dyno above.
heroku run -a APP_NAME -e "FILE_CONTENTS=\"$FILE_CONTENTS\"" rails c
Then in your console, you can get the csv by running:
csv_contents = ActiveSupport::Gzip.decompress(
Base64.decode64(
ENV['FILE_CONTENTS']
)
)
We have 2 Heroku apps - the first one is production and the second one is staging. I would like to pull data from one table from the production app (it's table users with all user's data) and push it to the staging database.
After a little research I found the addon called pgbackups - I have just a few concerns:
Does this addon also allow to get data only from one table, not from the whole database?
The second thing is this - let's say that on production are users with IDs from 1 to 300. On the staging version users with IDs from 1 to 10. How to put those 300 users from the production to the staging version the way, that these 300 users would be counted from the ID 11 (we would like to keep our staging users in the staging database as well).
Thank you
There are ways to do this in straight SQL. If you're comfortable with that, go for it. This way is for devs comfortable in Rails -- so we pull data out using JSON, and create users with a new ID in the new database from that JSON.
Since you're pulling only 1 table, AND you want to reset the IDs to the new database, I recommend:
bring a copy of the database locally with pbackups
File.open('yourfile.json', 'wb') {|file| file << User.all.to_json }
Connect to your new database, and move yourfile.json up there
then:
users_json = JSON.parse(File.read('yourfile.json'))
users_json.each do |json|
json.delete("id")
User.create(json)
end
This script pulls data from your Heroku database into a local postgres database. You need the pgbackups addon installed. Execute it from the root directory of your Heroku app.
#!/bin/bash -ex
# This script asks Heroku to backup the current production data, then downloads that dump and imports it into the local db.
# It is at least 3 times quicker than heroku db:pull, and we can only do it because we are dumping from postgres to postgres.
heroku pgbackups:capture --expire
curl -o tmp/latest.dump `heroku pgbackups:url`
pg_restore --verbose --clean --no-acl --no-owner -d your_db_name tmp/latest.dump
You can check my answer in this thread. You can use a library called forceps to do exactly what you are asking for.
I've just deployed my application to heroku and pointed my custom domain to the heroku servers. How can I check the records in my heroku database?
You can use heroku run rails console and look at your records with Model.all or any other method.
If you want to backup the database look at heroku PG backups, you then can import your database on your local machine and look at it there. Depending on your db adapter you could use sqlite browser for sqlite3 or phpmyadmin for MySQL.
I found a similar question like this and here is what #Chowlett says:
"You could run heroku pg:psql to fire up a Postgres console, then issue \d to see all tables, and \d tablename to see details for a particular table."
You can also type select * from tablename; to view the table contents.
How to view current database schema for Heroku app in Terminal?
heroku db:pull to pull your production DB locally to take a peek in it.
I'll give the method for connecting via a GUI tool
Run the following command to get the database credentials from Heroku:
heroku pg:credentials DATABASE_URL
Then you can use a GUI tool like PG Commander or PGAdmin to connect to the db
Heroku now has an add-on named PostgreSQL Studio (currently free & in beta) that would let you access your database from within the browser, without having to use CLI, much like PHP MyAdmin.
To attach this add-on to your application,
heroku addons:create pgstudio
Then go to the list of add-ons on Heroku, select PostgreSQL Studio, authorize it, select the database to connect with from the dropdown list of all databases and it will take you to the web-based interface to handle your selected database.
You may refer to this official article on Heroku:
https://devcenter.heroku.com/articles/pgstudio
The easy answer is:
heroku pg:info
You can also download a client side Postgres, like Postico, and using the information provided in that URL to enter password and database name etc, then you can create locally, just like phpMyAdmin.
I use the admin_data gem, works well in Heroku.
You can use heroku dataclips that allows to run queries online. Here you can find documentation https://devcenter.heroku.com/articles/dataclips.
Connect to the database using Sequel Pro. You can find your ClearDB url using heroku config command. The structure for connecting is as follows:
CLEARDB_DATABASE_URL => mysql://[username]:[password]#[host]/[database name]?reconnect=true