Heroku db:pull 'db:pull is not a heroku command' - ruby-on-rails

I'm getting this message even though I've used heroku db:pull a million times. For some reason it's no longer working even though I haven't even touched my code. Any ideas?
The full error message is
db:pull is not a heroku command.
Perhaps you meant pg:pull
See heroku help for a list of available commands.

For now, we can still use heroku-legacy-taps until the taps gods decide to deprovision the taps servers.
Run: heroku plugins:install https://github.com/heroku/heroku-legacy-taps.git
Then continue your db:push and db:pull workflow as usual.
(thanks to GantMan for the hint)

Since the taps servers will be decommissioned at some future point, the plugin is probably not the best long term solution. But of course you can run your own taps server.
Steps
Step 1: Start Your taps server
taps server `heroku config:get DATABASE_URL` db db
Step 2: Run the taps client
In a different shell:
taps pull sqlite://db/development.sqlite3 http://db:db#localhost:5000
Step 3: Shut down the taps server
Once the import is done you can shutdown the server with the normal Ctrl-C key combination.
Notes
This will pull my production database down into a local SQLite database. Of course if you are using MySQL or something locally just replace the sqlite URI with the equivalent MySQL URI.
Taps requires you to set a username/password. Since I am just running it locally for a short time I just use "db" for both. If you change these you will need to also update the username/password in the URL for step 2.
You can also use taps push to copy data to the production server although you should obviously do that with caution.
Taps has a number of bugs it has acquired over time due to the lack of activity by the maintainer:
The biggest annoyance is the fact that it stopped working after rack incorporated OKJson into Rack. The OKJson in Rack conflicts with the modified version that is included in taps. I created a patch to resolve this but no activity has been done to merge it. In the meantime workarounds include forcing taps to use an earlier rack. Paxa suggested an easy approach by modifying the bin/taps file. If you don't want to modify packaged installed files on your system you can also follow hax8or's instructions which use bundler to force the right version of Rack.
The progress bar does not render correctly. fd fixed this in his pull request but it has also not been merged in. Since this is purely cosmetic you can just ignore the bad output.
#wijet recently forked taps and incorporated some of the most important patches. He has named his gem "taps-taps" if you are looking for an easy out-of-the-box install.

This is still possible. Just run
heroku plugins:install https://github.com/heroku/heroku-taps.git
You'll be able to do your classic stuff, it's just a plugin now.
If you're having trouble still, you may need to make sure some other gems are installed. You can also run the following to be sure:
gem install heroku taps sequel
I hope this helps! I love db:push/pull just like the rest of the world, and I'm sad to see it go.
If you're still having problems take a look at this one: https://github.com/heroku/heroku-legacy-taps
GOODLUCK!

I used to use db:pull and it worked fine. After it was removed, I tried pg:pull but it is just not working for me.
I found a different solution.
If your local database is PostgreSQL, and you have the pgbackups addon enabled, this is the sequence of commands I'm using to copy the remote DB to my local machine:
$ wget "`heroku pgbackups:url --app app-name`" -O backup.dump
$ # Create the local database first, if it's not created yet. Then:
$ pg_restore -d database-name -c backup.dump -U database-user-name -O --no-acl -h localhost
Replace app-name, database-name and database-user-name with your own info.
You'll likely want to ask heroku to make a backup just before you pull your data:
heroku pgbackups:capture --expire
otherwise you get the data from whenever it did its own backup.

This is the error message I got when I tried db:pull.
db:pull is not a heroku command.
Perhaps you meant pg:pull.
See heroku help for a list of available commands.
Have you tried pg:pull?
Usage: heroku pg:pull <REMOTE_SOURCE_DATABASE> <LOCAL_TARGET_DATABASE>

Looks like db:pull etc is being deprecated & moved
See here https://github.com/heroku/heroku-pg-extras/issues/42
I found that the ability of db:push & pull to move single eg static tables of data up & down from dev to staging to production was invaluable - now looks like you need to create a new empty database and do an entire dump into it and then run pg commands to move an individual table

I found my answer here, but I put it in a rake task. I think this is a sensible way to deal with this situation. If you're running a local instance of postgres to work with your postgres on Heroku, you might try something like this:
# lib/tasks/database.rake
namespace :database do
desc "Gets the database from heroku and restores it to development"
task :pull => :environment do
dumpfile = 'tmp/latest.dump'
db_config = Rails.application.config.database_configuration[Rails.env]
File.delete(dumpfile) if File.exist?(dumpfile)
`heroku pgbackups:capture --app app-name-here`
system("curl -o #{dumpfile} `heroku pgbackups:url --app app-name-here`")
`pg_restore --verbose --clean --no-acl --no-owner -h localhost -d #{db_config['database']} #{dumpfile}`
end
end
Now, anytime I wish to pull my production data into dev I just run rake database:pull
This is a very rudimentary solution, but I only need it to do this one thing in my case.

Related

Heroku run rake db:migrate command doesn't update database

I'm reading railstutorial.org Chapter 2 (about toy_app) using cloud IDE and I've run into a problem. I'm newbie so I would appreciate a simple sollution and explanation what I did wrong.
I followed every step as described in tutorial: I created my 'hello world' index page and I pushed it to heroku. I renamed my heroku site as well (it wasn't required in tutorial). Everything was fine at that moment.
I added user and micropost models. I've started a local server (on second terminal by typing rails server -b $IP -p $PORT) and added some users and microposts. Again, everything was working as expected.
I didn't close local server. I committed changes to git and I pushed it to Bitbucket and then to heroku. Finally, I typed heroku run rake db:migrate. It was the final step of this tutorial.
Everything seemed to be Ok, but I noticed that posts and users I added on local server weren't visible on my heroku site. Since I didn't close my local server, I checked it and posts I created were still available. As I said, situation on my heroku site was different.
I didn't get any errors during that process, only one warning after typing git push: Warning: Permanently added the RSA host key for IP address ... to the list of known hosts. Next time it didn't show up.
After my first try of every step I got promising outputs. After second and next tries it returned
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean`
and
Everything up-to-date
So still nothing worrying.
I tried to restart server, to follow those steps again after closing my local server but it didn't help. I searched for solution but only what I found were problems with massive errors or different problems.
Actually, It is problem I can go along with because I don't need it updated at this stage, but I would like to hear an answer and find out if there is something wrong with heroku or it it normal behavior or even if it is newbie's blindness.
First your local database and heroku database are different. Any user created on local app won't be available on heroku app unless you use a common database.
rake db:migrate, in its most basic form, just runs the change or up method for all the migrations that have not yet been run. If there are no such migrations, it exits. It will run these migrations in order based on the date of the migration.
Try creating user and post on heroku app, it should work all fine.

how to relaunch rails app on heroku

My app runs properly locally and has a crash when it runs on Heroku. The app does fine until I run a new_post_path call. I tried git push to see if a change to the new.html.erb file or the _form.html.erb file was the issue - but I think during the last push to heroku - my database may have taken a hit. At the time I was adding a t.text field (longtext) and it seems to be the issue.
So I wanted to roll back or totally re-start at heroku. But I have not found anyway to do it. Since this is not near production yet. Is there anyway to just create a new instance at heroku and push up to it?
Again, local version runs perfect. Thanks in advance for any ideas.
I would say there are several ways to handle this problem
restarting heroku
Run $ heroku restart in the rails dir. Sometimes I believe in magic.
debugging
Use $ heroku log to debug what is causing the error.
migrate database
Perhaps you made a database change in local rails, while heroku rails didn't.
Run heroku run rake db:migrate
drop database and remigrate database
See this thread. Sometimes old migration might be in conflict with current migration. Do a clean migration, if the data in your database isn't important.
recreate heroku instance
run $ heroku apps:destroy –app appname and replace appname with your appname. Then delete the current heroku remote repository location from git. git remote rm heroku. Then recreate heroku. run $ heroku create. go through pushing your code and migrating database like usual.
Relaunching might be the least of your problems, so I suggest a few things:
Review this Heroku article on releases, including the Heroku rollback feature.
Add the PG Backups add-on to your app.
I also recommend using something like the exception_notification gem in combination with the SendGrid add-on, so that you can get an email with details on the error as well.

Syncing Heroku and localhost databases

Hey so I am following the One Month Rails guide to learning Ruby and I have hit a wall on one of the lessons. I have just finished uploading an image with Paperclip, and as I finished my work on my localhost and checked it on Heroku, something went wrong. The pin/image appears to have been pushed to my Heroku account, the only problem is that the username and password that works for my localhost:3000 won't work for my Heroku account. The same password should work for both, but for some reason something is wrong. I wish I could give you the action that is going on in my terminal, but the ruby rails is the only thing that has a continuous status flow. The problem may have been when I switched my password after not using my account on localhost for a few weeks, but i thought that once i "git pushed" that to heroku master, it would've synced. I have tried heroku run rake db:setup which didn't seem to do too much as well as wrestled by way through "Importing a Heroku Postgres Database with PG Backups", but I had some trouble working through that. Any ideas? Thanks for the help.
Your 'database.yml' should not be sent to Heroku, they take care of that, creating a new database.yml config file with the proper DB access details.
Try logging into your Heroku instance and deleting the file.
Edit: nevermind, assumed you were not able to connect to the DB, not to login into the website.
So if I'm understanding you correctly, you've deployed your application to Heroku and the login (to your application) that was working locally doesn't work on Heroku.
Deploying your application doesn't deploy data. Assuming you've run heroku run rake db:migrate then your database schemas will at least match.
At this point, you've got a couple of options.
Use a seeds.rb file which you can load with heroku run db:seed to setup some 'seed' data so that you can login.
Push your local database to Heroku - either via heroku db:push or using heroku pg:transfer provided by https://github.com/ddollar/heroku-pg-transfer
Use heroku run console to create your user account via the command line
User.create(email: 'someemail.com', password: 'somepassword', password_confirmation: 'somepassword')
I'd be inclined to go with the later option.
How did the user get in their in the first place? Perhaps going back to that step in the tutorial - just remember, if you are using rails console locally to use heroku run console on Heroku.

Easiest way to dump Heroku database for use in local seed.rb?

I can dump a heroku database with $ heroku pgbackups:capture. Also, this SO post shows that there are tools for taking a development database and dumping it to seed.rb.
I'm wondering if there is an easy way to combine the two processes, effectively dumping the data from a production Heroku database into my local seeds.rb for more realistic development testing.
If this is possible, what's the cleanest way to do this?
Update:
Based on the insightful answer from dB', I may consider using PGSQL locally. I am still interested, however, in the seed.rb aspect of the question if there is a way to do that easily.
There are a couple ways to accomplish such a thing. #dB' has outlined one of them - using the PG Backups add-on to export your database. It's a great options but involves a few (trivial) manual commands.
I would recommend using the pg:transfer Heroku CLI plugin to transfer the data in a single step. Under the covers it's still very much the same thing happening as with using PG Backups, but it's packaged a bit nicer and has some useful defaults.
From your app's directory, copy your production database locally (assuming a local PG db), by installing the plugin and execute the pg:transfer command.
$ heroku plugins:install https://github.com/ddollar/heroku-pg-transfer
$ heroku pg:transfer
There are a couple options you can set as well. See my writeup for more details.
Hope that helps! And yes, please do use the same database during development as you do in production.
Not sure if it's what you're looking for, but have you tried copying the database to your local machine using pgbackups:capture and pg_restore? This approach doesn't use seeds.rb, but still recreates your production database on your local machine. It looks something like this.
$ heroku pgbackups:capture
$ curl -o latest.dump `heroku pgbackups:url`
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U myuser -d mydb latest.dump
(This code is copied liberally from the explanation at the Heroku dev center.)

transfer db from one heroku app to another faster

Is there a faster way to transfer my production database to a test app?
Currently I'm doing a heroku db:pull to my local machine then heroku db:push --app testapp but this is becoming time consuming. I have some seed data but it is not nearly as accurate as simply testing with my real-world data. And since they're both stored on a neighboring AWS cloud, there must be a faster way to move the data?
I thought about using a heroku bundle, but I noticed the animate command is gone?
bundles:animate <bundle> # animate a bundle into a new app
It's quite common to migrate databases between staging, testing and production environments for Rails Apps. And heroku db:pull/push is painfully slow. The best way I have found so far is using Heroku PG Backups add-on and it's free. I followed following steps to migrate
production database to staging server:
1) Create the backup for the production-app db
heroku pg:backups capture --app production-app
This will generate b001 backup file from the main database (usually production db in database.yml)
2) To view all the backups (OPTIONAL)
heroku pg:backups --app production-app
3) Now use the pg:backups restore command to populate staging server database from the last backup file on production server
heroku pg:backups restore $(heroku pg:backups public-url --app production-app) DATABASE_URL --app staging-app
Remember that restore is a destructive operation, it will delete existing data before replacing it with the contents of the backup file.
So things are even easier now .. checkout the transfer command as part of pgbackups
heroku pgbackups:transfer HEROKU_POSTGRESQL_PINK sushi-staging::HEROKU_POSTGRESQL_OLIVE -a sushi
https://devcenter.heroku.com/articles/upgrading-heroku-postgres-databases#4b-alternative-transfer-data-between-applications
This has worked beautifully for me taking production code back to my staging site.
The correct answer has changed again as of March 11, 2015.
heroku pg:backups restore $(heroku pg:backups public-url --app myapp-production) DATABASE_URL --app myapp-staging
Note specifically that the argument is now public-url.
https://blog.heroku.com/archives/2015/3/11/pgbackups-levels-up
Update for mid-2015...
The pgbackups add-on has been deprecated. No more pgbackups:transfer. pg:copy is ideal for this scenario.
To copy a database from yourapp (example db name: HEROKU_POSTGRESQL_PINK_URL to yourapp_staging (example db name: HEROKU_POSTGRESQL_WHITE_URL)
# turn off the web dynos in staging
heroku maintenance:on -a yourapp-staging
# if you have non-web-dynos, do them too
heroku ps:scale worker=0 -a yourapp-staging
# backup the staging database if you are paranoid like me (optional)
heroku pg:backups capture -a yourapp-staging
# execute the copy to splat over the top of the staging database
heroku pg:copy yourapp::HEROKU_POSTGRESQL_PINK_URL HEROKU_POSTGRESQL_WHITE_URL -a yourapp-staging
Then when it's complete, turn staging back on:
# this is if you have workers, change '1' to whatever
heroku ps:scale worker=1 -a yourapp-staging
heroku maintenance:off -a yourapp-staging
Reminder: you can use heroku pg:info -a yourapp-staging (and yourapp) to get the database constants.
(source: https://devcenter.heroku.com/articles/upgrading-heroku-postgres-databases#upgrade-with-pg-copy-default)
psql -h test_host -c 'drop database test_db_name; create database test_db_name;'
pg_dump -h production_host production_db_name | psql -h test_host test_db_name`
This can be done on production_host or on test_host — will work both ways.
Have not tested this, but it might work.
Do this to get the URL of your source database:
heroku console "ENV['DATABASE_URL']" --app mysourceapp
Then try executing db:push with that.
heroku db:push database_url_from_before --app mytargetapp
This might not work if Heroku doesn't allow access to the DB machines from outside their network, which is probably the case. You could, perhaps, try using taps (gem that heroku db commands use internally) from within your app code somewhere (maybe a rake task). This would be even faster than the above approach because everything stays completely within AWS.
Edit:
Here's an (admittedly hacky) way to do what I described above:
Grab the database URL as in the first code snippet above. Then from a rake task (you could do it on console but you risk running into the 30 second timeout limit on console commands), execute a shell command to taps (couldn't easily determine whether it's possible to use taps directly from Ruby; all docs show use of the CLI):
`taps pull database_url_from_source_app #{ENV['DATABASE_URL']}`
The backticks are important; this is how Ruby denotes a shell command, which taps is. Hopefully the taps command is accessible from the app. This avoids the problem of accessing the database machine from outside Heroku, since you're running this command from within your app.
Heroku enables you to fork existing applications in production. Use heroku fork to copy an existing application, including add-ons, config vars, and Heroku Postgres data.
Follow the instructions on Heroku: https://devcenter.heroku.com/articles/fork-app
Update for mid-2016...
Heroku now have a --fast flag when creating forks, however they will be up to 30 hours out-of-date.
$ heroku addons:create heroku-postgresql:standard-4 --fork HEROKU_POSTGRESQL_CHARCOAL --fast --app sushi
https://devcenter.heroku.com/articles/heroku-postgres-fork#fork-fast-option

Resources