I want to use pg_dump on a Heroku multi-schema pgSQL app using the apartment gem, but am afraid I have too many schemas to make it work.
I looked into it and found the following from the creator of the apartment gem at the following link
I'm the author of the Apartment gem that's referenced here. We've been
using it for over a year on Heroku and it utilizes schemas.
It's definitely fine to use, we still get great performance with well
over 100 schemas in an application with 50+ tables per schema.
The article mentioned by #4ware talks about issues with heroku's
pg:backups command. (I'm pretty sure that article came about from our
support queries to them)
It DEFINITELY has issues, but this is not a shortcoming of Postgresql,
schemas or multi-tenancy with schemas, but rather the heroku tool
itself. Now that Postgresql has ingres support on their dbs, you don't
really need to use their built in tools. We just pg_dump when we need
to and it works just as fast as one would expect.
How can I use external tools? Can someone elaborate on this?
I don't know what that person means by "ingres"- certainly not the product, Ingres. Perhaps "ingress", i.e. "inbound"?
In any case: You can make a direct SSL connection to a PostgreSQL database on Heroku using any tool that supports the libpq protocol. That includes psql ... and pg_dump.
Simply:
pg_dump -Fc -f mydb.dbbackup "sslmode=require host=my.heroku.host.name port=5432 dbname=my.heroku.db.name user=my.heroku.user"
(with any other options you want).
See the Heroku documentation for more details.
Note that Heroku database credentials may change, per the docs link above. So if you're automating this or doing it regularly you should use your Heroku account to dynamically fetch the database credentials per the Heroku manual:
pg_dump -Fc -f mydb.dbbackup $(heroku pg:credentials DATABASE)
Related
I have deployed an application in Heroku. It is Rails applications. The DB is postgres. So this is the app in heroku. Now I want to create new app which will be a clone of the previous app. But it will have separate DB. How should I get going? I would also like to know about configuring the DB from heroku. I am not so techy so please go easy on me.
Thanks!
But it will have separate DB
So, just create a new heroku app, add PG add-on to it, and that's it. No more steps. It will have a separate database. I think you don't even have to add PG add-on, it probably is added by default.
If you wanted two apps use the same DB, then you'd have to do extra steps.
You can use Heroku Fork to clone your application. This will copy the app, environment variables and any postgres databases (and their data if you want it) to a separate Heroku application.
heroku fork --from old-app-name --to new-app-name
I have crm rails application deployed in heroku so my database is postgresql. I have an iphone app which is using mysql database. Now what i need to do is to synch my iphone database with my heroku database.
Can anyone please help me on this is it possible to sync two different databases in heroku
Thanks in Advance
The way we did this was to do it manually
Yep, it's a real pain, but at least you know it's done correctly
--
Process
For migrating from Postgres to MYSQL is as follows:
"Backup" the database using pg_backups
Download the backup locally
In MYSQL (we used PHPMyAdmin), create a series of SQL calls to insert the respective data into the correct tables
Migrating from MYSQL to Postgres is, as much as I can determine, a similar process:
Download the SQL dumps of your MYSQL DB (I only know how to do this in PHPMyAdmin)
Configure those files to be "PG" ready
Send the SQL to your Heroku postgres db's
--
Notes
For specifics, I only have the following:
If you dump from Heroku, you'll not get a traditional SQL file; you'll receive a .dump file. To change this, you'll need to use pg_restore in the following way:
$ cd c:/your/postgres/install/pg_restore/location
$ pg_restore path_to_your_dump.dump > path_to_your_sql.sql
This will give you the ability to "read" the file, uploading to your db & importing as you wish. That's when the fun begins :)
Importing MYSQL
The way to import MYSQL into a PG db will have to be done manually
Although we've never done this directly (we only migrated from PG), there are several things you can do. Firstly, you can use the PG2MYSQL converter to give you an idea as to the syntax differences between MYSQL & PG
Then I would perform a "dump" from MYSQL in SQL format. This will give me the data required for PG. This will allow me to insert into my PG database (you might have to do this locally to give you the ability to determine the syntax), from which you can then patch in the MYSQL data
I understand it's not the most elaborate explanation - I hope it gives you some ideas
I can dump a heroku database with $ heroku pgbackups:capture. Also, this SO post shows that there are tools for taking a development database and dumping it to seed.rb.
I'm wondering if there is an easy way to combine the two processes, effectively dumping the data from a production Heroku database into my local seeds.rb for more realistic development testing.
If this is possible, what's the cleanest way to do this?
Update:
Based on the insightful answer from dB', I may consider using PGSQL locally. I am still interested, however, in the seed.rb aspect of the question if there is a way to do that easily.
There are a couple ways to accomplish such a thing. #dB' has outlined one of them - using the PG Backups add-on to export your database. It's a great options but involves a few (trivial) manual commands.
I would recommend using the pg:transfer Heroku CLI plugin to transfer the data in a single step. Under the covers it's still very much the same thing happening as with using PG Backups, but it's packaged a bit nicer and has some useful defaults.
From your app's directory, copy your production database locally (assuming a local PG db), by installing the plugin and execute the pg:transfer command.
$ heroku plugins:install https://github.com/ddollar/heroku-pg-transfer
$ heroku pg:transfer
There are a couple options you can set as well. See my writeup for more details.
Hope that helps! And yes, please do use the same database during development as you do in production.
Not sure if it's what you're looking for, but have you tried copying the database to your local machine using pgbackups:capture and pg_restore? This approach doesn't use seeds.rb, but still recreates your production database on your local machine. It looks something like this.
$ heroku pgbackups:capture
$ curl -o latest.dump `heroku pgbackups:url`
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U myuser -d mydb latest.dump
(This code is copied liberally from the explanation at the Heroku dev center.)
I have a Rails app that includes pg_search and queue_classic gems. Both of these make use of PostgreSQL extensions e.g., unaccent, pg_trgm and ps-something-something (sorry, not as my dev machine and can not remember).
I'm deploying to Heroku, and having trouble running my migrations that exceute these extensions
e.g.,
def up
execute "create extension unaccent"
execute "create extension pg_trgm"
end
I get the impression that Heroku supports these from here https://devcenter.heroku.com/articles/full-text-search and the fact that Heroku uses queue_classic https://github.com/ryandotsmith/queue_classic.
I've been unable to find any information that explains how to make use of these postgres extensions on Heroku. Or even if they are available for shared databases or only dedicated.
So my questions:
How to I make these extensions available to my app on Heroku?
How do I handle migrations so that these extensions are available to dev
and test environments, but don't break migrations on staging or
production or environments if Heroku is restricting this type of
execution.
Really appreciate any ideas, especially those accompanied with pointers to relevant information/ instructions.
Thanks
If you're using the old standard shared database plans then these probably won't work so you need to look at bumping up to the new shared plans or the production plans.
I've just deployed my application to heroku and pointed my custom domain to the heroku servers. How can I check the records in my heroku database?
You can use heroku run rails console and look at your records with Model.all or any other method.
If you want to backup the database look at heroku PG backups, you then can import your database on your local machine and look at it there. Depending on your db adapter you could use sqlite browser for sqlite3 or phpmyadmin for MySQL.
I found a similar question like this and here is what #Chowlett says:
"You could run heroku pg:psql to fire up a Postgres console, then issue \d to see all tables, and \d tablename to see details for a particular table."
You can also type select * from tablename; to view the table contents.
How to view current database schema for Heroku app in Terminal?
heroku db:pull to pull your production DB locally to take a peek in it.
I'll give the method for connecting via a GUI tool
Run the following command to get the database credentials from Heroku:
heroku pg:credentials DATABASE_URL
Then you can use a GUI tool like PG Commander or PGAdmin to connect to the db
Heroku now has an add-on named PostgreSQL Studio (currently free & in beta) that would let you access your database from within the browser, without having to use CLI, much like PHP MyAdmin.
To attach this add-on to your application,
heroku addons:create pgstudio
Then go to the list of add-ons on Heroku, select PostgreSQL Studio, authorize it, select the database to connect with from the dropdown list of all databases and it will take you to the web-based interface to handle your selected database.
You may refer to this official article on Heroku:
https://devcenter.heroku.com/articles/pgstudio
The easy answer is:
heroku pg:info
You can also download a client side Postgres, like Postico, and using the information provided in that URL to enter password and database name etc, then you can create locally, just like phpMyAdmin.
I use the admin_data gem, works well in Heroku.
You can use heroku dataclips that allows to run queries online. Here you can find documentation https://devcenter.heroku.com/articles/dataclips.
Connect to the database using Sequel Pro. You can find your ClearDB url using heroku config command. The structure for connecting is as follows:
CLEARDB_DATABASE_URL => mysql://[username]:[password]#[host]/[database name]?reconnect=true