I want to pull some information from my database into a text file.
What's a good way to do that? I originally thought of running my heroku bash and rails console as I only need to do a simple loop to get the info I need. But I don't know the proper way to write to a file from heroku. It works in my local rails console
I've tried
File.open('text.txt', 'w') do |f|
User.all.each do |u|
f.puts u.email
end
end
or something like $stdout = File.new('/path/to/text.txt', 'w')
but I think these files don't end up in my local directory...
How do I do this?
Alternate simple solutions also welcome since I didn't think I was doing anything overly complex
Attempting Pakrash's answer
In heroku bash + rails c
I put in
require 'net/ftp'
ftp = Net::FTP.new('address.herokuapp.com','email#example.com', 'somepassword')
This just hangs though. No error.. I have to crl +c out of it
I previously had my full address https://adddress.herokuapp.com/ in there but it was saying getaddrinfo: Name or service not known
Should I be entering this in differently?
Heroku supports ftp in passive mode. So create the file on the server and download the file to local machine with ftp.
# FTPing from Heroku
ftp = Net::FTP.new(server, user, pass)
ftp.passive = true
ftp.getbinaryfile(remote_filename, tmp_filename)
References:
http://www.web-l.nl/posts/30-downloading-files-using-ftp-on-heroku
https://stackoverflow.com/a/5570645/429758
Perversely, I found the easiest thing to do was just to email the content to myself.
I don't know that it's advisable, but you can actually start a rails console instance locally while connecting to the remote database on Heroku. Get the database connection information from the Heroku dashboard and add it to your local database.yml file in place of your regular development stanza. Something like
development:
adapter: postgresql
encoding: unicode
database: agblah9dff3
host: ec2-44-333-444-100.compute-1.amazonaws.com
pool: 5
username: 8fk38hg72hd98d
port: 5462
password: 88dk3jblahblah8sk83df8sdfj23
timeout: 5000
But know that you're playing with production code, so...you know, it's pretty risky.
A better idea might be to just use a database IDE that creates exports for you, or write a rake task that collects the info in a file and dumps it on S3. You won't be able to write files to disk Heroku.
Related
I'm trying to connect my Rails app to an EC2 instance that contains a PG database. I've already checked with Navicat that I can connect to the database given the EC2 details. The issue is that when run locally the Rails app can't be viewed; it throws the error "database configuration does not specify adapter". A similar issue is thrown when I try a database migration. I haven't even tried to push this up to my Rails EC2 since it isn't working locally.
My database.yml file looks like this:
production:
adapter: postgresql
encoding: unicode
database: postgres
host: ec2-54-197-115-117.compute-1.amazonaws.com
pool: 10
port: 5432 (have both included and removed this line)
username: a database username for security
password: the password associated with that user
My gem files include the gem pg.
For the database name I just wrote what it had in Navicat, but perhaps there's an official name associated with it I should be using; if so, how would I find it? The host I got from the EC2 details. And the username and password were the ones I set with the postgres database via unix.
Thanks in advance for any insight!
Edit:
Fixed!
Fixed! I had forgotten to create an actual DB after setting up the PG; I changed the name in my database.yml file to reflect the new db name. Also, I needed to set on my Rails app environment directly (I thought Apache did this automatically w/Passenger) with "export RAILS_ENV=production". I thought it was still broken when I restarted my server and nothing had changed, but I just had to restart the console. Hope this helps someone else out too!
We're running a Rails app on Heroku and have it connected to a database on Amazon RDS. It works fine, the security zone is configured and the app is live.
Heroku requires you to provide a Database URL in the format of
mysql2://user:pass#rdsinstance.com/database
Since we're specifying the DB info in the add-on, what do we need to provide in the database.yml file, if anything?
Would the following suffice, or do we need even less than that? Maybe just the adapter name?
production:
adapter: mysql2
encoding: utf8
reconnect: false
pool: 5
Heroku automatically replaces the content of any database.yml file on deploy with the value of the shared database, normally stored in the SHARED_DATABASE_URL config variable.
I don't know if it's save to override that value. If you do it, you should be able to connect to the database from Rails without any additional effort.
If your app is working fine and you are just wondering what you need to write inside the default database.yml file, then you can put in whatever you want, Heroku will replace it in any case on deploy.
Let's say I have s different database connections...
Right now for the purposes of this example let's only be concerned with development. When I run 'rake db:setup' it only creates development because from the rake task perspective the other connection is another environment. I could pass in the other connection as the RAILS_ENV to create the database. However, the problem comes in with how we've defined our database connections. I'd rather not undo this as it makes management and deployment much easier. What we've done is this...
In our database.yml we've added the following code.
databases_file = File.join(Rails.root.to_s, "config", "databases", "#{Rails.env.to_s}.yml")
IO.read(databases_file) if File.exist?(databases_file)
Then in a databases sub-folder under config we've created the different environment yml files.
So in development.yml we have...
development:
... details ...
logging:
... details ...
Now if I set RAILS_ENV to 'logging' it will never load the file since the file is called development.yml. Thus it will never find the logging connection.
If I consolidate all these settings back into the database.yml file then we're back to not checking in the database.yml into the git repo and having to manually create it on the development machines for new hires and such. We'd also have to change our deploy process to start putting the file in place on the lower environments. (Our production process already supports this)
You could use environment variables to manage these. You could have a single database.yml with things defined such as:
development:
database: ENV['DEVELOPMENT_DATABASE']
And then set the environment variables in your .bashrc/.zshrc.
We have multiple databases and we use branches heavily... so I came up with this solution to create multiple databases and tie their names to the branch being worked on...
# config/database.yml
<%=
databases_file = File.join(Rails.root.to_s, "config", "databases", "#{Rails.env.to_s}.yml")
if Rails.env.development? || Rails.env.test?
branch = ENV['BRANCH'] || `git branch --no-color 2> /dev/null`.chomp.split("\n").grep(/^[*]/).first[/(\S+)$/,1].sub("-", "_")
puts "Using databases for #{branch}"
IO.read(databases_file).gsub!("<branch>", branch) if File.exist?(databases_file)
else
IO.read(databases_file) if File.exist?(databases_file)
end
%>
And then you create a database file for each environment. In the development.yml and test.yml you use the token to let this script look for what it needs to in order to do replacement. So an example file would look like this.
# config/databases/development.yml
development:
adapter: mysql2
host: 127.0.0.1
database: <branch>_dev
username: user
password: pass
versions:
adapter: mysql2
host: 127.0.0.1
database: <branch>_versions
username: user
password: pass
reporting:
adapter: mysql2
host: 127.0.0.1
database: <branch>_reporting
username: user
password: pass
ods:
adapter: mysql2
host: 127.0.0.1
database: <branch>_ods
username: user
password: pass
I then extended the rake tasks to handle multiple databases
This is where most of the work went in. Therefore, I'm going to leave it out of the answer and make you work for it! No actually, it's just a big large mess that I'm not proud of and would love time to fix it, but haven't found any. I don't want to lead anyone down the wrong path but if you have questions just get me a message and I'll be happy to help.
I have the following developement section of my development.yml file:
development:
adapter: postgresql
host: localhost
database: testtb
username: app_user
password: ENV['APP_USER_POSTGRES_PASSWORD'] <= Troublesome line
When I open a rails console via bundle exec rails console and type ENV['APP_USER_POSTGRES_PASSWORD'] I get back the DB password I've specified in my local profile. However, when I start my rails server, it can't connect to the DB, failing with
PGError FATAL: password authentication failed for user "app_user"
This was previously working when I had the DB password actually typed out in plain text, rather than trying to access it via ENV['...'], but for obvious reasons I want to keep the actual password out of this file entirely (and therefore out of the code repository) while still being able to commit other, non-secure changes to the database.yml file.
Is there something special about the syntax I'm missing, or are the environment variables for some reason not available when the database.yml file is being loaded?
Update: Some people report in the comments that this doesn't work as of Rails 4.2.x.x. I haven't tried it myself, so YMMV.
Ah, finally figured out the simple solution - it accepts embedded Ruby:
password: <%= ENV['APP_USER_POSTGRES_PASSWORD'] %>
Short and quick solution if you are running a Rails version > 4.2 Run the following command:
spring stop
..then run rails console or other rails command. My issue was that Spring server needed to be restarted in order to refresh/pickup my new ENV vars. I was starting up Rails console and it couldn't see them until I shut down Spring.
Previous versions of Rails didn't have this issue since they didn't use Spring server.
Another tool to help you troubleshoot -- Use the following command to print out your database.yml config. You can run it from the command line, but I prefer to run this within Rails console since then you can use awesome_print to make it pretty:
Within rails console:
puts ActiveRecord::Base.configurations
...or using awesome_print
ap ActiveRecord::Base.configurations
Or instead from the command line:
bin/rails runner 'puts ActiveRecord::Base.configurations'
I setup a remote connection locally and need to push it to heroku. When I pushed it to heroku I got an error saying:
RemoteDBName is not configured.
I'm just assuming (also searched and saw) heroku uses their own config.yml file.
Figured this out, for anybody connecting to a remote database on heroku that might see this:
Heroku replaces your database.yml file with their own, overwriting anything in yours.
To get around this:
Create a new file in your config folder, name it whatever.yml
Setup the connection string in this file.
Create a new file in your initializers folder, I called mine load_remote.rb. In this file write this line of code:
REMOTE_DB = YAML.load_file("#{RAILS_ROOT}/config/YOURNEWFILEHERE.yml")
Establish your connection in any of the remote models with this line of code:
establish_connection Remote_DB['Whatever you named your connection string in the yml file here']
Let me show you how database configuration is done when you work with Heroku. I think this might be a bit vague in the documentation, some people get confused over it. Lets utilize the console:
zero:~/Projects/crantastic $ heroku console
Ruby console for crantastic.heroku.com
>> puts File.read(Rails.configuration.database_configuration_file)
---
production:
encoding: unicode
adapter: postgresql
username: something_random
port: 5432
host: somewhere.at.heroku
database: something_random
password: something_random
=> nil
>>
Heroku in practice replaces your apps database.yml when you push your site to their servers. Your data will be stored in one of their fancy PostgreSQL servers no matter what you use locally - this means that you don't have to think about database.yml at all (except for development purpses, naturally). Taps makes sure that everything's db agnostic. If you want to push your latest development db to Heroku, simply run heroku db:push