I have a Rails application with a daemon that checks a mailbox for any new emails. I am using the Fetcher plugin for this task. The daemon file looks like this:
#!/usr/bin/env ruby
require File.dirname(__FILE__) + '/../config/environment.rb'
class MailFetcherDaemon < Daemon::Base
#config = YAML.load_file("#{RAILS_ROOT}/config/mail.yml")
#config = #config['production'].to_options
#sleep_time = #config.delete(:sleep_time) || 20
def self.start
puts "Starting MailFetcherDaemon"
# Add your own receiver object below
#fetcher = Fetcher.create({:receiver => MailProcessor}.merge(#config))
...
So I have it grab the new emails, parse them and create a resource from the parsed data. But when it tries to save the resource an exception is thrown. This is because the script is automatically assigned the development environment. So it is using my development database configuration instead of the production environment (which is the config that I want).
I have tried starting the script with:
rails-root$ RAILS_ENV=production; script/mail_fetcher start
but to no avail. It seems like when I load the environment.rb file it just defaults to the development environment and loads development.rb and the development database configuration from database.yml.
Thoughts? Suggestions?
Thanks
This is working in my app, the only difference I see is no semi-colon
RAILS_ENV=production script/mail_fetcher start
So when you say
RAILS_ENV=production; script/mail_fetcher start
do you mean
#!/bin/bash
export RAILS_ENV=production
cd /path/to/rails_root
./script/mail_fetcher start
You might try adding this to your script:
ENV['RAILS_ENV'] = "production"
Alternatively, it might work to add it to the command line.
#!/bin/bash
cd /path/to/rails_root
./script/mail_fetcher start RAILS_ENV=production
Related
I created a Rails application, using Rails 4.1, from scratch and I am facing a strange problem that I am not able to solve.
Every time I try to deploy my application on Heroku I get an error 500:
Missing `secret_key_base` for 'production' environment, set this value in `config/secrets.yml`
The secret.yml file contains the following configuration:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
On Heroku I configured the "SECRET_KEY_BASE" environment variable with the result of the rake secret command. If I launch heroku config, I can see the variable with the correct name and value.
Why am I still getting this error?
I had the same problem and solved it by creating an environment variable to be loaded every time I logged in to the production server, and made a mini-guide of the steps to configure it:
I was using Rails 4.1 with Unicorn v4.8.2 and when I tried to deploy my application it didn't start properly and in the unicorn.log file I found this error message:
app error: Missing `secret_key_base` for 'production' environment, set this value in `config/secrets.yml` (RuntimeError)
After some research I found out that Rails 4.1 changed the way to manage the secret_key, so if you read the secrets.yml file located at exampleRailsProject/config/secrets.yml you'll find something like this:
# Do not keep production secrets in the repository,
# instead read values from the environment.
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
This means that Rails recommends you to use an environment variable for the secret_key_base in your production server. In order to solve this error you should follow these steps to create an environment variable for Linux (in my case Ubuntu) in your production server:
In the terminal of your production server execute:
$ RAILS_ENV=production rake secret
This returns a large string with letters and numbers. Copy that, which we will refer to that code as GENERATED_CODE.
Login to your server
If you login as the root user, find this file and edit it:
$ vi /etc/profile
Go to the bottom of the file using Shift+G (capital "G") in vi.
Write your environment variable with the GENERATED_CODE, pressing i to insert in vi. Be sure to be in a new line at the end of the file:
$ export SECRET_KEY_BASE=GENERATED_CODE
Save the changes and close the file using Esc and then ":x" and Enter for save and exit in vi.
But if you login as normal user, let's call it "example_user" for this gist, you will need to find one of these other files:
$ vi ~/.bash_profile
$ vi ~/.bash_login
$ vi ~/.profile
These files are in order of importance, which means that if you have the first file, then you wouldn't need to edit the others. If you found these two files in your directory ~/.bash_profile and ~/.profile you only will have to write in the first one ~/.bash_profile, because Linux will read only this one and the other will be ignored.
Then we go to the bottom of the file using Shift+G again and write the environment variable with our GENERATED_CODE using i again, and be sure add a new line at the end of the file:
$ export SECRET_KEY_BASE=GENERATED_CODE
Having written the code, save the changes and close the file using Esc again and ":x" and Enter to save and exit.
You can verify that our environment variable is properly set in Linux with this command:
$ printenv | grep SECRET_KEY_BASE
or with:
$ echo $SECRET_KEY_BASE
When you execute this command, if everything went ok, it will show you the GENERATED_CODE from before. Finally with all the configuration done you should be able to deploy without problems your Rails application with Unicorn or some other tool.
When you close your shell and login again to the production server you will have this environment variable set and ready to use it.
And that's it! I hope this mini-guide helps you solve this error.
Disclaimer: I'm not a Linux or Rails guru, so if you find something wrong or any error I will be glad to fix it.
I'm going to assume that you do not have your secrets.yml checked into source control (ie. it's in the .gitignore file). Even if this isn't your situation, it's what many other people viewing this question have done because they have their code exposed on Github and don't want their secret key floating around.
If it's not in source control, Heroku doesn't know about it. So Rails is looking for Rails.application.secrets.secret_key_base and it hasn't been set because Rails sets it by checking the secrets.yml file which doesn't exist. The simple workaround is to go into your config/environments/production.rb file and add the following line:
Rails.application.configure do
...
config.secret_key_base = ENV["SECRET_KEY_BASE"]
...
end
This tells your application to set the secret key using the environment variable instead of looking for it in secrets.yml. It would have saved me a lot of time to know this up front.
Add config/secrets.yml to version control and deploy again. You might need to remove a line from .gitignore so that you can commit the file.
I had this exact same issue and it just turned out that the boilerplate .gitignore Github created for my Rails application included config/secrets.yml.
This worked for me.
SSH into your production server and cd into your current directory, run bundle exec rake secret or rake secret, you will get a long string as an output, copy that string.
Now run sudo nano /etc/environment.
Paste at the bottom of the file
export SECRET_KEY_BASE=rake secret
ruby -e 'p ENV["SECRET_KEY_BASE"]'
Where rake secret is the string you just copied, paste that copied string in place of rake secret.
Restart the server and test by running echo $SECRET_KEY_BASE.
While you can use initializers like the other answers, the conventional Rails 4.1+ way is to use the config/secrets.yml. The reason for the Rails team to introduce this is beyond the scope of this answer but the TL;DR is that secret_token.rb conflates configuration and code as well as being a security risk since the token is checked into source control history and the only system that needs to know the production secret token is the production infrastructure.
You should add this file to .gitignore much like you wouldn't add config/database.yml to source control either.
Referencing Heroku's own code for setting up config/database.yml from DATABASE_URL in their Buildpack for Ruby, I ended up forking their repo and modified it to create config/secrets.yml from SECRETS_KEY_BASE environment variable.
Since this feature was introduced in Rails 4.1, I felt it was appropriate to edit ./lib/language_pack/rails41.rb and add this functionality.
The following is the snippet from the modified buildpack I created at my company:
class LanguagePack::Rails41 < LanguagePack::Rails4
# ...
def compile
instrument "rails41.compile" do
super
allow_git do
create_secrets_yml
end
end
end
# ...
# writes ERB based secrets.yml for Rails 4.1+
def create_secrets_yml
instrument 'ruby.create_secrets_yml' do
log("create_secrets_yml") do
return unless File.directory?("config")
topic("Writing config/secrets.yml to read from SECRET_KEY_BASE")
File.open("config/secrets.yml", "w") do |file|
file.puts <<-SECRETS_YML
<%
raise "No RACK_ENV or RAILS_ENV found" unless ENV["RAILS_ENV"] || ENV["RACK_ENV"]
%>
<%= ENV["RAILS_ENV"] || ENV["RACK_ENV"] %>:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
SECRETS_YML
end
end
end
end
# ...
end
You can of course extend this code to add other secrets (e.g. third party API keys, etc.) to be read off of your environment variable:
...
<%= ENV["RAILS_ENV"] || ENV["RACK_ENV"] %>:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
third_party_api_key: <%= ENV["THIRD_PARTY_API"] %>
This way, you can access this secret in a very standard way:
Rails.application.secrets.third_party_api_key
Before redeploying your app, be sure to set your environment variable first:
Then add your modified buildpack (or you're more than welcome to link to mine) to your Heroku app (see Heroku's documentation) and redeploy your app.
The buildpack will automatically create your config/secrets.yml from your environment variable as part of the dyno build process every time you git push to Heroku.
EDIT: Heroku's own documentation suggests creating config/secrets.yml to read from the environment variable but this implies you should check this file into source control. In my case, this doesn't work well since I have hardcoded secrets for development and testing environments that I'd rather not check in.
You can export the secret keys to as environment variables on the ~/.bashrc or ~/.bash_profile of your server:
export SECRET_KEY_BASE = "YOUR_SECRET_KEY"
And then, you can source your .bashrc or .bash_profile:
source ~/.bashrc
source ~/.bash_profile
Never commit your secrets.yml
For rails6, I was facing the same problem as I was missing the following files. Once I added them the issue was resolved:
1. config/master.key
2. config/credentials.yml.enc
Make sure you have these files!
What I did :
On my production server, I create a config file (confthin.yml) for Thin (I'm using it) and add the following information :
environment: production
user: www-data
group: www-data
SECRET_KEY_BASE: mysecretkeyproduction
I then launch the app with
thin start -C /whereeveristhefieonprod/configthin.yml
Work like a charm and then no need to have the secret key on version control
Hope it could help, but I'm sure the same thing could be done with Unicorn and others.
I have a patch that I've used in a Rails 4.1 app to let me continue using the legacy key generator (and hence backwards session compatibility with Rails 3), by allowing the secret_key_base to be blank.
Rails::Application.class_eval do
# the key_generator will then use ActiveSupport::LegacyKeyGenerator.new(config.secret_token)
fail "I'm sorry, Dave, there's no :validate_secret_key_config!" unless instance_method(:validate_secret_key_config!)
def validate_secret_key_config! #:nodoc:
config.secret_token = secrets.secret_token
if config.secret_token.blank?
raise "Missing `secret_token` for '#{Rails.env}' environment, set this value in `config/secrets.yml`"
end
end
end
I've since reformatted the patch are submitted it to Rails as a Pull Request
I've created config/initializers/secret_key.rb file and I wrote only following line of code:
Rails.application.config.secret_key_base = ENV["SECRET_KEY_BASE"]
But I think that solution posted by #Erik Trautman is more elegant ;)
Edit:
Oh, and finally I found this advice on Heroku: https://devcenter.heroku.com/changelog-items/426 :)
Enjoy!
this is works good https://gist.github.com/pablosalgadom/4d75f30517edc6230a67
for root user should edit
$ /etc/profile
but if you non root should put the generate code in the following
$ ~/.bash_profile
$ ~/.bash_login
$ ~/.profile
On Nginx/Passenger/Ruby (2.4)/Rails (5.1.1) nothing else worked except:
passenger_env_var in /etc/nginx/sites-available/default in the server block.
Source: https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_env_var
Demi Magus answer worked for me until Rails 5.
On Apache2/Passenger/Ruby (2.4)/Rails (5.1.6), I had to put
export SECRET_KEY_BASE=GENERATED_CODE
from Demi Magus answer in /etc/apache2/envvars, cause /etc/profile seems to be ignored.
Source: https://www.phusionpassenger.com/library/indepth/environment_variables.html#apache
In my case, the problem was that config/master.key was not in version control, and I had created the project on a different computer.
The default .gitignore that Rails creates excludes this file. Since it's impossible to deploy without having this file, it needs to be in version control, in order to be able to deploy from any team member's computer.
Solution: remove the config/master.key line from .gitignore, commit the file from the computer where the project was created, and now you can git pull on the other computer and deploy from it.
People are saying not to commit some of these files to version control, without offering an alternative solution. As long as you're not working on an open source project, I see no reason not to commit everything that's required to run the project, including credentials.
I had the same problem after I used the .gitignore file from https://github.com/github/gitignore/blob/master/Rails.gitignore
Everything worked out fine after I commented the following lines in the .gitignore file.
config/initializers/secret_token.rb
config/secrets.yml
I'm using AWS Opsworks to host my Rails App (Ruby 2.0/Rails 3.2).
For assets compilation process, I am using AssetSync to upload the compiled assets automatically on S3. I used to store the credentials as environment variables.
Do you have any idea how can I do this with Chef/Opsworks?
Thanks.
I know this is an older post, but I'm posting this in case this helps someone else.
I found the easiest way actually was to use one of Chef's deploy hooks (http://docs.opscode.com/resource_deploy.html#deploy-phases).
Add a directory called 'deploy' at the Rails project root.
In it add a file called before_restart.rb, with the code:
Chef::Log.info("Running deploy/before_restart.rb")
# map the environment_variables node to ENV
node[:deploy].each do |application, deploy|
deploy[:environment_variables].each do |key, value|
Chef::Log.info("Setting ENV[#{key}] to #{value}")
ENV[key] = value
end
end
When you trigger the OpsWorks deploy, you should be able to see the ENV vars being set in the Rails App Server instance log.
I ended up using https://github.com/joeyAghion/opsworks_custom_env.
It works pretty well.
I used a slightly different approach, using OpsWorks hook to copy JSON to application.yml. you can read more about it here: http://zaman.io/how-to-import-aws-opsworks-json-into-rails-app/
Another option outside of environment variables is you can generate a file with the variables in it at deploy time.
For example, for a Rails app, the config/secrets.yml is a reasonable place to put these. I created a deploy/before_restart.rb deploy hook with the following content:
def create_secrets(secrets, release_path)
Chef::Log.info("Creating secrets")
file_path = ::File.join(release_path, 'config/secrets.yml')
::File.open(file_path, 'w') do |f|
f.write("production:\n")
secrets.each do |k,v|
f.write(" #{k}: #{v}\n")
end
end
end
node[:deploy].each do |application, deploy|
create_secrets(deploy[:secrets], release_path)
end
And then in your OpsWorks stack Custom JSON you can add your secrets:
"deploy": {
"super_cool_app": {
"secrets": {
"some_service_id": "foo",
"some_password": "bar"
}
}
You can create a deploy folder in the root of your application, create a file before_restart.rb inside it, then in your file run the precompile task
run "cd /srv/www/myapp/current && /usr/local/bin/bundle exec rake assets:precompile"
This file will run on every deployment
Source:
https://www.youtube.com/watch?v=nHu8fCp9GR4&list=WL&index=7
This can now be done directly from the AWS Console, on the application configuration, as per documentation : http://docs.aws.amazon.com/opsworks/latest/userguide/workingapps-creating.html#workingapps-creating-environment
I already answered here: AWS OpsWorks Environment variables not working
Is important to understand that from OpsWorks dashboard we can pass all declared environment variables to Chef, then we need to handle these variables with a Chef recipe to let them available to Rails environment.
Here you can find what you are looking for: https://medium.com/#diego_durante/opsworks-rails-and-environment-variables-30c6a143253c#.696grsgg9
I can't access env variables in the Rails console, while in the application they work.
In .powenv I have export SENDGRID_PASSWORD="123"
In config/initializers/mail.rb there is:
ActionMailer::Base.smtp_settings = {
:password => ENV['SENDGRID_PASSWORD']
}
So in the console when I type UserMailer.welcome_mail.deliver there is an error 'ArgumentError: SMTP-AUTH requested but missing secret phrase'. However from the application it sends the mail successfully.
How can I make the env variables available in the console?
try
. .powenv
then
rails c
(dot is a command to run script on current environment)
Your Rails console isn't able to access the environment variable because Pow passes information from the .powenv or .powrc file into Rails ... Rails doesn't read those files on its own.
In other words, you're setting the ENV['SENDGRID_PASSWORD'] variable in the .powenv file, but that file is not being touched when you start the Rails console.
You'll need to set up a before_filter in your Application Controller that sets the ENV['SENDGRID_PASSWORD'] (or come up with another, similar, way of reading in the .powenv file from within that before_filter in your Rails app).
For posterity, you can add something like this to either your environment.rb, development.rb, or an initializer (config/initializers/pow.rb) depending on what load order you want:
# Load pow environment variables into development and test environments
if File.exist?(".powenv")
IO.foreach('.powenv') do |line|
next if !line.include?('export') || line.blank?
key, value = line.gsub('export','').split('=',2)
ENV[key.strip] = value.delete('"\'').strip
end
end
I am pulling email via pop into my Rails 3 application so i have a file called dropbox_receiver.rb in the rails lib directory.
This pulls in all email and calls DropBox.receive(email)
I can run this using rails/runner but when i run it from the daemon_controller.rb file i get the error
initialized constant Object::DropBox
How do i boot rails 3 in this script ?
Also how do i log properly from this script to my production.log?
Put this at the top of your script:
ENV['RAILS_ENV'] = ARGV.first || ENV['RAILS_ENV'] || 'development'
require File.expand_path(File.dirname(__FILE__) + "/../../config/environment"
That's for a script in app/controllers. You'll need to adjust the path to config/environment if your daemon is elsewhere.
Also, you're probably going to run into the issue of open files as described here.
I have a cron job on an Ubuntu Hardy VPS that only half works and I can't work out why. The job is a Ruby script that uses mysqldump to back up a MySQL database used by a Rails application, which is then gzipped and uploaded to a remote server using SFTP.
The gzip file is created and copied successfully but it's always zero bytes. Yet if I run the cron command directly from the command line it works perfectly.
This is the cron job:
PATH=/usr/bin
10 3 * * * ruby /home/deploy/bin/datadump.rb
This is datadump.rb:
#!/usr/bin/ruby
require 'yaml'
require 'logger'
require 'rubygems'
require 'net/ssh'
require 'net/sftp'
APP = '/home/deploy/apps/myapp/current'
LOGFILE = '/home/deploy/log/data.log'
TIMESTAMP = '%Y%m%d-%H%M'
TABLES = 'table1 table2'
log = Logger.new(LOGFILE, 5, 10 * 1024)
dump = "myapp-#{Time.now.strftime(TIMESTAMP)}.sql.gz"
ftpconfig = YAML::load(open('/home/deploy/apps/myapp/shared/config/sftp.yml'))
config = YAML::load(open(APP + '/config/database.yml'))['production']
cmd = "mysqldump -u #{config['username']} -p#{config['password']} -h #{config['host']} --add-drop-table --add-locks --extended-insert --lock-tables #{config['database']} #{TABLES} | gzip -cf9 > #{dump}"
log.info 'Getting ready to create a backup'
`#{cmd}`
# Strongspace
log.info 'Backup created, starting the transfer to Strongspace'
Net::SSH.start(ftpconfig['strongspace']['host'], ftpconfig['strongspace']['username'], ftpconfig['strongspace']['password']) do |ssh|
ssh.sftp.connect do |sftp|
sftp.open_handle("#{ftpconfig['strongspace']['dir']}/#{dump}", 'w') do |handle|
sftp.write(handle, open("#{dump}").read)
end
end
end
log.info 'Finished transferring backup to Strongspace'
log.info 'Removing local file'
cmd = "rm -f #{dump}"
log.debug "Executing: #{cmd}"
`#{cmd}`
log.info 'Local file removed'
I've checked and double-checked all the paths and they're correct. Both sftp.yml (SFTP credentials) and database.yml (MySQL credentials) are owned by the executing user (deploy) with read-only permissions for that user (chmod 400). I'm using the 1.1.x versions of net-ssh and net-sftp. I know they're not the latest, but they're what I'm familiar with at the moment.
What could be causing the cron job to fail?
When scripts run correctly interactively but not when run by cron, the problem is usually because of the environment environment settings in place ... for example the PATH as alrady mentioned by #Ted Percival, but may be other environment variables.
This is because cron will not invoke .bash_profile, .bashrc or /etc/profile before executing.
The best way to avoid this is to ensure any scripts invoked by cron do not make any assumptions about the environment when executing. Over-coming this can be as simple as including a few lines in your script to make sure the environment is setup properly. For example, in my case I have all the significant settings in /etc/profile (for RHEL), so I will include the following line in any scripts to be run under cron:
source /etc/profile
Looks like your PATH is missing a few directories, most importantly /bin (for /bin/rm). Here's what my system's /etc/crontab uses:
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
Are you sure the temporary file is being created correctly when running as a cron job? The working directory for your script will either be specified in the HOME environment variable, or the /etc/passwd entry for the user that installed the cron job. If deploy does not have write permissions for the directory in which it is executing, then you could specify an absolute path for the dump file to fix the problem.
Is cron sending emails with logs?
If not, pipe the output of cron to a log file.
Make sure to redirect STDERR to the log.