AWS OpsWorks Environment variables not working - ruby-on-rails

I am running Ubuntu 14.04 LTS 64 bit Rails Application and I am unable to access my App environment variables.
In OpsWorks App panel, I set my environment variables, say:
MYKEY: 1234
Then I save and deploy my app again to make these visible.
In my Rails app, or the rails console I get nil:
$ bundle exec rails c production
>ENV["MYKEY"]
=> nil
I have tried restarting the server. I'm not sure what I am missing, I have been using environment variables in other services.
How can I trace where these should be set?

OpsWorks stores environmental variables in different places depending on what kind of app you're deploying. On Rails / Passenger they should be saved in the Apache config file #{your_app_name}.conf. (Source)
This means they aren't available in your normal shell environment.
I know the Node.js recipes stored everything in an /srv/www/#{app_name}/shared/app.env file... which is then sourced to pull in the environment to run the Node server. This implementation detail also meant you could write shell scripts that sourced that app.env file, then called some Node script or whatever.
Of course, Rails isn't Node. I have no idea if the environmental variables are also stored somewhere else or not: a quick look at the Rails recipes in the OpsWorks cookbooks didn't find anything obvious, but maybe I missed something.
Depending on the amount of modifications you have going on in your OpsWorks cookbook, you could create a deploy recipe that does something like this:
application_environment_file do
user deploy[:user]
group deploy[:group]
path ::File.join(deploy[:deploy_to], "shared")
environment_variables deploy[:environment_variables]
end
(maybe adjusting the path)
Then to run your console, when you're SSHed into the server, do something like
sudo source /srv/www/my_app_name/shared/app.env; bundle exec rails console -e production or whatever.

AWS OpsWorks console lets you declare environment variables but to let them be available for our Rails app we need to use a Chef cookbook recipe plus some precautions.
In a nutshell we use the config/secrets.yml file combined with config/application.yml file, Figaro gem and a Chef cookbook recipe.
The chef cookbook recipe read the variables defined in OpsWorks console and let them available to Rails app writing the config/application.yml file.
I have published a detailed guide to explain how exactly do it. Link here.
These are the core points that I covered:
Use config/secrets.yml file (added from Rails 4.1)
Use Figaro gem to load variables in the environment
Declare environment variables inside AWS OpsWorks Console
Use a custom Chef recipe to create a config/application.yml file that Figaro will use to let variables available

I (with some help from Bruno at the AWS PopUp Loft in NYC) added some custom Chef code inside the after_restart.rb deploy hook, simply add the folder "deploy" to your apps root directory and inside add "after_restart.eb." In it ....
Chef::Log.info("Running deploy/after_restart.rb")
contents = []
node[:deploy].each do |application, deploy|
deploy[:environment_variables].each do |key, value|
contents << "export #{key}=\"'#{value}'\""
end
end
Chef::Log.info("Adding the environment variables to /etc/profile.d/startup_env_config.sh")
bash "create_startup_env_config.sh" do
user "root"
cwd "/etc/profile.d"
code <<-EOH
echo \''#{contents.join(" ")}\'' > startup_env_config.sh
source startup_env_config.sh
cd #{release_path}
EOH
end
And that's it. If you update the environment variables inside the OpsWorks panel remember to restart your instances.

Related

how to add environment variable in linux for rails app?

i need to set an environment variable for the rails app to use
SECRET_KEY_BASE=9941144eb255ff0ffecasdlkjqweqwelkjasdlkjasd
the config settings for production is as shown below
# Do not keep production secrets in the repository,
# instead read values from the environment.
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
how can i set the environment variable using the linux command
export VARNAME="my value"
I tried to set the variable but looks like it needs to be for the right user. Sorry i am not an expert in linux.
I appreciate any help! Thanks!
export VARNAME="my value"
Well the above works for your current terminal session. After this command, all the subsequent commands can access this variable. Try running this:
echo $VARNAME
It will print the value my value in the console. If you want this behaviour to be persisted, you need to place the export command in your OS' config file (~/.bashrc in case of Ubuntu).
After editing this file, either restart your terminal, or run this:
source ~/.bashrc
This will reload the file in your current terminal session. Alternatively, you can try running your Rails server (or a rake command) as follows:
VARNAME="my value" rails s
For your local development I suggest you to use dotenv (https://github.com/bkeepers/dotenv) or figaro (https://github.com/laserlemon/figaro) and follow the README you find in the gem itself. This gives you much more flexibility than using directly environment variables because you set them only for this specific project and each project can have different of them.
You need to have either a .env file or a application.yml file where you will define your environment variables.
Remember to not commit or push this file to your repository because it contains sensible information!
When you will deploy to production you can use real environment variables or use admin panel control (on Heroku for example)

Capistrano and API keys in ENV variables?

I'm deploying my rails app with Capistrano. I want to save some API keys on the server as an environment variable. Those API keys should be accessible to my rails app that is deployed with Capistrano. Those API keys should also be accessible to a separate ruby file that is run as a daemon.
setting the API keys in environment variables seems like the ideal solution, however, I can't access them in my rails app with ENV["SOME_KEY"].
According to this post, because capistrano runs as non interactive and non login, ~/.bashrc and ~/.bash_profile are not loaded. The flowchart suggests that I should use $BASH_ENV.
Can I just add my api keys in $BASH_ENV and access them in my rails app and in the ruby file that is a daemon with ENV["SOME_KEY"]?
I'm also thinking of just adding the api keys to a file somewhere on the server and symlinking it to the ruby file dir and rails dir and just open and reading it. Would this be possible?
There are a few ways that work well with Capistrano, in my experience.
rbenv-vars
If you use Ruby via Rbenv on your server, then you are in luck. There is a Rbenv plugin called rbenv-vars that automatically injects environment variables into any Ruby process, which would include your Rails app. Just add your variables to ~/.rbenv/vars on the server using KEY=value syntax. That's it.
dotenv
The dotenv gem is a similar solution, but it works as a gem you add to your Rails app and doesn't require Rbenv or any other supporting tools. Add dotenv-rails to your Gemfile and deploy. Dotenv will automatically look for a .env.production file in the root of your Rails app. For Capistrano, create a .env.production file on the server inside Capistrano's shared directory, and then add .env.production to :linked_files. Now every deploy will link to it. Declare your variables using KEY=value syntax.
.bashrc
Declare your variables with export KEY=value syntax at very top of the ~/.bashrc file on the server. On Ubuntu, this file is evaluated even during an non-interactive SSH session. Just make sure you place the declarations on the top, before this case statement:
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
CentOS may be a different story, so YMMV.
I made a Capistrano plugin capistrano-env_config some time ago for managing and syncing environment variables across a Capistrano cluster which works by modifying the /etc/environment file to make environment variables available throughout the system. It's easy to use and is similar to how you can set environment variables with the Heroku toolbelt. Here are some examples:
cap env:list
cap env:get[VARIABLE_NAME, VARIABLE_NAME, ...]
cap env:unset[VARIABLE_NAME, VARIABLE_NAME, ...]
cap env:set[VARIABLE_NAME=VALUE, VARIABLE_NAME=VALUE, ...]
cap env:sync

Environmental variables in /etc/environment are not accessible by rails

I have been trying to deploy a rails project and it requires some environmental variables so I did some research and it turns out the best place to add them is inside /etc/environment which i did so and then sourced the file.
I now can get/access the environmental variables by running echo $variablename; however the rails application can still not see these environmental variables?
/etc/environment is not automatically sourced by all shells in all login contexts. To get the settings into the rails environment, there are a couple options. The easiest is probably to modify (or create) the .bashrc (assuming the rails user's shell is bash, which is the Ubuntu default shell) in the home directory of the user rails runs as and add the following line:
source /etc/environment
If you want the variables to be available to all users on the system, you can create a file called /etc/profile.d/environment.sh (or something similar) and add the above line to that.
For either solution, log in again as the rails user and verify the environment.
You might want to use this gem for env variables:
https://github.com/bkeepers/dotenv

How to set up and use Rails environment variables in production server?

I need to set up an environment variable for my rails app. Both in my local machine and in the production server. I read some tutorials on the internet but NONE has given the complete instruction on how to set and use these variable in the actual production server. I use digital ocean and linux server to host my rails app.
I have spent days trying to figure this out, but still haven't found a clear and complete instruction from setting the variables on my local machine -> push it to git repo -> set and use the variables in production server. So, hope somebody can help me here, thanks!
UPDATE:
This is how I currently setup the environment variables in my rails app by using figoro gem:
You can set system-wide environment variables in the /etc/rc.local file (which is executed when the system boots). If your Rails app is the sole user of the Linux system, that is a good place to store credentials such as API keys because there is no risk of including this file in a public Git repository, as it is outside the application directory. The secrets will only be vulnerable if the attacker gains shell access to your Linux server.
Set the environment variables within /etc/rc.local (do not include the <> characters):
export SOME_LOGIN=<username>
export SOME_PASS=<password>
To see the value of an environment variable, use one of the following commands in the Linux shell:
printenv MY_VAR
echo $MY_VAR
To access those environment variables within Rails, use the following syntax:
Inside .rb files or at the rails console
ENV['MY_VAR']
Inside .yml files:
<%= ENV['MY_VAR'] %>
For anyone still having this issue, figaro now has an easy method in setting the production variables in heroku. Just run:
$ figaro heroku:set -e production
ryzalyusoff.
For Unix
You can use LINUX ENV in rails application.
# .env
GITHUB_SECRET_KEY=SECRET
TWITTER_ACCESS_KEY=XXXXXXXXXXXX
# in rails code
puts ENV["TWITTER_ACCESS_KEY"] # => SECRET
Create .env files for local machine and your production server. Export environment variables like this(on server with ssh):
export GITHUB_SECRET_KEY="XXXXXXXXXXXXXXXXXX"
Anyway, storing keys in config - bad idea. Just add .env.example, others keys configs add to .gitignore. Goodluck.
Example with Rails
For Windows
Syntax
SET variable
SET variable=string
SET /A "variable=expression"
SET "variable="
SET /P variable=[promptString]
SET "
Key
variable : A new or existing environment variable name e.g. _num
string : A text string to assign to the variable.
expression : Arithmetic expression
Windows CMD
I believe we should not push a secret file on git.
To ignore such file use gitignore file and push other code on the git.
On the server side just copy the secret file and create a symlink for that file.
You can find demo here http://www.elabs.se/blog/57-handle-secret-credentials-in-ruby-on-rails
You can set your environment variables in production in the same way, you do it for local system. However, there are couple of gems, which make it easier to track and push to production. Have a look at figaro. This will help you in setting up and deployment of env vars.
You can do this with figaro gem
or in rails 4 there is a file named secret.yml in config folder where you can define your environment variables this file is by default in .gitignore file.For production you need to manually copy that file to server for security reason so that your sensitive information is not available to any one
First create your variable like:
MY_ENV_VAR="this is my var"
And then make it global:
export MY_ENV_VAR
You can check if the process succeeded with:
printenv
Or:
echo MY_ENV_VAR

Set environment variables with AWS Opsworks

I'm using AWS Opsworks to host my Rails App (Ruby 2.0/Rails 3.2).
For assets compilation process, I am using AssetSync to upload the compiled assets automatically on S3. I used to store the credentials as environment variables.
Do you have any idea how can I do this with Chef/Opsworks?
Thanks.
I know this is an older post, but I'm posting this in case this helps someone else.
I found the easiest way actually was to use one of Chef's deploy hooks (http://docs.opscode.com/resource_deploy.html#deploy-phases).
Add a directory called 'deploy' at the Rails project root.
In it add a file called before_restart.rb, with the code:
Chef::Log.info("Running deploy/before_restart.rb")
# map the environment_variables node to ENV
node[:deploy].each do |application, deploy|
deploy[:environment_variables].each do |key, value|
Chef::Log.info("Setting ENV[#{key}] to #{value}")
ENV[key] = value
end
end
When you trigger the OpsWorks deploy, you should be able to see the ENV vars being set in the Rails App Server instance log.
I ended up using https://github.com/joeyAghion/opsworks_custom_env.
It works pretty well.
I used a slightly different approach, using OpsWorks hook to copy JSON to application.yml. you can read more about it here: http://zaman.io/how-to-import-aws-opsworks-json-into-rails-app/
Another option outside of environment variables is you can generate a file with the variables in it at deploy time.
For example, for a Rails app, the config/secrets.yml is a reasonable place to put these. I created a deploy/before_restart.rb deploy hook with the following content:
def create_secrets(secrets, release_path)
Chef::Log.info("Creating secrets")
file_path = ::File.join(release_path, 'config/secrets.yml')
::File.open(file_path, 'w') do |f|
f.write("production:\n")
secrets.each do |k,v|
f.write(" #{k}: #{v}\n")
end
end
end
node[:deploy].each do |application, deploy|
create_secrets(deploy[:secrets], release_path)
end
And then in your OpsWorks stack Custom JSON you can add your secrets:
"deploy": {
"super_cool_app": {
"secrets": {
"some_service_id": "foo",
"some_password": "bar"
}
}
You can create a deploy folder in the root of your application, create a file before_restart.rb inside it, then in your file run the precompile task
run "cd /srv/www/myapp/current && /usr/local/bin/bundle exec rake assets:precompile"
This file will run on every deployment
Source:
https://www.youtube.com/watch?v=nHu8fCp9GR4&list=WL&index=7
This can now be done directly from the AWS Console, on the application configuration, as per documentation : http://docs.aws.amazon.com/opsworks/latest/userguide/workingapps-creating.html#workingapps-creating-environment
I already answered here: AWS OpsWorks Environment variables not working
Is important to understand that from OpsWorks dashboard we can pass all declared environment variables to Chef, then we need to handle these variables with a Chef recipe to let them available to Rails environment.
Here you can find what you are looking for: https://medium.com/#diego_durante/opsworks-rails-and-environment-variables-30c6a143253c#.696grsgg9

Resources