Encountered a problem with environmental variables for the Rails project to pass gitlab CI. Currently, I’m using dotenv gem to store the credentials of my project. Also, I’ve assigned the environment variables in gitlab CI environment. For example, database.yml:
host: <%= ENV['DATABASE_HOST'] %>
.env file:
DATABASE_HOST=somehost
gitlab CI variable:
DATABASE_HOST=somehost
I put .env file in .gitignore and guessed Rails would use variables from gitlab CI. But getting an access error to database. Found a way around, to create local .env files and shared ones as the instruction of dotenv gem suggest. Then put local files in .gitignore and let shared files with credentials for gitlab CI/CD be pushed to repository.
But struggling to understand how secure this approach is? And, in general, what is the best practice for using environment variables/credentials for Rails project and gitlab CI/CD?
Ideally .env will include sensitive information in most of the cases. so its not a good practice to commit these into any version control system.
https://dev.to/somedood/please-dont-commit-env-3o9h - Detailed guide here of the risks involved with .env file
I usually try to avoid dotenv for CIs because it may represent an overhead for the setup. You can conditionally load dotenv just for some environments but exclude it from the CI/CD. This could be done using a custom ENV variable like so:
Dotenv::Railtie.load unless ENV['GITLAB_CI']
And setting it up in Gitlab envs like GITLAB_CI = true
Regarding your original question, if you really want to have a .env file, you can follow the recommendation from this post answer https://stackoverflow.com/a/55581164/992000, for documentation reasons I'll post them here too:
Create your environment variables in your gitlab repo config
Create setup_env.sh:
#!/bin/bash
echo DATABASE_HOST=$DATABASE_HOST >> .env
Modify your .gitlab-ci.yml. Upsert below to your before_script: section
- chmod +x ./setup_env.sh
- ./setup_env.sh
Related
Context
Hi! I have multiple Heroku apps deployed by the same repository on github. The only difference is that they have different build args, as defined in my Dockerfile:
ARG REACT_APP_ENV
Problem
I thought of using heroku's config vars, but they are not available at build-time.
The problem with using the heroku build-time config vars on heroku.yml is that I want it to be different depending on the heroku app (also I don't want to write them on the code or push them to github).
How can I set multiple build-time config vars depending on the app?
As of the time of this answer, there is no way to obtain config vars at build time, so the solution would be to build the image locally.
If I'm not wrong, it seems like defined with Figaro variables are not available in Docker container.
I have env files to configure Postgresq DB:
POSTGRES_USER=ENV['db_user']
POSTGRES_PASSWORD=ENV['db_password]
POSTGRES_DB=ENV['db_name']
I have application.yml file copied with all the other Rails app files to the container (I could check it with ls in the container shell).
Is it a normal behaviour ?
I also faced this issue when working on a Rails application.
The thing to note here is that for Docker, environment variables are placed in a .env file or an env_file (my_variables.env). Environment files placed in .yml or .yaml files (application.yml) are not often recognized by Docker during runtime. Here's a link to Docker's official documentation that further explains it: The “.env” file
An example of such environment variables are Database connection strings which are required during the application startup process like:
database_name
database_username
database_password
However, you can still use the Figaro gem for defining variables that are not required during application startup, like:
RAILS_MASTER_KEY
That's all.
I hope this helps
I have a continuous integration which takes rails app and package it as a docker image.
As one of the steps of this packaging process, I want to do asset precompilation.
I was doing this on Rails 5.1. I had to provide some dummy SECRET_KEY_BASE to let it go through.
SECRET_KEY_BASE=1 RAILS_ENV=production rails assets:precompile
I am moving to Rails 5.2 now and want to start using credentials. I am trying following command:
RAILS_ENV=production rails assets:precompile
If I don't RAILS_MASTER_KEY then it will show me an error:
Missing encryption key to decrypt file with. Ask your team for your
master key and write it to /home/config/master.key or put it in the
ENV['RAILS_MASTER_KEY'].
If I provide dummy (incorrect) RAILS_MASTER_KEY, it will complain that it can't decode credentials.
I don't want to give a real RAILS_MASTER_KEY to CI.
As result, the question is. How to compile asset without it or what are the workarounds?
I'm not seeing a solution either. Another way would be to continue to set config/environments/production.rb to have the line:
config.require_master_key = false
and continue to use your SECRET_KEY_BASE=1 rails assets:precompile
I haven't found a better way. At least this way seems better than maintaining a fake master key.
I created a fake credentials.yml.enc and RAILS_MASTER_KEY associated with it and I use them when I precompile asset.
We can easy solve the issue in Docker 1.13 and higher (assuming your CI is also running in a Docker container) by passing the master.key to the CI container using the docker secrets. Note that this works only in docker swarm, but also a single docker container can act as a docker swarm node. To change a single node (for example, on your local development system) to a swarm node use the init command and follow the instructions:
docker swarm init
https://docs.docker.com/engine/reference/commandline/swarm_init/
Then in your docker-compose.yml of your CI container declare the master.key as docker secret and target it to the right position in the container:
version: '3.4'
services:
my_service:
...
secrets:
- source: master_key
target: /my_root/config/master.key
uid: '1000'
gid: '1000'
mode: 0440
...
security_opt:
- no-new-privileges
...
secrets:
master_key:
file: config/master.key
https://docs.docker.com/compose/compose-file/
As you can see, we can also assign dedicated access rights to the master.key in the container and protect it from privilege escalation. For further explanations to docker swarm visit:
https://docs.docker.com/engine/swarm/secrets/#how-docker-manages-secrets
Why should this be the preferred solution to the issue? Your secret master.key is no longer stored in the CI Docker container, but secure kept in the encrypted Raft log of the Docker Swarm infrastructure, and you do not have to make any acrobatic contortions with faked keys.
BTW, I use this approach to protect my domain-specific expertise in public Docker containers: Using the target parameter, each docker secret can carry generic strings or binary content up to 500 kb in size, especially pieces of code that contain sensitive knowledge.
I need to set up an environment variable for my rails app. Both in my local machine and in the production server. I read some tutorials on the internet but NONE has given the complete instruction on how to set and use these variable in the actual production server. I use digital ocean and linux server to host my rails app.
I have spent days trying to figure this out, but still haven't found a clear and complete instruction from setting the variables on my local machine -> push it to git repo -> set and use the variables in production server. So, hope somebody can help me here, thanks!
UPDATE:
This is how I currently setup the environment variables in my rails app by using figoro gem:
You can set system-wide environment variables in the /etc/rc.local file (which is executed when the system boots). If your Rails app is the sole user of the Linux system, that is a good place to store credentials such as API keys because there is no risk of including this file in a public Git repository, as it is outside the application directory. The secrets will only be vulnerable if the attacker gains shell access to your Linux server.
Set the environment variables within /etc/rc.local (do not include the <> characters):
export SOME_LOGIN=<username>
export SOME_PASS=<password>
To see the value of an environment variable, use one of the following commands in the Linux shell:
printenv MY_VAR
echo $MY_VAR
To access those environment variables within Rails, use the following syntax:
Inside .rb files or at the rails console
ENV['MY_VAR']
Inside .yml files:
<%= ENV['MY_VAR'] %>
For anyone still having this issue, figaro now has an easy method in setting the production variables in heroku. Just run:
$ figaro heroku:set -e production
ryzalyusoff.
For Unix
You can use LINUX ENV in rails application.
# .env
GITHUB_SECRET_KEY=SECRET
TWITTER_ACCESS_KEY=XXXXXXXXXXXX
# in rails code
puts ENV["TWITTER_ACCESS_KEY"] # => SECRET
Create .env files for local machine and your production server. Export environment variables like this(on server with ssh):
export GITHUB_SECRET_KEY="XXXXXXXXXXXXXXXXXX"
Anyway, storing keys in config - bad idea. Just add .env.example, others keys configs add to .gitignore. Goodluck.
Example with Rails
For Windows
Syntax
SET variable
SET variable=string
SET /A "variable=expression"
SET "variable="
SET /P variable=[promptString]
SET "
Key
variable : A new or existing environment variable name e.g. _num
string : A text string to assign to the variable.
expression : Arithmetic expression
Windows CMD
I believe we should not push a secret file on git.
To ignore such file use gitignore file and push other code on the git.
On the server side just copy the secret file and create a symlink for that file.
You can find demo here http://www.elabs.se/blog/57-handle-secret-credentials-in-ruby-on-rails
You can set your environment variables in production in the same way, you do it for local system. However, there are couple of gems, which make it easier to track and push to production. Have a look at figaro. This will help you in setting up and deployment of env vars.
You can do this with figaro gem
or in rails 4 there is a file named secret.yml in config folder where you can define your environment variables this file is by default in .gitignore file.For production you need to manually copy that file to server for security reason so that your sensitive information is not available to any one
First create your variable like:
MY_ENV_VAR="this is my var"
And then make it global:
export MY_ENV_VAR
You can check if the process succeeded with:
printenv
Or:
echo MY_ENV_VAR
I have an app being developed on github. I want to opensource it. Currently we use a capistrano script to deploy to our staging & production servers.
I am trying to figure out how we can put our config files in a separate repo, and still use capistrano to execute one touch deploys. The goal is that we can open up our repo for anyone to use.
You have some options.
Use environment variables: You can set environment variables at both your local and production machines. In your app you'll read these vars by doing: <%= ENV['my_var'] %>. Doing this you can commit your app to a public repo without worrying about expose sensitive information like passwords and keys. For example, set a db_password environment var to store your database password and in your database.yml you could read it doing: password: <%= ENV['db_password'] %>
You can use gems link dotenv (https://github.com/bkeepers/dotenv) or figaro (https://github.com/laserlemon/figaro): By using these gems you won't need to set environment variables manually on your machines, you will define them in a .env file instead. You'll be able to read them the same way using <%= ENV['my_var'] %>. You will have to ignore your .env files at your .gitignore and tell capistrano to create the environment variables at your production server reading from your .env file.
Another alternative would be make different config files for development and production and ignore them in your .gitignore. You can store your config production files in a different repo and have it updated on your machine at the time of your deploy. You'll just need to copy your config files from your local machine to the production server (https://coderwall.com/p/wgs6gw/copy-local-files-to-remote-server-using-capistrano-3) after updating the app repo on your production server.
The last alternative is the one I use most (in my case I use ansible in place of capistrano).
If you wanna see an example I have an application and a deployment task currently running in production that you can checkout:
Application: https://github.com/josuelima/bolao
Provisoning and Deployment: https://github.com/josuelima/ansible-sandbox
Hope I made myself clear and you got the idea.