im trying to read Grails environment for war build from heroku application config, is that possible, in that case how to do it? is it set in an ENV variable ?
No, this is not possible. Buildpacks run independently of the configuration. (To convince yourself of this, run heroku config:set and watch the output: there's no slug compiler.) This is consistent with the 12-Factor App's Build, Release, Run aspect, which strictly separates building (the slug compiler and buildpack), releasing (attaching a build to its configuration), and running (spinning up dynos for a given release).
The usual solution is to have your buildpack emit files that reads the environment and acts appropriately at runtime. For example, Rails reads database configuration from config/database.yml, while Heroku specifies this in ENV['DATABASE_URL']. Thus, the buildpack generates a file using this template which causes Rails to use the database specified in the config, allowing the configuration to change without re-building the entire application.
Related
Is it possible to use the Ruby on Rails asset pipeline in Google App Engine when the app is deployed to a standard (not flexible) environment? I know precompilation happens when deploying to a flexible environment but I can't get it to work for the standard environment.
The problem is that the default configuration of app.yaml is preventing a crucial file being uploaded to GAE.
Specifically the skip_files section has some defaults that are preventing all dot-files from being uploaded, including the sprockets manifest file: /public/assets/.sprockets-manifest-5y483543959430890.json. Without this file, Rails assumes that assets haven't been precompiled.
You need to override the default skip_files configuration with something that doesn't prevent the sprockets manifest from being uploaded, but still blocks things like .git/*.
This is working for me now, but I'm sure it could be further refined:
skip_files:
- ^(.*/)?#.*#$
- ^(.*/)?.*~$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?\.git/.*$
It is possible. Check the full documentation here: Ruby in the App Engine Standard Environment.
Note that the Ruby Standard Environment it's on Beta stage, so keep in mind that it might get changed overtime.
Instead of using the skip_files section of app.yaml, you can instead create a .gcloudignore file and add a line for /public/assets. If this directory is not present, the docs say:
The Ruby runtime executes rake assets:precompile during deployment to generate static assets and sets the RAILS_SERVE_STATIC_FILES environment variable to enable static file serving in production.
I was reading from this article that you can create a config/local_env.yml with environment variables and then use config/application.rb to read it to replace/take priority before the environment variables that you export in your .bashrc. What then is the .env used for then? Does it serve the same purpose as the config/local_env.yml?
All of these methods are used to feed environment variables to your rails application. So, from an app's point of view, it serves the same purpose whether you export it from .env or .bashrc or config/local_env.yml files.
The differences in these methods are really a matter of personal choice among the team members involved in maintaining the app's development and deployment environments. However, here are few things to consider while opting for either of these choices.
.bashrc - Use this if you manually configure deployment servers and really comfortable with linux/unix command line system administration. This configuration file is specific to bash shell (Bourne Again Shell). You need to configure different file if your server uses different shell (for example: .zshrc if it uses Z Shell)
.env - Use this if you want to keep your app centric environment variables within the app itself while maintaining different variations of environment variables for different runtime environments of your rails app. For example: .env.development and .env.test files with specific values of the environment variables for your development and test environments respectively. This gives you more control of your app's environment variables and do not have to rely on the platform (terminal shell) you want to deploy your app.
config/local_env.yml - This is similar to .env approach, which is provided out of the box by rails gem that allows you to configure environment variables for your app in yml format. This method also keeps your app's configuration within the app irrespective of the shell you are using to run your app.
In addition to the previous answer, another downside of using .bashrc is that it is specific to only one user, so if you're e.g. starting your app server as a systemd service then I believe it won't see your variables.
Meanwhile, .env's Github readme says that it is not the most recommended thing to use outside of the development environment, although it is OK for that purpose.
Another two options to consider are:
Rails secrets. The benefit of this approach is that you get to commit it to git since it's encrypted, so when working in a team you will all have access to the same file. The downside is that it is coupled to the RAILS_ENV variable, so you can't use it to set that e.g. to production on your production app (but you can manually pass it every time it's invoked). Another downside is that if you have a staging environment, then apparently Heroku discourages using RAILS_ENV=staging, which you sometimes really need, so if you need it then you can either do it anyway, or you'll need to set the differing variables via a different mechanism - for example my app has a variable which points to the URL of another part of my app, this URL needs to point to its staging variant on staging, and to its production variant on production, so it needs to differ between my production and staging environment.
Using an /etc/environment file - note that systemd services don't have access to it by default so you'd have to add the line EnvironmentFile=/etc/environment. Also if you're running a shell script from a non-login shell (which happens sometimes), they it won't load them either, but the solution is to just include in your script this: set -a; source /etc/environment; set +a. You should be careful not to commit this to git. If you're working in a team and you need to manage this file then it gets a little complicated since it's not committed to git, but maybe there's a way to have it encrypted. (systemd services note: you might optionally use LoadCredential= for sensitive variables such as private keys, so that other processes won't have access to them; if you do this then you can also commit to git /etc/environment which should now contain non-sensitive variables only).
In my opinion both of those are valid and it's fine to pick whatever is easier to do in your app.
How can I make my rails app aware of the new environment variables after I have edited my /etc/environment file on my remote EC2 instance?
I frequently add new (minor) things in my secrets.yml but I don't want to restart my server for it, nor do I want to use an existing secret.
In linux every process inherits envvars from its parent process and the values are passed by value, not by reference. Also, they don't behave like closures. So, child process (your rails/ruby app process) will not get any new environment variables of its parent (the shell process where you started your rails/ruby app).
That's why it is not possible. However, you can use gems like dotenv and figaro to watch some file with your environment variables and reload them when they are changed.
You should be able to add a line to your config/spring.rb:
Spring.watch "config/secrets.yml"
This will allow Spring to detect when changes have occurred in your secrets.yml file.
However, if you're actually asking about how to make your app aware that you've changed environment variables in a file, then it's not possible. Config values may be detected in files, but environment variables are detected in the shell environment. You have to load those into your shell for them to take any effect, and this would require stopping your server, sourcing the new changes into the environment, and starting the server again.
Understanding the difference between a config value in a file (.yml, .xml, .ini, etc) versus an environment variable in a shell script is important, because how it's applied and made usable is entirely different.
To give you some context, I'm trying to use Figaro to safely add in environment variables without having to worry about security risks. The problem is is that I can't seem to get Engine Yard to play nice with production.
I went and did a touch application.yml and then vim application.yml, i, and then command+v to insert that api keys and what not. I know the ENV['VARIABLES'] work because of development and all my rspec and cucumber tests (which utilize the APIs), passed.
When I've got everything ready, I add within the .gitignore:
# Ignore application configuration
/config/application.yml
Afterwards, I deploy the site. I open it up and data isn't going to the APIs anymore. OK...
cd into config and discover application.yml isn't there anymore. Paste it back in... Redeploy the site since now it understands it has to ignore that file and I'm not seeing changes on production. Check back... and its gone again!
Stumped on what's going on.
Simply putting a file into your deployed application's filesystem will not work because you get a clean environment each time you deploy. EngineYard cannot know that you want that particular file copied to that particular location without a little bit of extra work.
Their official recommendation is to put your YAML configuration files in /data/<app>/shared/config and symlink them to /data/<app>/current/config each time you deploy using deploy hooks.
I've enjoyed using Rails on Heroku, and like that I can adjust the configuration property of a Heroku app without having to commit a change to xyz.yml and redeploy.
It would be nice to completely do away with the Yaml config files in my Rails app, and rely as much as possible on storing configuration in ENV. This goes along with the 12-factor config principle.
However, there are some trade-offs in switching from a Yaml-based configuration management to a Heroku/12-factor-based one.
While it's true that a proliferation of deployments (qa, stage, prod, dev, demo, labs) can lead to a proliferation of Yaml files, it's very easy to copy-paste to create a new configuration profile. I don't see a way to 'copy' configuration profiles from one deployment to another in Heroku.
Storing configuration data in the repo means that, in the case of Heroku, deploying and configuring and application are accomplished in a single operation. If I were to move my configuration out of Yaml files and into ENV variables, I would have to configure my application in a separate step after deployment.
Would like to hear from people who have used 12-factor style configuration in their private applications, and how they have managed lots of configuration variables across lots of deployments.
How do you quickly configure a new deployment?
Where do you keep your authoritative source of configuration variables, if not the repo? How do you distribute it among developers?
Thanks!
What I typically use is Yaml using the ENV and provide defaults. For instance, YAML can be ERB'ed happily to include your ENV vars:
foo:
var: ENV["MY_CONFIG"] || "default_value"
You just need to make sure that you load the Yaml with ERB when you read it:
YAML.load(ERB.new(File.read("#{Rails.root}/config/app_config.yml")).result)
By doing this your code works fine in dev, but also allows you to set config vars in the environment as well.
You can accomplish this relatively easy with some simple shell scripts, iterate existing variables via heroku config or heroku release:info v99, and then set heroku config:set k=v --app
But if its a problem/pain/friction perhaps you have too much inside your env var configuration.
A bit of a late answer, but I believe this is what you are looking for.
I developed a gem called settei, allowing you to use YAML files to configure the app. However the gem will serialize the YAML file as one environment variable during deploy. This way one get the best of both worlds: YAML for ease of management/creating derivative environment, and ENV for 12-factor compliance.