I have a Rails app which has been running for quite some time; however, occasionally it will decide to create files and directories under its "tmp" directory (particularly "tmp/cache") that are world-writable.
As this is an intranet, our system admins find these and complain.
I can certainly set up a cron job to remove the world-write permission, but I'd prefer to address the problem at the source.
For whatever it's worth, this is running under nginx using the Passenger 5.0.9 gem.
Thanks!
Does the umask of the app user and the nginix user affect this?
How should secret files be pushed to an EC2 Ruby on Rails application using amazon web services with their elastic beanstalk?
I add the files to a git repository, and I push to github, but I want to keep my secret files out of the git repository. I'm deploying to aws using:
git aws.push
The following files are in the .gitignore:
/config/database.yml
/config/initializers/omniauth.rb
/config/initializers/secret_token.rb
Following this link I attempted to add an S3 file to my deployment:
http://docs.amazonwebservices.com/elasticbeanstalk/latest/dg/customize-containers.html
Quoting from that link:
Example Snippet
The following example downloads a zip file from an Amazon S3 bucket and unpacks it into /etc/myapp:
sources:
/etc/myapp: http://s3.amazonaws.com/mybucket/myobject
Following those directions I uploaded a file to an S3 bucket and added the following to a private.config file in the .ebextensions directory:
sources:
/var/app/current/: https://s3.amazonaws.com/mybucket/config.tar.gz
That config.tar.gz file will extract to:
/config/database.yml
/config/initializers/omniauth.rb
/config/initializers/secret_token.rb
However, when the application is deployed the config.tar.gz file on the S3 host is never copied or extracted. I still receive errors that the database.yml couldn't be located and the EC2 log has no record of the config file, here is the error message:
Error message:
No such file or directory - /var/app/current/config/database.yml
Exception class:
Errno::ENOENT
Application root:
/var/app/current
The "right" way to do what I think that you want to do is to use IAM Roles. You can see a blog post about it here: http://aws.typepad.com/aws/aws-iam/
Basically, it allows you to launch an EC2 instance without putting any personal credential on any configuration file at all. When you launch the instance it will be assigned the given role (a set of permissions to use AWS resources), and a rotating credential will be put on the machine automatically with Amazon IAM.
In order to have the .ebextension/*.config files be able to download the files from S3, they would have to be public. Given that they contain sensitive information, this is a Bad Idea.
You can launch an Elastic Beanstalk instance with an instance role, and you can give that role permission to access the files in question. Unfortunately, the file: and sources: sections of the .ebextension/*.config files do not have direct access to use this role.
You should be able to write a simple script using the AWS::S3::S3Object class of the AWS SDK for Ruby to download the files, and use a command: instead of a sources:. If you don't specify credentials, the SDK will automatically try to use the role.
You would have to add a policy to your role which allows you to download the files you are interested in specifically. It would look like this:
{
"Statement": [
{
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mybucket/*"
}
]
}
Then you could do something like this in your .config file
files:
/usr/local/bin/downloadScript.rb: http://s3.amazonaws.com/mybucket/downloadScript.rb
commands:
01-download-config:
command: ruby /usr/local/downloadScript.rb http://s3.amazonaws.com/mybucket/config.tar.gz /tmp
02-unzip-config:
command: tar xvf /tmp/config.tar.gz
cwd: /var/app/current
It is possible (and easy) to store sensitive files in S3 and copy them to your Beanstalk instances automatically.
When you create a Beanstalk application, an S3 bucket is automatically created. This bucket is used to store app versions, logs, metadata, etc.
The default aws-elasticbeanstalk-ec2-role that is assigned to your Beanstalk environment has read access to this bucket.
So all you need to do is put your sensitive files in that bucket (either at the root of the bucket or in any directory structure you desire), and create a .ebextension config file to copy them over to your EC2 instances.
Here is an example:
# .ebextensions/sensitive_files.config
Resources:
AWSEBAutoScalingGroup:
Metadata:
AWS::CloudFormation::Authentication:
S3Auth:
type: "s3"
buckets: ["elasticbeanstalk-us-east-1-XXX"] # Replace with your bucket name
roleName:
"Fn::GetOptionSetting":
Namespace: "aws:autoscaling:launchconfiguration"
OptionName: "IamInstanceProfile"
DefaultValue: "aws-elasticbeanstalk-ec2-role" # This is the default role created for you when creating a new Beanstalk environment. Change it if you are using a custom role
files:
/etc/pki/tls/certs/server.key: # This is where the file will be copied on the EC2 instances
mode: "000400" # Apply restrictive permissions to the file
owner: root # Or nodejs, or whatever suits your needs
group: root # Or nodejs, or whatever suits your needs
authentication: "S3Auth"
source: https://s3-us-west-2.amazonaws.com/elasticbeanstalk-us-east-1-XXX/server.key # URL to the file in S3
This is documented here: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/https-storingprivatekeys.html
Using environment variables is a good approach. Reference passwords in the environment, so in a yaml file:
password: <%= ENV['DATABASE_PASSWORD'] %>
Then set them on the instance directly with eb or the console.
You may be worried about having such sensitive information readily available in the environment. If a process compromises your system, it can probably obtain the password no matter where it is. This approach is used by many PaaS providers such as Heroku.
From there security document Amazon EC2 supports TrueCrypt for File Encryption and SSL for data in transit. Check out these documents
Security Whitepaper
Features
Risk and Compliance
Best Practices
You can upload a server instance with an encrypted disk, or you can use a private repo (I think this costs for github but there are alternatives)
I think the best way is not to hack AWS (set hooks, upload files). Just use ENV variables.
Use gem 'dot-env' for development (i.e. <%= ENV['LOCAL_DB_USERNAME'] %> in 'config/database.yml') and default AWS console to set variables in Beanstalk.
I know this is an old post but I couldn't find another answer anywhere so I burned the midnight oil to come up one. I hope it saves you several hours.
I agreed with the devs that posted how much of a PITA it is to force devs to put ENV vars in their local dev database.yml. I know the dotenv gem is nice but you still have to maintain the ENV vars, which adds to the time it takes to bring up a station.
My approach is to store a database.yml file on S3 in the bucket created by EB and then use a .ebextensions config file to create a script in the server's pre hook directory so it would be executed after the unzip to the staging directory but before the asset compilation--which, of course, blows up without a database.yml.
The .config file is
# .ebextensions/sensitive_files.config
# Create a prehook command to copy database.yml from S3
files:
"/opt/elasticbeanstalk/hooks/appdeploy/pre/03_copy_database.sh" :
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
set -xe
EB_APP_STAGING_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)
echo EB_APP_STAGING_DIR is ${EB_APP_STAGING_DIR} >/tmp/copy.log
ls -l ${EB_APP_STAGING_DIR} >>/tmp/copy.log
aws s3 cp s3://elasticbeanstalk-us-east-1-XXX/database.yml ${EB_APP_STAGING_DIR}/config/database.yml >>/tmp/copy.log 2>>/tmp/copy.log
Notes
Of course the XXX in the bucket name is a sequence number created by EB. You'll have to check S3 to see the name of your bucket.
The name of the script file I create is important. These scripts are executed in alphabetical order so I was careful to name it so it sorts before the asset_compilation script.
Obviously redirecting output to /tmp/copy.log is optional
The post that helped me the most was at Customizing ElasticBeanstalk deployment hooks
posted by Kenta#AWS. Thanks Kenta!
I have a dedicated CentOS server. I manage all my users, apps, and virtual hosts manually. I'm using Apache 2.2.3 and Passenger 3.0.7 to serve my apps. I have a typical httpd.conf file in /etc/httpd/conf/ that includes all *.conf files in my /etc/httpd/conf/vhosts/ directory. Normally for each app I create a new sample-domain.com.conf file in the vhosts directory.
However, I have a particular app that needs frequent apache configuration changes, and I'd like it to be more a part of my app and its version control. So I've moved my apache configuration file into the apps config directory. I added a script to my Capistrano deploy.rb that sets the permissions on the apache conf file in the app to 755. I added an include line in my vhosts that includes the symlinked current version of the apache conf file from the app.
Which brings me to my simple question: is this safe or a bad idea?
For frequent config changes in Apache, consider using a .htaccess file instead. Changing .conf files requires bouncing/hupping the server, and if a .conf file has an error, that'll kill the whole server and take down all sites. A .htaccess error will take down just the one site/directory where the file is.
I have a Rails app that works fine on my local environment. But when I upload it to my ISP the app is not loading any of the assets in the public directory. Because the ISP uses .htaccess rewrites for Rails apps, I suspect this is the problem, but I'm pretty green on that sort of thing.
On the server, my rails app is deployed at ~/etc/rails_apps/myapp
In my home dir there's a symlink www -> etc/rails_apps/myapp/public
Isn't the idea that the symlink loads the assets from etc/rails_apps/myapp/public? Am I missing something else?
The server is Mongrel, if that matters.
The public/stylesheets and public/javascripts dirs both have 755 permissions, if that matters.
Please specify the Hosting environment that you are using whether its shared host like Bluehost or Hosmonster or Dedicated Hosting Service .
If you are hosting in a Shared hosting environment , Then your rails root directory should be in /home/user-name/rails_apps. Since the shared host provider doesn't allow you to change the permission of the /etc folder .
if your railsapps is in /home/user-name/rails_apps/yourapp then in the created symbolic link chmod 777 permission to the created symobilic folder which will be your domain name inside the public_html folder .
ln -s ~/rails/your-rails-app-name/public your-domain-name-without-extension
example : ln -s ~/rails/your-rails-app-name/public stackoverflow
$ chmod 777 /public_html/stackoverflow
Hope this helps !
I'm uploading files to my public/files folder of a Rails application on a constant basis through a web interface.
I don't want to keep these in source control since they go for almost 2 GBs, so every time I do a cap deploy it will save those files away in releases/ and replace the directory with the pristine copy stored in the repository.
I'm wondering what's the best way to keep those files in the server, in the current directory. Some of my ideas are:
Remove the directory from source control and replace it with a link to an external directory that's not managed by Capistrano.
Create a Capistrano task to copy the directory to /tmp before deploying and then copying it back to /public after it's done deploying.
Is there standard way to do this?
For the future record, this is the task I used to do it with a shared directory:
task :link_shared_directories do
run "ln -s #{shared_path}/files #{release_path}/public/files"
end
after "deploy:update_code", :link_shared_directories
You could make files a symlink to another directory on your machine, for examples the /shared directory at the same level as /current and /releases.
Check out capistrano manages the /log and /tmp directories.
Now we can simply use :linked_files in deploy.rb:
set :linked_files, %w{config/database.yml}
In this case, the file [target_dir]/shared/config/database.yml must exist on the server.