When Capistrano deploys a Rails app, it creates a shared/ directory to store files that should be shared across releases and not re-exported every time. In my application I have several things in the shared/ directory that rarely change (so they belong there rather than in the application tree), but I'd still like them to be version controlled for the times when they do change.
What is the best way to approach version controlling those files but keeping them separate from the repository Capistrano is exporting from?
The /shared directory is really for un-versioned data. For example, you might store bundled gems so that you don't have to re-install all your gems every release. You can also store you logs there so they don't get overwritten every time you deploy. You can store pid files there so you don't loose the process ids of critical processes during a deploy. You might even store user generated or partially processed data there so that it is not removed during a release. If a file is meant to be versioned and has the chance of changing though, I would recommend keeping it with the rest of your files and out of the shared directory.
That said, you can always also write deploy scripts to pre-populate data in your shared directory, like database configuration files. These scripts will get run on each deploy and can be entirely customized. For example, your database config script might only write the config file if it doesn't already exist.
Another common use of the shared directory is for configuration files. Versioning and source control for configuration files is a very good idea, but should managed in a system configuration management tool. In my environment, I manage code releases with Capistrano and system configuration with Puppet. That way, there is still source control over configuration files, but they are kept distinct from the code deploy process. In turn, the code deploy process is kept independent of system configuration.
Related
I have a directory under /public folder with the use of CarrierWave I store all my PDF files under this dir. But the problem is all the time I deploy new version of my Rails app this directory gets cleaned up and the all files are missing. This directory is was set under my uploader.
I also have a directory named "private" which I created manually in order to not to serve sensitive files globally on WEB. Those files also gone after new deployment process.
How can I prevent files from deleting on deploy process?
Thanks.
I assume you are using some automation for deployment. Because if you are updating your code on server instance manually then you can preserve pre uploaded file, but using manually method to update code is not a good practice.
So in automation deployment we generally follow this kind of flow.
Whenever you deploy that create a new deploy version and set that as current version.
Simply that means it's creating a new directory and placing your rails project in it. Now the files you are storing inside the project directory are there in the previous version those are not gone if you are using any linux instance.(Only if you have setup that way to preserve last few versions to restore incase of new deploy is exploded)
Clear till now?
Not suppose you are not keeping any previous version, your files are gone forever.
So it's not a good idea to store any file under project repository.
Best practice is to use bucket system like AWS bucket or Google cloud bucket, where we store all the uploaded file. If having bucket is not in budget, you can choose a different directory on linux server instance outside of project directory. But you have to setup all those upload paths and directory system to be used as bucket.
This problem I am facing with is happening because of capistrano. Every time I run cap production deploy command on my server, the capistrano deployment tool syncs every file with git repo. And the files added by end-users are not stored under my git repos of course, so capistrano overwriting the empty public folder from my repo to the server. Adding the path to :linked_dirs variable under deploy.rb solved my problem.
Another approach could be using a directory which is somewhere else than your project root path (such as /home/files) to store all your files. By doing this you will be seperating your files from project and also prevent capistrano's overwriting problem.
Hope this information will be useful for someone or future me :) ..
When you deploy with capistrano, a new deploy(folder) is created from the repository.
Any files not in the repository are not carried over.
If you want to persist files in public, you need to create a directory in your server first and then create a symlink with capistrano inside public to that folder.
Then have your carrierwave uploads saved to that location.
During each deployment cap will symlink to that directory and your files will be there
I'm converting my RoR app to use capistrano v3. I have a number of configuration file that are generated by ERB. Most of these files, like like /etc/logrotate.d/app_name, are referenced external to my app. So I like the idea of linking them to my shared/config directory. Capistrano supports managing linked files via the linked_files array. So far so good. But, the files to be linked don't technically exist until I run ERB. And capistrano runs :deploy:check:linked_files as the first step of :starting, at which point the files don't exist and the check fails.
So my question is, what's a really good way to handle this? Do I check in empty config files in my config directory, let capistrano link them to shared, and then overwrite them via ERB at a later stage? That doesn't feel good. I can't generate them before the :starting task because on an initial deploy the source tree isn't there yet. Any suggestions?
I have a Redmine (www.redmine.org) installation pushed up onto Heroku (cedar stack). On my local instance of Redmine, the way file uploads work is that the database simply stores some data about the file including a name and the location of the file on disk, and the file itself is just stored on disk under [app-location]/files (Redmine is a ruby-on-rails application). When my Redmine project is pushed to Heroku, the files directory is nowhere to be found. From what I've read about Heroku's filesystem, this is no surprise. But what is surprising and confusing, is that file uploads still work and I didn't setup s3 which is the common recommendation for file uploads on Heroku. I checked the Heroku database to get the data about the file upload.
Here are the steps I took to locate the file.
heroku run rails c
and – to get the location of the most recent file – ran:
Attachment.last.diskfile
which returned:
=> "/app/files/2014/06/140610184025_Very-Basic-Globe-icon.png"
This path simply does not exist on the Heroku instance (using heroku run bash and listing directories or running a find). I also downloaded a dump of the Heroku database and imported it locally. The database data shows up on my local instance, but the file can't be found (no surprise).
So my questions are:
Where is the Heroku instance storing the files really?
Is there a way for me to back those files up locally without relying
on Amazon s3?
This app should remain fairly small, so I am not concerned about massive scalability, I just want to be able to get the file uploads if one day needed.
I know this question is a bit old, and you may have already found a solution, but just in case other people stumble on this question:
Heroku really is storing the files where it says it is. What's happening when you run heroku run bash is Heroku is spinning up a one-off dyno to run the command. This means that you will not be given a command prompt in the dyno that is actually running your app. This is why you are not able to find the file you're looking for.
There are currently no official addons that support backing up physical files (only databases), however you could write your own custom script to back up your data to where ever you choose (s3 or otherwise). To do so, you will likely need to use Heroku Scheduler to run your backup script in a cron-like way.
To give you some context, I'm trying to use Figaro to safely add in environment variables without having to worry about security risks. The problem is is that I can't seem to get Engine Yard to play nice with production.
I went and did a touch application.yml and then vim application.yml, i, and then command+v to insert that api keys and what not. I know the ENV['VARIABLES'] work because of development and all my rspec and cucumber tests (which utilize the APIs), passed.
When I've got everything ready, I add within the .gitignore:
# Ignore application configuration
/config/application.yml
Afterwards, I deploy the site. I open it up and data isn't going to the APIs anymore. OK...
cd into config and discover application.yml isn't there anymore. Paste it back in... Redeploy the site since now it understands it has to ignore that file and I'm not seeing changes on production. Check back... and its gone again!
Stumped on what's going on.
Simply putting a file into your deployed application's filesystem will not work because you get a clean environment each time you deploy. EngineYard cannot know that you want that particular file copied to that particular location without a little bit of extra work.
Their official recommendation is to put your YAML configuration files in /data/<app>/shared/config and symlink them to /data/<app>/current/config each time you deploy using deploy hooks.
Using Rails 3.0.7 and git, deploying with capistrano. I'm using different machines as web and app servers. I cannot deploy the application code to the web server, only the static assets--basically the public/ folder.
This would seem common but no luck searching for a best practice.
Is anything build around capistrano to handle this case? Otherwise I'm thinking that adding tasks to create the structure, but scp the public directory from the app server would be the solution.
So I assume there's a business reason you can't deploy the app to the other server?
If there isn't then just deploy the whole code
and configure your web server to just serve the public folder.
(in Apache/Passenger the configs would be exactly the same, you just wouldn't enable passenger on the static server)
That is the only simple way to do it..
otherwise you're going to cause yourself a load of headaches..
Nevertheless I'm going to make up a way to solve this.
If you do need to deploy just the static code
then I suggest you create two repositories
the app (eg. git#myserver:app.git
the static files (eg. git#myserver:static.git)
Now in your app include git#myserver:static.git as a submodule mounted at public/
Having done this, you should search standard capistrano recipes for deploying with git submodules (in particular I guess you'll want to store a local cache of the submodules, update it, then git submodule init somehow with that)
You can then have two capistrano recipes
I suggest you check out capistrano multi-stage... defining app and static as two stages
You can therefore just specify git#myserver:app.git as the repository for "app"
and git#myserver:static.git as the repository for "static"
then a simple cap app deploy:migrations && cap static deploy should do it.
but remember these will not be simultaneous
I too wish there were more established practices published. We've done ours based on the Django book which recommends making your public app directory a networked directory.
This is much better as scp only works if your public directory is static. Many apps will write things to the public directory, e.g. image generation on-the-fly. These files also need to be copied to the web server immediately.
I recommend using a NFS, Samba Share or similar, so that your public directory is actually just a networked folder, so when you write to it, it's like writing to the remote folder.
To integrate it into capistrano we do the following:
create this networked folder in shared/public
After deploy:update_code:
move content from current/public to shared/public (overriding files as needed)
remove or rename current/public then symlink current/public to shared/public
Downsides:
* doesn't remove old files (like someone earlier said)
* no real rollback option (apart from redeploying older version)
Best approach I've come up with is to in fact scp files over to the web server.