I have a scheduled job running every 12 hrs that unzips image files from an FTP server into my tmp folder. The problem is that due to Heroku's ephemeral filesystem, whenever I deploy new code, the tmp folder is cleared when the dyno's restart and the files are no longer there. I'd like to be able to deploy code at will, without this concern.
I have thought of creating a second app that runs this task and connects to the same database. As per this SO answer. This way I can deploy code updates unrelated to this task to my production server, and can chose more selectively when to deploy to the second server.
Does anyone have any experience with having two apps running on the same database? Or is there a better way to solve my issue? I've read that Heroku may change database URL's at any time, so the second app may lose its connection. How common is this? Thanks!
I would create a folder under public e.g. public/storage and save unzipped files here.
I believe that it is possible using an app on Heroku.
Check this out: https://devcenter.heroku.com/articles/s3
Related
I see everywhere that heroku deletes changes you made to the filesystem once you deploy or scale your application but in my case it seems files disappear immediately.
My rails application uploads files to public/uploads and then a delayed job tries to read those files, when it tries to read them they are not found.
If I do everything in the same thread it works, but when I try to use the delayed job or check the filesystem using heroku run bash the files are gone.
Why this happens?
heroku is a read only file system. so actually you don't even write the files but just keep them in memory while in one thread.
if you want to use some free storage system i recommend google drive. you'll need to do some searching of how to use that since not too long ago they changed they're login policy only with Oauth, no more password/username login
I've done 2 Rails apps involving image uploads with paperclip. The first one I did a while ago worked fine locally but I had issues when I deployed to Heroku; I realized I needed to use AWS to enable image uploads.
I did that project a while ago. I recently started another project and tried to incorporate similar functionality. Before I enabled AWS with paperclip when I tried to deploy, I just wanted to test what would happen if I tried to upload an image. To my surprise, it worked without AWS! I was hoping to understand why it didn't work the first time and why it does work now. How does paperclip work with heroku and rails behind the scenes?
It's likely the first application was hosted on the legacy Bamboo stack, that had a read-only file system.
In the new Cedar stack, the file system is no longer read-only, so the upload will not fail. However, you should keep using AWS or any other external storage because Heroku distributes your compiled application across several machines, and it's not guaranteed that the image will be visible from another request. They call it ephemeral filesystem.
In other words, with the Bamboo stack it was easy to understand that you could not store files on the file system, because you were experiencing an immediate crash. With Cedar the upload succeeds, but the uploaded image will be unavailable at anytime in the future.
The moral of the story is: you should keep using AWS or any other storage outside Heroku file-system.
My rails app allows users to edit a certain json file through the browser. This data file is saved in app/assets/data/thefile.json (the site is only used internally)
I tested the front-end locally and it worked fine, the data gets updated and saved. Then I pushed the code to Heroku and tested it there as well. It worked. However after about 1 day when I go back to the site, I realized that the data has reverted to its original state before it was edited.
This happened numerous times and I'm not so sure why it happened. Maybe because Heroku does not allow files in the app folder to be edited?
Any advice would be greatly appreciated!
Probably has something to do with the fact that Heroku has a read only file system.
There's also a note here on the ephemeral file system
Each dyno gets its own ephemeral filesystem, with a fresh copy of the
most recently deployed code. During the dyno’s lifetime its running
processes can use the filesystem as a temporary scratchpad, but no
files that are written are visible to processes in any other dyno and
any files written will be discarded the moment the dyno is stopped or restarted.
As the quantity of our pages increases, the number of images in the rails project has increased greatly. Does anyone have a good strategy for keeping track of these images?
We have experimented with a couple ideas but nothing seems to solve everything. Here are the high level requirements we have come up with:
Images should be available to any developer locally (given they have code access).
Updated images can be automatically sent to amazon s3 (and out to cloud front) on deploy
An easy way to "expire" an image when it changes forcing browsers to download instead of using a cached version
We actually deployed a Rake Task to archive this and keep all the files between our application and (in our case) Cloudfiles in sync.
The rake tasks checks for new files or files that have been changed and uploads them to Cloudfiles. If a developer adds an asset, they can just run the rake task and update the cloud. They also check in the file into our version control so other dev's have access to it.
I have some apps running on Heroku. I will be upgrading my OS in two weeks. The last time I upgraded though (6 months ago) I ran into some problems.
Here's what I did:
copied all my rails apps onto DVD
upgraded OS
transferred rails apps from DVD to new OS
Then, after setting up new SSH-keys I tried to push to some of my heroku apps and, whilst I can't remember the exact error message off-hand, it more or less amounted to "fatal exception the remote end hung up"
So I know that I'm doing something wrong here.
First of all, is there any need for me to be putting my heroku hosted rails apps onto DVD? Would I be better just pulling all my apps from their heroku repos once I've done the upgrade? What do others do here?
The reason I stuck them on DVD is because I tend to push a specific production branch to Heroku and sometimes omit large development files from it...
Secondly, was this problem caused by SSH keys? Should I have backed up the old keys and transferred them from my old OS to my new one too, or is Heroku perfectly happy to let you change OS's like that?
My solution in the end was to just create new heroku apps and reassign the custom domain names in heroku add-ons menu... I never actually though of pulling from the heroku repos as I tend to push a specific branch to heroku and that branch doesn't always have all the development files in it...
I realise that the error message I mentioned doesn't particularly help anyone but I didn't think to remember it 6 months ago. Any advice would be appreciated
PS - when I say upgrade, I mean full install of the new version with full format of the HDD.
One of the reasons for using a (D)VCS (Version Control System) is reproducibility (usually "reproducibility of builds", but here also the ability to reproduce a given state of a system versioned at a given time)
So it could be a good test to see if you have:
all your system properly committed
all the process to regenerate the extra content (if need be) you need from your restored data
Making a bundle of your repo (meaning dealing with only one file to backup somewhere else, DVD or otherwise) is a good start.
And regarding your "hung up" error message, if SSH is involved, that means you need also to save SSH keys (see comments) usually located under the user's homedir (as in ~/.ssh/authorized_key)