I write a script to fetch content from other site, and save it to my public/ directory. Due to my network's poor environment, I deployed it to Heroku and wish it do it instead of me doing it locally.
Just something simple like this
movie_file = "#{Rails.root}/public/movie_list#{year}.json"
File.open(movie_file, "w"){|f| f.write(JSON.pretty_generate($movie_list))}
However, when I run it in heroku(just a rake task), it seems like it can't write into the public/ directory, I get no such page error. And I find this answer: Problems with public directory when deploying Node.js app with Heroku
But the original article is unavaible in heroku, and I'm not sure if its still true.
I'm wondering:
If there is any workaround that I can save it to the server (maybe something other than the public/), and then I can download it to my computer?
Or, Instead of wrting the file into public/, maybe I can upload it to other free space?
======================
UPDATE:
Finally, I first save file to tmp, and then save it to Qiniu(An China counterpart as AWS), and you can save it to AWS
The storage on an Heroku dyno should be regarded as ephemeral, as a dyno restart will cause saved files to disappear, and the file would not be visible from other dynos.
You should use the dyno to upload your files to permanent storage, such as AWS, from which you can download it through your browser.
No permanent filesystem for Heroku?
Related
I have a rails app hosted on Heroku. I added a "Audio" folder in Assets. The "Audio" folder is not included into my git pushes. My app generates some audio files that are automatically added to the assets/audio folder. That's why I don't want Heroku to clean the assets inside this particular folder each time I push a modification.
One solution would be to host the generated audio files with AWS S3 but it's quite a lot of work to set up.
I wonder if, instead, there is a way to tell Heroku not to clean the assets in the "Audio" folder?
I searched on the internet but didn't find anything so far...
So far I know Heroku will clean you temporary assets everytime you deploy your application and the build starts.
Check this answer: Store file in directory tmp on heroku Rails
If your application is Rails 5.2 based you can implement your upload to AWS via Active Storage, check: https://evilmartians.com/chronicles/rails-5-2-active-storage-and-beyond
I deployed "Harrys Prelauncher" on Heroku and try to do the teardown (currently just testing). See here: https://github.com/harrystech/prelaunchr#teardown
After running the rake task ...
heroku run rake prelaunchr:create_winner_csvs
... a csv file is created in "/lib/assets", but I dont know how to access the file (it works locally in development).
How can I download or access the file?
Heroku uses "ephemeral" filesystem that is not guaranteed to preserve changes made at runtime. Simply put, if it's not pushed to git (I assume you're using git with heroku), it's not guaranteed to exist in all the instances of your app. It may exist in one of them, but you may have no simple way of accessing that specific filesystem. And you shouldn't, really.
It's done like that so that multiple instances of the same app can be fired up seamlessly. Of course, that requires some discipline: storage of any meaningful state outside: in the database, on external disk, anywhere. The benefit of this is horizontal scalability: should you be short on resources, you can fire up another web dyno that would (normally) behave exactly the same way. New dynos are started from bundles that are packed on git push and thus do not contain any changes you may have made in another instance.
A workaround may be running heroku run bash, so that you end up in an interactive shell linked to another instance of your bundle.
Then you can make that file (by running your rake task) and access its contents in any way you deem reasonable. Text files can be echoed into the console with cat and copy-pasted anywhere else. That's a dirty way.
A much cleaner way would be rigging the app to send the file in question via email. and it's one of the few reasonable ways if that rake task is invoked by the Rails app itself.
I ran into this problem recently while developing the Prelaunchr campaign for a client. Assuming you have a local version of your app, you can "pull" your Heroku database down to your local machine, set that as your development database in database.yml, and run the rake task from your local app, which should now have the same database as your heroku version. Here is the command to pull the db (subbing out name_for_database & heroku_app_name with your own):
heroku pg:pull HEROKU_POSTGRESQL_COPPER_URL name_for_database --app heroku_app_name
Make sure to restart your local server to see the new database info populated.
I have a Redmine (www.redmine.org) installation pushed up onto Heroku (cedar stack). On my local instance of Redmine, the way file uploads work is that the database simply stores some data about the file including a name and the location of the file on disk, and the file itself is just stored on disk under [app-location]/files (Redmine is a ruby-on-rails application). When my Redmine project is pushed to Heroku, the files directory is nowhere to be found. From what I've read about Heroku's filesystem, this is no surprise. But what is surprising and confusing, is that file uploads still work and I didn't setup s3 which is the common recommendation for file uploads on Heroku. I checked the Heroku database to get the data about the file upload.
Here are the steps I took to locate the file.
heroku run rails c
and – to get the location of the most recent file – ran:
Attachment.last.diskfile
which returned:
=> "/app/files/2014/06/140610184025_Very-Basic-Globe-icon.png"
This path simply does not exist on the Heroku instance (using heroku run bash and listing directories or running a find). I also downloaded a dump of the Heroku database and imported it locally. The database data shows up on my local instance, but the file can't be found (no surprise).
So my questions are:
Where is the Heroku instance storing the files really?
Is there a way for me to back those files up locally without relying
on Amazon s3?
This app should remain fairly small, so I am not concerned about massive scalability, I just want to be able to get the file uploads if one day needed.
I know this question is a bit old, and you may have already found a solution, but just in case other people stumble on this question:
Heroku really is storing the files where it says it is. What's happening when you run heroku run bash is Heroku is spinning up a one-off dyno to run the command. This means that you will not be given a command prompt in the dyno that is actually running your app. This is why you are not able to find the file you're looking for.
There are currently no official addons that support backing up physical files (only databases), however you could write your own custom script to back up your data to where ever you choose (s3 or otherwise). To do so, you will likely need to use Heroku Scheduler to run your backup script in a cron-like way.
I have a rails 3.1 application where users upload pictures. I am storing them in /assets/images since that is the path image_tag looks for instead of public/images.
Everything works fine in development but I deployed to Heroku and it gives me this error:
ActionView::Template::Error (image_name.jpeg isn't precompiled)
What is the right way to handle such a situation? Is there a way to compile images after uploading or should I store them somewhere else?
You must not use the filesystem on Heroku to store uploads.
You should not use image_path with uploaded images, since that assumes it is looking at the filesystem. If you use image_tag, you must pass a complete URL, not just an image name.
Are you using carrierwave for your images uploads? You can store them on amazon S3 reasonably easy with carrier wave. Carrierwave instructions Other solutions have S3 storage easily accessible as well.
Heroku will NOT let you store files in the filesystem. Run
RAILS_ENV=production bundle exec rake assets:precompile
to compile you assets locally, add to git, and push to heroku but you cannot add images later via your application on heroku. If you upload them to the /temp folder they will stay there for a short while or until you re-deploy/update your code I believe.
I'm trying to upload a file using paperclip in a production environment in Heroku and the log files show:
Errno::EACCES (Permission denied - /app/public/system/photos/1/small/081811-2012-honda-cbr1000rr-leaked-003.jpg):
Will I have to use s3 or similar to handle file uploads, or can I configure path permissions to store the files on Heroku?
Yes Heroku does not allows you to add files dynamically to its server.
Though if you need upload feature on a app on heroku you need to configure s3 or other similar services
Refer this for details
http://devcenter.heroku.com/articles/read-only-filesystem
Yes, you must use S3 or another persistent store like Rackspace cloudfiles, Dropbox etc.
Whilst you can write to the tmp on all the stacks, the Cedar stack does let you write to the file system but it's not shared across dynos or dyno stop/restarts.
See http://devcenter.heroku.com/articles/dyno-isolation#ephemeral_filesystem
Yeah, it is true that Heroku does not allow you to upload files directly onto their servers. However, you can shove your files into your database. I used a gem created by Pat Shaughnessy:
http://patshaughnessy.net/2009/2/19/database-storage-for-paperclip
It worked well.