I wonder how manage on a rails-api the upload of a "big" file (arround 100m). Is it possible to stream to s3 the rack tempfile during the upload?
I'm creating a new service how just need to receive post request with parameters and files ask the original app if it's everything is ok, and process them. Lot's people over blogs tell that ruby is not the best language for that, it's not a problem to change.
So I would like to know if making a rails api who will received post, return status, and communicate with the other rails app is a good thing.
Thanks in advance
Yes it's possible. Amazon has their own AWS SDK Gem which provides the s3 functionality.
s3 = Aws::S3::Client.new
Here's how to get started using it.
This question however is a little off-topic because you show no proof of trial and error.
Related
I'd like to use Rails as a backend to a mobile app (Android and maybe iOS), and one of the requirements is file upload.
I haven't been able to find any resources on using Activestorage with direct upload in this way – does anyone have an example or tips?
The alternative – I suppose – is to reverse-engineer activestorage.js and do exactly what it does. But perhaps I'm not the first one with this need...
Rails does not care if your data is sent from a mobile-app, html-form, react-app, server-to-server...
If you can authenticate the request and send a multi-part file, then you do active-storage from your app.
Probably smart to look at gems/apps that do this already https://github.com/cbothner/react-activestorage-provider
I have a image url like
#img = 'http://www.google.com/images/logos/ps_logo.png'
Is there any process to push this image directly into aws without using paperclip/saving into localhost in ruby on rails.
Thanks in advance
There is. AWS has a JS library that allows direct upload from the browser. There is a blog post here:
http://blog.littleblimp.com/post/53942611764/direct-uploads-to-s3-with-rails-paperclip-and
And here using JQuery
http://pjambet.github.io/blog/direct-upload-to-s3/
However, a little more googling should find a wealth of advice and gems.
There will be a bit of wrangling involved as the method of authentication is non-trivial.
So here's the thing. I believe my case is pretty particular at this point, and some help from the experts it's highly advisable.
I have an API built on Rails (3.2.6) and what I want to able to do is receive a video file (mostly .mp4, .avi) and upload it to s3 through a Process Queue (Using Resque).
Now I'm kind of lost on how to do this. To my understanding, I would be receiving a byte[] (Array of bytes) through the request which is the video and send that as a param to my Resque job in order to upload it (Resque Job params can only be strings, not objects)?
Has anyone had any experience doing this sort of procedure. We're pretty much trying to mimic the http://docs.brightcove.com/en/media/ create_video method. Where a Video object can be created either by sending the direct file in the request or the link the file....
Any suggestions?
Try using the CarrierWave gem. You should allow someone to HTTP POST the file data to your API, and then save that file data on the backend and upload it to S3 using CarrierWave.
I have a rails app hosted on heroku and a mobile app made with rhodes.
I'd like to send images from the mobile app to my rails app using an HTTP POST request. Since heroku doesn't allow you to store files, I'm using amazon s3.
I can't send the file from heroku to s3 because it takes more than 30 seconds and causes a timeout. I've seen plenty of examples of uploading a file direct to s3 when the user has a form, but this obviously won't work in this case.
I tried using the suggestion here:
rails 3, heroku, aws-s3, simply trying to upload a file to S3 that is POSTed (http/multipart) to our app
but I still get a 503 request timeout.
I don't want to put my amazon s3 keys on the app.
Right now, I feel like my only option is to host my app on EC2 which I would rather not do as I like the simplicity of Heroku.
Also, it seems strange that these uploads would take so long regardless. I'm only posting images from a mobile phone camera, so they're not huge files.
I was getting the same error in a project in my job. Some people says that the only way to solve this is by uploading files directly to the S3 bucket. This is difficult in our case, because we are using Paperclip Gem for Rails and different size versions of the image.
Some other people says that "The Heroku timeout is a set in stone thing that you need to work around. Direct upload to S3 is the only option, with some sort of post-upload processing required", so I recomend to do the next:
Maybe this is not a solution but, it could be very useful, it was for me in a Rails App:
Worker Dynos, Background Jobs and Queueing
Perhaps you should move this heavy lifting into a background job which can run asynchronously from your web request.
Regards!
So I finally figured out how to do this.
After lots of back and forth with AWS reps and Cloudfiles reps and pulling my hair out, I realized it would be a lot less work to just get another rails server that could write to the filesystem.
So, I started another rails app on openshift. It's just as easy as Heroku to get started (in fact, I might consider moving my rails app there, but it's too new for my taste right now and doesn't have the community around it that Heroku does).
Then, I just had to have communications between my two rails apps.
I know it's not the best/scalable/elegant fix, but it got the job done, and that's what matters in the end!
I'm building a rails app that interacts with a 3rd party API
When a user uploads a file to rails, it should be forwarded on to the 3rd party site via an HTTP POST.
In some cases, the upload can be several hundred MBs.
At the moment, I've just been re-posting to the API using Net::HTTP and accessing the multipart form object like so
#tempfile = params[:video][:file_upload].tempfile
This is hella slow though and feels kinda dirty.
Is there a better way to do this?
Is it possible to have the user post directly to the 3rd party service or do you have to handle the API through your Rails stack? Ideally you would be able to do this and would not have to load the file into your stack and then re-post it to the API. If you can't post directly, I would recommend seeing if the API has a streaming service so that you can send parts of the file instead of the entire thing at once. Either way I think you'll start running into Timeout errors on your side and on the API side with large files, so you'll have to increase your own timeouts or create a different type of streaming file uploader.
Spin up a background job using DelayedJob. In the delayed job, you could try rails redirect_to.
https://github.com/tobi/delayed_job
http://apidock.com/rails/ActionController/Base/redirect_to