I'm trying to upload image to Amazon S3 with this Ruby code:
require 'net/http/post/multipart'
url = URI.parse('http://public.domain.com/')
File.open("/tmp/uup_1114.jpg") do |jpg|
req = Net::HTTP::Post::Multipart.new url.path,
'key' => s3_key,
'acl' => s3_acl,
'content_type' => s3_content_type,
'AWSAccessKeyId' => s3_AWSAccessKeyId,
'policy' => s3_policy,
'signature' => s3_signature,
"file" => UploadIO.new(jpg, "image/png", "image.jpg")
res = Net::HTTP.start(url.host, url.port) do |http|
http.request(req)
end
end
And I'm getting error from Amazon:
InvalidArgument: Bucket POST must contain a field named 'key'. If it is specified, please check the order of the fields.
Looks like 'file' field goes first in query and that causes an error above. I can't figure out how to post file field at the end of query.
I have successfully used the AWS-SDK in ruby to create post forms. But in my case I was getting users to upload from a browser into an AWS account. Still this may help:
the aws-sdk has a call on a bucket called presigned_post(options) that creates a pre signed post that works fine.
See also
https://forums.aws.amazon.com/thread.jspa?messageID=296867񈞣
It's better to use AWS::S3 (http://amazon.rubyforge.org/)
and S3Object
If you're experiencing some problems try to check if your local computer time is valid (it's really important) and try setting
AWS::S3.const_set('DEFAULT_HOST', "s3-eu-west-1.amazonaws.com")
if you're working with bucket(s) located in Europe.
Related
So here is the thing: currently our files, when user downloads them, have names like 897123uiojdkashdu182uiej.pdf. I need to change that to file-name.pdf.
And logically I go and change paperclip.rb config from this:
Paperclip::Attachment.default_options.update({
path: '/:hash.:extension',
hash_secret: Rails.application.secrets.secret_key_base
})
to this:
Paperclip::Attachment.default_options.update({
path: "/attachment/#{SecureRandom.urlsafe_base64(64)}/:filename",
hash_secret: Rails.application.secrets.secret_key_base
})
which works just fine, filenames are great. However, old files are now unaccessable due to the change in the path. So I came up with the following decision
First I made a rake task which will store the old paths in the database:
namespace :paperclip do
desc "Set old urls for attachments"
task :update_old_urls => :environment do
Asset.find_each do |asset|
if asset.attachment
attachment_url = asset.attachment.try!(:url)
file_url = "https:#{attachment_url}"
puts "Set old url asset attachment #{asset.id} - #{file_url}"
asset.update(old_url: file_url)
else
puts "No attachment found in asset #{asset.id}"
end
end
end
end
Now the asset.old_url stores the current url of the file. Then I go and change the config, making the file unaccessable.
Now it's time for the new rake task:
require 'uri'
require 'open-uri'
namespace :paperclip do
desc "Recreate attachments and save them to new destination"
task :move_attachments => :environment do
Asset.find_each do |asset|
unless asset.old_url.blank?
url = asset.old_url
filename = File.basename(asset.attachment.path)
file = File.new("#{Rails.root}/tmp/#{filename}", "wb")
file.write(open(url).read)
if File.exists? file
puts "Re-saving asset attachment #{asset.id} - #{filename}"
asset.attachment = file
asset.save
# if there are multiple styles, you want to recreate them :
asset.attachment.reprocess!
file.close
else
puts "Missing file attachment #{asset.id} - #{filename}"
end
File.delete(file)
end
end
end
end
But my plan didn't work at all, I didn't get access to the files, and the asset.url still isn't equal to asset.old_url.
Would appreciate help very much!
With S3, you can set the "filename upon saving" as a header. Specifically, the user will get to an url https://foo.bar.com/mangled/path/some/weird/hash/whatever?options and when the browser will offer to save, you can control the filename (not the url).
The trick to that relies on the browser reading the Content-Disposition header from the response, if it reads Content-Disposition: attachment; filename="filename.jpg" it will save (or ask the user to save as) filename.jpg, independently on the original URL.
You can force S3 to add this header by adding one more parameter to the URL or by setting a metadata on the file.
The former can be done by passing it to the url method:
has_attached_file :attachment,
s3_url_options: ->(instance) {
{response_content_disposition: "attachment; filename=\"#{instance.filename}\""}
}
Check https://github.com/thoughtbot/paperclip/blob/v6.1.0/lib/paperclip/storage/s3.rb#L221-L225 for the relevant source code.
The latter can be done in bulk via paperclip (and you should also configure it to do it on new uploads). It will also take a long time!!
Asset.find_each do |asset|
next unless asset.attachment
s3_object = asset.attachment.s3_object
s3_object.copy_to(
s3_object,
metadata_directive: 'REPLACE',
content_disposition: "attachment; filename=\"#{asset.filename}\")"
)
end
# for new uploads
has_attached_file :attachment,
s3_headers: ->(att) {
{content_disposition: "attachment; filename=\"#{att.model.filename}\""}
}
I'm trying to make a request to an API sending an image and some other data, and getting the response. That's my code:
file = "assets/images/test.jpg"
conn = Faraday.new(:url => "api_url" ) do |faraday|
faraday.request :multipart
end
payload = { :profile_pic => Faraday::UploadIO.new(file, 'image/jpeg') }
conn.post "/test", payload
My first problem is that I'm always getting the following error:
Errno::ENOENT (No such file or directory - assets/images/test.png)
I've tried all the paths I could imagine. Where should be saved the image in directories to be found by Faraday?
The second question is about the response, how can I get the response and handle it?
The third one is that, I haven't understand what's the utility of the first parameter of the last call:
conn.post "/hello", payload
I've written "/hello" but don't have any idea about what's the real usage.
And the last one. Could I send a raw image saved in a variable instead of sending a path to Faraday?
EDIT
Now it's working, this is the solution:
Be aware that url must be only until .com, the rest of the path must go on conn.post like this example /v1/search.
c.adapter :net_http was needed too.
Message response is correctly handled in json variable.
Solution:
url = 'http://url.com'
file = Rails.root.to_s + "/app/assets/images/test.jpg"
conn = Faraday.new(:url => url ) do |c|
c.request :multipart
c.adapter :net_http
end
payload = { :image => Faraday::UploadIO.new(file, 'image/jpeg'), :token => token}
response = conn.post '/v1/search', payload
json = JSON.parse response.body
You should try this for your first question :
file = Rails.root.to_s + "/app/assets/images/test.jpg"
For your third question, the first parameters allows you to construct the right URL from the base "api_url". Please see the example from the Readme.
## POST ##
conn.post '/nigiri', { :name => 'Maguro' } # POST "name=maguro" to http://sushi.com/nigiri
Does anyone know how to post a JSON to a Rails server with a file attached? Would the content be base64 encoded? Multipart? I honestly have no idea and havent really found anything here to help. Idea is to have a client posting a JSON to a rails API with the file attached, as well as having the Rails (with paperclip would be perfect) getting the JSON and saving the file properly. Thanks in advance
Here is how I solved this problem. First I created a rake task to upload the file within the json content:
desc "Tests JSON uploads with attached files on multipart formats"
task :picture => :environment do
file = File.open(Rails.root.join('lib', 'assets', 'photo.jpg'))
data = {title: "Something", description: "Else", file_content: Base64.encode64(file.read)}.to_json
req = Net::HTTP::Post.new("/users.json", {"Content-Type" => "application/json", 'Accept' => '*/*'})
req.body = data
response = Net::HTTP.new("localhost", "3000").start {|http| http.request(req) }
puts response.body
end
And then got this on the controller/model of my rails app, like this:
params[:user] = JSON.parse(request.body.read)
...
class User < ActiveRecord::Base
...
has_attached_file :picture, formats: {medium: "300x300#", thumb: "100#100"}
def file_content=(c)
filename = "#{Time.now.to_f.to_s.gsub('.', '_')}.jpg"
File.open("/tmp/#{filename}", 'wb') {|f| f.write(Base64.decode64(c).strip) }
self.picture = File.open("/tmp/#{filename}", 'r')
end
end
JSON is a data serializing format. There is no standard pattern for uploading data or files as data in the serialized object. JSON has expectations that the data fields will be basic objects so you probably want to use Base64 encoding of the file to turn it into a string.
You are free to define your structure however you want, and processing it is your responsibility.
Well my problem is that I'm using send_data on my Rails 3 application to send to the user a file from AWS S3 service with something like
Base.establish_connection!( :access_key_id => 'my_key', :secret_access_key => 'my_super_secret_key')
s3File = S3Object.find dir+filename, "my_unique_bucket"
send_data(open(s3File.url).read,:filename=>filename, :disposition => 'attachment')
but seems like the browser is buffering the file and before buffering it sends the file to download taking no time on the download but at the buffering time it's taking as long as the file size .... but what i need is the user to view the download process as normal, they won't know what happening with the loader only on the browsers tab:
They'd rather see a download process i guess to figure out there's something happening there
is there any way i can do this with send_data?
It's not the browser that's buffering/delaying, it's your Ruby server code.
You're downloading the entire file from S3 before sending it back to the user as an attachment.
It may be better to serve this content to your user directly from S3 using a redirect. Here's a link to building temporary access URLs that will allow a download with a given token for a short period of time:
http://docs.amazonwebservices.com/AmazonS3/latest/dev/S3_QSAuth.html
Base.establish_connection!( :access_key_id => 'my_key', :secret_access_key => 'my_super_secret_key')
s3File = S3Object.find dir+filename, "my_unique_bucket"
redirect_to s3File.url(:expires_in => 30)
Set Your Content Disposition
You'll need to set the content-disposition of the S3 url for it download instead of opening up in the browser. Here is my basic implementation:
Think of attachment as your s3file.
In your attachment.rb
def download_url
s3 = AWS::S3.new.buckets[ 'bucket_name' ]
s3.url_for( :read,
expires_in: 60.minutes,
use_ssl: true,
response_content_disposition: "attachment; filename='#{file_name}'" ).to_s
end
In your views
<%= link_to 'Download Avicii by Avicii', attachment.download_url %>
Thanks to guilleva for his guidance.
I got the following back from delayed_job:
[Worker(XXXXXX pid:3720)] Class#XXXXXXX failed with URI::InvalidURIError: bad URI(is not URI?): https://s3.amazonaws.com/cline-local-dev/2/attachments/542/original/mac-os-x[1].jpeg?AWSAccessKeyId=xxxxxxxx&Expires=1295403309&Signature=xxxxxxx%3D - 3 failed attempts
The way this URI comes from in my app is.
In my user_mailer I do:
#comment.attachments.each do |a|
attachments[a.attachment_file_name] = open(a.authenticated_url()) {|f| f.read }
end
Then in my attachments model:
def authenticated_url(style = nil, expires_in = 90.minutes)
AWS::S3::S3Object.url_for(attachment.path(style || attachment.default_style), attachment.bucket_name, :expires_in => expires_in, :use_ssl => attachment.s3_protocol == 'https')
end
That being said, is there some type of URI.encode or parsing I can do to prevent a valid URI (as I checked the URL works in my browser) for erroring and killing delayed_job in rails 3?
Thank you!
Ruby has (at least) two modules for dealing with URIs.
URI is part of the standard library.
Addressable::URI, is a separate gem, and more comprehensive, and claims to conform to the spec.
Parse a URL with either one, modify any parameters using the gem's methods, then convert it using to_s before passing it on, and you should be good to go.
I tried ' open( URI.parse(URI.encode( a.authenticated_url() )) ' but that errord with OpenURI::HTTPError: 403 Forbidden
If you navigated to that page via a browser and it succeeded, then later failed going to it directly via code, it's likely there is a cookie or session state that is missing. You might need to use something like Mechanize, which will maintain that state while allowing you to navigate through a site.
EDIT:
require 'addressable/uri'
url = 'http://www.example.com'
uri = Addressable::URI.parse(url)
uri.query_values = {
:foo => :bar,
:q => '"one two"'
}
uri.to_s # => "http://www.example.com?foo=bar&q=%22one%20two%22"