I am very new to Amazon web services. I am able to generate presigned url for Amazon S3 but not for Amazon SQS.
Thanks in advance for your time and help.
Amazon SQS does not support the creation of a presigned URL. You can check out this forum post for more information.
You can generate presigned urls for SQS using the REST api. Here's an example of a presigned url being creating for reading a message off a queue in Ruby using the aws sdk.
require "httparty"
require("aws-sdk")
message_body = { "hello": "world" }
msg_body_encoded = CGI.escape(message_body.to_json)
signer = Aws::Sigv4::Signer.new(
service: "sqs",
region: "us-east-1",
access_key_id: "ACCESS KEY ID",
secret_access_key: "SECRET ACCESS KEY"
)
presigned_url = signer.presign_url(
http_method: "get",
url: "https://sqs.us-east-1.amazonaws.com/123456789/SomeQueueNmae.fifo/?Action=SendMessage&MessageBody=#{msg_body_encoded}&MessageDeduplicationId=#{dedup_id}&MessageGroupId=Default",
expires_in: 1000.day.to_i
)
response = HTTParty.get(presigned_url, headers: { "Accept": "application/json" })
puts presigned_url
puts response
# {"SendMessageResponse":{"ResponseMetadata":{"RequestId":"12345678910-100c-510b-96af-12345678910"},"SendMessageResult":{"MD5OfMessageAttributes":null,"MD5OfMessageBody":"fbc24bcc7a1794758fc1327fcfebdaf6","MD5OfMessageSystemAttributes":null,"MessageId":"12345678910-db6c-4e17-b6ac-12345678910","SequenceNumber":"12345678910"}}}
I've written an example for ReceiveMessage and SendMessage via Ruby in this gist.
Related
I have Rails app with embedded images. What I want is to upload these images to s3 and serve theme from there instead of form original source Do I have to download the img to my server before upload it to s3?
Short answer: If you're scraping someone's content, then...yes, you need to pull the file down before uploading to to S3.
Long answer: If the other site (the original source) is working with you, you can give them a Presigned URL that they can use to upload to your S3 bucket.
From Amazon's docs: https://docs.aws.amazon.com/AmazonS3/latest/dev/UploadObjectPreSignedURLRubySDK.html
#Uploading an object using a presigned URL for SDK for Ruby - Version 3.
require 'aws-sdk-s3'
require 'net/http'
s3 = Aws::S3::Resource.new(region:'us-west-2')
obj = s3.bucket('BucketName').object('KeyName')
# Replace BucketName with the name of your bucket.
# Replace KeyName with the name of the object you are creating or replacing.
url = URI.parse(obj.presigned_url(:put))
body = "Hello World!"
# This is the contents of your object. In this case, it's a simple string.
Net::HTTP.start(url.host) do |http|
http.send_request("PUT", url.request_uri, body, {
# This is required, or Net::HTTP will add a default unsigned content-type.
"content-type" => "",
})
end
puts obj.get.body.read
# This will print out the contents of your object to the terminal window.
I used to write this to generate a presigned URL in aws-sdk V1 :
AWS.config(S3Config::S3_CONFIG)
bucket = AWS.s3.buckets[S3Config::S3_CONFIG[:bucket]]
presigned_url = bucket.presigned_post(
key: "attachments/#{SecureRandom.uuid}/${filename}",
success_action_status: 201, acl: 'public-read'
)
Which sent an OPTIONS/POST working request like this :
https://gist.github.com/gotoAndBliss/cdd8818b8adce58d1b625f68e2633199
The biggest difference is that the Status Code is 201 Created
I then updated to V2 and rewrote it to this :
presigned_url = Aws::S3::PresignedPost.new(aws_creds, aws_region, S3Config::BUCKET, {
key: "attachments\/#{SecureRandom.uuid}\/\${filename}",
metadata: {"original-filename" => "${filename}"},
acl: 'public-read', success_action_status: ['201']
})
To which I'm pretty sure was properly written. But this generates this request :
https://gist.github.com/gotoAndBliss/43a4a88adc5c2be0b70b66d551a72a84
The biggest difference is the Status Code : 204 No Content
I went through this line for line and it seems like everything else is identical. Would anyone know why these are failing? Or what sets them apart?
Below is my process for doing presigned uploads. One thing I'm noticing right off the bat is you are escaping the / in your key. I don't and it works fine. I use ENV variables for the secret stuff.
config/initializers/aws.rb:
Aws.config.update(
{region: 'us-west-2',
credentials: Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'],
ENV['AWS_SECRET_ACCESS_KEY']),
}
)
S3_BUCKET = Aws::S3::Resource.new.bucket(ENV['S3_BUCKET_NAME'])
In my controller I generate the url like this:
#s3_direct_post = S3_BUCKET.presigned_post(
key: "my_bucket_folder/#{SecureRandom.uuid}/${filename}",
success_action_status: '201',
acl: 'public-read')
When a heroku creates resources:
a) Make a call to the endpoint of authentication:
POST https://app.starving.cloud.io/v1/profile/users/email/{email}/sign-in
with the body
(Content-Type: application / json)
{
"password": "{password}"
}
The endpoint returns the token as json:
{
"token": "{token}"
}
Please help me. how to create this is? Or what i should to read about it. Thanks very much.
If this is an API you need to consume (hit, etc), you can use an HTTP Client library like typhoeus to write code that does the POST request and allows you to read the token from the response's body:
email = 'foo#example.com'
password = 'password' # Never use this password!
response = Typhoeus.post("https://app.starving.cloud.io/v1/profile/users/email/#{email}/sign-in",
headers: { 'Content-Type'=> 'application/json' },
body: { password: password }.to_json)
token = JSON.parse(response.body)['token']
Please read the gem's documentation to build the best version of this code, but this should give you an idea and a starting point.
I am using Ruby on Rails and AWS gem.
I can get pre-signed URL for upload and download.
But when I get the URL there is no file, and so setting acl to 'public-read'
on the download-url doesn't work.
Use case is this: 1, server provides the user a path to upload content to my bucket that is not readable without credentials. 2, And that content needs to be public later: readable by anyone.
To clarify:
I am not uploading the file, I am providing URL for my users to upload. At that time, I also want to give the user a URL that is readable by the public. It seems like it would be easier if I uploaded the file by myself. Also, read URL needs to never expire.
When you generate a pre-signed URL for a PUT object request, you can specify the key and the ACL the uploader must use. If I wanted the user to upload an objet to my bucket with the key "files/hello.txt" and the file should be publicly readable, I can do the following:
s3 = Aws::S3::Resource.new
obj = s3.bucket('bucket-name').object('files/hello.text')
put_url = obj.presigned_url(:put, acl: 'public-read', expires_in: 3600 * 24)
#=> "https://bucket-name.s3.amazonaws.com/files/hello.text?X-Amz-..."
obj.public_url
#=> "https://bucket-name.s3.amazonaws.com/files/hello.text"
I can give the put_url to someone else. This URL will allow them to PUT an object to the URL. It has the following conditions:
The PUT request must be made within the given expiration. In the example above I specified 24 hours. The :expires_in option may not exceed 1 week.
The PUT request must specify the HTTP header of 'x-amz-acl' with the value of 'public-read'.
Using the put_url, I can upload any an object using Ruby's Net::HTTP:
require 'net/http'
uri = URI.parse(put_url)
request = Net::HTTP::Put.new(uri.request_uri, 'x-amz-acl' => 'public-read')
request.body = 'Hello World!'
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
resp = http.request(request)
Now the object has been uploaded by someone else, I can make a vanilla GET request to the #public_url. This could be done by a browser, curl, wget, etc.
You have two options:
Set the ACL on the object to 'public-read' when you PUT the object. This allows you to use the public url without a signature to GET the object.
Let the ACL on the object default to private and provide pre-signed GET urls for users. These expire, so you have to generate new URLs as needed. A pre-signed URL allows someone to send GET request to the object without credentials themselves.
Upload a public object and generate a public url:
require 'aws-sdk'
s3 = Aws::S3::Resource.new
s3.bucket('bucket-name').object('key').upload_file('/path/to/file', acl:'public-read')
s3.public_url
#=> "https://bucket-name.s3.amazonaws.com/key"
Upload a private object and generate a GET url that is good for 1-hour:
s3 = Aws::S3::Resource.new
s3.bucket('bucket-name').object('key').upload_file('/path/to/file')
s3.presigned_url(:get, expires_in: 3600)
#=> "https://bucket-name.s3.amazonaws.com/key?X-Amz-Algorithm=AWS4-HMAC-SHA256&..."
I am working on a Rails App that Uses OmniAuth to gather Oauth/OAuth2 credentials for my users and then posts out to those services on their behalf.
Creating simple posts to update status feeds work great.. Now I am to the point of needing to upload files. Facebook says "To publish a photo, issue a POST request with the photo file attachment as multipart/form-data." http://developers.facebook.com/docs/reference/api/photo/
So that is what I am trying to do:
I have implemented the module here: Ruby: How to post a file via HTTP as multipart/form-data? to get the headers and data...
if appearance.post.post_attachment_content_type.to_s.include?('image')
fbpost = "https://graph.facebook.com/me/photos"
data, headers = Multipart::Post.prepare_query("title" => appearance.post.post_attachment_file_name , "document" => File.read(appearance.post.post_attachment.path))
paramsarray = {:source=>data, :message=> appearance.post.content}
response = access_token.request(:post, fbpost, paramsarray, headers)
appearance.result = response
appearance.save
end
I but I am getting a OAuth2::HTTPError - HTTP 400 Error
Any assistance would be Incredible... As I see this information will also be needed for uploading files to SoundCloud also.
Thanks,
Mark
Struggled with this myself. The oauth2 library is backed by Faraday for it's HTTP interaction. with a little configuration it supports uploaded files out of the box. First step is to add the appropriate Faraday middleware when building your connection. An example from my code:
OAuth2::Client.new client_id, secret, site: site do |stack|
stack.request :multipart
stack.request :url_encoded
stack.adapter Faraday.default_adapter
end
This adds the multipart encoding support to the Faraday connection. Next when making the request on your access token object you want to use a Faraday::UploadIO object. So:
upload = Faraday::UploadIO.new io, mime_type, filename
access_token.post('some/url', params: {url: 'params'}, body: {file: upload})
In the above code:
io - An IO object for the file you want to upload. Can be a File object or even a StringIO.
mime_type - The mime type of the file you are uploading. You can either try to detect this server-side or if a user uploaded the file to you, you should be able to extract the mime type from their request.
filename - What are are calling the file you are uploading. This can also be determined by your own choosing or you can just use whatever the user uploading the file calls it.
some/url - Replace this with the URL you want to post to
{url: 'params'} - Replace this with any URL params you want to provide
{file: upload} - Replace this with your multipart form data. Obviously one (or more) of the key/value pairs should have an instance of your file upload.
I'm actually using successfully this code to upload a photo on a fb page :
dir = Dir.pwd.concat("/public/system/posts/images")
fb_url = URI.parse("https://graph.facebook.com/#{#page_id}/photos")
img = File.open("myfile.jpg")
req = Net::HTTP::Post::Multipart.new(
"#{fb_url.path}?access_token=#{#token}",
"source" => UploadIO.new(img, "application/jpg", img.path),
"message" => "some messsage"
)
n = Net::HTTP.new(fb_url.host, fb_url.port)
n.use_ssl = true
n.verify_mode = OpenSSL::SSL::VERIFY_NONE
n.start do |http|
#result = http.request(req)
end