I followed this blog
https://aws.amazon.com/blogs/compute/resize-images-on-the-fly-with-amazon-s3-aws-lambda-and-amazon-api-gateway/
and this repo
https://github.com/awslabs/serverless-image-resizing
To be able to resize images on AWS S3 on the fly. The bucket is created successfully. Whenever I upload image to the bucket, I can resize it from the url as indicated in the blog, everything works fine.
The problem is when an image is uploaded from the code (Ruby on Rails using paperclip), then the image can be accessed normally, however when trying with resizing url (e.g. /100x200/photo.png) I receive this in the browser as json response
{
"message": "Internal server error"
}
The response headers are:
Request URL:<url>
Request Method:GET
Status Code:502
Remote Address:<IP>
Referrer Policy:no-referrer-when-downgrade
Response Headers
content-length:36
content-type:application/json
date:Thu, 23 Nov 2017 09:59:24 GMT
status:502
via:1.1 <hash>.cloudfront.net (CloudFront)
x-amz-cf-id:<id>
x-amzn-requestid:<id>
x-cache:Error from cloudfront
Request Headers
:authority:<url>
:method:GET
:path:/prod?key=<path/to/photo>
:scheme:https
accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9,de;q=0.8,ar;q=0.7
cache-control:max-age=0
upgrade-insecure-requests:1
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.62 Safari/537.36
Query String Parameters
view source
view URL encoded
key:<path/to/photo>
The very strange thing is that, after the error occurs, when I upload the same image manually again under any place in the bucket then I try again using the original resize url, not the new uploaded image url that caused a problem, it works normally and I can resize to any size !
So it seems like uploading manually triggers something related to this very image data ? I have no clue!
The error here that you put the required size before file name directly, you should put it before the whole file path.
Make sure that you follow this pattern:
http://$BucketWebsiteHost/$size/$imagePath
Example:
http://mybucketname.s3-website.mybucketregion.amazonaws.com/60x50/photos/attachments/000/002/002/original/image.jpg
Related
I am trying to read the contents of a web page using a Groovy script. The page contains the readings from one of my temperature sensors that I want to save regularly. I have tried the simplest variant:
def url = "https://measurements.mobile-alerts.eu/Home/MeasurementDetails?deviceid=021B5594EAB5&vendorid=60122a8b-b343-49cb-918b-ad2cdd6dff16&appbundle=eu.mobile_alerts.mobilealerts&fromepoch=1674432000&toepoch=1674518400&from=23.01.2023%2000:00&to=24.01.2023%2000:00&command=refresh"
def res = url.toURL().getText()
println( res)
The result is:
Caught: java.io.IOException: Server returned HTTP response code: 403 for URL: (my url)
In any browser, this URL works without problems.
I would be very grateful for any tips on how to solve this problem.
HTTP code 403 means that a client is forbidden from accessing a valid URL. In other words, the server knows that you are not making a request via a web browser. To bypass this restriction, you need to specify a User-Agent in the request header.
For example:
def url = 'https://measurements.mobile-alerts.eu/Home/MeasurementDetails?deviceid=021B5594EAB5&vendorid=60122a8b-b343-49cb-918b-ad2cdd6dff16&appbundle=eu.mobile_alerts.mobilealerts&fromepoch=1674432000&toepoch=1674518400&from=23.01.2023%2000:00&to=24.01.2023%2000:00&command=refresh'
def res = url.toURL().getText(requestProperties:
['User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0'])
println res
You can switch to other valid user-agent values.
I've been using Radview's Webload IDE tool for a couple of test simulation projects and it has worked well. But for this one scenario where I have a client web session for a login a screen, it would always fail with a 500 Response error for a particular HTTP post as the page loads.
When I try the scenario to load the page manually with a browser it works fine with no issues.
During the recording I would set clear browser cache and cookies and no luck. And I've also tried out many configuration combinations from the "Recording and Script Generatinon Options: Post Data" settings.
/***** WLIDE - URL : http://192.168.2.2/ - ID:2 *****/
wlGlobals.GetFrames = false
wlGlobals.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko"
wlHttp.Get("http://192.168.2.2/")
// END WLIDE
/***** WLIDE - URL : http://192.168.2.2/Api.ashx?c=Images&action=GetSettings - ID:3 *****/
wlHttp.Header["Referer"] = "http://192.168.2.2/"
wlHttp.FormdataEncodingType = 1
wlHttp.ContentType = "application/x-www-form-urlencoded"
wlHttp.FormData["c"] = "Images"
wlHttp.FormData["action"] = "GetSettings"
wlHttp.Post("http://192.168.2.2/Api.ashx"+"?c=Images&action=GetSettings")
// END WLIDE
Anybody with experience with Radview's Webload can give me some suggestions?
I noticed that commenting out the formdata "c" and "actions" lines works. but later I notice a similar error which requires a sessionID in the URL so I'm not sure if I can comment out the formdata "sessionID" line.
To run the API from Webload you need to specify the authorization if its secured.
Using wlHttp.FormData is not the same as adding a parameter to the URL for a POST request.
FormData will be send as part of the post-data request body, while adding it to the URL will send it as a query string - your sever probably expects one form but not the other.
Contact RadView support if you can't get it to work and they'll help you
I am building a React.js front-end portion of a web application using an already existing back-end that is built with ASP.NET MVC Framework. The method in the API's controller is named Post (which should default it to a post method) and even has the [HttpPost] decorator. I have recently enabled CORS, but I am not sure if that matters.
The API also has Swagger (it's like postman if you haven't used it before) enabled, and when I call the method as POST from the swagger, it will work and return the correct data.
However, when I try to call the method from my UI component, the API endpoint will give a 405 Method Not Allowed status code.
It says that the method only allows GET, which is baffling since it is supposed to only allow POST and it works that way on swagger or if I manually travel to the URL using a browser. Has anyone had a problem like this using React components or working with ASP.NET MVC/Web API 2?
I am completely stuck...
EDIT:
I am adding the API's method for the post endpoint.
[Route("{id}/merchant/SearchMerchants")]
[HttpPost]
[AuthorizeUserPermissionsToken(Cookie)]
public Response Post(Request rq)
{ // I put a breakpoint here to debug
// some code
}
I had to redact a lot of the code but here is the post method.
But I don't think it has to do with the API endpoint.
This is because when I debug, it will not even reach the breakpoint or call the method at all, when I am using the UI. However, it will reach the breakpoint when I travel to the endpoint on the browser or use swagger.
EDIT 2:
EDIT 3: (removed EDIT 2 and moved the original headers to bottom to reduce confusion)
Pre-flight OPTIONS request headers
Accept:*/*
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8
Access-Control-Request-Headers:content-type
Access-Control-Request-Method:POST
Cache-Control:no-cache
Connection:keep-alive
Host: *redacted domain*
Origin:http://localhost:49702
Pragma:no-cache
Referer:http://localhost:49702/FindMerchants
User-Agent:Mozilla/5.0 (Linux; Android 5.1.1; Nexus 6 Build/LYZ28E) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Mobile Safari/537.36
response headers to OPTIONS request:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Headers:Origin, X-Requested-With, Content-Type, Accept
Access-Control-Allow-Origin:http://localhost:49702
Allow:OPTIONS, TRACE, GET, HEAD, POST
Content-Length:0
Date:Mon, 07 Aug 2017 18:41:42 GMT
Public:OPTIONS, TRACE, GET, HEAD, POST
Server:Microsoft-IIS/10.0
X-Content-Type-Options:nosniff
X-Frame-Options:SAMEORIGIN
then the real POST request headers after pre-flight:
Accept:*/*
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:keep-alive
Content-Length:154
Content-Type:application/json
Cookie: *redacted a bunch of cookies*
Host:a *redacted domain*
Origin:http://localhost:49702
Pragma:no-cache
Referer:http://localhost:49702/FindMerchants
User-Agent:Mozilla/5.0 (Linux; Android 5.1.1; Nexus 6 Build/LYZ28E) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Mobile Safari/537.36
The real response headers are like so:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Headers:Origin, X-Requested-With, Content-Type, Accept
Access-Control-Allow-Origin:http://localhost:49702
Allow:GET
Cache-Control:no-cache
Content-Length:73
Content-Type:application/json; charset=utf-8
Date:Mon, 07 Aug 2017 17:58:28 GMT
Expires:-1
Pragma:no-cache
Server:Microsoft-IIS/10.0
X-Content-Type-Options:nosniff
X-Frame-Options:SAMEORIGIN
EDIT 4:
Request URL:*redacted domain*/380/merchant/SearchMerchant
Request Method:POST
Status Code:405 Method Not Allowed
Remote Address:127.0.0.1:80
Referrer Policy:no-referrer-when-downgrade
EDIT #9999 (jk):
My react component simply calls a custom data service that I wrote, which has a post method like this.
post(relativeUrl, data, apiVersion) {
var url = this.getUrl(relativeUrl, apiVersion);
var res = $.ajax({
type: 'POST',
url: url,
contentType: 'application/json',
xhrFields: {
withCredentials: true
},
crossDomain: true,
data: JSON.stringify(data)
});
return res;
}
I simply pass relativeUrl to fetch a list related to a user account.
Before I make a call that fails (which shows 405), I use this same method to log in and update the global redux state for the app. After updating the global state, I render a different component and use this method again with another relativeUrl to fetch the list. Then I get the error. I also make sure that the login call is finished and returns and updates the state correctly before moving on.
To follow my comment,
in web.config file:
<httpProtocol>
<customHeaders>
<add name="Access-Control-Allow-Methods" value="POST, GET, OPTIONS, HEAD" />
</customHeaders>
</httpProtocol>
The code: I got the following piece of code in some HAML file.
= image_tag "https://s3.amazonaws.com/my_bucket/my_image.jpg"
It sends a request to s3 and loads the image in the browser. I got the following CORS configuration on the bucket:
<CORSConfiguration>
<CORSRule>
<AllowedOrigin>https://www.my_site.com</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
</CORSRule>
</CORSConfiguration>
The problerm:
In order to be able to manipulate the images on client side, the images are supposed to be served with the following headers:
Access-Control-Allow-Origin: https://www.my_site.com
Access-Control-Allow-Methods: GET
but this does not happen.
The cause: My browser does not send 'Origin' request header, and therefore s3 does not respond with the desired headers.
Why I think that missing "Origin" header is the cause:
Because the response of:
wget --server-response --header "Origin:https://www.my_site.com" "https://s3.amazonaws.com/my_bucket/my_image.jpg"
is the following:
HTTP/1.1 200 OK
x-amz-id-2: kQV8HEChV1...QHmHC1Gt/
x-amz-request-id: A626...4A2
Date: Wed, 03 Jul 2013 10:10:38 GMT
Access-Control-Allow-Origin: https://www.my_site.com
Access-Control-Allow-Methods: GET
Access-Control-Allow-Credentials: true
Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
...
i.e. the 'Access-Control-Allow-Origin' and 'Access-Control-Allow-Methods' are present.
The supposed solution:
Is there a way to add manually the desired headers to the image_tag in the HAML file? Something like:
= image_tag "https://s3.amazonaws.com/my_bucket/my_image.jpg", :headers=>["Origin"]
Not using image_tag, no. Remember all image_tag does is generate the HTML tag. The end user's browser is responsible for loading the image source and displaying it. Unless I've missed something major, there isn't any way for you to tell the browser to pass additional headers via HTML.
You might be able to change your method and load those images using Javascript. Maybe you can pass headers that way.
Check How to debug CORS error for a solution.
I spent a lot of time thinking the problem was server side but finally it was due to a browser issue.
You can force Chrome to user the proper CORS protocol by loading your image through JavaScript.
Try changing your html to an empty img tag and loading your image into it. Like:
<span id="source-url" class="hidden"><%= #product.images.first.attachment.url(:large) %></span>
<img id="art-image">
and in a .js file
artImage = document.getElementById('art-image');
$(artImage).attr('crossOrigin', '');
$(artImage).attr("src", $("#source-url").text());
$(artImage).one('load', function() {
// image processing
});
Hope it helps!
Previously i was able to download YouTube videos as mp3 via youtube-mp3.org Using this method:
http://www.youtube-mp3.org/api/pushItem/?item=http%3A//www.youtube.com/watch%3Fv%3D<VIDEOID>&xy=_
Then it returned the video id and they started converting the video on their servers. Then this request would return a JSON string with info about the video and the current conversion status:
http://www.youtube-mp3.org/api/itemInfo/?video_id=<VIDEOID>&adloc=
After repeating the request until the value for status is 'serving' I then started the last request by taking the value for key h from the JSON response from the previous request, and this would download a the mp3 file.
http://www.youtube-mp3.org/get?video_id=<VIDEOID>&h=<JSON string value for h>
Now the first request always returns nothing. The second and third requests only succeed if the requested video is cached on their servers (like popular music videos). If thats not the case then the second request would return nil and so the 3rd request can't be started because of the missing hvalue from the second request. Could anybody help me with getting the website to start a conversion something needs to be wrong with the first URL i just dont know what. Thanks
I just tested it. For the first request, you need to send with it a header of:
Accept-Location: *
Otherwise, it will return a 500 (Internal Server Error). But with that header, it will return a string of the youtube video id, and you can use the 2nd api for checking the progress.
Here's the C# code I used for testing:
HttpWebRequest wr = (HttpWebRequest)WebRequest.Create("FIRST_API_URL");
wr.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.75 Safari/535.7";
wr.Headers.Add("Accept-Location", "*");
string res = (new StreamReader(wr.GetResponse().GetResponseStream())).ReadToEnd();
Btw, you can keep track of the headers in the browser's Network (Chrome) debug tab.
Regards