Previously i was able to download YouTube videos as mp3 via youtube-mp3.org Using this method:
http://www.youtube-mp3.org/api/pushItem/?item=http%3A//www.youtube.com/watch%3Fv%3D<VIDEOID>&xy=_
Then it returned the video id and they started converting the video on their servers. Then this request would return a JSON string with info about the video and the current conversion status:
http://www.youtube-mp3.org/api/itemInfo/?video_id=<VIDEOID>&adloc=
After repeating the request until the value for status is 'serving' I then started the last request by taking the value for key h from the JSON response from the previous request, and this would download a the mp3 file.
http://www.youtube-mp3.org/get?video_id=<VIDEOID>&h=<JSON string value for h>
Now the first request always returns nothing. The second and third requests only succeed if the requested video is cached on their servers (like popular music videos). If thats not the case then the second request would return nil and so the 3rd request can't be started because of the missing hvalue from the second request. Could anybody help me with getting the website to start a conversion something needs to be wrong with the first URL i just dont know what. Thanks
I just tested it. For the first request, you need to send with it a header of:
Accept-Location: *
Otherwise, it will return a 500 (Internal Server Error). But with that header, it will return a string of the youtube video id, and you can use the 2nd api for checking the progress.
Here's the C# code I used for testing:
HttpWebRequest wr = (HttpWebRequest)WebRequest.Create("FIRST_API_URL");
wr.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.75 Safari/535.7";
wr.Headers.Add("Accept-Location", "*");
string res = (new StreamReader(wr.GetResponse().GetResponseStream())).ReadToEnd();
Btw, you can keep track of the headers in the browser's Network (Chrome) debug tab.
Regards
Related
I am trying to read the contents of a web page using a Groovy script. The page contains the readings from one of my temperature sensors that I want to save regularly. I have tried the simplest variant:
def url = "https://measurements.mobile-alerts.eu/Home/MeasurementDetails?deviceid=021B5594EAB5&vendorid=60122a8b-b343-49cb-918b-ad2cdd6dff16&appbundle=eu.mobile_alerts.mobilealerts&fromepoch=1674432000&toepoch=1674518400&from=23.01.2023%2000:00&to=24.01.2023%2000:00&command=refresh"
def res = url.toURL().getText()
println( res)
The result is:
Caught: java.io.IOException: Server returned HTTP response code: 403 for URL: (my url)
In any browser, this URL works without problems.
I would be very grateful for any tips on how to solve this problem.
HTTP code 403 means that a client is forbidden from accessing a valid URL. In other words, the server knows that you are not making a request via a web browser. To bypass this restriction, you need to specify a User-Agent in the request header.
For example:
def url = 'https://measurements.mobile-alerts.eu/Home/MeasurementDetails?deviceid=021B5594EAB5&vendorid=60122a8b-b343-49cb-918b-ad2cdd6dff16&appbundle=eu.mobile_alerts.mobilealerts&fromepoch=1674432000&toepoch=1674518400&from=23.01.2023%2000:00&to=24.01.2023%2000:00&command=refresh'
def res = url.toURL().getText(requestProperties:
['User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0'])
println res
You can switch to other valid user-agent values.
I've been using Radview's Webload IDE tool for a couple of test simulation projects and it has worked well. But for this one scenario where I have a client web session for a login a screen, it would always fail with a 500 Response error for a particular HTTP post as the page loads.
When I try the scenario to load the page manually with a browser it works fine with no issues.
During the recording I would set clear browser cache and cookies and no luck. And I've also tried out many configuration combinations from the "Recording and Script Generatinon Options: Post Data" settings.
/***** WLIDE - URL : http://192.168.2.2/ - ID:2 *****/
wlGlobals.GetFrames = false
wlGlobals.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko"
wlHttp.Get("http://192.168.2.2/")
// END WLIDE
/***** WLIDE - URL : http://192.168.2.2/Api.ashx?c=Images&action=GetSettings - ID:3 *****/
wlHttp.Header["Referer"] = "http://192.168.2.2/"
wlHttp.FormdataEncodingType = 1
wlHttp.ContentType = "application/x-www-form-urlencoded"
wlHttp.FormData["c"] = "Images"
wlHttp.FormData["action"] = "GetSettings"
wlHttp.Post("http://192.168.2.2/Api.ashx"+"?c=Images&action=GetSettings")
// END WLIDE
Anybody with experience with Radview's Webload can give me some suggestions?
I noticed that commenting out the formdata "c" and "actions" lines works. but later I notice a similar error which requires a sessionID in the URL so I'm not sure if I can comment out the formdata "sessionID" line.
To run the API from Webload you need to specify the authorization if its secured.
Using wlHttp.FormData is not the same as adding a parameter to the URL for a POST request.
FormData will be send as part of the post-data request body, while adding it to the URL will send it as a query string - your sever probably expects one form but not the other.
Contact RadView support if you can't get it to work and they'll help you
I followed this blog
https://aws.amazon.com/blogs/compute/resize-images-on-the-fly-with-amazon-s3-aws-lambda-and-amazon-api-gateway/
and this repo
https://github.com/awslabs/serverless-image-resizing
To be able to resize images on AWS S3 on the fly. The bucket is created successfully. Whenever I upload image to the bucket, I can resize it from the url as indicated in the blog, everything works fine.
The problem is when an image is uploaded from the code (Ruby on Rails using paperclip), then the image can be accessed normally, however when trying with resizing url (e.g. /100x200/photo.png) I receive this in the browser as json response
{
"message": "Internal server error"
}
The response headers are:
Request URL:<url>
Request Method:GET
Status Code:502
Remote Address:<IP>
Referrer Policy:no-referrer-when-downgrade
Response Headers
content-length:36
content-type:application/json
date:Thu, 23 Nov 2017 09:59:24 GMT
status:502
via:1.1 <hash>.cloudfront.net (CloudFront)
x-amz-cf-id:<id>
x-amzn-requestid:<id>
x-cache:Error from cloudfront
Request Headers
:authority:<url>
:method:GET
:path:/prod?key=<path/to/photo>
:scheme:https
accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9,de;q=0.8,ar;q=0.7
cache-control:max-age=0
upgrade-insecure-requests:1
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.62 Safari/537.36
Query String Parameters
view source
view URL encoded
key:<path/to/photo>
The very strange thing is that, after the error occurs, when I upload the same image manually again under any place in the bucket then I try again using the original resize url, not the new uploaded image url that caused a problem, it works normally and I can resize to any size !
So it seems like uploading manually triggers something related to this very image data ? I have no clue!
The error here that you put the required size before file name directly, you should put it before the whole file path.
Make sure that you follow this pattern:
http://$BucketWebsiteHost/$size/$imagePath
Example:
http://mybucketname.s3-website.mybucketregion.amazonaws.com/60x50/photos/attachments/000/002/002/original/image.jpg
I'm trying out http requests to download a pdf file from google docs using google document list API and OAuth 1.0. I'm not using any external api for oauth or google docs.
Following the documentation, I obtained download URL for the pdf which works fine when placed in a browser.
According to documentation I should send a request that looks like this:
GET https://doc-04-20-docs.googleusercontent.com/docs/secure/m7an0emtau/WJm12345/YzI2Y2ExYWVm?h=16655626&e=download&gd=true
However, the download URL has something funny going on with the paremeters, it looks like this:
https://doc-00-00-docs.googleusercontent.com/docs/securesc/5ud8e...tMzQ?h=15287211447292764666&\;e=download&\;gd=true
(in the url '&\;' is actually without '\' but I put it here in the post to avoid escaping it as '&').
So what is the case here; do I have 3 parameters h,e,gd or do I have one parameter h with value 15287211447292764666&ae=download&gd=true, or maybe I have the following 3 param-value pairs: h = 15287211447292764666, amp;e = download, amp;gd = true (which I think is the case and it seems like a bug)?
In order to form a proper http request I need to know exectly what are the parameters names and values, however the download URL I have is confusing. Moreover, if the params names are h,amp;e and amp;gd, is the request containing those params valid for obtaining file content (if not it seems like a bug).
I didn't have problems downloading and uploading documents (msword docs) and my scope for downloading a file is correct.
I experimented with different requests a lot. When I treat the 3 parameters (h,e,gd) separetaly I get Unauthorized 401. If I assume that I have only one parameter - h with value 15287211447292764666&ae=download&gd=true I get 500 Internal Server Error (google api states: 'An unexpected error has occurred in the API.','If the problem persists, please post in the forum.').
If I don't put any paremeters at all or I put 3 parameters -h,amp;e,amp;gd, I get 302 Found. I tried following the redirections sending more requests but I still couldn't get the actual pdf content. I also experimented in OAuth Playground and it seems it's not working as it's supposed to neither. Sending get request in OAuth with the download URL responds with 302 Found instead of responding with the PDF content.
What is going on here? How can I obtain the pdf content in a response? Please help.
I have experimented same issue with oAuth2 (error 401).
Solved by inserting the oAuth2 token in request header and not in URL.
I have replaced &access_token=<token> in the URL by setRequestHeader("Authorization", "Bearer <token>" )
Is there a way to get the following data from the Application_Error event in the Global.ascx file?
action error came from,
ipaddress error came from,
browser error came from,
browser version error came from,
hostName error came from
??
All that information is contained in the Context.Request property.
Context.Request.Url; // /controller/action?foo=bar so up to you to extract the action
Context.Request.UserHostAddress; // 123.456.789.0123
Context.Request.UserAgent // Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13
And once you are sick of parsing all this crap manually and repeating this code all over again among all your applications you might consider using ELMAH.
hostName error came from
Not sure what you mean here. Isn't that the IP of the client?