I want download pictures from twitter direct messages.
using (var cli = new WebClient())
{
cli.DownloadFile(file.Address, file.Path);
}
where file.Address is uri of picture.
But instead of .jpg file I get html-page.
Thank you!
Maybe it is because you miss some HTTP headers in your request to behave like a browser and not like a bot. You may also have to provide an auth_token cookie if the images are from a private conversation.
Here are the headers I use:
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0'
'Accept': 'application/json, text/javascript, */*; q=0.01'
'X-Requested-With': 'XMLHttpRequest'
You can have a look at a tool I have created to download my direct messages, with the ability to also download the uploaded images and GIFs (as MP4).
The images are downloaded in a separate folder with a reference in the filename to the tweet ID.
Related
YouTube videos have different resolution default thumbnails to use. We use the "default" version (ie: http://img.youtube.com/vi/UNIQUECODE/default.jpg) to preview youtube videos before showing them.
The problem is that Google PageSpeed then dings the page as having a non-compressed image. (ie: Compressing http://img.youtube.com/vi/UNIQUECODE/default.jpg could save 1.1KiB (23% reduction). ).
Is there anyway to call their API and get a compressed image to show?
There are only the default options offered, they don't have a processing option. One possiblity is a 3rd-party compression service like TinyPNG. Their developer API lets you remotely compress an image:
Features: "You can either upload images directly or specify a URL to the image you want to compress."
A POST request using JSON will compress the Youtube image on TinyPNG's server:
POST /shrink HTTP/1.1
Host: api.tinify.com
Authorization: Basic YXBpOmFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6MDEyMzQ1
Content-Type: application/json
{
"source": {
"url": " http://img.youtube.com/vi/UNIQUECODE/default.jpg)"
}
}
The (JSON) response would give you a new compressed image link to use instead of the one from Youtube.
I have a web application that makes file conversion. I've checked usage statistics today and I realized 20-30 files uploaded and converted every second from the same ip. I checked the browser's header info (I wrote every request's http header info to my database) and found this:
Mozilla/5.0 (iPad; CPU OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko)
I think someone is using my web service in an iOS application for converting files. How can I know which application is this or how can I stop this?
If it's your own form, you may use a captcha but this may create problems if you or any of your allowed customers want to use it by using a an application instead of using web view. (I'm not sure if you have such a case). My basic solution would be adding basic authentication to the page on web server level (IIS, etc...)...
I have a web app written in ASP.NET MVC which usually works as it should, but upon submitting the first login page, I occasionally see the error page below. Nothing is logged in ELMAH, and a Google search has returned nothing. I've had similar results with both Chrome and Firefox. Anyone have any ideas what could be causing this?
If I click on the 'Webmaster' mailto link, the email body is filled out with the following:
CacheHost: tx22rrpep37a
ErrPage: ERR_CONNECT_FAIL
Err: (99) Cannot assign requested address
TimeStamp: Sat, 01 Dec 2012 01:52:08 GMT
ClientIP: 10.5.0.99
ServerIP: benjineering.com
HTTP Request:
POST /cloud-chm/Account/Login?ReturnUrl=%2Fcloud-chm HTTP/1.1
Host: benjineering.com
Connection: keep-alive
Content-Length: 488
Cache-Control: max-age=0
Origin: http://benjineering.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11
Content-Type: application/x-www-form-urlencoded
Accept: text/html,application/xhtml xml,application/xml;q=0.9,/;q=0.8
Referer: http://benjineering.com/cloud-chm/Account/Login?ReturnUrl=%2fcloud-chm
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: _jsuid=782392611; ASP.NET_SessionId=i1u4twd1usymcgtppcejhsxt; __RequestVerificationToken_L2Nsb3VkLWNobQ2=F97jJ8pEE3oalgEJFpNCLMU68UE8nG2AdHeI_KDQSfiN2yOj0fV4-psVMtt59sDddJc4XdHM8DLppmdR-fen1xWHp4wkE22KwLaoCKYplWyrlBC3OmeClTcTTKXPdCTC_A-YEhEOVM6FO55fHIBCow2
This looks like a customised error page for an error at web server or infrastructure level (ie. not the application). At face value, it looks as though the your logon url cannot be found, possibly to a resource being unavailable. What type of authentication does the application use? Does it rely on another server?
Are you in control of the hosting, or do you have a 3rd party provider? If the latter you might ask them for their interpretation of the error message.
I appreciate there's no answer here, but I hope the observations and questions might help you arrive at one.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
How do short URLs services work?
I often see shortened urls from bitly.com such as http://bit.ly/abcd. How is this "bit.ly" realized at server side? Is it some DNS trick inside?
Yes.. actually if you go to https://bitly.com/ you will notice that it provides this URL shortening service.
Going to http://bit.ly/abcd just redirects it to a URL of your choice. You can figure it by looking at the HTTP request and response headers
Request URL:http://bit.ly/abcd
Request Method:GET
Status Code:301 Moved
Request Headersview source
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Host:bit.ly
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.24 Safari/535.1
Response Headersview source
Cache-control:private; max-age=90
Connection:keep-alive
Content-Length:145
Content-Type:text/html; charset=utf-8
Date:Thu, 16 Jun 2011 21:14:04 GMT
Location:http://macthemes2.net/forum/viewtopic.php?id=16786044
MIME-Version:1.0
Server:nginx
Set-Cookie:_bit=4dfa721c-001f7-011f8-c8ac8fa8;domain=.bit.ly;expires=Tue Dec 13 16:14:04 2011;path=/; HttpOnly
http://www.w3.org/Protocols/HTTP/HTRESP.html talks about status codes and 301 is what you should be looking for
No, it's just an HTTP server that looks up abcd in a database, finds http://example.com/long/url, and sends an HTTP redirect answer, like
HTTP/1.1 301 Moved Permanently
Location: http://example.com/long/url
Have you gone to http://bit.ly/? The url shortener stores the long url in a database, then when the short url is used, the url shortener service performs an http redirect to the long url.
LY is the top-level domain for Libya, which is distinct from bitly.com.
bit.ly is just a domain like any other (ie: .com, .net. .fr)
In this case .ly belongs to Libya.
It looks like they use A-Za-z0-9 for generating their URLs, and if my calculations are right, this means at any one time they can probably store a database of 61,474,519 of those codes mapped onto the long URLs. Assuming certain links can expire, or people can delete links they have made, it's safe to assume they won't run out of possibilities soon...and hey if they do, just make the links up to 8 characters- then you get 3,381,098,545 possibilities =P
I have a REST API in Rails 3 being accessed sometimes from an AIR application and sometimes from the browser.
I think this is a Rails 3 problem but it might be a Flex/AIR problem.
The Rails app uses omniauth for authentication, cancan for authorization, and active_record_store. I use the session model to store the identity of the user.
(There is a reason I'm not using cookie sessions, having to do with AIR for Android, OAuth, and StageWebView.)
I'm using Charles to monitor HTTP traffic.
Most requests work fine. The browser (or the AIR client) sends the session ID to the server using the Cookie http header, like this:
_session_id=950dee7eca6732aa62b5f91876f66d15
And Rails finds the session, figures out who the user is, and does its thing.
But under certain circumstances, Rails generates a new session before sending the response. It adds a session to the sessions table, and returns a Set-Cookie header to the client with a new session ID. Like this:
_session_id=e1489a6b610c0a1d13cec1454228ae47; path=/; HttpOnly
The circumstances under which this happens are:
The request comes from the AIR client
The request is a POST
This is obviously a problem, because on subsequent requests, Rails can't find the user information. It created a new session without that information.
So I'm looking at the HTTP headers for the POST request. Here's a copy/paste from Charles; I inserted the colon after the header name to make it readable.
Host: localhost.seti.hg94.com:3000
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/531.9 (KHTML, like Gecko) AdobeAIR/2.6
Referer: app:/AndroidApplication.swf
X-Flash-Version: 10,2,152,22
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Cookie: _session_id=950dee7eca6732aa62b5f91876f66d15
Content-Length: 84
Connection: keep-alive
Does anyone have any insight into why Rails would generate a new session under those circumstances? It seems to be happening after my controller code executes, since I have the correct session information in the controller.
I'm busy trying to isolate the problem further, control the headers from within AIR, and so on. I've been working on this bug for almost a week. So any insight or suggestions from the community would be greatly appreciated.
Only a guess, but it seems like you're not bringing across the CSRF token that Rails generates for all POST-based requests:
http://guides.rubyonrails.org/security.html#cross-site-request-forgery-csrf