I'm developing an HttpLitener C# application that receive requests from client & count them.
But the problem is there is some automatic requests are sent from the user which he didn't click them like applications, Gadgets, CSS file requests... etc
I want to detect the difference between the user requests & automatic requests,
through IP Address or any request header, it doesn't matter, BUT HOW & IS there a difference?
Related
In our application we observe multiple ( two ) these same requests send from mobile application to server in milliseconds apart.
As we discuss the problem with dev team, they said they don't send two requests from an application perspective, but on the server-side, we see exactly these same two requests.
Does anybody know if iOS has this type of functionality to keep resending this same request in case of a lost connection or any other case? ( This is milliseconds that server doesn't respond yet )
The application should send only one request, wait for response success/failure, and then resend as needed. So far as we know, there is no logic in the application itself that will trigger sending two requests from the app to the server in milliseconds apart.
Thank you for any suggestions.
It's hard to tell without looking at the code or knowing your network infrastructure.
What I'd suggest to do first is to run the app through a debugging proxy server like Charles, Proxyman or mitmproxy. If it shows multiple requests, most likely the app is to blame, I'd bet on a concurrency bug.
If the debugging proxy shows just one request but your server observes two, you'll have to check your network infrastructure, it might be that some load balancer or reverse proxy is configured incorrectly.
Here is my need:
I have to displays some information from a web page.
The web browser is actually on the same machine (localhost).
I want the data to be updated dynamically by the server initiative.
Since HTTP protocol is actually a request/response protocol, I know that to get this functionality, the connection between the server and the client (which is local here) should be kept open in some way (Websocket, Server-Sent Events, etc..)
Yes, "realtime" is really a fashion trend nowadays and there are many frameworks out there to do this (meteor, etc...)
And indeed, it seems that Rails supports this functionnality too in addition to using Websockets (Server-Sent Events in Rails 4 and ActionCable in Rails 5)
So achieving this functionnality would not be a big deal, I guess...
Nevertheless what I really want is to trigger an update of the webpage (displayed here locally) from a request made by another client..
This picture will explain that better :
At the beginning, the browser connects to the (local) server (green arrows).
I guess that a thread is executed where all the session data (instance variables) are stored.
In order to use some "realtime" mechanisms, the connection remains open and therefore the thread Y is not terminated. (I guess this is how it works)
A second user is connecting (blue arrows) to the server (could be or not be the same web page) and make some actions (eg. posting a form).
Here the response to that external client does not matter. Just an HTTP OK response is fine. But a confirmation web page could also be returned.
But in anyway the thread X (and/or the connection) has no particular reason to be kept.
Ok, here is my question (BTW thank you for reading me thus far).
How can I echo this new data on the local web browser ?
I see 2 differents ways to do this :
Path A: Before terminating, the thread X passes the data (its instance variables) to the thread Y which has its connection still open. Thus the server is able to update the web browser.
Path B: Before terminating the thread X sends a request (I mean a response since it is the server) directly to the web browser using a particular socket.
Which mechanisms should I use in either method to achieve this functionnality ?
For method A, how can I exchange data between threads ?
For method B, how can I use an already opened socket ?
But which of these two methods (or another one) is actually the best way to do that?
Again thank you for reading me thus far, and sorry for my bad english.
I hope I've been clear enough to expose my need.
You are overthinking this. There is no need to think of such low-level mechanisms as threads and sockets. Most (all?) pub-sub live-update tools (ActionCable, faye, etc.) operate in terms of "channels" and "events".
So, your flow will look like this:
Client A (web browser) makes a request to your server and subscribes to events from channel "client-a-events" (or something).
Client B (the other browser) makes a request to your server with instructions to post an event to channel "client-a-events".
Pub-sub library does its magic.
Client A gets an update and updates the UI accordingly.
Check out this intro guide: Action Cable Overview.
I want to make an application to redirect websites.
It has a table with "domains" and "redirect domains".
Once it matched domain, it redirect to redirect domain.
If didn't matched, it redirect to default page.
So I created a Delphi application with IdHTTPProxyServer.
I have configured it to even work with https using "ssleay32.dll" and "libeay32.dll".
Everything works great.
It use "IdHTTPProxyServerHTTPBeforeCommand" event to redirect like this:
with AContext.Connection.IOHandler do
begin
WriteLn('HTTP/1.0 302 Moved Temporarily');
WriteLn('Location: ' + RedirectURL);
WriteLn('Connection: close');
WriteLn;
end;
But how do I distinguish the event call by main URL (user typed in the address bar) and other URLs?
"IdHTTPProxyServerHTTPBeforeCommand" event called lots of times when a page is loading for stat counters, facebook like buttons, etc. I don't want to redirect all of them to default page.
If this is not possible with IdHTTPProxyServer, is there any other options in Delphi or any other language (which can generate native executable. C++ preferred)?
Thank You
From the perspective of a proxy (or the target HTTP server, for that matter), there is no difference whatsoever between a user-typed URL and other URLs. Every HTTP request is self-contained and independent of every other HTTP request. They have to processed as-is on a per-request basis.
If you want to ignore dependent URLs (images, scripts, etc), you will have to know ahead of time what the initial URL is, parse the data that is retrieved from that URL, keep track of any URLs the data refers to, and then ignore those URLs if you see them being requested later. However, there is nothing in the HTTP protocol to tell you what the initial URL is. There is a Referer request header that may help at times, as it is filled in when a browser is requesting dependent resource files, but it is also filled in when the user navigates around from one page to another, so you can't rely on the Referer by itself. You will have to implement your own discovery logic to figure out the initial URL based on more analysis of the URLs being requested by a given client over time.
Only the client really knows what it is requesting and why, a proxy is just a gateway to reach it. So there is only so much smart filtering you can do in a proxy without knowing what the client is actually doing.
Hoping someone has some knowledge on this one. I have a system which allows users to post to facebook or to send a link via email using an ESP called dotMailer. When creating the wall post / email campaign however, both Facebook and dotMailer 'test' the URL embedded in the content to see if it's valid.
I am storing a viewed_on date for the URLs, and as such I want to be able to ignore the HTTP requests by Facebook and dotMailer rather than storing the viewed_on date that they trigger by hitting the URL.
In terms of what I've tried / won't work:
IP Filtering - cannot rely on IP being same each time
Time-based delay - depends on how quickly dotMailer/Facebook processes the requests, so cannot rely on this
Thanks!
I'm a dev with dotMailer - for us, you can rely on the request coming from one of two different places: 94.143.104.0/21, 80.249.97.113, or 80.249.97.114. With Facebook, you can simply check the UserAgent. We use an IE useragent, because a surprising amount of sites behave differently when presented with a non-standard useragent and thus make link checking less reliable.
We've got a great forum, so stop on by if you have any more questions! https://support.dotmailer.com/forums
I have an app running where a socket connection is constantly maintained (using socket.io). Data that needs to be sent is similar to that which you might see in a chat application. Would it be better to have it sent through POST (essentially, post that data, prevent page redirect, and then return the new page state with websockets), or just send it through websockets? What are the advantages to each?
(You might want to explain what you're trying to accomplish in more detail. Do you want to implement chat-like functionality).
A WebSocket gives you a TCP-like connection protocol over an HTTP connection. It's full duplex and lets you push and pull content in both directions. The connection is initiated from HTTP which "upgrades" the connection type. It gives you flexibility with some added complexity. I don't think it works across old HTTP 1.0 proxies.
A simple HTTP POST is more brute force. Unless you use ajax-ish techniques it pushes data to a web service and responds with a new web page to replace whatever's in your browser.