I want write a program in Delphi to watch the internet connection, and if a certain response received (in response to request from a program), send request again encoded to another server, get a new encoded response, decode it, and pass it as response to the program who sent the main request. But I don't now how to hook internet connection. I want to use this program to pass through filter my country governments made using a private program to avoid blocking it. Is there any idea?
Thanks for your answer.
Magenta Systems released a free set of Delphi components that let you see the network traffic on your computer and examine the content. If you see the response you are looking for, your monitoring program can send a request to another server.
Off the top of my head, I'm not sure if it will let you alter the content of the original packet. If not, then Marcus' suggestion of using a proxy might suit you better.
You can either try to hook stuff at the Winsock level (there's plenty of examples for that around), but I suggest you go one level deeper and use a Layered Service provider (LSP).
I have used Komodia's redirector from http://www.komodia.com. Commercial, but well worth it.
See also this post
Is it possible to intercept dns queries using LSP/SPI?
Related
In a World of Warcraft Vanilla Lua Addon Development, how can I issue an HTTP call to receive data back? If not, how can I get data from a web source into the game while playing?
I have a feeling the answer is tragically short, but would like the question asked and answered on Stack Overflow. My research came up lacking, and I recall doing some LUA in ~2007 and was disappointed.
Well, tragically short is an understatement. You simply can't. There was never any APIs that interacted directly with connections, let alone create any, let alone to arbitrary URLs.
Most of them just broadcast game events that occur from the game's connection, and the closest thing you can get to a "data stream" is add-on chat channels. But since bots violate the ToS, you wouldn't be able to make an account that responds to your addon's inquiries.
The closest thing you can get is building an "asynchronous mesh network", but that's only good if your addon has a considerable user base, and it's not guaranteed you'll get information timely.
The general idea is that your addon will have a public key (as in encryption), and you (only you) will detain a private key. Your addon emits a message to any connected peers, which store it on cross-realm SavedVariables, and you hope that someone will have characters on more than one realm. Upon login, the client addon will broadcast its latest packet (still encrypted) to that realm's addon channel, and hopefully within a week or so you might get the updated information across all clients.
A disadvantage is that you'll only get "push" notifications, the client won't be able to send any data back to you*.
That, or you could release a patch for the addon on Curse :P
BUT WAIT!
You mention vanilla, so I can presume you're developing this for a private server. Private servers often have one or a very small amount of realms, making the above mesh network much simpler. Instead of a mesh, just have encryption and manually login&broadcast on each realm every time you want to update the information retrieved.
Plus, you might even be able to contact the server devs to allow you an API that sends messages to the appropriate ingame addon channel (you'd have to ask).
Of course, if you pretend to make your addon server-agnostic, instead of tailored to a specific server, you're back to square one.
* Unless you are really dedicated to make that happen, because it's a ton of work.
There is no web API in vanilla WoW. There is a web browser widget in the game currently though, albeit very limited in usage.
If you have access to the server software code, you may be able to hook listening on specific game channels for user messages in a defined format, and let the server respond in a way for the addon to parse it.
there is similar question (how to process a HTTP stream with Dart) about processing streams in dart2js. I am focused on vm.
I can read from spec that:
The content of message can be: primitive values (null, num, bool, double, String), instances of SendPort, and lists and maps whose elements are any of these. List and maps are also allowed to be cyclic.
In the special circumstances when two isolates share the same code and are running in the same process (e.g. isolates created via spawn), it is also possible to send object instances (which would be copied in the process). This is currently only supported by the dartvm. For now, the dart2js compiler only supports the restricted messages described above.
I ve learnt I cannot send to isolates and back following objects: HttpRequest and HttpResponse objects, I cannot send streams.
Q: I cannot really understand how should I process big chunk of data in isolate and then send it back to main isolate and in turn it can be send back to client.
Normally if I want to read a file. I can obtain a stream, apply transforms and then pipe a stream to http response. what is best practice to do that using isolates?
thanks for help!
I have provided an example but Ill give a quick overview on how you can achieve this although it may not necessarily be the best practise - its simply the only way I personally know how.
We are given a method called Isolate.spawnUri(Uri uri, List<String> args, dynamic message)
The message parameter can hold any of the values the extract in your first post mentions. What we want to do is in the main thread create a new ReceivePort and listen for incoming data, then we want to spawn an isolate with the message as the ReceivePorts .sendPort.
The isolate should then create its own ReceivePort and use the message value to send back its own sendPort and listen on its receive port. What this essentially does is creates a 2 way communication. Allowing us to keep the isolate alive and send work back and forth.
Any response from your isolate will come through your original receive port and any data you want to send to the isolate will be through the send port the isolate just sent back.
You can then directly send the stream data to the isolate for it to process at will and it can send back data as its made (or you can just close the ports after isolate has sent its response - up to you how you want to go about it).
I have provided an example on GitHub:
https://github.com/TomCaserta/ExampleIsolate
Please note, if you want your isolates to work in dart2js its important that you run dart2js on each of your isolate files via the following command:
dart2js -o <filename>.dart.js <filename>.dart
RFC 3501 states in section 6.1.2. that you should use the NOOP command for polling.
Though in TIdIMAP4 there's only the KeepAlive method using it, which is implemented as a procedure, i.e. doesn't return anything.
So how to check for status updates like e.g. new messages or read status changes? I.e. how can I do manual polling with TIdIMAP4? Which methods and properties are involved in doing that? And how to get the (U)IDs these messages?
Or is it even possible to use the IDLE command specified in RFC 2177 to avoid polling and to get updates automatically?
IMAP is technically an asynchronous protocol, but TIdIMAP4 is currently implemented as a synchronous client. As such, unexpected/out-of-order data is either discarded, treated as untagged data, or treated as error data, depending on timing and context. Untagged/extra data is accessible from the TIdIMAP4.LastCmdResult property, which you can type-cast to TIdReplyIMAP4 to access its Extra sub-property.
IDLE is not currently supported in TIdIMAP4. There are tickets in Indy's issue trackers (see here and here) to add IDLE support in a future release, maybe in Indy 11. Until then, you will have to poll the mailbox envelopes periodically, keeping track of messages you have already seen so you can detect new messages.
Yes, you can use IDLE to avoid NOOP and in general it's a good idea.
However, that won't give you any results. In a way, IMAP commands don't have results. They tell the server what you want, and the server tells you things. The server is free to tell you things for other reasons as well, including the goodness of its heart.
You might say that NOOP means "hi server, now is a good time to tell me things, I'm listening" and IDLE means "hi server, I'm listening all the time, so just tell me whatever you want whenever you want". Both also mean "and btw, restart your inactivity timeout if you have one".
The server will send you EXISTS, FETCH and other responses, which I expect TIdIMAP4 forwards to you in some way. (Yes, they're called responses even though they're not in response to any command of yours. They may be sent in response to someone else having sent you mail, for instance. Stupid naming.)
Can any one tell me which is more stable? I know each has their own advantages and disadvantages. But which one is better for http, etc?
In my previous application I used indy9 but I wasn't satisfied with it, as I would sometimes get strange errors.
Can anyone recommend one?
I use Indy in a lot of projects. I used both 9 and 10 mainly as HTTP server and proxy. The projects get very intense traffic at times (HTTP). Indy never did let me down. It works very stable.
But I also had some "strange" situations where I had to dig deep to find the underlying problem. I also do not like the way Indy tends to handle a lot of things through exceptions. In general I like the ICS coding style more. But let me go to ICS.
ICS uses non-blocking sockets, while indy uses blocking. While non-blocking is ok and seems to be better at first sight, I found it irritating in a lot of situations. The problem is that the natural flow of the code gets lost because of the callback functions. This makes it harder to write procedural type of libraries. Furthermore I do not like how everything is handled through messages. For me it gets messy real quick when mixed with multithreading. And multithreading is mainstream these days.
So while I like the coding style and quality of the code in ICS, I prefer the simplicity of use and blocking mode of Indy. What you like more is up to you, but both libraries are mature and stable.
These are my two cents.
I also use both Indy and ICS.
Most of the time I prefer Indy because implementing sequential type of protocols with it is very easy (the request runs in it's own thread so you simply Read/Write to the connection, really easy). Using Indy requires solid knowledge of threading and synchronization. Unlike Runner I like how Indy uses Exceptions to handle "exceptional" stuff because it allows me to concentrate on the normal flow of the protocol (I use try-finally blocks to make sure I deallocate resources).
I also used ICS in a application where Indy simply failed: I used it for an application that implements an TCP/IP proxy. Using ICS was simpler because of it's non-blocking nature. I was able to "proxy" TCP/IP protocols that I know nothing about, so I have no idea how bytes would flow from one end to the other. Indy failed in that scenario because in Indy you're ether reading or you're writing, you can't do both at the same time. Using ICS to implement an sequential-type protocol is a bit of pain: you essentially need to use state-machine logic, brake the protocol in small bits, keep flags laying around so you know where you are in the protocol. An big plus: François Piette, the author of ICS, is active and very helpful on a number of forums and mailing list, and is very prompt to help with anything related to ICS.
For me, if I need to do something with TCP/IP, the decision path is very simple: Can it be done with Indy? Then it's Indy. If it can't be done with Indy then it'll be done with ICS!
I've used Indy 9 and 10 for TCP, HTTP and FTP with very few problems. ICS is a good choice, too. It's non blocking, which will change how you use it.
I haven't used it, but I've heard good things about Synapse, which is also blocking.
We test Indy10 IdTCPClient to receive video stream from remote server, it's OK. But when it's receiveing stream, the same time use it send some data to the server, after some minute, the received stream data began lost data bytes. We use sniffer tool to trace this problem, confirmed that IdTCPClient lost some bytes in receiveing stream.
So, we test Indy9.018, the same problem happend but a few times VS. Indy 10.
Remember that Indy is always in beta stage. Sometimes you need to work with night builds.
Which one is better really depends on specific use case, but I was unhappy with indy as an http client and ICS ended up being exactly what I needed it to be, with a lot less random quirks.
Note though that it is non blocking, so it's not just a drop in replacement.
I use Indy 9 for stable, released, production code to over 1 million users, and have never received any strange errors.
I'd say the answer depends on what you want to do with the internet. Indy is fine if you are prepared to get involved with understanding how it works, and is very capable. ICS is a different take on things, and I've used it effectively for many-connection systems. But for day to day "grab a file or send an email" type stuff, where you want to do a basic task, I pretty much always use Clever Components Internet Suite as you just create the component,
set the options, and it works. The suite is quite comprehensive, and gets useful updates.
We have a requirement from a customer to provide a "lite" version for dial-up and all the bells-and-whistles for a broadband user.
The solution will use Flex / Flash / Java EJB and some jsp.
Is there a way for the web server to distinguish between the two?
You don't care about the user's connection type, you care about the download speed.
Have a tiny flash app that downloads the rest the of the flash, and times how long it takes. Or an HTML page that times how long an Ajax download takes.
If the download of the rich-featured app takes too long, have the initially downloaded stub page/flash redirect to the slow download page (or download the bare-bones flash app, or whatever).
The simplest and most reliable mechanism is probably to get the user to select their connection type from a drop down. Simple, I know, but it may save you a world of grief!
There's no way to distinguish between a broadband or dial-up as a connection type, but you can make an educated guess by connection speed.
Gmail does this and provides a link to a basic HTML version of their service if they detect it.
(source: nirmaltv.com)
My guess is that there is some client side javascript polling done on AJAX requests. If the turnaround time surpasses a threshold, the option to switch to "lite" appears.
The best part about this option is that you allow the user to choose if they want to use the lite version instead of forcing them.
Here's a short code snippet from a code who attempted something similar. It's in C#, but it's pretty short and it's just the concept that's of interest.
Determine the Connection Speed of your client
Of course, they could be a temporary speed problem that has nothing to do with the user's connection at the time you test, etc, etc.
I had a similar problem a couple of years ago and just let the user choose between the hi and lo bandwidth sites. The very first thing I loaded on the page was this option, so they could move on quickly.
I think the typical approach to this is just to ask the user. If you don't feel confidant that your users will provide an accurate answer, I suspect you'll have to write an application that runs a speed test on the client. Typically these record how long it takes the client to receive x number of bytes, and use that to determine bandwidth.
Actionscript 3 has a library to help you with this task, but I believe it requires you to deploy your flex/flash app on Flash Media Server. See ActionScript 3.0 native bandwidth detection for details.
#Apphacker (I'd comment instead of answering if I had enough reputation...):
Can't guarantee the reverse, either--I have Earthlink dial-up, soon to upgrade to Earthlink DSL (it's what's available here...).
You could check their IP and see if it resolves to/is assigned to a dial up provider, such as AOL, Earthlink, NetZero. Wouldn't guarantee that those that don't resolve to such a provider are broadband users.
you could ...
ask the user
perform a speed test and ask the user if the result you found is correct
perform a speed test and hope that the result found is correct
I think a speed test should be enough.
If you only have a small well known user group it is sometimes possible to determine the connection speed by the ip. (Some providers assign different subnets to dial-up/broadband connections)