This my be a silly question, but I'm trying to use non-blocking sockets for the first time,but what I noticed is that when I run the server program it imediately gives me error in accept()(WSAEWOULDBLOCK, what i have read, this is an usual thing). My question is then,how can I connect to the server then ?
I am using ioctlsocket in Windows. Thanks.
By using non-blocking sockets, accept() will immediately return if there's no client connection waiting. You need to check if the error is WSAEWOULDBLOCK, ignore it if it is, and use a polling loop to check again later. The select function may be useful.
Related
I've done a lot of reading lately about calling my C code from Erlang. My C code will be making a remote call and I expect to have a timeout of ~10 seconds. In order to keep Erlang responsive and handle additional requests while someone else is waiting for the initial call to C, what are the best practices in 2022?
We can use async port drivers or a NIF with ERL_NIF_DIRTY_JOB_IO_BOUND to run it on the dirty IO scheduler.
My understanding is that the async port driver would be the best option, but looking for more insight here.
Let's say we have an Erlang web server handling two callers A and B. A makes a call which then enters into C and blocks for 10 seconds. While that is waiting, B makes their call which will also need to call C. With an async port driver and a dirty IO NIF what are the differences in execution and blocking? Will B get blocked in Erlang, waiting for the port driver or NIF to be ready to start working on this new request (if that's a thing), or waiting for an available thread (as I understand the async port driver will use its own single thread (unless I do something else to spawn another thread)? And the NIF would use a threadpool).
If we have even more callers calling in where will we bottleneck?
As you can see I have quite a few questions and would highly appreciate help from those with Erlang familiarity!
I have a application with client and server sockets using Indy, compiled with Delphi 10.2.
The application have a working thread that processes requests from the different ports and write responses within the same thread using a call to something like this:
procedure TMyCommManager.WriteResponse(AHandler: TIdIOHandler; SomeData: SomeType);
var
idBytes: TidBytes;
PacketSize: Integer;
begin
SomeData.GetBytes(idBytes, PacketSize);
AHandler.Write(DataBuffer, PacketSize);
end;
Almost all the time everything is working as expected, but we noticed the worker thread is freezing from time to time in production. After various iterations, we finally got to the point that all this is occurring in the call to the TidIOHandler.Write() and I'm quite sure it is happening because the other end for a single port is not reading the responses from the socket.
After a reset to the port from within the other end the worker thread unfreezes and keeps working as expected.
I found this answer from Remy Lebeau to the question Delphi (Indy) Server Freezing on Write, where he mentions in comments (emphasis mine):
Indy uses blocking sockets, so if the client is not reading inbound data on its end, eventually the socket's internal send buffer will fill up and the socket will become blocked on the server side waiting for the client to empty the buffer. The only way to avoid a deadlock in that scenario would be to set a socket-level send timeout using the socket API directly. Indy does not implement send timeouts in its logic.
I'm looking for the correct way to set that timeout, via INDY or via a direct API call in Windows, but I'm stuck in that, so I come here looking for help.
If that's not possible, I could implement a Timeout mechanism on a secondary thread, but I'm not sure what's the correct way to reset the connection from this secondary thread on my end to let the worker thread continue it's work.
I'm looking for the correct way to set that timeout, via INDY or via a direct API call in Windows
In my previous comment, when I said "set a socket-level send timeout using the socket API directly", I was refering to the SO_SNDTIMEO socket option via the setsockopt() function.
In terms of Indy, you can call setsockopt() by using the TIdSocketHandle.SetSockOpt() method, eg:
// Windows
SomeConnection.Socket.Binding.SetSockOpt(Id_SOL_SOCKET, Id_SO_SNDTIMEO, TimeoutInMS);
// 'Nix
var tv: timeval;
...
SomeConnection.Socket.Binding.SetSockOpt(Id_SOL_SOCKET, Id_SO_SNDTIMEO, Integer(#timeval));
or:
GBSDStack.SetSocketOption(SomeConnection.Socket.Binding.Handle, Id_SOL_SOCKET, Id_SO_SNDTIMEO, timeval, sizeof(timeval));
(the TIdTCPConnection.Socket property is a shorthand for accessing TIdIOHandlerSocket when the TIdTCPConnection.IOHandler has been assigned a TIdIOHandlerSocket or descendant).
Just know that if a timeout does occur, you can't know exactly which data was actually transmitted to the peer, so recovery is usually impossible in most protocols. All you can really do is just close the connection and reconnect.
We have an application server developed with Delphi 2010 and Indy 10. This server receives more than 50 requests per second and it works well. But in some cases, it seems to me that Indy is very obscure. Their components are good, but sometimes I found myself digging into the source code only to understand a simple thing. Indy lacks on good documentation and good support.
The last thing that i came across was a big problem for me: I must detect when a client disconnects non gracefully (When the the client crashes or shutdown, for instance. Not telling the server that it will disconnect) and indy was not able to do that. If I want that, I will have to develop a algorithm like heartbeat, pooling or TCP keep-alive. I do not want to spend more time doing a, at least I think, component job. After a few study, I found out that this is not Indy's fault, but this is an issue of all blocking sockets components.
Now I am really thinking of changing the core of the Server to another good suite. I must admit I am tending to use a non-blocking socket. Based on that, I have some questions:
What do a benefit from changing from blocking to non-blocking sockets?
Will I be able to detect client disconnects (non gracefully)?
What component suite has the best product? By best product I mean: fast, good support, good tools and easy to implement.
I know this must be a subjective question, but I really want to hear that from you. My first question is the one I care most. I do not care if I have to pay 100, 500, 1000, 10000 dollars, but I want a complete solution. For now, I am thinking about Ip*works .
EDIT
I think some guys are not understand what I want. I don't want to create my own socket. I have been working with sockets for a long time and I am getting tired of it. Really.
And non-blocking sockets CAN detect client disconnects. That is a fact and it has good documentation all over the internet. A non-blocking socket checks the socket state for new incoming data all the time, and it makes possible to detect that the socket is not valid. This is not a heartbeat algorithm. A heartbeat algorithm is used on client side and it sends periodically packets (aka keep-alive) to the server to tells it is still alive.
EDIT
I am not make myself clear. Maybe because English is not my main language. I am not saying that it is possible to detect a dropped connection without trying to send or receiving data from a socket. What I am saying is that every non-blocking socket is able to do that because they constantly tries to read from the socket for new incoming data. Why is that so hard to understand? If you guys download and run ip*works demos, in special, the echoserver and echoclient ones (both use TCP) you can test by yourselves. I already tested it, and it works like I expected to do. Even if you use the old TCPSocketServer and TCPSocketClient in a non-blocking mode you will see what I meant.
"What do a benefit from changing from blocking to non-blocking sockets? Will I be able to detect client disconnects (non gracefully)?"
Just my two cents to get the ball rolling on this question - I'm not a socket EXPERT, but I do have a good deal of experience with them. If I'm mistaken, I'm sure someone will correct me... :-)
I assume that since you're running a server using blocking sockets with 50 connections per second, you have a threading mechanism in place to handle client requests. If so, you don't really stand to gain anything from non-blocking sockets. On the contrary - you will have to change your server logic to be event driven- based on events fired in your main thread from the non-blocking sockets, or use constant polling to know what your sockets are up to.
Non-blocking sockets can't detect clients disconnecting without notification any more than blocking sockets can - they don't have telepathic powers... The nature of the TCP/IP 'conversation' between client and server is the same - blocking and non-blocking is only with respect to your application's interaction with the socket connection conducting the 'conversation'.
If you need to purge dead connections, you need to implement a heartbeat or timeout mechanism on your socket (I've never seen a modern socket implementation that didn't support timeouts).
What do a benefit from changing from blocking to non-blocking sockets?
Increased speed, availability, and throughput (from my experience). I had an IndySockets client that was getting about 15 requests per second and when I went directly to asynchronous sockets the throughput increased to about 90 requests per second (on the same machine). In a separate benchmark test on a server at a data-center with a 30 Mbit connection I was able to get more than 300 requests per second.
Will I be able to detect client disconnects (non gracefully)?
That's one thing I haven't had to try yet, since all of my code has been on the client side.
What component suite has the best product? By best product I mean: fast, good support, good tools and easy to implement.
You can build your own socket client in a couple of days and it can be very robust and fast... much faster than most of the stuff I've seen "off the shelf". Feel free to take a look at my asynchronous socket client: http://codesprout.blogspot.com/2011/04/asynchronous-http-client.html
Update:
(Per Mikey's comments)
I'm asking you for a generic, technical explanation of how NBS increase throughput as opposed to a properly designed BS server.
Let's take a high load server as an example: say your server is supposed to handle 1000 connections at any given time, with blocking sockets you would have to create 1000 threads and even if they're mostly idle, the CPU will still spend a lot of time context switching. As the number of clients increases you will have to increase the number of threads in order to keep up and the CPU will inevitably increase the context switching. For every connection you establish with a blocking socket, you will incur the overhead of spawning of a new thread and you eventually you will incur the overhead of cleaning up after the thread. Of course, the first thing that comes to mind is: why not use the ThreadPool, you can reuse the threads and reduce the overhead of creating/cleaning-up of threads.
Here is how this is handled on Windows (hence the .NET connection): sure you could, but the first thing you'll notice with the .NET ThreadPool is that it has two types of threads and it's not a coincidence: user threads and I/O completion port threads. Asynchronous sockets use the IO completion ports which "allows a single thread to perform simultaneous I/O operations on different handles, or even simultaneous read and write operations on the same handle."(1) The I/O completion port threads are specifically designed to handle I/O in a much more efficient way than you would ever be able to achieve if you used the user threads in ThreadPool, unless you wrote your own kernel-mode driver.
"The completion port uses some special voodoo to make sure only a specific number of threads can run at once — if one thread blocks in kernel-mode, it will automatically start up another one."(2)
There are other advantages also: "in addition to the nonblocking advantage of the overlapped socket I/O, the other advantage is better performance because you save a buffer copy between the TCP stack buffer and the user buffer for each I/O call." (3)
I am using Indy and Synapse TCP libraries with good results for some years now, and did not find any showstoppers in them. I use the libraries in threads - client and server side, stability and performance was not a problem. (Six thousand request and response messages per second and more with the server running on the same system are typical.)
Blocking sockets are very useful if the protocol is more advanced than a simple 'send a string / receive a string'. Non-blocking sockets cause a higher coupling of message protocol handlers with the socket read / write logic, so I quickly moved away from non-blocking code.
No library can overcome the limitations of the TCP/IP protocol regarding detection of connection loss. Only trying to read or send data can tell wether the connection is still present.
In Windows, there is a third option which is overlapped I/O. Non-blocking sockets are essential a model using Windows messages developed to avoid single-threaded GUI apps to become "blocked" while waiting for data. A modern application IMHO would be better designed using threads and overlapped I/O.
See for example http://support.microsoft.com/kb/181611
Aahhrrgghh - the myth of being able to always detect "dropped" connections. If you pull the power on a machine with a client connection then the server cannot tell, without sending data, that the connection is "dead". The is through the design of the TCP protocol. Don't take my word for it - read this article (Detection of Half-Open (Dropped) TCP/IP Socket Connections).
This article explains the main differences between blocking and non-blocking:
Introduction to Indy, by Chad Z. Hower
Pros of Blocking
Easy to program - Blocking is very easy to program. All user code can
exist in one place, and in a
sequential order.
Easy to port to Unix - Since Unix uses blocking sockets, portable code
can be written easily. Indy uses this
fact to achieve its single source
solution.
Work well in threads - Since blocking sockets are sequential they
are inherently encapsulated and
therefore very easily used in threads.
Cons of Blocking
User Interface "Freeze" with clients - Blocking socket calls do not
return until they have accomplished
their task. When such calls are made
in the main thread of an application,
the application cannot process the
user interface messages. This causes
the User Interface to "freeze" because
the update, repaint and other messages
cannot be processed until the blocking
socket calls return control to the
applications message processing loop.
He also wrote:
Blocking is NOT Evil
Blocking sockets have been repeatedly
attacked with out warrant. Contrary to
popular belief, blocking sockets are
not evil.
It is not is an issue of all blocking sockets components that they are unable to detect a client disconnect. There is no technical advantage on the side of non-blocking components in this area.
I am trying to simulate the "new identity" button in Vidalia (the Tor GUI) from my program. I asked about that, based on Rob Kennedy's answer, I tried this in my application:
IdTelnet1.Host:='127.0.0.1';
IdTelnet1.Port:=9051;
IdTelnet1.Connect(-1);
IdTelnet1.SendCmd('SIGNAL NEWNYM');
But it has not worked for me. Even after I send the command, I get the same proxy.
I am using Indy 9.
I don't know whether I don't know how to use TIdTelnet or don't know how to send that specific command.
You cannot use the SendCmd() method with TIdTelnet. TIdTelnet uses an internal reading thread that continuously reads from the socket (since Telnet is an asynchronous protocol that can receive data at any time). SendCmd() does its own internal reading to receive the sent command's response. The two reading operations interfer with each other (this issue also exists in Indy 10's TIdCmdTCPClient component for the same reason).
To send an outgoing command with TIdTelnet, you must use its SendCh() method to send each character individually (if you upgrade to Indy 10, TIdTelnet has a SendString() method whch handles that for you) and then wait until the OnDataAvailable event to process the response as needed.
Unless TOR is actually using the real Telnet protocol (Telnet sequences and all), then you are better off using TIdTCPClient instead of TIdTelnet. TIdTelnet is a Telnet-specific client, not a general purpose TCP/IP client like TIdTCPClient is.
I am at a point in my Erlang development where I need to create a C-Node (see link for C-Node docs). The basic implementation is simple enough, however, there is a huge hole in the doc.
The code implements a single threaded client and server. Ignoring the client for the moment... The 'c' code that implements the server is single threaded and can only connect to one erlang client at a time.
Launch EPMD ('epmd -daemons')
Launch the server application ('cserver 1234')
Launch the erlang client application ('erl -sname e1 -setcookie secretcookie') [in a different window from #2]
execute a server command ('complex3:foo(3).') from the erlang shell in #3
Now that the server is running and that a current erlang shell has connected to the server try it again from another window.
open a new window.
launch an erlang client ('erl -sname e2 -setcookie secretcookie').
execute a new server command ('complex3:foo(3).').
Notice that the system seems hung... when it should have executed the command. The reason it is hung is because the other erlang node is connected and that there are no other threads listening for connections.
NOTE: there seems to be a bug in the connection handling. I added a timeout in the receive block and I caught some errant behavior but I did not get them all. Also, I was able to get the cserver to crash without warnings or errors if I forced the first erlang node to terminate after the indicated steps were performed.
So the question... What is the best way to implement a threaded C-Node? What is a reasonable number of connections?
The cnode implementation example in the cnode tutorial is not meant to handle more than one connected node, so the first symptom you're experiencing is normal.
The erl_accept call is what accepts incoming connections.
if ((fd = erl_accept(listen, &conn)) == ERL_ERROR)
erl_err_quit("erl_accept");
fprintf(stderr, "Connected to %s\n\r", conn.nodename);
while (loop) {
got = erl_receive_msg(fd, buf, BUFSIZE, &emsg);
Note that, written this way, the cnode will accept only one connection and then pass the descriptor to the read/write loop. That's why when the erlang node closes, the cnode ends with an error, since erl_receive_msg will fail because fd will point to a closed socket.
If you want to accept more than one inbound connection, you'll have to loop accepting connections and implement a way to handle more than one file descriptor. You needn't a multithread programme to do so, it would probably be easier (and maybe more efficient) to use the poll or select syscall if your OS supports them.
As for the optimum number of connections, I don't think there is a rule for that, you'd need to benchmark your application if you want to support high concurrency in the cnode. But in that case it would probably be better to re-engineer the system so that erlang copes with the concurrency, alleviating the cnode from that.