Could someone explain when is recv() method returning 0? Of course buffer longer than 0 bytes is provided. Method is blocking, so it should wait until it recieves something or throw an error and return -1. I have flag - 0 set. It is not indicating non-blocking mode.
MSDN recv Function:
... If the connection has been gracefully closed, the return value is
zero.
recv(2) - Linux man page:
... The return value will be 0 when the peer has performed an
orderly shutdown.
Related
I am still confused about the NumberOfConcurrentThreads parameter within CreateIoCompletionPort(). I have read and re-read the MSDN dox, but the quote
This value limits the number of runnable threads associated with the
completion port.
still puzzles me.
Question
Let's assume that I specify this value as 4. In this case, does this mean that:
1) a thread can call GetQueuedCompletionStatus() (at which point I can allow a further 3 threads to make this call), then as soon as that call returns (i.e. we have a completion packet) I can then have 4 threads again call this function,
or
2) a thread can call GetQueuedCompletionStatus() (at which point I can allow a further 3 threads to make this call), then as soon as that call returns (i.e. we have a completion packet) I then go on to process that packet. Only when I have finished processing the packet do I then call GetQueuedCompletionStatus(), at which point I can then have 4 threads again call this function.
See my confusion? Its the use of the phrase 'runnable threads'.
I think it might be the latter, because the link above also quotes
If your transaction required a lengthy computation, a larger
concurrency value will allow more threads to run. Each completion
packet may take longer to finish, but more completion packets will be
processed at the same time.
This will ultimately affect how we design servers. Consider a server that receives data from clients, then echoes that data to logging servers. Here is what our thread routine could look like:
DWORD WINAPI ServerWorkerThread(HANDLE hCompletionPort)
{
DWORD BytesTransferred;
CPerHandleData* PerHandleData = nullptr;
CPerOperationData* PerIoData = nullptr;
while (TRUE)
{
if (GetQueuedCompletionStatus(hCompletionPort, &BytesTransferred,
(PULONG_PTR)&PerHandleData, (LPOVERLAPPED*)&PerIoData, INFINITE))
{
// OK, we have 'BytesTransferred' of data in 'PerIoData', process it:
// send the data onto our logging servers, then loop back around
send(...);
}
}
return 0;
}
Now assume I have a four core machine; if I leave NumberOfConcurrentThreads as zero within my call to CreateIoCompletionPort() I will have four threads running ServerWorkerThread(). Fine.
My concern is that the send() call may take a long time due to network traffic. Hence, I could be receiving a load of data from clients that cannot be dequeued because all four threads are taking a long time sending the data on?!
Have I missed the point here?
Update 07.03.2018 (This has now been resolved: see this comment.)
I have 8 threads running on my machine, each one runs the ServerWorkerThread():
DWORD WINAPI ServerWorkerThread(HANDLE hCompletionPort)
{
DWORD BytesTransferred;
CPerHandleData* PerHandleData = nullptr;
CPerOperationData* PerIoData = nullptr;
while (TRUE)
{
if (GetQueuedCompletionStatus(hCompletionPort, &BytesTransferred,
(PULONG_PTR)&PerHandleData, (LPOVERLAPPED*)&PerIoData, INFINITE))
{
switch (PerIoData->Operation)
{
case CPerOperationData::ACCEPT_COMPLETED:
{
// This case is fired when a new connection is made
while (1) {}
}
}
}
I only have one outstanding AcceptEx() call; when that gets filled by a new connection I post another one. I don't wait for data to be received in AcceptEx().
I create my completion port as follows:
CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 4)
Now, because I only allow 4 threads in the completion port, I thought that because I keep the threads busy (i.e. they do not enter a wait state), when I try and make a fifth connection, the completion packet would not be dequeued hence would hang! However this is not the case; I can make 5 or even 6 connections to my server! This shows that I can still dequeue packets even though my maximum allowed number of threads (4) are already running? This is why I am confused!
the completion port - is really KQUEUE object. the NumberOfConcurrentThreads is corresponded to MaximumCount
Maximum number of concurrent threads the queue can satisfy waits for.
from I/O Completion Ports
When the total number of runnable threads associated with the
completion port reaches the concurrency value, the system blocks the
execution of any subsequent threads associated with that completion
port until the number of runnable threads drops below the concurrency
value.
it's bad and not exactly said. when thread call KeRemoveQueue ( GetQueuedCompletionStatus internal call it) system return packet to thread only if Queue->CurrentCount < Queue->MaximumCount even if exist packets in queue. system not blocks any threads of course. from another side look for KiInsertQueue - even if some threads wait on packets - it activated only in case Queue->CurrentCount < Queue->MaximumCount.
also look how and when Queue->CurrentCount is changed. look for KiActivateWaiterQueue (This function is called when the current thread is about to enter a wait state) and KiUnlinkThread. in general - when thread begin wait for any object (or another queue) system call KiActivateWaiterQueue - it decrement CurrentCount and possible (if exist packets in queue and became Queue->CurrentCount < Queue->MaximumCount and threads waited for packets) return packet to wait thread. from another side, when thread stop wait - KiUnlinkThread is called. it increment CurrentCount.
your both variant is wrong. any count of threads can call GetQueuedCompletionStatus(). and system of course not blocks the execution of any subsequent threads. for example - you have queue with MaximumCount = 4. you can queue 10 packets to queue. and call GetQueuedCompletionStatus() from 7 threads in concurrent. but only 4 from it got packets. another will be wait (despite yet 6 packets in queue). if some of threads, which remove packet from queue begin wait - system just unwait and return packet to another thread wait on queue. or if thread (which already previous remove packet from this queue (Thread->Queue == Queue) - so active thread) again call KeRemoveQueue will be Queue->CurrentCount -= 1;
I am using IO completion ports and AcceptEx() whilst learning about servers, and am studying Len Holgate's free server framework to do this. He has the following code:
// Basically calls AcceptEx() via a previously obtained function pointer
if (!CMSWinSock::AcceptEx(
m_listeningSocket,
pSocket->m_socket,
reinterpret_cast<void*>(const_cast<BYTE*>(pBuffer->GetBuffer())),
bufferSize,
sizeOfAddress,
sizeOfAddress,
&bytesReceived,
pBuffer))
{
const DWORD lastError = ::WSAGetLastError();
if (ERROR_IO_PENDING != lastError)
{
Output(_T("CSocketServerEx::Accept() - AcceptEx: ") + GetLastErrorMessage(lastError));
pSocket->Release();
pBuffer->Release();
}
}
else
{
// Accept completed synchronously. We need to marshal the data recieved over to the
// worker thread ourselves...
m_iocp.PostStatus((ULONG_PTR)m_listeningSocket, bytesReceived, pBuffer);
}
I am confused about the "Accept completed synchronously" else-case. I have tried many times to get this code path to be hit (by pausing the code before I issue the AcceptEx, connecting, then resuming the code), but whenever I try the call always fails with ERROR_IO_PENDING and I get my notification packet. Furthermore, I have read this MS knowledgebase article (which I may have misinterpreted) which states
Additionally, if a Winsock2 I/O call returns SUCCESS or IO_PENDING, it
is guaranteed that a completion packet will be queued to the IOCP when
the I/O completes
However, I am thinking this doesn't apply to AcceptEx() because the dox for AcceptEx() states of the parameter lpdwBytesReceived
This parameter is set only if the operation completes synchronously.
So it seems it can complete synchronously...can someone tell me how AcceptEx() can complete synchronously (i.e. how I can replicate it in my server?)
Additionally, if a Winsock2 I/O call returns SUCCESS or ERROR_IO_PENDING, it
is guaranteed that a completion packet will be queued to the IOCP when
the I/O completes
this is apply for any I/O request if completion port is associated with the file. but begin from windows vista this also depend from notification mode set for a file handle.
but need first begin look from native view.
by default, if FILE_SKIP_COMPLETION_PORT_ON_SUCCESS not set, exist 3 case by returned NTSTATUS status :
NT_SUCCESS(status) or status >= 0 - will be completion
NT_ERROR(status) or status >= 0xc0000000 - will be no completion
NT_WARNING(status) or status < 0xc0000000 - unclear - if this
error from I/O manager (say - STATUS_DATATYPE_MISALIGNMENT - will
be no completion). if this error from driver (say
STATUS_NO_MORE_FILES - will be completion).
the win32 layer usually separate check for STATUS_PENDING and return ERROR_IO_PENDING in this case (but exist and exceptions, like ReadDirectoryChangesW). otherwise in case NT_ERROR(status) api return fail and set error code. otherwise return success. visible that case NT_WARNING(status) considered as success, but in this case, if error from I/O manager, will be no completion. I/O usually return errors from NT_ERROR(status) range, if parameters is incorrect. only case which i know (for asynchronous api) - STATUS_DATATYPE_MISALIGNMENT can be returned in case wrong aligned buffers, when I/O manager have special knowledge about buffer align. in NtNotifyChangeDirectoryFile
(ReadDirectoryChangesW for win32) or NtQueryDirectoryFile (no corresponded win32 api). so only case which i know when will be no completion, when win32 return success - call ReadDirectoryChangesW with unaligned lpBuffer (it must be DWORD-aligned ) - in this case I/O manager just return STATUS_DATATYPE_MISALIGNMENT but win32 layer interpret this as success code and return true. but will be no completion in this case. however this is rarely case and you probably need use wrong align structures for this. so in general yes:
by default if I/O call returns SUCCESS or ERROR_IO_PENDING will be queued a completion entry to the port. (with special exception case which i try describe)
if we set FILE_SKIP_COMPLETION_PORT_ON_SUCCESS on file object (note this is per file object but not per file handle - documentation not exactly here) all become much more simply and efficient - completion entry will be queue to the port - when and only when I/O request return STATUS_PENDING. ERROR_IO_PENDING from win32 view (except ReadDirectoryChangesW (maybe some else api ?) where win32 layer simply lost return code information)
However, I am thinking this doesn't apply to AcceptEx()
you mistake. this, how i say, apply to any io request. "This parameter is set only if the operation completes synchronously." - and so what ?
if look to code snippet, clear visible that code assume - in case AcceptEx completed synchronous and no error occurs - will be no io completion. or SetFileCompletionNotificationModes(m_listeningSocket, FILE_SKIP_COMPLETION_PORT_ON_SUCCESS) called or code is wrong - will be io completion in this case and not need m_iocp.PostStatus - this is fatal error will be. however i doubt that code used FILE_SKIP_COMPLETION_PORT_ON_SUCCESS - so it wrong. but error never raised because driver side implementation of AcceptEx (underlining ioctl) never return STATUS_SUCCESS: it check parameters - if it wrong - just return some error, otherwise always return STATUS_PENDING. as result, for asynchronous sockets AcceptEx never return true and code never jump to error else case. but anyway code is wrong. also i think design not the best - in case we determinate will be no completion - better just direct call completion routine with returned error code instead Release() (this will be done in completion routine) or PostStatus - for what post ?! - call direct.
how AcceptEx() can complete synchronously
very easy - if m_listeningSocket is handle to synchronous file object. however in this case you can not bind IOCP to file (it can be bind only in case asynchronous file object).
about lpdwBytesReceived parameter - system copy Information member of IO_STATUS_BLOCK or if want OVERLAPPED.InternalHigh, in case operation is completed just. in case pending returned - this data simply not ready and not filled. you got actual number of bytes returned by io in completion
I have input and output NSStream's as part of a TCP connection after using NSStream.getStreamsToHostWithName(). If I call the close() method on those input and output streams, then by the time the functions return, will my TCP connection be in the CLOSED state?
If not, how could I determine the time at which the underlying TCP connection actually closes?
Closing the stream terminates the flow of bytes and releases system
resources that were reserved for the stream when it was opened. If the
stream has been scheduled on a run loop, closing the stream implicitly
removes the stream from the run loop. A stream that is closed can
still be queried for its properties.
var streamStatus: NSStreamStatus { get }
The receiver’s status
enum NSStreamStatus : UInt {
case NotOpen
case Opening
case Open
case Reading
case Writing
case AtEnd
case Closed
case Error
}
As far as I can tell, the answer is no, the TCP connection will not necessarily be in the CLOSED state. To make sure the TCP connection does a graceful close and find out when that happens, one must use a lower-level API than NSStream.
if I add an event for a connection socket which is returned by accept(), as below
event_set(&conn_ev, connfd, EV_READ|EV_PERSIST, on_recv, NULL);
event_base_set(base, &conn_ev);
event_add(&conn_ev, NULL);
if at sometime, the local program(not the peer) closes the socket, will the conn_ev be triggered?
if so, how to detect whether whether the event is due to the closing of the socket?
is it recv(connfd,..) returns -1 and errno is set EBAD or any other cases?
thanks!
All sockets are marked as readable if the socket is nicely closed by the other end, with read returning zero. When an error is received they are marked either read- or write-able, with read or write returning -1.
See e.g. the socket(7) manual page for a table of states.
I use file descriptors to find the readable sockets and go on to read. For some reasons, a socket that has no data on the wire, goes on to read and never returns. Is there a way I can come out of the receive after a timeout?
I am using winsock library..
http://tangentsoft.net/wskfaq/newbie.html#timeout
2.15 - How can I change the timeout for a Winsock function?
Some of the blocking Winsock functions (e.g. connect()) have a timeout embedded into them. The theory behind this is that only the stack has all the information necessary to set a proper timeout. Yet, some people find that the value the stack uses is too long for their application; it can be a minute or longer.
You can adjust the send() and recv() timeouts with the SO_SNDTIMEO and SO_RCVTIMEO setsockopt() options. .
For other Winsock functions, the best solution is to avoid blocking sockets altogether. All of the non-blocking socket methods provide ways for you to build custom timeouts:
Non-blocking sockets with select() – The fifth parameter to the select() function is a timeout value.
Asynchronous sockets – Use the Windows API SetTimer().
Event objects – WSAWaitForMultipleEvents() has a timeout parameter.
Waitable Timers – Call CreateWaitableTimers() to make a waitable timer, which you can then pass to a function like WSAEventSelect() along with your sockets: if none of the sockets is signalled before the timer goes off, the blocking function will return anyway.
Note that with asynchronous and non-blocking sockets, you may be able to avoid handling timeouts altogether. Your program continues working even while Winsock is busy. So, you can leave it up to the user to cancel an operation that’s taking too long, or just let Winsock’s natural timeout expire rather than taking over this functionality in your code.
your problem is in the while loop that you try to fill buffer
just put an if statement that check last index of every chunks for '\0'
then break your while loop
do {
len = rcv(s,buf,BUF_LEN,0);
for (int i = 0; i <= len; ++i) {
if (buf[i] >= 32 || buf[i] == '\n' || buf[i] == '\r') { //only write valid chars
result += buf[i]; //final string
}
}
if (buf[len] == '\0') { //buf[len] is the last position in the received chanks
break;
}
} while (inner_result > 0);