Is there any MFC(memory function completed) signal generated after write operation is completed?
I am reading a book on computer organisation by Carl Hamacher, in this book it is written that a MFC signal is generated after read operation completes, but it is not clear in the text(pg no. 293) whether it is generated in case of completion of write operation also.
Memory function complete or MFC is just a signal that tells the cpu that the current operation involving the memory is complete and by that definition it is obvious that MFC signal will be generated in case of write operation also as in the case of read operation.
Related
This might sound quite basic and stupid but it has been bothering me for a while. How can print be classified in terms of operation - main or background ?
As a small test, on putting print in a background task - web service call :
Webservice().loadHeadlinesForSource(source: source) { headlines in
print("background print")
self.headlineViewModels = headlines.map(HeadlineViewModel.init)
DispatchQueue.main.async {
print("main thread print")
completion()
}
}
Both the print statements get printed. From previous experience, if print was a main thread task, Xcode would have given me a warning saying that I need to put that in main thread. This is an evidence that print is not a main thread operation. Note that I am not saying print is a background task.
However, I have this understanding that since print displays output on Console, it is not a background operation. As a matter of fact all logging operations are not.
How would one justify the classification ?
It seems what you consider to be a main thread operation is a call that needs to be performed on the main thread. From that perspective you are correct and have found an evidence of this call not being a main thread operation.
But does this have anything to do with anything else? Internally if needed this method may still execute its real operation on the main thread or any other thread for what we care. So in this sense a main thread operation is a restriction that call needs to be performed on main thread but has nothing to do with its execution or multithreading.
Without looking into what print does in terms of coding we can see that it works across multiple "computers". You can run your app on your device (iPhone) while plugged and Xcode on your computer will print out logs. This makes a suspicion that print is much like call to the remote server in which case the server is responsible for serializing the events so it makes no difference what thread the client is on. There are other possibilities such as dropping logs into file and then sending it which really makes little difference.
So How can print be classified in terms of operation - main or background? The answer is probably none. The call is not restricted to any thread so it is not main. It will probably lock whatever thread it is on until the operation is complete so it is not background either. Think of it like Data(contentsOf: <#T##URL#>) which will block the thread until data from given URL is retrieved (or exception is thrown).
I am using IO completion ports and AcceptEx() whilst learning about servers, and am studying Len Holgate's free server framework to do this. He has the following code:
// Basically calls AcceptEx() via a previously obtained function pointer
if (!CMSWinSock::AcceptEx(
m_listeningSocket,
pSocket->m_socket,
reinterpret_cast<void*>(const_cast<BYTE*>(pBuffer->GetBuffer())),
bufferSize,
sizeOfAddress,
sizeOfAddress,
&bytesReceived,
pBuffer))
{
const DWORD lastError = ::WSAGetLastError();
if (ERROR_IO_PENDING != lastError)
{
Output(_T("CSocketServerEx::Accept() - AcceptEx: ") + GetLastErrorMessage(lastError));
pSocket->Release();
pBuffer->Release();
}
}
else
{
// Accept completed synchronously. We need to marshal the data recieved over to the
// worker thread ourselves...
m_iocp.PostStatus((ULONG_PTR)m_listeningSocket, bytesReceived, pBuffer);
}
I am confused about the "Accept completed synchronously" else-case. I have tried many times to get this code path to be hit (by pausing the code before I issue the AcceptEx, connecting, then resuming the code), but whenever I try the call always fails with ERROR_IO_PENDING and I get my notification packet. Furthermore, I have read this MS knowledgebase article (which I may have misinterpreted) which states
Additionally, if a Winsock2 I/O call returns SUCCESS or IO_PENDING, it
is guaranteed that a completion packet will be queued to the IOCP when
the I/O completes
However, I am thinking this doesn't apply to AcceptEx() because the dox for AcceptEx() states of the parameter lpdwBytesReceived
This parameter is set only if the operation completes synchronously.
So it seems it can complete synchronously...can someone tell me how AcceptEx() can complete synchronously (i.e. how I can replicate it in my server?)
Additionally, if a Winsock2 I/O call returns SUCCESS or ERROR_IO_PENDING, it
is guaranteed that a completion packet will be queued to the IOCP when
the I/O completes
this is apply for any I/O request if completion port is associated with the file. but begin from windows vista this also depend from notification mode set for a file handle.
but need first begin look from native view.
by default, if FILE_SKIP_COMPLETION_PORT_ON_SUCCESS not set, exist 3 case by returned NTSTATUS status :
NT_SUCCESS(status) or status >= 0 - will be completion
NT_ERROR(status) or status >= 0xc0000000 - will be no completion
NT_WARNING(status) or status < 0xc0000000 - unclear - if this
error from I/O manager (say - STATUS_DATATYPE_MISALIGNMENT - will
be no completion). if this error from driver (say
STATUS_NO_MORE_FILES - will be completion).
the win32 layer usually separate check for STATUS_PENDING and return ERROR_IO_PENDING in this case (but exist and exceptions, like ReadDirectoryChangesW). otherwise in case NT_ERROR(status) api return fail and set error code. otherwise return success. visible that case NT_WARNING(status) considered as success, but in this case, if error from I/O manager, will be no completion. I/O usually return errors from NT_ERROR(status) range, if parameters is incorrect. only case which i know (for asynchronous api) - STATUS_DATATYPE_MISALIGNMENT can be returned in case wrong aligned buffers, when I/O manager have special knowledge about buffer align. in NtNotifyChangeDirectoryFile
(ReadDirectoryChangesW for win32) or NtQueryDirectoryFile (no corresponded win32 api). so only case which i know when will be no completion, when win32 return success - call ReadDirectoryChangesW with unaligned lpBuffer (it must be DWORD-aligned ) - in this case I/O manager just return STATUS_DATATYPE_MISALIGNMENT but win32 layer interpret this as success code and return true. but will be no completion in this case. however this is rarely case and you probably need use wrong align structures for this. so in general yes:
by default if I/O call returns SUCCESS or ERROR_IO_PENDING will be queued a completion entry to the port. (with special exception case which i try describe)
if we set FILE_SKIP_COMPLETION_PORT_ON_SUCCESS on file object (note this is per file object but not per file handle - documentation not exactly here) all become much more simply and efficient - completion entry will be queue to the port - when and only when I/O request return STATUS_PENDING. ERROR_IO_PENDING from win32 view (except ReadDirectoryChangesW (maybe some else api ?) where win32 layer simply lost return code information)
However, I am thinking this doesn't apply to AcceptEx()
you mistake. this, how i say, apply to any io request. "This parameter is set only if the operation completes synchronously." - and so what ?
if look to code snippet, clear visible that code assume - in case AcceptEx completed synchronous and no error occurs - will be no io completion. or SetFileCompletionNotificationModes(m_listeningSocket, FILE_SKIP_COMPLETION_PORT_ON_SUCCESS) called or code is wrong - will be io completion in this case and not need m_iocp.PostStatus - this is fatal error will be. however i doubt that code used FILE_SKIP_COMPLETION_PORT_ON_SUCCESS - so it wrong. but error never raised because driver side implementation of AcceptEx (underlining ioctl) never return STATUS_SUCCESS: it check parameters - if it wrong - just return some error, otherwise always return STATUS_PENDING. as result, for asynchronous sockets AcceptEx never return true and code never jump to error else case. but anyway code is wrong. also i think design not the best - in case we determinate will be no completion - better just direct call completion routine with returned error code instead Release() (this will be done in completion routine) or PostStatus - for what post ?! - call direct.
how AcceptEx() can complete synchronously
very easy - if m_listeningSocket is handle to synchronous file object. however in this case you can not bind IOCP to file (it can be bind only in case asynchronous file object).
about lpdwBytesReceived parameter - system copy Information member of IO_STATUS_BLOCK or if want OVERLAPPED.InternalHigh, in case operation is completed just. in case pending returned - this data simply not ready and not filled. you got actual number of bytes returned by io in completion
A 3rd party data loss prevention driver when enabled driver verifier on it causes driver verifier bugcheck based on IrqlZwPassive Rule
The crash includes the following information:
ZwOpenKey should only be called at IRQL = PASSIVE_LEVEL.
What are some of the potential impacts to a Windows system if ZwOpenKey is used outside of IRQL=PASSIVE_LEVEL?
Is this always a serious problem that we should raise with a vendor, or only in certain scenarios.
all Zw api in kernel must be called only on PASSIVE_LEVEL. this is by design. if call it on APC_LEVEL this already will be UB some times this can work, some times produce hang or crash. say in case ZwOpenKey - registry manager can read key data from disk, if it still not in memory. so pass IRP to filesystem and wait for it complete. but Irp for completion can insert special APC (IopCompleteRequest) in calling thread. if thread on APC level - APC will not be executed, until IRQL of thread not lower to passive. but it never done - he wait on IRP complete..
another point - on exit from Zw service, system check - are UserApcPending in Thread and if yes, raise IRQL to APC_LEVEL, initiate user apc, and lower it back to PASSIVE_LEVEL (system assume that Zw called on PASSIVE_LEVEL) - so you can enter to Zw api at APC_LEVEL and exit on PASSIVE_LEVEL. can ask - why thread at some time have APC_LEVEL ? simply, because nothing to do IRQL raised ? or exist some requirements why at some point must be APC_LEVEL ? if yes, what is be if situation require stay on APC_LEVEL but thread ahead of time lower IRQL to PASSIVE_LEVEL ? really UB. in most case can be nothing. but in some case can be very nasty bug which very hard catch and research.
When pthread_exit(PTHREAD_CANCELED) is called I have expected behavior (stack unwinding, destructors calls) but the call to pthread_cancel(pthread_self()) just terminated the thread.
Why pthread_exit(PTHREAD_CANCELED) and pthread_cancel(pthread_self()) differ significantly and the thread memory is not released in the later case?
The background is as follows:
The calls are made from a signal handler and reasoning behind this strange approach is to cancel a thread waiting for the external library semop() to complete (looping around on EINTR I suppose)
I have noticed that calling pthread_cancel from other thread does not work (as if semop was not a cancellation point) but signalling the thread and then calling pthread_exit works but calls the destructor within a signal handler.
pthread_cancel could postpone the action to the next cancellation point.
In terms of thread specific clean-up behaviour there should be no difference between cancelling a thread via pthread_cancel() and exiting a thread via pthread_exit().
POSIX says:
[...] When the cancellation is acted on, the cancellation clean-up handlers for thread shall be called. When the last cancellation clean-up handler returns, the thread-specific data destructor functions shall be called for thread. When the last destructor function returns, thread shall be terminated.
From Linux's man pthread_cancel:
When a cancellation requested is acted on, the following steps occur for thread (in this order):
Cancellation clean-up handlers are popped (in the reverse of the order in which they were pushed) and called. (See pthread_cleanup_push(3).)
Thread-specific data destructors are called, in an unspecified order. (See pthread_key_create(3).)
The thread is terminated. (See pthread_exit(3).)
Referring the strategy to introduce a cancellation point by signalling a thread, I have my doubts this were the cleanest way.
As many system calls return on receiving a signal while setting errno to EINTR, it would be easy to catch this case and simply let the thread end itself cleanly under this condition via pthread_exit().
Some pseudo code:
while (some condition)
{
if (-1 == semop(...))
{ /* getting here on error or signal reception */
if (EINTR == errno)
{ /* getting here on signal reception */
pthread_exit(...);
}
}
}
Turned out that there is no difference.
However some interesting side effects took place.
Operations on std::iostream especially cerr/cout include cancellation points. When the underlying operation is canceled the stream is marked as not good. So you will get no output from any other thread if only one has discovered cancellation on an attempt to print.
So play with pthread_setcancelstate() and pthread_testcancel() or just call cerr.clear() when needed.
Applies to C++ streams only, stderr,stdin seems not be affected.
First of all, there are two things associated to thread which will tell what to do when you call pthread_cancel().
1. pthread_setcancelstate
2. pthread_setcanceltype
first function will tell whether that particular thread can be cancelled or not, and the second function tells when and how that thread should be cancelled, for example, should that thread be terminated as soon as you send cancellation request or it need to wait till that thread reaches some milestone before getting terminated.
when you call pthread_cancel(), thread wont be terminated directly, above two actions will be performed, i.e., checking whether that thread can be cancelled or not, and if yes, when to cancel.
if you disable cancel state, then pthread_cancel() can't terminate that thread, but the cancellation request will stay in a queue waiting for that thread to become cancellable, i.e., at some point of time if you are enabling cancel state, then your cancel request will start working on terminating that thread
whereas if you use pthread_exit(), then the thread will be terminated irrespective to the cancel state and cancel type of that particular thread.
*this is one of the differences between pthread_exit() and pthread_cancel(), there can be few more.
this has been bugging me all day. When a program sets itself up to call a function when it receives a certain interrupt, I know that the registers are pushed onto the stack when the program is interrupted, but what I can't figure out is: how do the registers get off the stack? I know that the compiler doesn't know if the function is an interrupt handler, and it can't know how many arguments the interrupt gave to the function. So how on earth does it get the registers off?
It depends on the compiler, the OS and the CPU.
For low level embedded stuff, where an ISR may be called directly in response to an interrupt, the compiler will typically have some extension to the language (usually C or C++) that flags a given routine as an ISR, and registers will be saved and restored at the beginning and end of such a routine. [1]
For common desktop/server OSs though there is normally a level of abstraction between interrupts and user code - interrupts are normally handled first by some kernel code before being passed to a user routine, in which case the kernel code takes care of saving and restoring registers, and there is nothing special about the user-supplied ISR.
[1] E.g. Keil 8051 C compiler:
void Some_ISR(void) interrupt 0 // this routine will get called in response to interrupt 0
{
// compiler generates preamble to save registers
// ISR code goes here
// compiler generates code to restore registers and
// do any other special end-of-ISR stuff
}