I was wondering what the correct way of issuing an asynchronous read operation on a page that has been swapped out is. And once this is done is there a way to execute a callback that happens so that I can execute some logic with that page?
madvise(start_address, length, MADV_WILLNEED) will do that.
A callback does not really exist, though you can poll a snapshot of the status with the mincore syscall, if you want.
Note that the opposite hint MADV_DONTNEED does not do what you think, nor does msync. The documentation is lying.
There is no non-broken-by-design way to start asynchronous writeback under Linux (although the documentation says so). You can choose between no-op, throwing pages away, synchronous writeback, or asynchronous writeback with purging the cache.
Related
We use an NSURLSession to download data in the background, and have timeoutIntervalForResource defined so it will timeout if it takes too long, but if, for whatever reason, the source server doesn't exist then it still sits and waits. Is there any way to get it to abort immediately, or 'ask' the NSURLSessionDownloadTask if anything has been downloaded yet?
Failing that, what would be the best way of performing a pre-check to ensure a server exists before trying to download data from it?
These servers may be out of our control so we can't place a small file to download to check availability. The only file we may not about could be a sizeable video, for example.
You can indeed ask the task about its status. First, check the response property. If that is nil, then you haven't gotten the first packet from the server. If that is non-nil, use countOfBytesExpectedToReceive and countOfBytesReceived as needed to determine progress.
I should also note that these properties all support KVO, AFAIK.
You could also perform an explicit DNS lookup prior to scheduling the background request if you'd prefer, with the caveat that doing so would prevent you from scheduling something that might actually work if the user's Internet connection comes back online in the meantime. :-)
I'm using ASINetworkQueue to execute multiple ASIHTTPRequests, and if any request fails I'd like the queue to cancel any pending requests and end. From reading the docs this should be the default behaviour. But I'm finding that even after a request fails, I still get 'requestStarted' for most of the remaining requests, and 'requestFailed' for all of them - is this how it is supposed to be? I'm guessing it's maybe because my requests are quite small and the requests start before it has chance to cancel them once a failure is detected. I tried implicitly setting setShouldCancelAllRequestsOnFailure:YES but this made no difference.
Without knowing the exact nature of your requests ... short answer: Yes, it's working how it's supposed to. Your requests are starting before a failure occurs. Longer answer: Try setting the queue's maxConcurrentOperationCount property. This may help you control the request pipeline a bit better if you need to test for failure.
hi
i'm going to set up a rails-website where, after some initial user input, some heavy calculations are done (via c-extension to ruby, will use multithreading). as these calculations are going to consume almost all cpu-time (memory too), there should never be more than one calculation running at a time. also i can't use (asynchronous) background jobs (like with delayed job) as rails has to show the results of that calculation and the site should work without javascript.
so i suppose i need a separate process where all rails instances have to queue their calculation requests und wait for the answer (maybe an error message if the queue is full), kind of a synchronous job manager.
does anyone know if there is a gem/plugin with such functionality?
(nanite seemed pretty cool to me, but seems to be only asynchronous, so the rails instances would not know when the calculation is finished. is that correct?)
another idea is to write my own using distributed ruby (drb), but why invent the wheel again if it already exists?
any help would be appreciated!
EDIT:
because of the tips of zaius i think i will be able to do this asynchronously, so i'm going to try resque.
Ruby has mutexes / semaphores.
http://www.ruby-doc.org/core/classes/Mutex.html
You can use a semaphore to make sure only one resource intensive process is happening at the same time.
http://en.wikipedia.org/wiki/Mutex
http://en.wikipedia.org/wiki/Semaphore_(programming)
However, the idea of blocking a front end process while other tasks finish doesn't seem right to me. If I was doing this, I would use a background worker, and then use a page (or an iframe) with the refresh meta tag to continuously check on the progress.
http://en.wikipedia.org/wiki/Meta_refresh
That way, you can use the same code for both javascript enabled and disabled clients. And your web app threads aren't blocking.
If you have a separate process, then you have a background job... so either you can have it or you can't...
What I have done is have the website write the request params to a database. Then a separate process looks for pending requests in the database - using the daemons gem. It does the work and writes the results back to the database.
The website then polls the database until the results are ready and then displays them.
Although I use javascript to make it do the polling.
If you really cant use javascript, then it seems you need to either do the work in the web request thread or make that thread wait for the background thread to finish.
To make the web request thread wait, just do a loop in it, checking the database until the reply is saved back into it. Once its there, you can then complete the thread.
HTH, chris
I created a class that handles serial port asynchronously. I use it to communicate with a modem. I have no idea why, but sometimes, when I close my application, I get the Blue Screen and my computer restarts. I logged my code step by step, but when the BSOD appeared, and my computer restarted, the file into which I was logging data contained only white spaces. Therefore I have no idea, what the reason of the BSOD could be.
I looked through my code carefully and I found several possible reasons of the problem (I was looking for all that could lead to accessing unallocated memory and causing AV exceptions).
When I rethought the idea of asynchronous operations, a few things came to my mind. Please verify whether these are right:
1) WaitCommEvent() takes a pointer to the overlapped structure. Therefore, if I call WaitCommEvent() inside a function and then leave the function, the overlapped structure cannot be a local variable, right? The event mask variable and event handle too, right?
2) ReadFile() and WriteFile() also take references or pointers to variables. Therefore all these variables have to be accessible until the overlapped read or write operations finish, right?
3) I call WaitCommEvent() only once and check for its result in a loop, in the mean time doing other things. Because I have no idea how to terminate asynchronous operations (is it possible?), when I destroy my class that keeps a handle to a serial port, I first close the handle, and then wait for the event in the overlapped structure that was used when calling the WaitCommEvent() function. I do this to be sure that the thread that waits asynchronously for a comm event does not access any fields of my class which is destroyed. Is it a good idea or is it stupid?
try
CloseHandle(FSerialPortHandle);
if Assigned(FWaitCommEvent) then
FWaitCommEvent.WaitFor(INFINITE);
finally
FSerialPortHandle := INVALID_HANDLE_VALUE;
FreeAndNil(FWaitCommEvent);
end;
Before I noticed all these, most of the variables mentioned in point one and two were local variables of the functions that called the three methods above. Could it be the reason of the BSOD or should I look for some other mistakes in my code?
When I corrected the code, the BSOD stopped occuring, but It might be a coincidence. How do you think?
Any ideas will be appreciated. Thanks in advance.
I read the CancelIo() function documentation and it states that this method cancells all I/O operations issued by the calling thread. Is it OK to wait for the FWaitCommEvent after calling CancelIo() if I know that WaitCommEvent() was issued by a different thread than the one that calls CancelIo()?
if Assigned(FWaitCommEvent) and CancelIo(FSerialPortHandle) then
begin
FWaitCommEvent.WaitFor(INFINITE);
FreeAndNil(FWaitCommEvent);
end;
I checked what happens in such case and the thread calling this piece of code didn't get deadlocked even though it did not issue WaitCommEvent(). I tested in on Windows 7 (if it matters). May I leave the code as is or is it dangerous? Maybe I misunderstood the documentation and this is the reason of my question. I apologize for asking so many questions, but I really need to be sure about that.
Thanks.
An application running as a standard user should never be able to cause a bug check (a.k.a. BSOD). (And an application running as an Administrator should have to go well out of its way to do so.) Either you ran into a driver bug or you have bad hardware.
By default, Windows is configured to save a minidump in %SystemRoot%\minidump whenever a bug check occurs. You may be able to determine more information about the crash by loading the minidump file in WinDbg, configuring WinDbg to use the Microsoft public symbol store, and running the !analyze -v command in WinDbg. At the very least, this should identify what driver is probably at fault (though I would guess it's your modem driver).
Yes, you do need to keep the TOverlapped structure available for the duration of the overlapped operation. You're going to call GetOverlappedResult at some point, and GetOverlappedResult says it should receive a pointer to a structure that was used when starting the overlapped operation. The event mask and handle can be stored in local variables if you want; you're going to have a copy of them in the TOverlapped structure anyway.
Yes, the buffers that ReadFile and WriteFile use must remain valid. They do not make their own local copies to use internally. The documentation for ReadFile even says so:
This buffer must remain valid for the duration of the read operation. The caller must not use this buffer until the read operation is completed.
If you weren't obeying that rule, then you were likely reading into unreserved stack space, which could easily cause all sorts of unexpected behavior.
To cancel an overlapped I/O operation, use CancelIo. It's essential that you not free the memory of your TOverlapped record until you're sure the associated operation has terminated. Likewise for the buffer you're reading or writing. CancelIo does not cancel the operation immediately, so your buffers might still be in use even after you call it.
I have not been able to find a way to cancel/terminate asynchronous read operation after successful HttpWebRequest. There is no way to set timeout, ThreadPool.RegisterWaitForSingleObject is not working too. And closing the underlying socket is not an option too because neither HttpWebRequest/Reponse provide access to it.
Edit:
Sadly this approach that Sunny suggestet works only for HttpWebRequest.BeginGetResponse. For the stream that you get after GetResponseStream() for some reason RegisterWaitForSingleObject is not working - the callback is never called.
The situation is the following:
I got an application that uses HttpGetRequest. It is build by using the default MSDN example for async httpwebrequest. Getting response is working like a charm. But on rare occasions the server that my httpwebrequest is connecting to forgets to close the socket. So I am hung on a infinite read from BeginRead.
In some rare occasions the other server forget
Why not RegisterWaitForSingleObject? You can use it for time out, and in the handler you can call request.Abort().
Here's an example. Btw, I have used simial approach before I found this article in the past, and it worked like a charm.
NOTE: The real end of the operation will happen on the next Stream.Read (of only the request/response was async) or Stream.EndRead (if the reading is async as well). You still need to capture the exception thrown in the EndXXXX handler.
EDIT: If the RegisterWaitForSingleObject never get called, then you have another issue, which is most probably a ThreadPool having not enough free threads. The callback for RegisterWaitForSingle object is called on a ThreadPool thread, and if there is no a free thread in the pool, it will never get called. You have some solutions:
Make your pages asynchronous. Why? How.
Change the max threads in the pool.
In any case, increasing the pool size will not help you too much. You have to inspect your code for resource (stream) leaks, and make sure that you not only call request.Abort(), but also close the response. Make sure than in your callbacks you use the proper EndXXX method, etc.
I nazdrave :)