I need to terminate long running SFTP put operations (sometimes the transfer hangs and takes 10-15 minutes to complete).
Is there a way to do this using Apache VFS? The SftpFileSystemConfigBuilder has a setTimeout method, but that only affects the connect operation. What I would need is to be able to terminate a PUT operation if it takes longer than X minutes.
Related
I am building a large project on a remote machine using Bazel. Clean build times are around 30 minutes. Incremental builds (changing code in 1-2 files) typically take around 10-20 seconds.
The problem I have is that when I log out of my machine and log back in again after 1-2 days the build command takes around 10 minutes even though I have not modified any source code.
If I call bazel shutdown and then call bazel build again the "no-build" op takes around 5-10 seconds (i.e. much better than the other "no-build" op).
If I log out and log back in again immediately I can see there is still a bazel process running in the background, which disappears when I call bazel shutdown. I am guessing that when I do not shut bazel down properly it gets killed in such a way that corrupts or deletes cached data. The long "no-build" op then spends a long time reconstructing data that was previously stored in the Bazel cache.
Is there a way to automatically shut down the bazel server when I am disconnected? Preferably this should work both when (i) I call exit from the command-line to log out, (ii) I get automatically disconnected through some kind of timeout or interruption in network connectivity.
Set up your development environment so that you sessions do not automatically exit / get killed, e.g., using a tool like screen or tmux. When you want to end a session call bazel shutdown prior to exit. Not completely automated but the point is that you should be in control of when your sessions end.
I'm currently trying to optimize and scale an API built on Ruby on Rails behind an AWS ALB that sends traffic to NGINX and then into Puma to our Rails application. Our API has a timeout option of 30 seconds maximum which is when we eventually timeout the request. Currently we have a controller action that queues a Sidekiq worker and then we poll a Redis key every 100ms for the first 1 second and then move to polling every 500ms for the remaining 29 seconds. Many of our requests can be completed in under 1 second, but some of them will take the full 30 seconds before they succeed or timeout, telling the user to retry in a little while.
We're currently trying to load test this API and scale it to 500-1000 RPS and we're running into problems where the slower requests will block up all of our connections. When a slow request is running shouldn't Puma be able to take other requests in during the sleep period of the slow requests?
If this was not an API we could easily just immediately respond after we queue the background worker, but in this case we need to wait for the response and hold the connection for up to 30 seconds for the API request.
My first thought is that you can have multiple redis queues and push specific tasks to certain queues.
If you have a queue for the quick tasks and a queue for the slower tasks, then both can run in parallel without the slow tasks holding everything else up.
I want to run a long-running job on cloud run. this task may execute more than 30 minutes and it mostly sends out API requests.
cloud run stops executing after about 20 minutes and from the metrics, it looks like it did not identify that my task is still in the running state. so it probably thinks it is in idling and closing the container. I guess I can run calls to the server while job run to keep the container alive, but is there a way to signal from to container to cloud run that job is still active and not to close the container?
I can tell it is closing the container since the logs just stop. and then, the next call I make to the cloud run endpoint, I can see the "listening" log again from the NodeJS express.
I want to run a long-running job on cloud run.
This is a red herring.
On Cloud Run, there’s no guarantee that the same container will be used. It’s a best effort.
While you don’t process requests, your CPU will be throttled to nearly 0, so what you’re trying to do right now (running a background task and trying to keep container alive by sending it requests) is not a great idea. Most likely your app model is not fit a for Cloud Run, I recommend other compute products that would let you run long-running processes as well.
According to the documentation, Cloud Run will time out after 15 minutes, and that limit can't be increased. Therefore, Cloud Run is not a very good solution for long running tasks. If you have work that needs to run for a long amount of time, consider delegating the work to Compute Engine or some other product that doesn't have time limits.
Yes, You can use.You can create an timer that call your own api after 5 minutes, so no timeout after 15 minutes.Whenever timer executes it will create a dummy request on your server.
Other option you can increase request timeout of container to 1 hour from 5 min, if your backend request gets complete in 1 hour
My app is hanging in the call to procedure TIdStackWindows.Connect. When the TCP/IP address exists there is no problem, but if it doesn't, I get the hang. The IP address is a literal- there is no DNS lookup involved. I was expecting the connection attempt to fail after the timeout (TCPClient.ConnectTimeout) I have set of 1 second but the app hangs for up to 30 seconds on this call (the call from my app isn't threaded. I intend to move the TCP connection to a thread, but the long connect timeout will still be an issue).
If I pause execution in the Delphi IDE when the app is unresponsive, I an positioned at:
ntdll.KiUserApcDispatcher:
7C90E450 8D7C2410 lea edi,[esp+$10]
I then F8 a couple of times until I see a stack frame. I am then at:
IdStack.TIdStack.RaiseSocketError(10038)
IdStack.TIdStack.RaiseLastSocketError
IdStack.TIdStack.CheckForSocketError(-1)
IdStackWindows.TIdStackWindows.Connect(912,'10.8.2.170',5001,Id_IPv4)
IdSocketHandle.TIdSocketHandle.Connect
IdIOHandlerStack.TIdConnectThread.Execute
:00451fc1 HookedTThreadExecute + $2D
Classes.ThreadProc($254B910)
System.ThreadWrapper($5456CB0)
:00451ea3 CallThreadProcSafe + $F
:00451f10 ThreadExceptFrame + $3C
:7c80b729 ; C:\WINDOWS\system32\kernel32.dll
I note after a bit of poking around that this topic has received a bit of traffic. The common answer seems to be "put it in a thread". I intend to, but the long timeout will still be problematic. Why does the connect timeout not work? I'm using Indy 10.5.5 and Delphi 2006 - if I upgrade to the latest build of Indy will there be much migration involved?
Blocking sockets have no concept of a connect timeout at the API layer, so Indy's ConnectTimeout is a manually implemented timeout. Indy calls TIdStack.Connect() in an internal worker thread while TIdTCPClient.Connect() runs a sleep loop that waits for that thread to terminate. If the loop detects the ConnectTimeout period has elapsed, it closes the socket, which is supposed to cause the blocked TIdStack.Connect() to exit immediately, but that is not a guarantee. There is OS overhead in creating and terminating a thread as well. It should definitely not take 30 seconds to react to a 1 second timeout, but on the other hand 1 second is usually too small. It is possible that the thread will not even begin running within 1 second. You should typically set the ConnectTimeout to 5-10 seconds at a minimum to give the OS enough time to do its work.
My application has a long running request that takes over a minute. If I'm using Chrome or Firefox I just need to be patient. If I use IE however, at the one minute mark I get the popup that says I've reached a Network Connection Timeout.
Why is that?
The default Internet Explorer time out is 1 minute. Since your process is a long-running one, IceFaces doesn't send the response and it times out.
You can avoid this by spawning a new thread for your long running process and returning the response immediately. IceFaces has plenty of polling or push options available to you to let your client know when the long-running process is done.