For Elastic Search version 6.0
to cancel a running task:
POST _tasks/node_id:task_id/_cancel
I am following the same statement, but in actual it is unable to cancel the task. After running this command if I run get command to fetch all running task, the particular task don't get cancel.
If I hit cancel statement again, it gives statement that task_id is already cancelled but in reality it doesn't get cancel.
Calling the _tasks/<id>/_cancel endpoint only signifies to ES that the task should be cancelled. ES will then cancel the task as soon as it possibly can.
It might not be immediate, though, for instance, if you have a very complex search that takes a long time (e.g. wildcard queries with leading wildcards).
If you check the ContextIndexSearcher class and look for all places where checkCancelled.run() is called, those are the only phases when a query can be cancelled, but sometimes a phase can take a long time before being eligible for cancellation.
If your task is a search task, then you might try to leverage the timeout parameter to see if it helps (it might not always for the same reasons as mentioned above).
Related
I'm using Neo4j. For large data imports from external csvs, parquets, etc. there is a very handful command for "fire and forget", the apoc.periodic.submit. There is also the apoc.periodic.list that list the background jobs.
During the execution of the background job it appears in the output of apoc.periodic.list. But after it finishes, either by an error or by a successful execution, it will disappear from this list without any feedback from the completion status.
Is there a general way to check if a background job finish status? Is there a more suitable API for my purposes?
If there is a way to directly check error status on the fire&forget routines, I don't see it in the documentation (they are fire&forget, so it comes with the territory?)
Ideas
don't background the query itself, background a process/task that waits for a blocking Cypher execution to finish and capture the error code...
check for success instead of failure? (if it didn't succeed you know it failed right?), this may be evident based on what the Cypher does, or you could add a graph content update for this purpose. E.g. Update property on a NODE with last_updated. Do that last so that if the cypher fails, the property is not updated
You could enable query log and then check there to see what happened, most likely this query has a unique signature and the last execution could be found easily in the log (with status/error code)
I have a project in which when I try to update some attribute, a long and exhausting before_update function runs. This function runs some scripts, and when they're finished successfully the attribute is changed.
The problem is that I want a way to reflect to current status of the currently running scripts (to display some sort of 2/5...3/5... process), but I can't figure out a solution. I tried saving the last running command in the DB, but because the scripts are running in a before_update scope the commit is done only after all script are finished.
Is there any elegant solution to this kind of problem?
In general, you should avoid running expensive, cross-cutting code in callbacks. A time will come when you want to update one of those records without running that code, and then you'll start adding flags to determine when that callback should run, and all sorts of other nastiness. Also, if the record is being updated during a request, the expensive callback code will slow the whole request down, and potentially time out and/or block other visitors from accessing your application.
The way to architect this would be to create the record first (perhaps with a flag/state that tells the rest of your app that the update hasn't been "processed" yet - meaning that related code currently in your callback hasn't run yet). Then, you'd enqueue a background job that does whatever is in your callback. If you are using Sidekiq, you can use the sidekiq-status gem to update the job's status as it's running.
You'd then add a controller/action that checks up on the job's status and returns it in JSON, and some JS that pings that action every few seconds to check up on the status of the job and update your interface accordingly.
Even if you didn't want to update your users on the status of the job, a background job would probably still be in order here - especially if that code is very expensive, or involves third-party API calls. If not, it likely belongs in the controller, and you could run it all in a transaction. But if you need to update your users on the status of that work, a background job is the way to go.
Unlike Oracle's solution, I'm not so sure but I think Intalio can't handle tasks with a deadline.
What I want is the task to be canceled once 48 hours have passed and to follow a different sequence flow in this case. Just like this.
Is there any way this purpose could be done with Intalio? Thanks
Intalio lets you attach an interrupter timer to a sub-process(group of task in the process flow). When that timer is hit (48 hours in your case) execution jumps out of the sub-process to the next task in line. I think this might be exactly what you are looking for.
Human interaction tasks (like completing a form) also have a deadline option that you can set in the mapper. When the deadline is hit the human interaction task is canceled and the process automatically moves on.
Edit. Added a picture to show how it looks in Intalio designer. When timer is hit process moves on to Task 3. You can also execute a separate stream (optional task in this example)
http://i.stack.imgur.com/2AmrN.png
Hope this helps. Cheers.
I use ASP.Net MVC 5 and I have a long running action which have to poll webservices, process data and store them in database.
For that I want to use TPL library to start the task async.
But I wonder how to do 3 things :
I want to report progress of this task. For this I think about SignalR
I want to be able to left the page where I start this task from and be able to report the progression across the website (from a panel on the left but this is ok)
And I want to be able to cancel this task globally (from my panel on the left)
I know quite a few about all of technologies involved. But I'm not sure about the best way to achieve this.
Is someone can help me about the best solution ?
The fact that you want to run long running work while the user can navigate away from the page that initiates the work means that you need to run this work "in the background". It cannot be performed as part of a regular HTTP request because the user might cancel his request at any time by navigating away or closing the browser. In fact this seems to be a key scenario for you.
Background work in ASP.NET is dangerous. You can certainly pull it off but it is not easy to get right. Also, worker processes can exit for many reasons (app pool recycle, deployment, machine reboot, machine failure, Stack Overflow or OOM exception on an unrelated thread). So make sure your long-running work tolerates being aborted mid-way. You can reduce the likelyhood that this happens but never exclude the possibility.
You can make your code safe in the face of arbitrary termination by wrapping all work in a transaction. This of course only works if you don't cause non-transacted side-effects like web-service calls that change state. It is not possible to give a general answer here because achieving safety in the presence of arbitrary termination depends highly on the concrete work to be done.
Here's a possible architecture that I have used in the past:
When a job comes in you write all necessary input data to a database table and report success to the client.
You need a way to start a worker to work on that job. You could start a task immediately for that. You also need a periodic check that looks for unstarted work in case the app exits after having added the work item but before starting a task for it. Have the Windows task scheduler call a secret URL in your app once per minute that does this.
When you start working on a job you mark that job as running so that it is not accidentally picked up a second time. Work on that job, write the results and mark it as done. All in a single transaction. When your process happens to exit mid-way the database will reset all data involved.
Write job progress to a separate table row on a separate connection and separate transaction. The browser can poll the server for progress information. You could also use SignalR but I don't have experience with that and I expect it would be hard to get it to resume progress reporting in the presence of arbitrary termination.
Cancellation would be done by setting a cancel flag in the progress information row. The app needs to poll that flag.
Maybe you can make use of message queueing for job processing but I'm always wary to use it. To process a message in a transacted way you need MSDTC which is unsupported with many high-availability solutions for SQL Server.
You might think that this architecture is not very sophisticated. It makes use of polling for lots of things. Polling is a primitive technique but it works quite well. It is reliable and well-understood. It has a simple concurrency model.
If you can assume that your application never exits at inopportune times the architecture would be much simpler. But this cannot be assumed. You cannot assume that there will be no deployments during work hours and that there will be no bugs leading to crashes.
Even if using http worker is a bad thing to run long task I have made a small example of how to manage it with SignalR :
Inside this example you can :
Start a task
See task progression
Cancel task
It's based on :
twitter bootstrap
knockoutjs
signalR
C# 5.0 async/await with CancelToken and IProgress
You can find the source of this example here :
https://github.com/dragouf/SignalR.Progress
I have a long running operation in my Grails application. My first solution is simply to perform the operation in the controller and let the user wait until the action is finished. This is not an acceptable solution, I want to show the user the progress of the action. The progress is simply text. The action can take from 10 seconds to roughly 30 minutes.
How can I show the progress to the user for my long running action?
First you might want to try the Executor plugin so you can run the job in the background. This works quite well.
Then I guess you have 2 options. Have the browser poll the server via Ajax for an update (as Tim pointed out the JProgress plugin would do something like this for you) or get even more bleeding edge and consider HTML5 WebWorkers for a kind of server push approach. WebWorkers are not available in
You will need something like a Task or Job domain class with a field percentageComplete. The controller will create and save the Task or Job and then spawn a new thread to execute it. Perhaps place the execution code in a service.
It will be up to your execution code to update the Task or Job's percentageComplete field as it completes its task. Then you can poll the job (via ajax) to see how the job is progressing.
Note: determining that percentage which is complete is very much to up to your specific code. You will probably have to just come up with a best guess based on the knowledge you have. Even for an operation where it is obvious how to determine percentage complete (like a file download), it is not certain (network issues, etc.)
Can you determin the state of your progress? Let's say in percent?
I would create the long operation as a quarz job (background) and query the state of job/ long running progress via ajax.
The JProgress plugin might help, but I've never tried it out...