I am trying to understand which of the %exec, system, IO.popopen commands I can use in Ruby for what circumstances.
These pages explains it but I dont think so I quite understand what does starting a new application etc mean.
http://humblelittlerubybook.com/book/html/chapter4.html
From my understanding :
%exec: takes over your application process and runs the new command. So does that mean that once the system command is finished, it just exits OR does it resumes the parent application process ?
io.popopen: spawns a new thread and returns output and errs back to parent application ?. Is this safe command to use when you want to spawn a thread and do something in parallel ?
system: spawns a sub-process. This probably means that the control returns back to parent once the spawned process is done ? Does the parent process halts until the child completes execution ? [that sounds dangerous]
Can someone explain to me in English, in what circumstances do we use these commands?
Thanks
exec() is used to replace the currently executing program with the program specified as a parameter to exec(). The program you specify as a parameter to exec() becomes the currently executing program. If you're using RoR, you probably do not want to do this. If you're using plain old Ruby for a program, you might want to do this under certain circumstances.
system() is used to invoke a new program in a subshell. The currently executing program will block, or wait, until the program you invoked has finished running. You are right - this can be dangerous if you have a long-running program. If you have a program which can run-away but you want to use system, you can wrap the system call in the child part of a fork() call and have the parent kill the child process if it runs for too long. More on fork(): http://en.wikipedia.org/wiki/Fork_%28operating_system%29
system() will not give you the STDOUT or STDERR of the application in its return value. Instead, you'll get the exit status code. It's a general practice that programs will exit with a status code of 0 if they are successful, and anything else if they fail.
The backticks are an alternative to system() which return not the exit status code but the actual STDOUT of the program which was executed. You can obtain the exit status code of the program through the "$?" variable.
IO.popopen() (and Open3 and Open4) allows you to work with the STDIN and the STDOUT of a subprocess.
This seems to be a good place to get a little more familiar with some of these concepts as they are implemented in Ruby: http://tech.natemurray.com/2007/03/ruby-shell-commands.html
exec is something that most people don't need to use. The executed program becomes the active program, your original program looses execution, file handles etc.
popen is about standard input and output, meaning it's good to be used with filters, i.e. programs that write or read from standard input/output.
system is better used to execute a program, your original program keeps executing but waits till that program invoked by system finishes.
Related
I'm starting a new instance of another application using CreateProcess from Example and I end up saving the PID so that I can later check if that process is still running.
I'm using to following method to check if it's running or not:
procedure TfrmRSM.Button1Click(Sender: TObject);
begin
var
ahandle := OpenProcess(PROCESS_ALL_ACCESS, true, aPID);
if ahandle = 0 then
ShowMessage('is not running')
else
ShowMessage('is running');
CloseHandle(ahandle);
end;
The code above should return 0 when the process is no longer running but it still returns a number greater than 0
I am closing the handle after using CreateProcess
Whats to propper way to check if a PID is running if the method I'm using is incorrect? I'm only able to find methods that use the application name.
I'm starting a new instance of another application using CreateProcess from Example and I end up saving the PID so that I can later check if that process is still running.
The correct way to handle this is to keep open the HANDLE that CreateProcess() gives you, and then you can query it via WaitForSingleObject() or GetExitCodeProcess() to see if the process has terminated or not, and then close the HANDLE when you no longer need it.
In comments, you mention that your launching app may terminate and be restarted separate from the target process. In that case, you could close the HANDLE if you still have it open, saving its PID and creation date/time somewhere you can get it back from, and then when your app restarts it can enumerate running processes (alternatively) to see if the target EXE is still running and has a matching PID and date/time, and if so then open a new HANDLE to that PID. Just be careful, because this does introduce a small race condition where the target process might terminate after you detect its presence and its PID could get recycled before you have a chance to open it. So you might need to re-validate the HANDLE's info again after opening it.
Otherwise, during your app's shutdown (or even before), you can off-load the open HANDLE from CreateProcess() to a separate helper process that stays running in the background monitoring the HANDLE, and then your main app can get the HANDLE back from that helper after restarting. Or, perform the actual CreateProcess() call in the helper to begin with, so the HANDLE monitoring stays within a single process at all times, and let your main app query the helper for status when needed.
I'm using to following method to check if it's running or not:
That will not work, as you don't know whether the PID is still valid, or even still refers to the same process you are interested in. Once that process has terminated, its PID can be recycled at any time for use with a new process.
The code above should return 0 when the process is no longer running but it still returns a number greater than 0
The only way OpenProcess() can return non-zero is if the specified PID is actually running. But that does not guarantee it is the same process you are interested in. At the very least, after OpenProcess() returns a non-zero HANDLE, you can query that HANDLE for its info (EXE file path, creation date/time, etc) to see if it is the same process you are expecting. If the info does not match, the PID was recycled.
Why does OpenProcess() return a non 0 value when the process ID is no
longer running?
This is probably an undefined behavior or the PID has been recycled by Windows.
I want to check if a process I created is still running later even
after I quit my application and opened it again.
You said you cannot change the process. One work around is to use an intermediate and very simple process to launch the target process. The intermediate process can use any IPC (for example shared memory of shared file) to talk to the main process to inform it about running target process. This intermediate process will run the target process and update a flag that the main process can query (In my example in shared memory).
The intermediate process can also simply keep a file open for exclusive access while the target process is running. The main process can try to open the file and if it succeed, then the target process is done (Intermediate process use WaitForSingleObjector WaitForMultipleObject to wait for target process termination).
To wait for a process to end, simply use WaitForSingleObject() on the process handle: when the process ends, the handle is signaled and the function returns WAIT_OBJECT_0. This ensures that you "look" at the handle the whole time it lives, instead of only inspecting it from time to time and otherwise leave it unattended.
Most likely you put this into a separate thread, and upon its ending you have your event to react to when the watched process ended. If you have multiple handles to look for, use WaitForMultipleObjects() - be aware that it can't handle more than 64 (MAXIMUM_WAIT_OBJECTS) handles at once.
Edit: when not being able to track a process handle entirely you can at least use GetProcessTimes() to check if the process you're looking at still started at the same time you looked at it last time - that should make it pretty distinctive: when the process handle is recycled by a new process then at least its starting time should differ.
using delphi XE I am trying to execute an exe file multiple times with different parameters
but i will need to close/restart each one separately for various reasons.
so i thought if i start that example.exe and get its pid
i will be able to kill it later using that unique pid value.
see if i simply execute the example.exe THEN try to get the PID of that process using process name or the process file path it will end up giving me wrong result because there are like 4 processes with that name.
any suggestions or ideas ?
my question might seem similar to some others but i need to return the pid value so keep that in mind
Use the Win32 API CreateProcess() function. It outputs a PROCESS_INFORMATION struct that contains the IDs and handles of the launched process and its main thread. You can use the process handle to wait for the process to exit.
To terminate the process, you can pass the process handle to TerminateProcess().
Or, you can be more civil and:
enumerate the process's UI windows using EnumWindows() or EnumThreadWindows(), posting a WM_CLOSE message to each one.
And/Or:
post a WM_QUIT message to the main thread.
If that does not work, then use TerminateProcess() as a last resort.
Look into using CreateProcess. There are multiple examples on StackOverflow including: Hide process window with 'CreateProcess'
If the call is successful, you will have the handle of the Process in the TProcessInformation parameter which you pass into CreateProcess.
I am trying to understand how I shall port my Java chess engine to dart.
So I have understood that I should use Isolates and/or Futures to run my engine in parallell with the GUI but how can I force the engine to terminate the search.
In java I just set some boolean that where shared between the engine thread and the gui thread.
You should send a message to the isolate, telling it to stop. You can simply do something like:
port.send('STOP');
To be clear, isolates and futures are two different things, and you use them differently.
Use an isolate when you want some code to truly run concurrently, in a separate "isolated memory heap". An isolate is like a mini program, running separately from your main program. You send isolates messages, and you can receive messages from isolates.
Use a future when you want to be notified when a value is available later. "Later" is defined as "a future tick in the event loop". Each isolate has its own event loop. It's important to understand that just asking a Future to run a function doesn't make the function run in parallel. It just puts the function onto the event loop to be run "later".
Answering the implied question 'how can I get a long running task in an isolate to cease running?' rather than more explicitly asked 'how can I cause an isolate to terminate, release it's resources and generally cease to be?'
Break the long running task up into smaller, shorter running units.
Execute each unit with a Future. Chain futures as appropriate.
Provide a flag that each unit should check before executing its logic. If the flag is set, bail.
Listen for a 'stop' message and set the flag if/when received.
Splitting the main processing task up into Futures allows processing of the stop message to get onto the event queue ahead of units of processing of the main task.
There is now iso.Isolate.kill()
WARNING: This method is experimental and not handled on every platform yet.
I've been looking at lua and lvm.c. I'd very much like to implement an interface to allow me to control the VM interpreter state.
Cooperative multitasking from within lua would not work for me (user contributed code)
The debug hook gets me only about 50% of the way there, instruction execution limits, but it raises an exception which just crashes the running lua code - but I need to be able to tweak it even further.
I want to create a system where 10's of thousands of lua user scripts are running - individual threads would not work, and the execution limits would cause headache for beginning developers, I'm going to control execution speeds too. but ultimately
while true do
end
will execute forever, and I really don't care that it is.
Any ideas, help or other implementations that I could look at?
EDIT: This is not about sandboxing pretend I'm an expert in that field for this conversation
EDIT: I do not want to use an internally ran lua code coroutine based controller.
EDIT: I want to run one thread, and manage a large number of user contributed lua scripts, an external process level control mechansim would not scale at all.
You can search for Lua Sandbox implementations; for example, this wiki page and SO question provide some pointers. Note that most of the effort in sandboxing is focused on not allowing you to execute bad code, but not necessarily on preventing infinite loops. For better control you may need to combine Lua sandboxing with something like LXC or cpulimit. (not relevant based on the comments)
If you are looking for something Lua-based, lightweight, but not necessarily 100% foolproof, then you can try running your client code in a separate coroutine and set a debug hook on that coroutine that will be triggered every N-th line. In that hook you can check if the process you are running exceeded its quotes. You also need to take care of new coroutines started as those need to have their own hooks set (you either need to disable coroutine.create/wrap or to replace them with something that sets the debug hook you need).
The code in this case may look like:
local coro = coroutine.create(client_func)
debug.sethook(coro, debug_hook, "l", 1000) -- trigger hook on every 1000th line
It's not foolproof, because it may block on some IO operation and the debug hook will not help there.
[Edit based on updated question and comments]
Between "no lua code coroutine based controller" and "no external process control mechanism" I don't think you are left with much choice. It may be that your only option is to run one VM per user script and somehow give ticks to those VMs (there was a recent question on SO on this, but I can't find it). Before going this route, I would still try to do this with coroutines (which should scale to tens of thousands easily; Tir claims supporting 1M active users with coroutine-based architecture).
The mechanism would roughly look like this: you install the debug hook as I shown above and from that hook you yield back to your controller, which then decides what other coroutine (user script) to resume. I have this very mechanism working in the Lua debugger I've been developing (although it only does it for one client script). This doesn't protect you from IO calls that can block and for that you may still need to have a watchdog at the VM level to see if it's been blocked for longer than needed.
If you need to serialize and deserialize running code fragments that preserve upvalues and such, then Pluto is probably your only option.
Look at implementing lua_lock and lua_unlock.
http://www.lua.org/source/5.1/llimits.h.html#lua_lock
Take a look at lulu. It is lua VM written on lua. It's for Lua 5.1
For newer version you need to do some work. But it's then you really can make a schelduler.
Take a look at this,
https://github.com/amilamad/preemptive-task-scheduler-for-lua
I maintain this project. It,s a non blocking preemptive scheduler for running lua code. Suitable for long running game scripts.
Lets say I am running a script and the game client waits for the script to be finished before it updates. Can Lua do somthing of a 'timeout'? Like, can i set a priority on the update so it leaves the script to do the update and then after words could it go back to where it was in the script?
You can also set a count hook with a suitable count for timeout and abort execution of the script in the hook.
Lua uses collaborative multithreading, so the script must know how long it has taken before it passes control back to the caller. Its not hard to figure out how long it has run using os.time and getting the difference. In some cases this might be more difficult, but if the script is a loop it shouldn't be hard. Once you've figured out that you've run for too long, do a coroutine.yield() and when you want to resume the script, simply call lua_resume from your update loop.
You can run your whole lua_State and lua script in another thread. When the Lua script accesses functions you implemented which need to modify things in the main thread, use mutexes and other things to make that access thread-safe.
This way, you can easily have your Lua script hang or do whatever while your main thread can continue to operate normally, however, it also requires you to make all your implemented functions accessing anything the main thread probably takes care of normally (like graphics) to be threading-aware.