I am using parallel_tests gem and more specifically parallel_rspec. I have 2 sets of tests that can't run in parallel as they interfere with the state of some other tests.
Currently I am doing
parallel_rspec spec --single 'spec/set_A'
I now have the need to also run set_B non-parallely but how do I ensure that it runs in its own process and not with set_A's process above?
I have tried parallel_rspec spec --single 'spec/set_A|set_B' but it runs both sets in a single process which makes that process run for a really long time. Passing to separate --single flags also doesn't seem to achieve that.
Related
I'm running tests from the command line via:
cargo test --workspace --tests --jobs 1
I'm testing a server service, where each tests starts and stops the server and interacts with a service. So, running in parallel will not work. I observe multiple tests trying to start the server at the same time. I've resorted to guarding against this, where I'm observing multiple tests attempting to start the server, with the guard eventually passing when the owning test stops the server. My understanding is that the --jobs 1 prevents tests from running in parallel, which is exactly what I want. However, it seems to not be working. Is this a known issue? Have I done something wrong? Or did I misunderstand the usage of --jobs n ?
I can provide more details, that lead me to this conclusion, if needed.
Found that in addition to --jobs 1 I needed to add -- --test-threads=1
I currently have a build of an application that is set to run infinitely. It is designed to run on a Raspberry Pi as a service, so it will continuously be running.
Whenever I try to test it on Travis-CI, the infinite loop portion draws an error even though the file builds correctly since it is running infinitely. Is there any way to stop this error, or do I have to remove the ability to run the build from the .travis.yml?
language: cpp
compiler:
- clang
- g++
script:
- make
- cd main
- ./jsonWeatherPrediction
I would expect it to error, I'm just not sure of a current way to stop it without removing - ./jsonWeatherPrediction
I don't know if this will help, but the build is located at https://travis-ci.org/DMoore12/json-weather-prediction
Thanks in advance :)
In most any reasonable CI workflow, the job should have well-defined start and finish. Your software you are testing may run forever, but your tests should not. So, first, I suggest re-thinking how you run your build.
Looking at build such as https://travis-ci.org/DMoore12/json-weather-prediction/jobs/474719832, I see that you are simply running your command (which raises a different question: The command is printing the same output forever in a tight loop. Is this the desired behavior?).
For testing, you need a different kind of behavior, one that can be tested (e.g., take input from STDIN or a command-line flag, print, and terminate).
I'm executing the following command which executes a group of scripts with each script being a curl download.
parallel --resume-failed --joblog logshd.log {1} ::: SH/*.sh
The set of files downloaded is quite large. I've noticed some files don't download.
I hoped that the resume-failed parameter would ensure that all the downloads that fail resume and complete.
I'm not clear on if that means I need to run the process again a second time or if that should occur when I run the one time.
From the gnu documentation
Where --resume-failed reads the commands from the command line (and
ignores the commands in the joblog), --retry-failed ignores the
command line and reruns the commands mentioned in the joblog.
I'm not clear on what ignoring the command line or ignores the commands in the job log means. Could that be clarified.
Can --resume-failed and --retry-failed be declared within the same command and if so what is the effect of that?
Regards
Conteh
If we assume the download fails intermittently then your answer is --retries 10. It will run the command 10 times before giving up.
--resume-failed and --retry-failed are both used when GNU Parallel has finished, and you then figure out that you want to retry some of the jobs again.
The difference between the two is in how to retry the command.
--retry-failed will run exactly the same command as failed before. It does that by looking in the joblog for the command. This is typically what you want.
--resume-failed is used if you figure out that the failing command actually needed some other parameter: i.e. GNU Parallel should not run exactly the same command, but it should run a (typically slightly changed) command with the same parameters instead.
I frequently run "rake", see some tests fail, and have to manually cut-and-paste failures into a new command, "ruby SomeTest -n some_test_method" to run individual tests. Is there any way to automate that?
It feels like standard behaviour for IDEs to show errors and allow quick re-playing, so I wonder if anyone has figured out how to do it quickly on the command line.
Guard is what you're looking for:
https://github.com/guard/guard
Guard watches the filesystem for changes and automatically triggers a command. With guard you can automatically run tests the second they're saved.
For minitest use:
https://github.com/guard/guard-minitest
For test unit:
https://github.com/guard/guard-test
For Rspec:
https://github.com/guard/guard-rspec
In a rails application (or sinatra), if I make a call to a shell command, under what context does this command run?
I'm not sure if I am asking my question correctly, but does it run in the same thread as the rails process?
When you shell out, is it possible to make this a asychronous call? If yes, does this mean at the operating system level it will start a new thread? Can it start in a pool of threads instead of a new thread?
If you are using system('cmd') or simply backticks:
`cmd`
Then the command will be executed in the context of a subshell.
If you wish to run multiple of these at a time, you can use Rubys fork functionality:
fork { system('cmd') }
fork { system('cmd') }
This will create multiple subprocessess which run the individual commands in their respective subshells.
Read up on forking here: http://www.ruby-doc.org/core-2.0/Process.html#method-c-fork
It's more than just a new thread, it's a completely separate process. It will be synchronous and control will not return to Ruby until the command has completed. If you want a fire-and-forget solution, you can simply background the task:
$ irb
irb(main):001:0> system("sleep 30 &")
=> true
irb(main):002:0>
$ ps ax | grep sleep
3409 pts/4 S 0:00 sleep 30
You can start as many processes as you want via system("foo &") or`foo &`.
If you want more control over launching background processes from Ruby, including properly detaching ttys and a host of other things, check out the daemons gem. That's more suitable for long-running processes that you want to manage, with PID files, etc., but it's also possible to just launch tasks with it.
There are alternative solutions for managing background processes depending on your needs. The resque gem is popular for queuing and managing background jobs. It requires Redis and some setup, but it's good if you need that level of control.