If I want to start all tests within a module, I simply write:
> import Example;
> :test
and all of the test bool functions run. However, I want to start them using the Rascal .jar for CI purposes. Is there any flag that I can use? For example:
$ rascal.jar TestsModule --test
Or any alternative solution so I can run my Rascal tests for CI purposes?
At the moment, not yet from the command line. For the (almost finished) compiler backend we now have --rascalTests <modules> but for now that won't solve your problem.
If your CI supports JUnit style tests, we have added a JUnit compatible layer around the rascal tests. For example this class runs all the tests in the Rascal modules in the lang::rascal::tests::basic package.
Another way to make it work is to pipe commands into the rascal-shell?
$ echo -e "import IO;\n println(\"Hello World\");\n:quit\n" | java -jar rascal-shell-unstable.jar
Version: 0.8.0.201604121603
rascal>import IO;
ok
rascal> println("Hello World");
Hello World
ok
rascal>:quit
Quiting REPL
$
Related
Running all tests (works)
When I run all tests inside PhpStorm, then I go to "Run" >> "MyProject PhpUnit Test". This outputs and runs a command like this:
[docker-compose://[/path/to/my/project/docker-compose.yml]:php/]:/usr/local/bin/php /var/www/docroot/bin/phpunit --configuration /var/www/docroot/phpunit.xml.dist --teamcity
That runs all tests inside PhpStorm, so far so good!
Running single test (does not work)
If I go to a single test, right-click on the test-name and click:
"Run testAppendShortToAnswers..."
Shortcut: CTRLShiftR
then it outputs and runs a command like this:
[docker-compose://[/path/to/my/project/docker-compose.yml]:php/]:/usr/local/bin/php /var/www/docroot/bin/phpunit --configuration /var/www/docroot/phpunit.xml.dist --filter "/(App\\Tests\\Entity\\CardTest::testAppendShortToAnswers)( .*)?$/" --test-suffix CardTest.php /path/to/my/project/tests/Entity --teamcity
And throws an error immediately:
Cannot open file "/path/to/my/project/tests/Entity.php".
Process finished with exit code 1
I'm assuming that it's because it tries to access that file, not from inside my Docker-container, but rather from my host-machine.
How do I fix this, so I can run a single test in PhpStorm?
Info about the setup
I struggled a bit setting up PhpStorm to run the tests, but you can see the big problem I had (and the solution) alongside links and whatnot here.
I'm on a Mac. It's a Symfony-project, where PhpUnit is located in a folder in the project-root (/bin/phpunit). So it's not installed via Composer.
I'm not really sure why it's setup this way.
Solution attempt 1: Running single test from the terminal
If I go to the terminal and run this command:
docker-compose exec php /usr/local/bin/php /var/www/docroot/bin/phpunit --filter testAppendShortToAnswers
The it runs as it should. Hmm... But I ideally I would like to run it straight from PhpStorm, so I can utilize the CTRLShiftR-shortcut.
#LazyOne pointed me in the right direction.
It was because I had a path mapping setup incorrectly.
So by changing that, then it works now:
I had it like this:
/Users/MYUSERNAME/Code/MYPROJECTNAME/some-wrong-project -> /var/www/docroot
And it fixed it to change it to:
/Users/MYUSERNAME/Code/MYPROJECTNAME/the-project-I-am-working-on -> /var/www/docroot
I am writing a few kong custom plugins in Lua. I am using Kong 2.3.3 and Lua 5.1.
I have some test cases (unit tests + integration tests) and i am running them with pongo run -coverage option. I have already installed luacov (and also cluacov, both with luarocks install) and all my tests are passing but no luacov files are being generated with coverage data. I am not running pongo from Docker, i have installed and configured it in my local machine (which is Linux Ubuntu 20.04).
I have already tried a few things as follows:
my .busted file is setting coverage = true, verbose = true and output = "gtest" (already tried utfTerminal, tap and json too)
tried adding luacov as a dependency to my rockspec file... the build does not fail but no coverage file is generated
i even tried running the tests without pongo, using busted directly but this is a very bad option because things like spec.helpers, or the cjson lib are not set in my LUAPATH
A quick way to do this is to modify pongo
Edit your pongo.sh file to:
add coverage flag to busted --coverage
call luacov to generate the report luacov
display the report cat luacov.report.out
locate where busted is called, line 959 for me:
"/bin/sh" "-c" "bin/busted --coverage --helper=bin/busted_helper.lua ${busted_params[*]} ${busted_files[*]};luacov;cat luacov.report.out"
Install luacov, edit assets/Dockerfile
after busted installation add luacov:
&& luarocks install busted-htest \
&& luarocks install luacov \
pongo run will give you
[...]
==============================================================================
Summary
==============================================================================
File Hits Missed Coverage
-------------------------------------------------------------------------------------------------------
/kong-plugin/kong/plugins/myplugin/schema.lua 105 1 99.06%
/kong-plugin/spec/myplugin/01-schema_spec.lua 199 5 97.55%
[...]
You can create a docker image based on pongo
spec/unit/docker/Dockerfile
FROM kong-pongo-test:2.3.2
USER root
RUN luarocks install luacov
WORKDIR /kong-plugin
COPY . .
spec/unit/docker/run.sh
#!/bin/sh
busted --coverage spec/unit
luacov
cat luacov.report.out
Run
docker build -f spec/unit/docker/Dockerfile -t my-coverage .
docker run my-coverage sh spec/unit/docker/run.sh
Pongo gained some support for this (still a PR). Note that it only covers unit tests, not integration ones.
See https://github.com/Kong/kong-pongo/pull/184
btw: the other anwers are too complex imo, you can add .pongo/pongo-setup.sh to install LuaCov, and move the .luacov file from /kong-plugin to /kong. That should be all that is necessary.
Running tests with coverage can be simply done by passing the flag, without any need to edit pongo or the dockerfile. Try pongo run -- --coverage for example.
I have the following section in my lua script:
newoption {
trigger = "build-tests",
description = "Build tests."
}
configuration "build-tests"
print("bbbbbbbbb")
include("Src/tests/tests.lua")
configuration "not build-tests"
print("aaaaaaaaaaa")
running premake5 gmake --help gives:
Usage: premake5 [options] action [arguments]
OPTIONS - General
--build-benchmarks Build benchmarks.
--build-tests Build tests.
--debugger Start MobDebug remote debugger. Works with ZeroBrane Studio
--fatal Treat warnings from project scripts as errors
--file=FILE Read FILE as a Premake script; default is 'premake5.lua'
--help Display this information
However running premake5 --build-tests gmake outputs:
bbbbbbbbb
aaaaaaaaaaa
Building configurations...
Running action 'gmake'...
Done (362ms).
BOTH versions are running, I don't understand. I am trying to get a toggle to select whether I want to build tests or not.
configuration() is a function, not an if-then. Your example above is equivalent to...
configuration("build-tests")
print("bbbbbbbbb")
include("Src/tests/tests.lua")
configuration("not build-tests")
print("aaaaaaaaaaa")
It has no effect on which code runs or does not run in your script; all the code in your script is always run, and then the configuration conditions are evaluated later.
Instead look at the actual generated project output to see if things are working or not.
I am unable to run repo non-interactively inside a container as part of a freestyle job.
It prompts for the user-name and email. I got round that by doing a git config --global inside the job.
But then it does the color test, and that hangs indefinitely.
Looking at the source code for repo I see this
if os.isatty(0) and os.isatty(1) and not self.manifest.IsMirror:
if opt.config_name or self._ShouldConfigureUser():
self._ConfigureUser()
self._ConfigureColor()
So, I ran the following inside the container:
python -C "import os; print os.isatty(0), os.isatty(1)"
and, sure enough, it printed out True True
Looking at the Jenkins log, it launches the container with --tty specified, and there seems no way to configure that option.
I can't find a bash option to force a script to be run in a non-interactive shell. If I put the above python line in a file and execute it with almost any combination of commands and options, it still prints out True True
The only way I see something different is if I use I/O redirection
bash <a.sh
which prints out False True - i.e. stdin is not a tty, and
bash <a.sh >a.log
which prints False False.
For a complex script, are there any problems using the bash <script approach?
Does anyone know any jenkins magic to prevent docker being launched using --tty?
I know that the --tty is the culprit. I built the container locally and ran the following
$ docker run repotest python -c "import os;print os.isatty(0), os.isatty(1)"
False False
$ docker run --tty repotest python -c "import os;print os.isatty(0), os.isatty(1)"
True True
Running Versions:
repo: 1.12.37 (per Ubuntu 16.04 apt-get)
Jenkins: 2.149
Cloudbees Docker Plugin: 1.7.3
Container base is ubuntu:xenial
I'm using the "Build inside a docker container" option.
To run bash script repo_script.sh "non-interactively", or more exactly speaking without having terminals associated with standard streams, you could run your script simply as
repo_script.sh < /dev/null 2>&1 | cat
assuming you want to see the output the way you would see it running simply as repo_script.sh. By piping the standard output and error to a different process the file descriptor appears as a pipe and not TTY to repo_script.sh. You could also direct output to a file, or even to /dev/null if you do not care about the output:
log_file=/dev/null
repo_script.sh < /dev/null > "${log_file}" 2>&1
Running the script as
bash < repo_script.sh | cat
might would work too, though it is very unorthodox and to my mind hackish way of running a script just to break the association of TTY to the standard input. From script engine point of view, it is different to read a script program from a file than from standard input (which typically, if it is a terminal, is not seekable), so there might be some subtle differences that could possibly bite you in unexpected ways. This way does not as clearly communicate your intention to the next person that need to understand your code, and may lead to partial hair loss in that person due to extraneous head scratching.
There is no need for any bash options, just using the output directions from within the interpreting shell as above described is an easy-to-comprehend, multi-platform compatible standard convention for changing the standard stream associations.
P.S. I think it should be enough for your repo script to just test if the standard input is a TTY. It looks to me like the author of that script did not think deeply enough there. There is simply no use waiting for input if you do not have terminal device associated with standard input, and you could determine that everything needs to run without user interaction from there or stop with an error if that is not possible.
After creating an escript app:
$ rebar3 new escript hello
Can I run with with rebar3, instead of calling escriptize first?
I.e. something like this?
$ rebar3 run
One solution might be to add a "run" directive to your makefile instead - this way you can call make run which does the escriptize step and runs the result.
Depending on what you really want to achieve running ./rebar3 shell might help you - I do this sometimes so I can debug portions of code just by running the required functions.
Finally you could always write a rebar plugin that extends the escriptize step ;) - never done it myself for rebar3 but has to be easier than the old rebar...