I faced a problem, where PyCharm doesn't show report on pycharm with coverage run. I use Docker Compose interpreter and I tried it:
with pytest-cov installed.
with pytest-cov uninstalled.
with addopts = -s -v in pytest.ini.
Nothing seems to work. The test coverage report is not shown. No error messages, no warnings. It runs as usually.
Is there some specific actions that should be do if you want to run pytest coverage in PyCharm on remote (Docker Compose) interpreter?
Related
I have a selenium java test automation framework in my Mac os . Now , I want to execute my automation testcases in Ubuntu Docker container using a docker file which should automatically install java, selenium , TestNG, Maven in ubuntu docker container .
Docker require shell commands so, before thinking on docker, you need to be able to run your selenium tests using the shell. If your test cannot be executed using the shell on your mac, it will be difficult to execute it with docker.
If you are able to run the tests using the shell and al of your reports are well generated, you are ready to docker.
Selenium tests are not live applications, so your docker container will be use just to run the tests and after that, you should destroy it.
As you using java, is there an option to run your tests as a single jar, instead maven. If you achieve this, your flow will be more easy or light.
If you achieve the dockerization of your test, you could run your tests developed in you mac on any machine on-premise or cloud with this line:
docker run --name tests -d \
-e PARAM1=FOO \
-e PARAM2=BAR \
tests:1.0.0
Running tests with maven (source code level)
If you pom.xml is well configured, you could run your the testng test with : mvn clean test
So you docker file will be
FROM maven:3.3-jdk-8
RUN mkdir /usr/test
COPY . /usr/test
WORKDIR /usr/test
CMD["mvn","clean", "test"]
Note: I'm not tested this Dockerfile yet
The execution will be a little slower because compilation is performed at docker run phase.
Run tests using jar
According to this you can run testng with pure java:
java -cp F:\Selenium\SampleTestNG\lib\*;F:\Selenium\SampleTestNG\bin org.testng.TestNG testng.xml
As you can see, you need the testng framework jars. That will complicate the dockerization.
If you are able to use the maven-assembly-plugin, maven will merge all the jars in just one. If you achieve this, your automation flow will be:
mvn clean package
java -jar selenium-test.jar org.testng.TestNG testng.xml
If you achieve this, your Dockerfile could be:
FROM maven:3.3-jdk-8
RUN mkdir /usr/test
COPY . /usr/test
WORKDIR /usr/test
RUN mvn clean package
CMD["java","-jar", "selenium-test.jar","org.testng.TestNG","testng.xml"]
Note: I'm not tested this Dockerfile yet
In this approach, the compilation is done at docker build phase, so it is more fast than previous approach
Common mistake
If your selenium tests opens a browser in you developer machine, you could not achieve this with a single docker container.
Selenium needs a operative system with desktop interface and a browser installed. At developer phase, all of this is performed on you developer machine. On real environments, you have these options
- ubuntu with desktop
Is not common but it is possible. If you choose this, you will need to install many browsers (and its selenium drivers) as much as you can test. As this approach is not a shell solution, you will need to install an agent to be executed remotely. Also you will need emulators to be able to launch browser of specific os like safari or microsoft edge. This will be a nigthmare
So basically this is the same of your developer machine but in another network or in a cloud.
- Selenium grid server
Similar to the previous option, but more elegantly. Check:
https://digital.ai/catalyst-blog/set-up-cross-browser-testing-with-our-selenium-grid-tutorial
https://github.com/SeleniumHQ/docker-selenium
You are not saved of browser installation, but it is free.
Your test code will be the same, just the configuration change a little bit:
https://www.mstsolutions.com/technical/execution-of-test-in-remote-machine-using-selenium-grid/
- BrowserStack ($)
Basically, is a Selenium grid server private service ready to use which needs a payment (Sometime the time is worthier than money). Just need to point your selenium test to its url. You just pick your browsers and run:
https://www.browserstack.com/docs/automate/selenium/getting-started/java#run-your-first-test
Also with this service, docker may not necessary because you just need a simple mvn test or java -jar. These commands could be launched with a Jenkins or a simple shell script in your devops server.
- headless browser
Basically are browsers that run in background mode in your shell using your ram. This is perfect if you cannot pay browserstack o configure your own selenium grid server.
The only disadvantage is that some latest javascript features may not be work in this kind of virtual browsers. Also don't support features like printers, camera, or another low level requirement or in which a real UI is required.
Here some options:
https://phantomjs.org/
https://hacks.mozilla.org/2017/12/using-headless-mode-in-firefox/
https://developers.google.com/web/updates/2017/04/headless-chrome
I've installed the PyLint PyCharm plugin (https://plugins.jetbrains.com/plugin/11084-pylint). I can get the linting to work if I choose the default project interpreter to be the one installed on my Windows laptop, but my project interpreter is the one on my attached Docker image. When the program default interpreter is the one on the Docker image, the PyLint plugin complains of:
The project interpreter is missing Pylint, which is needed to properly check the imports.
I've installed pylint on the docker image, it does not however show up package list when looking in File -> Settings -> Project Interpreter
Does anyone know if the PyLint plugin should work with this workflow?
To run Pylint in a Docker container I configured it as an external tool.
Edit Tool window screenshot
Program:
docker-compose
Arguments:
run --rm django pylint --msg-template="$ProjectFileDir$/{path}:{line}:{column}: {msg_id}: {msg} ({symbol})" $FilePathRelativeToProjectRoot$
msg-template argument makes each file path clickable so you can easily navigate to a line with a problem.
I did functional tests in JMeter (backend application).
I have in Jmeter 42 functional tests and I launch this in one docker container
Now I configuration JMeter with Jenkins by docker container -
My dockerfile is following:
[https://pastebin.com/Aq9A9eqh][1]
To automatically run these tests regullary,
I created docker - container and when I run build execution on Jenkins, I had a following error:
https://pastebin.com/365kLWXB
What is wrong?
How is the best way to improve this?
What can I improve further?
I know that I must add a java jdk, but I don't know how to add to my dockerfile?
https://pastebin.com/Aq9A9eqh
When I added:
FROM openjdk:8u162-jre-slim-stretch
ARG GITHUB_OAUTH_TOKEN
it didn't run.
With regards to your current setup your Docker image doesn't have Java installed therefore you cannot launch JMeter.
You can fix it by replacing these line:
RUN apt-get install -y git
with this one:
RUN apt-get install -y git default-jre
With regards to "improve further" be aware that according to JMeter Best Practices you should always be using the latest version of JMeter so consider replacing JMeter version 3.3 with somethine more recent, as of now it's JMeter 5.0, you can always check JMeter Downloads page for current release details.
I have a custom Docker container in which I perform build and test of a project. It is somehow integrated with Travis CI. Now I want to run the Coverity scan analysis from within the Travis CI as well, but the tricky part is (if I understand the Coverity docs correctly), that I need to run the build. The build, however, runs in the container.
Now, according to the cov-build --help
The cov-build or cov-build-sbox command intercepts all calls to the
compiler invoked by the build system and captures source code from the
file system.
What I've tried:
cov-build --dir=./cov docker exec -ti container_name sh -c "<custom build commands>"
With this approach, however, Coverity apparently does not catch the calls to the compiler (it is quite understandable considering Docker philosophy) and emits no files
What I do not want (at least while there is hope for a better solution):
to install locally all the necessary stuff to build in the container
only to be able to run Coverity scan.
to run cov-build from within the container, since:
I believe this would increase the docker image size significantly
I use Travis CI addon for the Coverity scan and this would
complicate things a lot.
The Travis CI part just FWIW, tried all that locally and it doesn't work either.
I am thrilled for any suggestions to the problem. Thank you.
Okay, I sort of solved the issue.
I downloaded and modified ( just a few modification to fit my
environment ) the script that Travis uses to download and run Coverity
scan.
Then I installed Coverity to the host machine (in my case Travis
CI machine).
I ran the docker container and mounted the directory where the
Coverity is installed using docker run -dit -v <coverity-dir>:<container-dir>:ro .... This way I avoided increasing the docker image size.
Executed the cov-build command and uploaded the analysis using
another part of the script directly from docker container.
Hope this helps someone struggling with similar issue.
If you're amenable to adjusting your build, you can change your "compiler" to be cov-translate <args> --run-compile <original compiler command line>. This is effectively what cov-build does under the hood (minus the run-compile since your compiler is already running), and should result in a build capture.
Here is the solution I use:
In "script", "after_script" or another phase in Travis job's lifecycle you want
Download coverity tool archive using wget (the complete Command to use can be found in your coverity scan account)
Untar the archive into a coverity_tool directory
Start your docker container as usual without needing to mount coverity_tool directory as a volume (in case you've created coverity_tool inside the directory from where the docker container is started)
Build the project using cov-build tool inside docker
Archive the generated cov-int directory
Send the result to coverity using curl command
Step 6 should be feasible inside the container but I usually do it outside.
Also don't forget the COVERITY_SCAN_TOKEN to be encrypted and exported as an environment variable.
A concrete example is often more understandable than a long text; here is a commit that applies above steps to build and send results to coverity scan:
https://github.com/BoubacarDiene/NetworkService/commit/960d4633d7ec786d471fc62efb85afb5af2bed7c
i try to integrate my Robotium Tests on our Jenkins Server. I implemented an android Test Project that based on our app Project. Everything works like a charm when i run the Tests on my local Machine on Windows out of Eclipse and also by calling from shell by call
'adb shell am instrument -w <our-package>/android.test.InstrumentationTestRunner'
So far, so good, but after i set up a build Job in Jenkins, let the Projects build by maven clean install sign etc. none of my tests will be proceeded. I also tried to call 'adb shell am instrument -w <our-package>/android.test.InstrumentationTestRunner' as shell command after the build step but also no result. The build ended up with just 2 Tests shown log below
[workspace] $ /bin/sh -xe /tmp/hudson3571502822112946903.sh
+ /home/jenkins/tools/android-sdk-linux/platform-tools/adb shell pm list instrumentation
instrumentation:com.android.emulator.connectivity.test/android.test.InstrumentationTestRunner (target=com.android.emulator.connectivity.test)
instrumentation:com.android.emulator.gps.test/android.test.InstrumentationTestRunner (target=com.android.emulator.gps.test)
instrumentation:com.zeppelin.zemos.test.addispo/android.test.InstrumentationTestRunner (target=com.zeppelin.zemos.addispo)
+ /home/jenkins/tools/android-sdk-linux/platform-tools/adb shell am instrument -w com.zeppelin.zemos.test.addispo/android.test.InstrumentationTestRunner
android.test.AndroidTestCase:.
android.test.LoaderTestCase:.
Test results for InstrumentationTestRunner=..
Time: 0.031
OK (2 tests)
Just the 2 Tests AndroidTestCase and LoaderTestCase are shown up and it seems that all of my other Testcases (i've implemented 11) are not processed by Jenkins.
I spent a long time googling around but found no solution for this. Could this be a ant Problem? I have Version 1.8 local and 1.6 on the Jenkins Server. Or is there another Problem i cannot see.....
Thanks a lot
You can try uninstalling the package from device/ emulator, using
"adb uninstall your_package"
Then, do a clean debug install again and then run the tests.