I was just very shocked to see that one of my web projects coverage dropped quite a bit after I introduced end-to-end tests. Those tests do not contribute anything to line coverage as they test against another environment and thus pytest cannot know which lines were covered.
I've marked those lines with #pytest.mark.e2e and I have two stages in my GitlabCi pipeline:
stages:
- test
- route-tests
Unit Tests:
stage: test
script:
- pytest -m "not e2e"
artifacts:
reports:
cobertura: cobertura-coverage.xml
Route Tests:
stage: route-test
script:
- pytest -m e2e
Surprisingly, both GitlabCI stages coverage is in the analytics:
Although only the test stage has the cobertura reports section, somehow both are in gitlab. How can I disable the coverage reporting for the route-test stage? Or is it possible to restrict the test coverage report badge to the test stage?
As a work-around, I just made sure that only a single gitlab stage produces a coverage report / file.
Related
Im totally new to this Devops field basically for Jenkins, Groovy file is used to maintain preparation-build-Deploy, Similarly for Bamboo which script is used?
I got to know bamboo plan is used. But how the plan is generated though any script or any file.
And i have pipeline for Jenkins similarly how it can be done for Bamboo plan.
the groovy file for Jenkins is
node {
stage('Preparation') { // for display purposes
// Get EDM code from a GitHub repository
cleanWs()
checkout scm
sh "python $WORKSPACE/common/deployment_scripts/abc.py --localFolder $WORKSPACE --env dev"
}
stage('Build') {
// Run the maven build
sh "mvn clean install -f $WORKSPACE/pom.xml -Dmaven.test.skip=true"
}
stage('Deploy') {
//Run the deployment script
sh "python $WORKSPACE/common/deployment_scripts/ase.py $WORKSPACE lm-edm-builds-ndev ${env.BUILD_NUMBER} dev"
sh "python $WORKSPACE/common/deployment_scripts/qwert.py --JsonParameterFile $WORKSPACE/common/deployment_scripts/my_properties.json --BuildVersion ${env.BUILD_NUMBER} --WorkSpace $WORKSPACE --environment dev"
}
}
For Bamboo, you can do so with Bamboo Specs. The Bamboo Specs allows you to define Bamboo configuration as code, and have corresponding plans/deployments created or updated automatically in Bamboo. Read more about the Bamboo Specs here.
Bamboo Specs recognize two ways of creating plans, with Java or YAML. Select the one that matches your needs best. The syntax for both can be found in their official reference documentation.
A sample YAML Specs to define a plan can look like below as detailed in this page:
---
version: 2
plan:
project-key: MARS
key: ROCKET
name: Build the rockets
# List of plan's stages and jobs
stages:
- Build the rocket stage:
- Build
#Job definition
Build:
tasks:
- script:
- mkdir -p falcon/red
- echo wings > falcon/red/wings
- sleep 1
- echo 'Built it'
- test-parser:
type: junit
test-results: '**/junit/*.xml'
# Job's requirements
requirements:
- isRocketFuel
# Job's artifacts. Artifacts are shared by default.
artifacts:
- name: Red rocket built
pattern: falcon/red/wings
You may start with this tutorial for Creating a simple plan with Bamboo Java Specs
We are using stages to deploy and run our different kinds of regression tests in parallel using stages. Is there a way I can skip certain scripts within stages based on an environment variable that is available when the travis job kicks off.
I read the travis-ci documentation and SO questions and this is the best I could come up with. This is a snippet of our current setup.
jobs:
include:
- stage: Setup
script:
- reset_db
- reset_es_index
name: Setting up Environment
if: type IN (push, api)
- stage: Tests
script:
- run_fast
name: Fast tests
if: type IN (push, api)
allow_failure: true
- script:
- run_api_tests
name: Api tests
if: type = api AND env(TRIGGER_REPO) != com_ui_project
- script:
- run_slow_tests
name: Slow tests
if: type = api AND env(TRIGGER_REPO) != com_ui_project
As you can see I want to run the Api tests and the Slow tests scripts in the Test stage only if the travis job was triggered by an API call AND the custom environment variable is NOT com_ui_project. Otherwise run the Fast tests script and skip the other scripts.
Right now if the travis was triggered by the api call, all the scripts in the stage Tests run. How can I avoid that?
I tried the following too
$TRIGGER_REPO != com_ui_project
env(TRIGGER_REPO) != "com_ui_project"
$TRIGGER_REPO!="com_ui_project" (Thinking that using shell formatting might help. It did not)
I am running pytest tests in a Jenkins scheduled job and generating a junitxml report as the following
pytest --junitxml=report.xml
Then I am using a post build action of publish junit test result and I can see the results but I don't see output of the passed tests (Even if I check the checkbox of "Retain long standard output/error"
Anyone succeeded in showing the output of passed tests when using pytest + junitxml publisher in Jenkins ?
It's due to Pytest that doesn't add end of line in the .xml.
How do I display "npm lint" results in Jenkins? I know how to display "pytest" results in Jenkins:
Run pytest like this: pytest --junitxml=./path/to/pytestFile.xml ./path/to/code
Run this Jenkins command: junit testResults: "./path/to/pytestFile.xml"
Once I do this, my test results appear in a "Test Results" window in Jenkins. I want to do something similar with my angular lint results. I tried doing this, but it didn't work:
Run linter like this: npm lint --junitxml=./path/to/lintFile.xml
Run this Jenkins command: junit testResults: "./path/to/lintFile.xml"
The problem is that npm doesn't have a --junitxml switch. How can I do this? I'm using the Jenkins declarative pipeline.
JUnit is a format for test results, not lint results. It makes no sense to try to use it.
The best plugin to use is Warnings Next Generation. It lists tslint as a supported format.
steps {
sh 'npm run-script --silent -- ng lint --format=checkstyle >checkstyle-result.xml'
}
post {
always {
recordIssues tool: tsLint(pattern: 'checkstyle-result.xml'),
enableForFailure: true
}
}
If there are multiple projects in your workspace, you may be affected by #14659. Either specify a single project or pipe the output through split -l 1.
As you can see, you then get a report specifically for TSLint, separate from other analysis issues and test results. It also understands the different severity levels, which cannot be expressed with JUnit.
Using Protractor/Jasmine conjunction automation framework I want to run test suite. When my Jenkins job runs I do not want to fail my job if any test case fails.
Pytest in python provides #pytest.mark.xfail feature to mark expected failures and this does not impact the jenkins job.
Is there any such feature in Protractor which can mark test cases as expected to fail?
I saw xit and xdescribe features but it skips the test case rather then expected failure