I'm running my cypress tests on Jenkins inside a dockerized container and I generate cypress mocha awesome report, but I don't know how to display it inside Jenkins.
This is my cypress.json content
{
"integrationFolder": "test/specs",
"supportFile": "test/support/index.js",
"video": true,
"reporter": "node_modules/cypress-multi-reporters",
"reporterOptions": {
"reporterEnabled": "mochawesome",
"mochawesomeReporterOptions": {
"reportDir": "results/mocha",
"overwrite": false,
"html": false,
"json": true,
"timestamp": "mmddyyyy_HHMMss",
"showSkipped": true,
"charts": true,
"quite": true,
"embeddedScreenshots": true
}
},
"screenshotOnRunFailure": true,
"screenshotsFolder": "results/mochareports/assets/screenshots",
"videosFolder": "results/mochareports/assets/videos",
"baseUrl": "http://testurl.com",
"viewportWidth": 1920,
"viewportHeight": 1080,
"requestTimeout": 10000,
"responseTimeout": 10000,
"defaultCommandTimeout": 10000,
"watchForFileChanges": true,
"chromeWebSecurity": false
}
And here is my scripts which I run locally.
"clean:reports": "rm -R -f results && mkdir results && mkdir results/mochareports",
"pretest": "npm run clean:reports",
"cypress:interactive": "cypress open",
"scripts:e2e": "cypress run",
"combine-reports": "mochawesome-merge results/mocha/*.json > results/mochareports/report.json",
"generate-report": "marge results/mochareports/report.json -f report -o results/mochareports -- inline",
"posttest": "npm run combine-reports && npm run generate-report",
"test:e2e": "npm run pretest && npm run scripts:e2e || npm run posttest",
I can view my generated report successfully in the local environment.
Here is my jenkinsfile content
#!groovy
pipeline {
agent any
stages {
stage('Checkout') {
steps {
echo 'Checking out the PR'
checkout scm
}
}
stage('Build') {
steps {
echo 'Destroy Old Build'
sh 'make destroy'
echo 'Building'
sh 'make upbuild_d'
}
}
stage('Test') {
steps {
echo 'Running Tests'
sh 'make test-e2e'
}
}
stage('Destroy') {
steps {
echo 'Destroy Build'
sh 'make destroy'
}
}
}
}
The make test-e2e actually runs the test:e2e script inside a docker container, the tests actually run and I can see the reports get generated on Jenkins but I don't know how to view it.
I need to view it in a separate inside Jenkins, also I don't know why I can't access it via Jenkins workspace.
btw. I'm adding the results file in .gitignore
This is my local report preview
You can use the HTML publisher plugin for Jenkins for this:
https://plugins.jenkins.io/htmlpublisher/
Within your Jenkinsfile add a stage to publish the HTML reports
e.g.
publishHTML([
allowMissing: false,
alwaysLinkToLastBuild: false,
keepAll: true,
reportDir: 'cypress/cypress/reports/html',
reportFiles: 'index.html',
reportName: 'HTML Report',
reportTitles: ''])
I used the HTML Publisher plugin as the mentioned solution above however my problem was that my results file was in the docker container not in Jenkins workspace and I fixed this problem by copying the folder from a docker container to Jenkins workspace.
docker cp container_name:/app/results ./results
Related
Currently , my cypress testes are runnning in docker container on one stage
stage('Run E2E tests') {
steps {
withCredentials([
sshUserPrivateKey(credentialsId: '*********', keyFileVariable: 'SSH_KEY_FILE', usernameVariable: 'SSH_USER')
]) {
sh """
eval `ssh-agent -s`
ssh-add ${SSH_KEY_FILE}
~/earthly \
--no-cache \
--config=.earthly/config.yaml \
+e2e
eval `ssh-agent -k`
"""
}
}
}
And publishing the test report to via publishHTML.
post {
always {
echo "-- Archive report artifacts"
archiveArtifacts artifacts: 'results', allowEmptyArchive: 'true'
echo "-- Publish HTLM test result report"
publishHTML (target: [
allowMissing: false,
alwaysLinkToLastBuild: false,
keepAll: true,
reportDir: 'results/html/',
reportFiles: 'mochawesome-bundle.html',
reportName: "Test Result Report"
])
}
}
But i need to make the build failure if any of the TC failure in the cypress mocha report
what can be the solution for this..?
Thanks in advance
I run my regression tests on docker container and I am trying to publish Test Results in jenkins-pipline using HTML-Publisher. This doesn't work properly, thought I get a mistake by trying to copy the result-file from docker container (Error type: such file does not exist).
My Jenkinsfile looks like this:
//https://www.jenkins.io/doc/book/pipeline/syntax/
pipeline {
agent any
stages {
stage('Deploy webstore') {
steps {
//start and run an application container using .yml file
sh "docker compose -f webstore-compose.yml up -d"
}
}
stage ('Regression Tests') {
//setting up docker container for regression tests
agent {
docker {
image 'localhost:5000/dotnet_s3'
args '--add-host=host.docker.internal:host-gateway'
reuseNode true
}
}
steps {
//running tests located in /guiautomationtask directory in the top layer and logging into testResults.html file
sh 'id; cd /guiautomationtask; dotnet test --logger "html;logfilename=testResults.html"'
sleep(time: 10, unit: "SECONDS")
/*To Do:
copy logfile from container to local*/
//console output
echo "++++++++++++++++++++++++++++++++++ Display Test Results in the Console +++++++++++++++++++++++++++++++++++++++++"
echo "Running build ${env.BUILD_ID} on jenkins ${env.JENKINS_URL}"
echo "current docker container ID is ${hostname}"
sh "id; cd /guiautomationtask; dotnet test -v normal"
echo "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
/*sh "dotnet publish /guiautomationtask/GuiTest/GuiTest.csproj"
sleep(time: 10, unit: "SECONDS")*/
}
}
stage ('Publish results') {
steps {
//view test-logs via HTML Publisher plugin
publishHTML(target:[
allowMissing: false,
alwaysLinkToLastBuild: true,
keepAll: false,
reportDir: "", //here should be report directory with saved html report file
reportFiles: "testResults.html",
reportName: 'HTML-Report',
//reportTitles: ''
])
echo "artifacts saved in zip";
}
}
}
post {
always {
//stop an application container
sh "docker compose -f webstore-compose.yml stop"
}
}
}
In my pipeline, I run testcases in the docker container then I copy some directories from the docker container to the Jenkins workspace.
It isn't necessary that all directories will exist in the docker container (for example screenshot dir may exists or not according to failing tests). How can I check for directory or file existence before copying it from docker container.
Here the part I mention in the pipeline
post {
always {
echo 'Generating Test Reports ...'
sh 'make posttest'
echo('Copying Test Files ...')
sh 'docker cp container-name:/app/results/mochareports/assets/videos ./results'
sh 'docker cp container-name:/app/results/mochareports/assets/screenshots ./results'
sh 'docker cp container-name:/app/results/mochareports/report.html ./results'
echo 'Publish Test Reports ...'
publishHTML (target : [allowMissing: false,
alwaysLinkToLastBuild: true,
keepAll: true,
reportDir: 'results/mochareports',
reportFiles: 'report.html',
reportName: 'Cypress Test Reports',
reportTitles: 'The test report'])
echo 'Destroy Build'
sh 'make destroy'
cleanWs()
}
}
Groovy-way is to use fileExists https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#fileexists-verify-if-file-exists-in-workspace
if(fileExists('/app/results/mochareports/assets/screenshots/')) {
...
}
While running the Cypress tests on jenkins, I am getting the below error. Our jenkins is integrated with Docker container and devs asked me to install the pdf-parse library in docker container which will solve the issue. How do I install pdf-parse in docker container, which file does that ? Could some one please advise ?
Note: I am unable to see a docker file in my project root directory
11:38:29 Or you might have renamed the extension of your `pluginsFile`. If that's the case, restart the test runner.
11:38:29
11:38:29 Please fix this, or set `pluginsFile` to `false` if a plugins file is not necessary for your project.
11:38:29
11:38:29 Error: Cannot find module 'pdf-parse'
docker file:
FROM cypress/browsers:node12.14.1-chrome85-ff81
COPY package.json .
COPY package-lock.json .
RUN npm install --save-dev cypress
RUN $(npm bin)/cypress verify
# there is a built-in user "node" that comes from the very base Docker Node image
# we are going to recreate this user and give it _same id_ as external user
# that is going to run this container.
ARG USER_ID=501
ARG GROUP_ID=999
# if you want to see all existing groups uncomment the next command
# RUN cat /etc/group
RUN groupadd -g ${GROUP_ID} appuser
# do not log creating new user, otherwise there could be a lot of messages
RUN useradd -r --no-log-init -u ${USER_ID} -g appuser appuser
RUN install -d -m 0755 -o appuser -g appuser /home/appuser
# move test runner binary folder to the non-root's user home directory
RUN mv /root/.cache /home/appuser/.cache
USER appuser
jenkins file:
pipeline {
agent {
docker {
image 'abcdtest'
args '--link postgres:postgres -v /.composer:/.composer'
}
}
options {
ansiColor('xterm')
}
stages {
stage("print env variables") {
steps {
script {
echo sh(script: 'env|sort', returnStdout: true)
}
}
}
stage("composer install") {
steps {
script {
withCredentials([usernamePassword(credentialsId: 'bitbucket-api', passwordVariable: 'bitbucketPassword', usernameVariable: 'bitbucketUsername')]) {
def authProperties = readJSON file: 'auth.json.dist'
authProperties['http-basic']['bitbucket.sometest.com']['username'] = bitbucketUsername
authProperties['http-basic']['bitbucket.sometest.com']['password'] = bitbucketPassword
writeJSON file: 'auth.json', json: authProperties
}
}
sh 'php composer.phar install --prefer-dist --no-progress'
}
}
stage('unit tests') {
steps {
lock('ABCD Unit Tests') {
script {
try {
sh 'mv codeception.yml.dist codeception.yml'
sh 'mv tests/unit.suite.yml.jenkins tests/unit.suite.yml'
sh 'php vendor/bin/codecept run tests/unit --html'
}
catch (err) {
echo "unit tests step failed"
currentBuild.result = 'FAILURE'
}
finally {
publishHTML (target: [
allowMissing: false,
alwaysLinkToLastBuild: false,
keepAll: true,
reportDir: 'tests/_output/',
reportFiles: 'report.html',
reportName: "Unit Tests Report"
])
}
}
}
}
}
}
post {
success {
slackSend color: 'good', channel: '#jenkins-abcdtest-ci', message: "*SUCCESSED* - CI passed successfully for *${env.BRANCH_NAME}* (<${env.BUILD_URL}|build ${env.BUILD_NUMBER}>)"
}
failure {
slackSend color: 'danger', channel: '#jenkins-abcdtest-ci', message: "*FAILED* - CI failed for *${env.BRANCH_NAME}* (<${env.BUILD_URL}|build ${env.BUILD_NUMBER}> - <${env.BUILD_URL}console|click here to see the console output>)"
}
}
}
I suppose you use cypress/base:10 as the image to new a container in jenkins. If you don't have dockerfile, you may have to write your own dockerfile extends from cypress/base:10.
Dockerfile:
FROM cypress/base:10
RUN npm install pdf-parse
Then, docker build -t mycypress ., docker push mycypress to push the image to dockerhub(You may need an account) to let your jenkins use your new image to setup container.
NOTE: you will have to find how your project choose image to start your container, with this, you can find suitable way to install pdf-parse. One possible maybe next:
pipeline {
agent {
docker { image 'cypress/base:10' }
}
stages {
stage('Test') {
steps {
sh 'node --version'
}
}
}
}
Then, you may change docker { image 'cypress/base:10' } to docker { image 'mycypress' }.
I'm trying to install and run Postman's Newman tests collection with HTML reporter (on a jenkins podTemplate container with docker image from Postman's account) but it keeps failing because no suitable Newman version is found:
"npm WARN newman-reporter-htmlextra#1.19.6 requires a peer of newman#>=4 but none is installed. You must install peer dependencies yourself"
Newman image docker is "postman/newman:5.2-alpine".
And the Run command is
sh "newman run tests/collection.json -r htmlextra --reporter-htmlextra-export var/reports/newman/html/index.html";
I've also tried to install with (the "sh" prefix is because it's in groovy script..in Jenkins) :
sh "npm install -g newman#4.6.1"
sh "npm install -g newman-reporter-htmlextra"
and then executing the same run command as above.
sh "newman run tests/collection.json -r htmlextra --reporter-htmlextra-export var/reports/newman/html/index.html";
But the results are the same.
What's weird is that right after I get the error mentioned above - the jenkinsfile executes the "newman run" command and successfully creates the tests report file:
Using htmlextra version 1.19.6
Created the htmlextra report in this location: var/reports/newman/html/index.html
But then exits the script/job with FAILURE.
What am I missing?
Any advice?
Thank you!
Thats a npm bug, https://github.com/npm/npm/issues/12905
for newman-reporter-htmlextra , newman is a peer dependency.
In npm peer dependency is not detected for global packages if the dependency and the package are not installed together
In this case you can fix it by installing it together using
npm install -g newman newman-reporter-htmlextra
Try :
podTemplate(label: "newmanPodHtmlExtra", containers: [
containerTemplate(name: "newman", image: "dannydainton/htmlextra", command: "cat", ttyEnabled: true),
]) {
node("newmanPodHtmlExtra") {
def testsFolder = "./tests";
container("newman") {
stage("Checkout") {
checkout scm;
}
try{
stage("Install & run Newman") {
sh "npm install -g newman newman-reporter-htmlextra";
sh "newman run ${testsFolder}/collection.json -r htmlextra --reporter-htmlextra-export var/reports/newman/html/index.html";
}
}catch(e){}finally{
stage("Show tests results") {
publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'var/reports/newman/html', reportFiles: 'index.html', reportName: 'API Tests', reportTitles: ''
])
}
}
}
}
}