Postman is used as a popular testing tool for API testing, you can write bunch of unit tests using Postman and execute them as a part of your build process to perform unit testing. The following covers the Jenkins integration of Postman tests.
In order to do that you should have
Exported Postman tests as a collection
Make the APIs available at run time when the tests are performed. (Using docker or by creating a separate build pipeline.)
Node module newman can be used to execute Postman collections. Refer the following Package.json file. Here we are executing the postman collection inside the unit_tests folder using newman, also newman dependency is defined.
package.json
{
"name": "postman-newman-jenkins",
"version": "1.0.0",
"description": "My Test Project",
"directories": {
"tests": "tests"
},
"scripts": {
"newman-tests": "newman run unit_tests/my-collection.postman_collection.json --reporters cli,junit --reporter-junit-export newman.xml --insecure"
},
"author": "Test Author",
"dependencies": {
"newman": "^3.5.2"
}
}
The following is the content of the Jenkinsfile. We are using NPM to install the dependencies and execute tests.
Jenkinsfile
pipeline {
agent { label 'LinuxSlave' }
stages {
stage ('Checkout') {
steps {
checkout scm
}
}
stage('Test'){
steps {
sh 'npm install'
sh 'npm run newman-tests'
junit 'newman.xml'
}
}
}
}
you can avoid installing newman on the machine (slave/ master) and use docker
example pipeline script:
pipeline {
agent any stages {
stage('Test') {
steps {
sh 'docker run -t postman/newman_ubuntu1404 run https://www.getpostman.com/collections/8a0c9bc08f062d12dcda'
}
}
}
}
more info on docker & newman here
Related
I am struggling, to get the right configuration for my Jenkins Pipeline.
It works but I could not figure out how to seperate test & build stages.
Requirements:
Jenkins Pipeline with seperated test & build stage
Test stage requires chromium (I currently use node alpine image + adding chromium)
Build stage is building a docker image, which is published later (publish stage)
Current Setup:
Jenkinsfile:
pipeline {
environment {
...
}
options {
...
}
stages {
stage('Restore') {
...
}
stage('Lint') {
...
}
stage('Build & Test DEV') {
steps {
script {
dockerImage = docker.build(...)
}
}
}
stage('Publish DEV') {
steps {
script {
docker.withRegistry(...) {
dockerImage.push()
}
}
}
}
Dockerfile:
FROM node:12.16.1-alpine AS build
#add chromium for unit tests
RUN apk add chromium
...
ENV CHROME_BIN=/usr/bin/chromium-browser
...
# works but runs both tests & build in the same jenkins stage
RUN npm run test-ci
RUN npm run build
...
This works, but as you can see "Build & Test DEV" is a single stage,
I would like to have 2 seperate jenkins stages (Test, Build)
I already tried using Jenkins agent docker and defining the image for the test stage inside the jenkins file, but I dont know how to add the missing chromium package there.
Jenkinsfile:
pipeline {
agent {
docker {
image 'node:12.16.1-alpine'
//add chromium package here?
//set Chrome_bin env?
}
}
I also thought about using a docker image that already includes chromium, but couldnt find any official images
Would really appreciate your help / insights how to make this work.
You can either build your customized image (which includes the installation of Chromium) and push it to a registry and then pull it from that registry:
node {
docker.withRegistry('https://my-registry') {
docker.image('my-custom-image').inside {
sh 'make test'
}
}
}
Or build the image directly with Jenkins with your Dockerfile:
node {
def testImage = docker.build("test-image", "./dockerfiles/test")
testImage.inside {
sh 'make test'
}
}
Builds test-image from the Dockerfile found at ./dockerfiles/test/Dockerfile.
Reference: Using Docker with Pipeline
So in general I would execute the npm run commands inside the groovy syntax and not inside the dockerfile. So your code would look something like that:
pipeline {
agent {
docker {
image 'node:12.16.1-alpine'
args '-u root:root' // better would be to use sudo, but this should work
}
}
stages {
stage('Preparation') {
steps {
sh 'apk add chromium'
}
}
stage('build') {
steps {
sh 'npm run build'
}
}
stage('test') {
steps {
sh 'npm run test'
}
}
}
}
I would also suggest that you collect the results within Jenkins with the warnings ng jenkins plugin
Newbie to jenkins.
Experimenting to run unit tests (not even sure if the step to run is correct)
and also need to make a report available within jenkins (any suggestions how i can make this possible?)
pipeline {
agent any
stages
{
stage("Build") {
steps {
echo 'Building the appication...'
}
}
stage ('Unit test') {
steps {
sh 'npm run test'
}
}
stage ("Deploy") {
steps {
echo 'Deploying the appication...'
}
}
}
}
Use for example xunit tool (can be invoked on post actions or simply as step) to record a report from tests.
Here is the documentation:
https://plugins.jenkins.io/xunit/
To create a report with npm run test command you have to define it in package.json of the project or wherever the command is defined
Rwmeber also to add script { after steps { in the stage where you call Bash script
How to Execute this below code in paralley.. please help iam new to jenkins.
parallel firstBranch: {
node{
newman run xyz.json
}
},
secondBranch:{
node{
newman run xyz123.json
}
}
},
failFast: true|false
for this i am gettig below error:
Also:
org.jenkinsci.plugins.workflow.steps.FlowInterruptedException
at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution.cancel(CpsBodyExecution.java:244)
at org.jenkinsci.plugins.workflow.steps.BodyExecution.cancel(BodyExecution.java:76)
at org.jenkinsci.plugins.workflow.cps.steps.ParallelStepExecution.stop(ParallelStepExecution.java:67)
at org.jenkinsci.plugins.workflow.cps.steps.ParallelStep$ResultHandler$Callback.checkAllDone(ParallelStep.java:147)
at org.jenkinsci.plugins.workflow.cps.steps.ParallelStep$ResultHandler$Callback.onFailure(ParallelStep.java:134)
at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$FailureAdapter.receive(CpsBodyExecution.java:349)
at com.cloudbees.groovy.cps.impl.ThrowBlock$1.receive(ThrowBlock.java:68)
try the Jenkins documentaiton exemple click in the link bellow :
Parallel stages with Declarative Pipeline 1.2
to use new man with jenkins it's very easy
install nodeJS and npm
install newman by `npm install -g newman
create json file newman-test:
{
"name": "postman-newman-jenkins",
"version": "1.0",
"description": "Project postman test",
"directories": {
"test": "tests"
},
"scripts": {
"api-tests-production": "newman run postman_collection.json --reporters cli,html --reporter-html-export ${Environment}-$BUILD_NUMBER-newman.html --reporter-html-template template.hbs
},
"author": "me or you",
"dependencies": {
"newman": "^3.5.2"
}
}
in you jenkins
try {
sh 'npm install'
sh 'npm run newman-test'
}
catch (Exception err) {
currentBuild.result = 'UNSTABLE'
}
to archive your report
archiveArtifacts 'jenkins/${Environment}-$BUILD_NUMBER-newman.html'
in the last you can format this code to respect declarative pipeline code
In my Jenkinsfile I have a stage Test where I run a npm test command step as well as a junit step to archive test results.
stage('Test') {
steps {
sh 'npm run test-ci'
junit 'test-results.xml'
}
}
How can I use try/finally correctly to run the junit step even if the sh 'npm run test-ci' step fails?
You want to use the post stage, https://jenkins.io/doc/book/pipeline/syntax/#post.
pipeline {
agent any
stages {
stage('Test') {
steps {
sh 'npm run test-ci'
}
}
post {
always {
junit 'test-results.xml'
}
}
}
Also have a look at this blog post, it explains it further, https://jenkins.io/blog/2017/02/10/declarative-html-publisher/
When our browser based tests fail, we take a screenshot of the browser window to better illustrate the problem. However, I don't understand how to archive them in my pipeline, because the pipeline stops after the failure. Same for the junit.xml, I'd also like to use it in error cases.
I've checked, the screenshots are generated and stored correctly.
My definition looks like this (irrelevant things mostly trimmed):
node {
stage('Build docker container') {
checkout([$class: 'GitSCM', ...])
sh "docker build -t webapp ."
}
stage('test build') {
sh "mkdir -p rspec screenshots"
sh "docker run -v /var/jenkins_home/workspace/webapp/rspec/junit.xml:/myapp/junit.xml -v /var/jenkins_home/workspace/webapp/screenshots:/myapp/tmp/capybara -v webapp bundle exec rspec"
}
stage('Results') {
junit 'rspec/junit*.xml'
archive 'screenshots/*'
}
}
You can use simple Java try/catch to avoid pipeline failure on test failure, or Jenkins catchError like this :
node {
catchError {
// Tests that might fail...
}
// Archive your tests artifacts
}
From here, you can use the post section in your pipeline:
pipeline {
agent any
stages {
stage('Build') {
...
}
stage('Test') {
...
}
}
post {
always {
archive 'build/libs/**/*.jar'
}
}
}