How to run Jenkins parallel cypress on different agents? - docker

I'm running parallel cypress in Jenkins on the same slave, and it's working,
I want to change the parallel stages so each stage will run on a different slave, how can I do it?
for example:
run "cypress tester A" on slave-1.
run "cypress tester B" on slave-2.
run "cypress tester C" on slave-3.
this is my current Jenkinsfile:
pipeline {
options {
timeout(time: 15, unit: 'MINUTES')
}
agent {
docker {
image 'cypress/base:12.18.2'
label 'slave-1'
}
}
parameters {
string(defaultValue: 'master', description: 'Branch name', name: 'branchName')
}
stages {
stage('build') {
steps {
echo 'Running build...'
sh 'npm ci'
sh 'npm run cy:verify'
}
}
stage('cypress parallel tests') {
environment {
CYPRESS_RECORD_KEY = 'MY_CYPRESS_RECORD_KEY'
CYPRESS_trashAssetsBeforeRuns = 'false'
}
parallel {
stage('cypress tester A') {
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run cypress:run"
}
}
stage('cypress tester B') {
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run cypress:run"
}
}
stage('cypress tester C') {
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run cypress:run"
}
}
}
}
}
post {
always {
cleanWs(deleteDirs: true)
echo 'Tests are finished'
}
}
}
The cypress:run command is:
cypress run --record --parallel --config videoUploadOnPasses=false --ci-build-id $BUILD_TAG

I was able to get this to work by explicityly defining the agent within each parallel stage:
parallel {
stage('cypress tester A') {
agent {
node: {
label "slave-1"
}
}
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run cypress:run"
}
}
stage('cypress tester B') {
agent {
node: {
label "slave-2"
}
}
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run cypress:run"
}
}
stage('cypress tester C') {
agent {
node: {
label "slave-3"
}
}
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run cypress:run"
}
}
}
However, one disadvantage I found is now that you're running cypress in each individual node/virtual machine, cypress needs to know where to find the running instance of your application. Cypress looks into cypress.json at baseUrl to see where to find your app. Its common to use a localhost address for development, which means cypress runnning on slave-1 will look for an app running on localhost of slave-1 - but there isn't one, so it will fail.
For simplicity's sake, I just did an npm install and npm start & npx wait-on http://localhost:3000 in each node:
stage('cypress tester A') {
agent {
node: {
label "slave-1"
}
}
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm install --silent"
sh "npm start & npx wait-on http://localhost:3000"
sh "npm run cypress:run"
}
}
This is obviously not very efficient because you have to install and run the app on each node. However, you could potentially set up a previous stage on a dedicated node (say, slave-0) to install and serve your project, and use that. Within your Jenkinsfile, you'll need to know the IP of that slave-0, or you could get it dynamically within your Jenkinsfile. Then instead of installing and running your project on slave-1, 2 and 3, you would install and run it just on slave-0, and use the CYPRESS_BASE_URL env variable to tell cypress where to find the running instance of your app. If the IP of slave-0 is 2222.2222.2222.2222, you might try something like this:
pipeline {
stage ('Serve your project'){
agent {
label 'slave-0'
}
steps {
sh 'npm install --silent'
sh 'npm start & npx wait-on http://localhost:3000'
}
}
stage('Cypress'){
environment {
CYPRESS_BASE_URL=2222.2222.2222.2222:3000
// other env vars
}
parallel {
stage {
agent {
label 'slave-1'
}
steps {
echo "Running build ${env.BUILD_ID}"
sh "npm run cypress:run"
}
}
// more parallel stages
}
}
}
There's a lot of variations you can do, but hopefully that will get you started.

Related

Run docker build inside Jenkins Docker Slave

Currently I've a CI pipeline with the following stages:
Build
Unit Tests
Static Code Analysis
This is how my Jenkinsfile looks like:
pipeline {
agent any
stages {
stage("Install") {
steps {
sh "npm install"
}
}
stage("Build") {
steps {
sh "npm run build"
}
}
stage("Format") {
steps {
sh "npm run format"
}
}
stage("Lint") {
steps {
sh "npm run lint"
}
}
stage("Test") {
steps {
sh "npm run test"
}
}
stage("Code Coverage") {
steps {
sh "npm run test:cov"
publishHTML(target: [
reportDir: "./coverage/lcov-report",
reportFiles: "index.html",
reportName: "Jest Coverage Report"
])
}
}
stage("End-To-End Testing") {
steps {
sh "npm run test:e2e"
}
}
}
}
I want to add more stages to my pipeline:
Build and tag Docker Image from Dockerfile
Push the image to the Docker Hub
Some more steps which would need Docker CLI
Example:
pipeline {
.
.
.
stage("Docker Build") {
steps {
sh "docker build -t [user_name]/[image_name]:[tag] .
}
}
}
I'm quite new to this, and I have tried multiple ways to install docker and it was unsuccessful and it is a bad practice too.
We can run docker run -v /var/run/docker.sock:/var/run/docker.sock ... but I can't use bind mounting while using docker build command.
Can someone please suggest me a way where I can use docker commands inside Jenkins SSH Agents?
Solution
Install Docker CLI without the Daemon in Jenkins Docker Slave. I have used this Docker Agent and installed Docker CLI inside it using this method
Then as a docker daemon I've used my remote docker host. (Also, you can configure the local docker host as remote using these steps). You can use docker remote host using --host flag. E.g. docker --host x.x.x.x:2375 build -t johndoe:calculator .
Syntax: docker --host [Docker_Host]:[Port] build -t [Image_Name]:[Image_Tag] .
My New Jenkinsfile is as follows:
pipeline {
agent any
stages {
stage("Install") {
steps {
sh "npm install"
}
}
stage("Build") {
steps {
sh "npm run build"
}
}
stage("Format") {
steps {
sh "npm run format"
}
}
stage("Lint") {
steps {
sh "npm run lint"
}
}
stage("Test") {
steps {
sh "npm run test"
}
}
stage("Code Coverage") {
steps {
sh "npm run test:cov"
publishHTML(target: [
reportDir: "./coverage/lcov-report",
reportFiles: "index.html",
reportName: "Jest Coverage Report"
])
}
}
stage("End-To-End Testing") {
steps {
sh "npm run test:e2e"
}
}
stage("Docker Build") {
steps {
withCredentials([string(credentialsId: 'Docker_Host', variable: 'DOCKER_HOST')]) {
sh 'docker --host $DOCKER_HOST build -t xxx/xxx .'
}
}
}
}
}
Note: I have stored Docker host URL on Jenkins as a credential and used it using withCredentials function.

How to run the build inside docker container in Jenkins?

In my application I have a build script in package.json.
The build makes dist folder and inside I have my application.
I set Jenkins master and Jenkins agent as say in boxboat setup jenkins with docker and watch the video in youtube.
But now after I did this, I don't think my bash commands running inside a container.
I want to clone the repo and run npm i and npm run build - inside the docker container.
How I modify this configuration to able to do that?
throttle(['throttleDocker']) {
node('docker') {
wrap([$class: 'AnsiColorBuildWrapper']) {
try{
stage('Build') {
checkout scm
sh '''
echo "in Setup"
docker ps -a
echo "after docker"
# ./ci/docker-down.sh
# ./ci/docker-up.sh
'''
}
stage('Test'){
parallel (
"unit": {
sh '''
echo "in unit"
# ./ci/test/unit.sh
'''
},
"functional": {
sh '''
echo "in functional"
# ./ci/test/functional.sh
'''
}
)
}
stage('Capacity Test') {
sh '''
echo "in Capacity Test"
# ./ci/test/stress.sh
'''
}
}
finally {
stage('Cleanup') {
sh '''
echo "in Cleanup"
# ./ci/docker-down.sh
'''
}
}
}
}
}
I tried to this codes but they don't work. I also add agent after try.
stage('Build') {
agent {
docker {
label 'docker'
image 'node:latest'
}
}
steps {
checkout scm
sh 'node -v'
}
...
You can try below scripted pipeline
node {
docker.image('yourimage').inside {
stage('Build'){
sh 'echo "Build stage inside container"'
}
stage('Test'){
sh 'echo "Test Stage inside container"'
}
}
}

Npm test in Jenkins build takes 8 hours

My Jenkins build is still not finished after 8hrs. I have a simple React project I want to implement Continuous Integration with.
My Jenkinsfile looks like this:
pipeline {
agent {
docker {
image 'node'
args '-u root'
}
}
stages {
stage('Build') {
steps {
echo 'Building...'
sh 'npm install'
sh 'npm install node'
}
}
stage('Test') {
steps {
echo 'Testing...'
sh 'npm test'
}
}
}
}
I think what is happening is npm test is testing ALL the node modules. The build itself takes 44s.
Also, I have not been able to get npm install to install the node modules? So far as I understand it should install node automatically?
How can I stop it taking so long?
Override docker entrypoint with command --entrypoint \'\'
agent will therefore look like
agent {
docker {
image 'node'
args '-u root --entrypoint \'\''
}
}
This is a wild guess, all I can do with so little information

How can I use agent docker in a declartive pipeline running on jenkins ssh-slave node?

I am running jenkins master and slave as docker container. I have setup a slave node using jenkins/ssh-slave image with label 'worker'. I can successfully run my pipeline on the worker node. However, when I am trying to run docker build command using the Jenkinsfile, I am getting error docker: not found.
pipeline {
agent { label 'worker' }
tools {nodejs "node"}
stages {
stage ('Build APP') {
steps {
echo 'BUILDING APPLICATION'
sh 'npm install'
}
}
stage ('Create Package') {
steps {
script{
echo 'BUILDING DOCKER IMAGE'
docker.build("package${env.BUILD_NUMBER}")
}
}
}
stage('Package Test') {
agent { docker }
steps {
echo 'RUNNING IMAGE IN CONATAINER'
sh "docker run -p 5050:4000 -d package${env.BUILD_NUMBER}"
echo 'CHECKING HEALTH STATUS'
script {
try {
sh "curl -s --head --request GET http://127.0.0.1:5050/ | grep '200'"
echo 'Health Check Passed!'
} catch(Exception e) {
echo "Health Check Failed!"
}
}
}
}
In the third step 'package test' I have placed agent docker in the file but it doesn't seem to work. How can I place agent docker in a declarative pipeline?

How to keep a Jenkins step 'alive' with declarative pipelines?

My use case is the following: I have a web application written in Node, and I've done a set of functional tests with Java and Selenium. I have configured a job in Jenkins using the new declarative pipelines syntax.
Here are the contents of the Jenkinsfile:
#!groovy
pipeline {
agent any
stages {
stage('Test') {
steps {
nodejs(nodeJSInstallationName: 'node:8.2.0') {
sh 'echo $PATH'
sh 'npm -v'
sh 'node -v'
dir('src/webapp') {
sh 'npm install'
sh 'nohup npm start &> todomvc.out &'
}
sh './gradlew clean test'
}
}
}
stage('Clean up') {
steps {
deleteDir()
}
}
}
}
As you can see, first I launch the webapp using npm start and I send it to the background (in order to continue to the next step which is the actual testing).
However when the tests run the webapp isn't available, making them fail.
I've tried replacing:
sh 'nohup npm start &> todomvc.out &'
with:
npm start
and when I go to the port I've specified there is an instance of the webapp as expected. However, this blocks the next steps.
What I want is to launch an instance of the webapp and then test it with ./gradlew clean test.

Resources