I'm migrating from Docker to Podman and I have not been able to replace the sentence
docker.withRegistry(DOCKER_REGISTRY_URL, DOCKER_REGISTRY_CREDENTIALS_ID).
Here's what I'm trying to move to Podman:
def mvnBuild(String command) {
script {
docker.withRegistry(DOCKER_REGISTRY_URL, DOCKER_REGISTRY_CREDENTIALS_ID) {
sh "docker run -v $HOME/.m2:$HOME/.m2 -u \$(id -u \${whoami}):\$(id -g \${whoami}) " +
"-e MAVEN_CONFIG=$HOME/.m2 -v ${pwd()}:${pwd()}:rw,z -w ${pwd()} ${DOCKER_REGISTRY}/${DOCKER_REPOSITORY_NAME}/maven:${MAVEN_VERSION} " +
"mvn -Duser.home=$HOME -B $command"
}
}
}
I would appreciate any help or pont in the right direction.
Cheers
Related
I am trying to execute my tests in parallel but it is not working.
I have this config in jeniks file .sh
docker run --rm -i --name integration-tests-$p --network=host $DOCKER_VOLUME -e CYPRESS_INCLUDE_TAGS=$TAG_TO_INCLUDES -e CYPRESS_EXCLUDE_TAGS=disabled -w $WORKING_DIR cypress/included:10.10.0 --record --group 4x-electron --key keyId --parallel --ci-build-id $BUILD_NUMBER
and the other config in the .jenkins file
stage('Integration Tests') {
when {
expression { return RUN_E2E != ''}
}
steps {
sh "$RUSH hc -t web" // Check if web is running
sh "$RUSH integration-tests --tags=smoke --to manager"
}
}
I need to run these tests in parallel, can anyone help?
i am trying to ssh into a remote host and then execute certain commands on the remote host's shell. Following is my pipeline code.
pipeline {
agent any
environment {
// comment added
APPLICATION = 'app'
ENVIRONMENT = 'dev'
MAINTAINER_NAME = 'jenkins'
MAINTAINER_EMAIL = 'jenkins#email.com'
}
stages {
stage('clone repository') {
steps {
// cloning repo
checkout scm
}
}
stage('Build Image') {
steps {
script {
sshagent(credentials : ['jenkins-pem']) {
sh "echo pwd"
sh 'ssh -t -t ubuntu#xx.xxx.xx.xx -o StrictHostKeyChecking=no'
sh "echo pwd"
sh 'sudo -i -u root'
sh 'cd /opt/docker/web'
sh 'echo pwd'
}
}
}
}
}
}
But upon running this job it executes sh 'ssh -t -t ubuntu#xx.xxx.xx.xx -o StrictHostKeyChecking=no' successfully but it stops there and does not execute any further commands. I want to execute the commands that are written after ssh command inside the remote host's shell. any help is appreciated.
I would try something like this:
sshagent(credentials : ['jenkins-pem']) {
sh "echo pwd"
sh 'ssh -t -t ubuntu#xx.xxx.xx.xx -o StrictHostKeyChecking=no "echo pwd && sudo -i -u root && cd /opt/docker/web && echo pwd"'
}
I resolve this issue
script
{
sh """ssh -tt login#host << EOF
your command
exit
EOF"""
}
stage("DEPLOY CONTAINER"){
steps {
script {
sh """
#!/bin/bash
sudo ssh -i /path/path/keyname.pem username#serverip << EOF
sudo bash /opt/filename.sh
exit 0
<< EOF
"""
}
}
}
There is a better way to run commands on remote using SSH. I know this is late answer but I just explored this thing so would like to share and this will help others to resolve this problem easily.
I just found this link helpful on how to run multiple commands on remote using SSH. Also we can run multiple commands conditionally as mentioned in above blog.
By going through it, I found the syntax:
ssh username#hostname "command1; command2;commandN"
Now, how to run command inside remote hots using SSH in Jenkins pipeline?
Here is the solution:
pipeline {
agent any
environment {
/*
define your command in variable
*/
remoteCommands =
"""java --version;
java --version;
java --version """
}
stages {
stage('Login to remote host') {
steps {
sshagent(['ubnt-creds']) {
/*
Provide variable as argument in ssh command
*/
sh 'ssh -tt username#hostanem $remoteCommands'
}
}
}
}
}
Firstly and optionally, you can define a variable that holds all commands separated by ;(semicolon) and then pass it as parameter in command.
Another way, you can also pass your commands directly to ssh command as
sh "ssh -tt username#hostanem 'command1;command2;commandN'"
I have used it in my code and it's working great!
see the output here
Happy Learning :)
I have a step in my pipeline which does this:
sh("shmig -m ${app_root}/${migration_folder} -t mysql -H $mysql_server -l $USERNAME -p $PASSWORD -d $schema up")
It works fine but sometime I get this error:
java.io.NotSerializableException: org.jenkinsci.plugins.workflow.job.WorkflowJob
Nothing change between build and I don't understand this error.
Have you any idea ?
For more information about the call, it is done like this:
node('docker') {
step('shmig') {
smhig()
}
}
def smhig() {
...
sh("shmig -m ${app_root}/${migration_folder} -t mysql -H $mysql_server -l $USERNAME -p $PASSWORD -d $schema up")
}
Are there any variable declarations/assignments before that 'sh("shmig -m ...)' line? I used to face the same error and now it is gone after I replaced all the variable declarations from
myVar = myVal
to
def myVar = myVal
Not sure if that can help but I hope so.
In my jenkinsfile I have this
stage ('Build Docker') {
steps {
script {
image1 = docker.build "docker1:${BRANCH_NAME}"
}
script {
image2 = docker.build "docker2:${BRANCH_NAME}"
}
}
}
stage ('Run Docker Acceptance Tests') {
steps {
script {
container1 = image1.run "-v /tmp/${BRANCH_NAME}:/var/lib/data"
container1Id = container1.id
container1IP = sh script: "docker inspect ${container1Id} | grep IPAddress | grep -v null| cut -d \'\"\' -f 4 | head -1", returnStdout: true
}
//let containers start up
sleep 20
script {
container2= image2.run("-v /tmp/${BRANCH_NAME}:/var/lib/data --add-host=MY_HOST:${container1IP}")
}
}
}
When it gets to run container2 I get this output.
[resources] Running shell script
00:01:33.775 + docker run -d -v /tmp/master:/var/lib/data --add-host=MY_HOST:172.17.0.3
00:01:33.775 "docker run" requires at least 1 argument(s).
00:01:33.775 See 'docker run --help'.
Clearly its not appending the container name when running the docker image.
I tried just hardcoding in the IP address to test if it worked like this
container2= image2.run("-v /tmp/${BRANCH_NAME}:/var/lib/data --add-host=MY_HOST:172.17.0.3")
And then it worked and ran the command correctly
00:00:29.386 [resources] Running shell script
00:00:29.641 + docker run -d -v /tmp/master:/var/lib/data --add-host=MY_HOST:172.17.0.3 docker-name:branch
I dont understand why its not picking up the container image name.
I have even tried doing this - getting the same error
container2= image2.run("-v /tmp/${BRANCH_NAME}:/var/lib/data --add-host=MY_HOST:${container1IP} docker2:${BRANCH_NAME}")
My final step I tried
sh "docker run -v /tmp/${BRANCH_NAME}:/var/lib/data --add-host=MY_HOST:${container1IP} docker2:${BRANCH_NAME}"
Again it seems like it is stripping off the final command after resolving ${container1IP}
managed to fix it, it was due to a hidden new line char
container1IP = sh (script: "docker inspect ${container1Id} | grep IPAddress | grep -v null| cut -d \'\"\' -f 4 | head -1", returnStdout: true).trim()
Trimming the var fixed it
I'm trying to execute an SSH command from inside a Docker container in a Jenkins pipeline. I'm using the CloudBees Docker Pipeline Plugin to spin up the container and execute commands, and the SSH Agent Plugin to manage my SSH keys. Here's a basic version of my Jenkinsfile:
node {
step([$class: 'WsCleanup'])
docker.image('node').inside {
stage('SSH') {
sshagent (credentials: [ 'MY_KEY_UUID' ]) {
sh "ssh -vvv -o StrictHostKeyChecking=no ubuntu#example.org uname -a"
}
}
}
}
When the SSH command runs, I get this error:
+ ssh -vvv -o StrictHostKeyChecking=no ubuntu#example.org uname -a
No user exists for uid 1005
I combed through the logs and realized the Docker Pipeline Plugin is automatically telling the container to run with the same user that is logged in on the host by passing a UID as a command line argument:
$ docker run -t -d -u 1005:1005 [...]
I decided to check what users existed in the host and the container by running cat /etc/passwd in each environment. Sure enough, the list of users was different in each. 1005 was the jenkins user on the host machine, but that UID didn't exist in the container. To solve the issue, I mounted /etc/passwd from the host to the container when spinning it up:
node {
step([$class: 'WsCleanup'])
docker.image('node').inside('-v /etc/passwd:/etc/passwd') {
stage('SSH') {
sshagent (credentials: [ 'MY_KEY_UUID' ]) {
sh "ssh -vvv -o StrictHostKeyChecking=no ubuntu#example.org uname -a"
}
}
}
}
The solution provided by #nathan-thompson is awesome, but in my case I was unable to find the user even in the /etc/passwd of the host machine! It means mounting the passwd file did not fix the problem. This question https://superuser.com/questions/580148/users-not-found-in-etc-passwd suggested some users are logged in the host using an identity provider like LDAP.
The solution was finding a way to add the proper line to the passwd file on the container. Calling getent passwd $USER on the host will provide the passwd line for the Jenkins user running the container.
I added a step running on the node (and not the docker agent) to get the line and save it in a file. Then in the next step I mounted the generated passwd to the container:
stages {
stage('Create passwd') {
steps {
sh """echo \$(getent passwd \$USER) > /tmp/tmp_passwd
"""
}
}
stage('Test') {
agent {
docker {
image '*******'
args '***** -v /tmp/tmp_passwd:/etc/passwd'
reuseNode true
registryUrl '*****'
registryCredentialsId '*****'
}
}
steps {
sh """ssh -i ********
"""
}
}
}
I just found another solution to this problem, that I want to share. It differentiates from the existing solutions in that it allows to run the complete pipeline in one agent, instead of per stage.
The trick is to, instead of directly using an image, refer to a Dockerfile (which may be build FROM the original) and then add the user:
# Dockerfile
FROM node
ARG jenkinsUserId=
RUN if ! id $jenkinsUserId; then \
usermod -u ${jenkinsUserId} jenkins; \
groupmod -g ${nodeId} jenkins; \
fi
// Jenkinsfile
pipeline {
agent {
dockerfile {
additionalBuildArgs "--build-arg jenkinsUserId=\$(id -u jenkins)"
}
}
}
agent {
docker {
image 'node:14.10.1-buster-slim'
args '-u root:root'
}
}
environment {
SSH_deploy = credentials('e99988ea-6bdc-45fc-b9e1-536b875bcac7')
}
stage('build') {
steps {
sh '''#!/bin/bash
eval $(ssh-agent -s)
cat $SSH_deploy | tr -d '\r' | ssh-add -
touch .env
echo 'REACT_APP_BASE_API = "//172.22.132.115:8080"' >> .env
echo 'REACT_APP_ADMIN_PANEL_URL = "//172.22.132.115"' >> .env
yarn install
CI=false npm run build
ssh -t -o StrictHostKeyChecking=no root#172.22.132.115 'rm -rf /usr/local/src/build'
scp -r -o StrictHostKeyChecking=no build root#172.22.132.115:/usr/local/src/
ssh -t -o StrictHostKeyChecking=no root#172.22.132.115 'systemctl restart nginx'
'''
}
From the solution provided by Nathan Thompson, I modified it this way for Jenkins DOCKER build container which runs inside a Jenkins DOCKER-slave. #docker in docker
if (validated_parameters.custom_gradle_image){
docker.image(validated_parameters.custom_gradle_image).inside(" -v /etc/passwd:/etc/passwd -v /var/lib/jenkins/.ssh/:/var/lib/jenkins/.ssh/ "){
sshagent(['jenkins-git-io']){
sh "${gradleCommand}"
}