Jenkinsfile: connect to a server and perform operations on the server - jenkins

In Jenkinsfile I am trying to login to a server and from there run docker-compose. However, I am getting an AccessDeniedException error. Root user has permissions to all folders on the server.
I assume I have a thinking error: I suspect that after ssh to the server, the dir command doesn't not run in the server but rather where the jenkins is run.
Jenkinsfile:
pipeline {
agent any
environment {
PATH = "$PATH:/usr/local/bin/"
}
stages {
stage('Deploy to digital ocean') {
steps {
sshagent(['my-private-ssh-key']) {
sh """
ssh root#host
"""
dir("/var/www/car_prices"){
sh "docker-compose down"
}
}
}
}
}
}
Error stack:
[Pipeline] dir
Running in /var/www/car_prices
[Pipeline] {
[Pipeline] sh
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 67144 killed;
[ssh-agent] Stopped.
[Pipeline] // sshagent
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.nio.file.AccessDeniedException: /var/www
Question:
1. Why does it throw an error?

You need to use ssh to do the job for you, which is rather straight forward:
...
sshagent(['my-private-ssh-key']) {
sh """
ssh root#host "cd /var/www/car_prices; docker-compose down"
"""
}
...
This is kind of ugly, but works. If you care, you can split commands into multiple lines for better readability. This approach becomes even trickier if you want to provide some more complex functionality, like error handling.

Related

Jenkins throws error when running pipeline

I got a little problem with Jenkins Pipeline... To be more specific it seems like it always fails, but without any reasons, (probably there is one, but cannot find it)
Snippet of the pipeline.... (Whole Pipeline)
pipeline {
environment {
DOCKERHUB_CREDENTIALS=credentials("Dockerhub")
APPLICATION_HOST="0.0.0.0:8000"
APPLICATION_PORT="8000"
}
stage("build"){
steps{
script {
def inspectDockerNetwork = sh script: "docker network create global_store_network", returnStatus: true
if (inspectDockerNetwork == 0) {
sh "echo 'Creating Docker Network...'"
sh "docker network create global_store_network"
sh "echo 'Network Has been Created..'"
}
}
dir("test_env"){
sh "docker-compose up -d"
sleep 10
sh "echo 'Docker Built Image Successfully! Running Container....'"
}
}
}
stage("test"){
steps{
load "./test_env/version_env.groovy"
sh "echo 'Running Test Pipeline'"
sh "echo 'Running Healtcheck Test...'"
sh "echo 'Sleeping until the Application will be fully ready...'"
sleep 10
script {
command = """curl -s -X GET -H 'accept: */*' 'http://${env.APPLICATION_HOST}:${env.APPLICATION_PORT}/healthcheck/'"""
responseStatus = sh(script: command, returnStdout: true).trim()
if (responseStatus != "200") {
sh "echo 'Application Responded with Failure, Not Ready for Production...'"
error "Health Check Stage Failure."
}
}
}
post {
always {
dir("test_env"){
sh "echo 'Removing Testing Environment'"
sh "docker-compose down"
}
}
}
}. /////////// It has not start an Execution of the Deployment, So the Error Is Somewhere above ////////////////////////////////////
stage("deployment"){
steps {
load "./test_env/version_env.groovy"
sh "echo 'Running Deployment Pipeline Stage...'"
sh "echo 'Tagging new Image Version'"
withCredentials([usernamePassword(
credentialsId: "DockerHub", // Credential Id that should be created at Jenkins Server...
usernameVariable: env.DOCKERHUB_CREDENTIALS_USR, // Credential Username that should be created at jenkins Server.
passwordVariable: env.DOCKERHUB_CREDENTIALS_PSW, // Credential Password that shoud be created at Jenkins Server..
)]){
sh "docker login -u ${env.DOCKERHUB_CREDENTIALS_USR} -p ${env.DOCKERHUB_CREDENTIALS_PSW}"
sh "echo 'Logged In.. Into Docker.'"
sh "echo 'Tagging An Image'"
sh "docker tag new_versioned_image ${env.DOCKERHUB_REPOSITORY_LINK}:latest"
sh "echo 'Tagged... Pushing onto docker repo.'"
sh "docker push ${env.DOCKERHUB_REPOSITORY_LINK}:latest"
sh "echo 'Tagged Successfully.. Pushing Image On Docker Hub..'"
sh "echo 'Image has been Pushed Successfully! Pipeline Finished.'"
}
}
}
}
So the Output of that snippet is following...
( I've Separated Stage Logs In order to make it easier to read )
/////////// Build Stage ///////////////
First time build. Skipping changelog.
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $DOCKERHUB_CREDENTIALS or $DOCKERHUB_CREDENTIALS_PSW
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build)
[Pipeline] dir
Running in /var/jenkins_home/workspace/Store Pipeline/test_env
[Pipeline] {
[Pipeline] sh
+ docker-compose up -d
Container test_postgres_store_database Creating
Container test_postgres_store_database Created
Container test_store_application_server Creating
Container test_store_application_server Created
Container test_postgres_store_database Starting
Container test_postgres_store_database Started
Container test_store_application_server Starting
Container test_store_application_server Started
[Pipeline] sleep
Sleeping for 10 sec
[Pipeline] sh
+ echo Docker Built Image Successfully! Running Container....
Docker Built Image Successfully! Running Container....
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (test)
[Pipeline] load
[Pipeline] { (./test_env/version_env.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] sh
////////////// Testing Stage goes there /////////////////////
+ echo Running Test Pipeline
Running Test Pipeline
[Pipeline] sh
+ echo Running Healtcheck Test...
Running Healtcheck Test...
[Pipeline] sh
+ echo Sleeping until the Application will be fully ready...
Sleeping until the Application will be fully ready...
[Pipeline] sleep
Sleeping for 10 sec
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ curl -s -X GET -H accept: */* http://0.0.0.0:8000/healthcheck/
[Pipeline] }
[Pipeline] // script
Post stage
[Pipeline] dir
Running in /var/jenkins_home/workspace/Store Pipeline/test_env
[Pipeline] {
[Pipeline] sh
+ echo Removing Testing Environment
Removing Testing Environment
[Pipeline] sh
+ docker-compose down
Container test_store_application_server Stopping
Container test_store_application_server Stopping
Container test_store_application_server Stopped
Container test_store_application_server Removing
Container test_store_application_server Removed
Container test_postgres_store_database Stopping
Container test_postgres_store_database Stopping
Container test_postgres_store_database Stopped
Container test_postgres_store_database Removing
Container test_postgres_store_database Removed
///// Deployment Stage Goes there.... ////////////
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deployment)
Stage "deployment" skipped due to earlier failure(s) ///// The Error Message Goes There....
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
//// ERROR /////
[Pipeline] End of Pipeline
ERROR: script returned exit code 7
Finished: FAILURE
The Problem is that it does not respond what's exactly the problem is, (just simply skipped due to earlier failure(s), above logs does not shows any of the errors.
So would really appreciate any help or any suggestions how to solve this Issue.
Thanks.

jenkins ssh agent fail to copy war to remote server

I am new to jenkin, I have created a jenkinFile to build a war and copy it to a remote machine on tomcat server based on the following tutorial:
https://thenucleargeeks.com/2020/05/31/declarative-jenkins-pipeline-to-deploy-java-web-application/
Please find below jenkinFile which i have created:
#!/usr/bin/env groovy
pipeline {
environment {
NAME = readMavenPom().getArtifactId()
}
agent any
options {
timeout(time: 1, unit: 'HOURS')
buildDiscarder(logRotator(daysToKeepStr: '10', numToKeepStr: '5', artifactNumToKeepStr: '2'))
}
stages {
stage("Git Checkout"){
steps{
git branch: 'develop',
credentialsId: 'jenkins', url: 'https://gitlab.gov/ih/ih-por.git'
}
}
stage('Maven build') {
steps {
sh "mvn clean package"
sh "mv target/*.war target/UI.war"
}
}
stage("deploy-dev"){
steps{
sshagent(['user-id-tomcat-deployment']) {
sh """
scp -o StrictHostKeyChecking=no target/UI.war
root#192.168.1.000:/opt/tomcat/webapps/
ssh root#192.168.1.000 /opt/tomcat/bin/shutdown.sh
ssh root#192.168.1.000 /opt/tomcat/bin/startup.sh
"""
}
}
}
}
}
I have also created added the private key on my jenkin server.
However when I build the project on jenkins the following error is displayed by ssh agent:
[Pipeline] { (deploy-dev)
[Pipeline] sshagent (hide)
[ssh-agent] Using credentials root (This defines the credential to login remote server where tomcat is installed for deployment purpose)
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-r2GnAq9cX2F8/agent.37244
SSH_AGENT_PID=37247
Running ssh-add (command line suppressed)
Identity added: /var/lib/jenkins/workspace/InfoHighway/portal-ui-deploy#tmp/private_key_6145932096571627059.key (root#localhost.localdomain)
[ssh-agent] Started.
[Pipeline] {
[Pipeline] sh
+ scp -o StrictHostKeyChecking=no target/portalUI.war
usage: scp [-12346BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
[-l limit] [-o ssh_option] [-P port] [-S program]
[[user#]host1:]file1 ... [[user#]host2:]file2
[Pipeline] }
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 37247 killed;
[ssh-agent] Stopped.
[Pipeline] // sshagent
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
Any idea what I am doing wrong please?
You have a new line after the target/UI.war so the scp command is missing the target parameter.
Try to run it with the full scp command in a single line:
stage("deploy-dev"){
steps{
sshagent(['user-id-tomcat-deployment']) {
sh """
scp -o StrictHostKeyChecking=no target/UI.war root#192.168.1.000:/opt/tomcat/webapps/
ssh root#192.168.1.000 /opt/tomcat/bin/shutdown.sh
ssh root#192.168.1.000 /opt/tomcat/bin/startup.sh
"""
}
}
}

jenkins canot cannot pass parameters to the remote server's shell script with 'publish over ssh' plugin

I executed a shell script command from the remote server via Jenkins and wanted to pass a defined environment variable.
my test pipeline script as follow
#!/usr/bin/env groovy
pipeline {
agent any
environment {
param = "param1"
}
stages{
stage('deploy project') {
steps {
sshPublisher(publishers: [sshPublisherDesc(configName: '192.168.1.112-ssh',
transfers: [sshTransfer(cleanRemote: true,
execCommand: '''
/root/test.sh hello ${param} world ${JOB_NAME}
''',
execTimeout: 120000, patternSeparator: '[, ]+',remoteDirectory: '/',
removePrefix: '', sourceFiles: '')])])
}
}
}
}
The contents of the remote script 'test.sh' are as follows
#! /bin/sh
echo "$1"
echo "$2"
echo "$#"
then,The jenkins console builds are logged as follows
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (deploy project)
[Pipeline] step
SSH: Connecting from host [2a132d40d769]
SSH: Connecting with configuration [192.168.1.112-ssh] ...
SSH: EXEC: STDOUT/STDERR from command [
/root/test.sh hello ${param} world pipeline-docker-demo
] ...
hello
world
hello world pipeline-docker-demo
SSH: EXEC: completed after 207 ms
SSH: Disconnecting configuration [192.168.1.112-ssh] ...
SSH: Transferred 0 file(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
You find that we can only pass one jenkins built-in variable with '${JOB_NAME}' and not the variable defined in this script with '${param}'
my app version:
jenkins:2.235.2
the plugin 'Publish Over SSH': '1.20.1'
What do I have to do to pass this custom variableļ¼Œplease help me,thanks a lot.

Jenkins pipeline fails to login to docker hub

I am trying to use pipeline to run a few things and when i started running my pipeline it failed to login to docker.
the weird thing is that i am able to login on the machine itself but when i run the pipeline is fails with this weird error:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] stage
[Pipeline] { (Front-end)
[Pipeline] node
Running on test-env in /var/www/test-env/workspace/client-e2e
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withDockerRegistry
Using the existing docker config file.Removing blacklisted property: auths$ docker login -u ***** -p ******** https://hub.docker.com/?namespace=******
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: login attempt to https://hub.docker.com/v2/ failed with status: 404 Not Found
[Pipeline] // withDockerRegistry
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
ERROR: docker login failed
Finished: FAILURE
I don't know why it performs a login when this image is publicly available for everyone.
can someone help me ?
this is the pipeline itself:
pipeline {
agent none
stages {
stage('Front-end') {
agent {
docker {
image 'node:8-alpine'
label "test-env"
}
}
steps {
sh 'node --version'
}
}
}
}
You can try with Credentials Binding Plugin in:
steps {
sh 'node --version'
}
You can do:
withCredentials([string(credentialsId: 'mytoken', variable: 'TOKEN')]) {
sh '''
docker login -u '<your_user>' -p '<$TOKEN>'
node --version
'''
}
There is an example here:
https://issues.jenkins-ci.org/browse/JENKINS-41051
Ok, so after a while i found out that it was as simple as doing this
pipeline {
agent none
stages {
stage('Front-end') {
agent {
docker {
image 'node:8-alpine'
registryUrl 'https://index.docker.io/v1/'
label "test-env"
}
}
steps {
sh 'node --version'
}
}
}
}
this was aded: registryUrl 'https://index.docker.io/v1/'

Jenkins, Host key verification failed, script returned exit code 255

I have a building-server where I have Jenkins 2.73.3 and another servers where I deploy my apps.
I have also set up a credential to connect from building-server to the other servers.
But everytime I add another server it is difficult to add it because I set up the authorized key in the new server and in the command line works, but not in Jenkins.
Here is a little recipe that fails:
pipeline {
agent any
stages {
stage('Set conditions') {
steps {
sshagent(['xxxx-xxxx-xxxx-xxxx-xxxx']) {
sh "ssh user#product.company.com 'echo $HOME'"
}
}
}
}
}
And here is the Log failure:
[ssh-agent] Started.
[Pipeline] {
[Pipeline] sh
[check] Running shell script
+ ssh user#product.company.com echo /var/lib/jenkins
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 12567 killed;
[ssh-agent] Stopped.
Host key verification failed.
[Pipeline] }
[Pipeline] // sshagent
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 255
Finished: FAILURE
It seems that the solution was to add the parameter StrictHostKeyChecking to the shell script line
sh "ssh -o StrictHostKeyChecking=no user#product.company.com 'echo $HOME'"

Resources