I need to use host ssh key inside docker , for this purpose i have build docker like
docker build -t example --build-arg ssh_prv_key="$(cat ~/.ssh/id_rsa)" -f dockerfile-dev .
if we use direct docker command it is working fine , but if I use inside the jenkins pipe-line script getting below error
Running in Durability level: MAX_SURVIVABILITY
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 92: expecting '}', found 'ssh_prv_key' # line 92, column 116.
ev:${GIT_COMMIT} "--build-arg ssh_prv_ke
Below step i have used in jenkins pipe-line
sh "docker build -t ${service_name}-dev:${GIT_COMMIT} --build-arg ssh_prv_key="$(cat ~/.ssh/id_rsa)" -f dockerfile-dev ."
And docker file used like below
ARG ssh_prv_key
# Authorize SSH Host
# Add the keys and set permissions
RUN mkdir -p /root/.ssh
RUN echo "$ssh_prv_key" > /root/.ssh/id_rsa && \
chmod 600 /root/.ssh/id_rsa
I solved a similar issue as follow:
Jenkins pipeline
sh "cp ~/.ssh/id_rsa id_rsa"
sh "docker build -t ${service_name}-dev:${GIT_COMMIT} -f dockerfile-dev ."
sh "rm id_rsa"
Dockerfile
# Some instructions...
ADD id_rsa id_rsa
# Now use the "id_rsa" file inside the image...
Related
I have created a docker container to be used in a gitlab-ci pipeline to build a java project. Properties files for an integration test are in an encrypted image I want to mount in the container during the execution of the integration test stage.
In my Dockerfile:
RUN apt install -y libncurses5-dev
RUN apt install -y cryptsetup
The docker image gets built and pushed without problems.
In my Gitlab CI File:
mkdir -p configs
echo -n $LUKSOPEN_PASS_PHRASE | /usr/sbin/cryptsetup luksOpen configs.img /home/java/config
But when the pipeline is run:
Executing "step_script" stage of the job script
00:01
$ -| echo 'integration tests' mkdir -p /home/java/config mkdir -p configs cryptsetup echo -n $LUKSOPEN_PASS_PHRASE | /usr/sbin/cryptsetup luksOpen configs.img /home/java/config mount /dev/mapper/configs configs cp configs/* /home/java/config ls /home/java/config gradle integrationTest
/usr/bin/bash: line 113: /usr/sbin/cryptsetup: No such file or directory
/usr/bin/bash: line 113: -: command not found
When I log into the container, cryptsetup is there:
sven#ixori:~/workspace/java-11-container$ docker run -it 9b733f66d757 /bin/bash
root#1193dc1f57b9:/# cryptsetup
Usage: cryptsetup [-?Vvyrq] [-?|--help] [--usage] [-V|--version] [-v|--verbose] [--debug]
[--debug-json] [-c|--cipher=STRING] [-h|--hash=STRING] [-y|--verify-passphrase]
What am I missing?
My Jenkins pipeline code successfully checks out my private git repo from bitbucket using
checkout([$class: 'GitSCM',
userRemoteConfigs: [[credentialsId: 'cicd-user', url:'ssh://git#bitbucket.myorg.co:7999/A/software.git']]
in same software.git I have a Dockerfile that I want to use to build various build targets present in software.git on Kubernetes and I am trying the below to pass jenkins credentials into a docker container that I want to build and run.
So in the same jenkins pipeline when I checked out software.git (above code), I try to do the following to get the docker container built
withCredentials([sshUserPrivateKey(credentialsId: 'cicd-user', keyFileVariable: 'FILE')]) {
sh "cd ${WORKSPACE} && docker build -t ${some-name} --build-arg USERNAME=cicd-user --build-arg PRIV_KEY_FILE=$FILE --network=host -f software/tools/jenkins/${some-name}/Dockerfile ."
}
in Dockerfile I do
RUN echo "$PRIV_KEY_FILE" > /home/"$USERNAME"/.ssh/id_rsa && \
chmod 700 /home/"$USERNAME"/.ssh/id_rsa
RUN echo "Host bitbucket.myorg.co\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config
But still from my Docker container I am not able to successfully checkout my private repo(s). What am I missing ? Any comments, suggestions ? Thanks.
Please read about Groovy String Interpolation.
In your expression
sh "cd ${WORKSPACE} && docker build -t ${some-name} \
--build-arg USERNAME=cicd-user \
--build-arg PRIV_KEY_FILE=$FILE --network=host \
-f software/tools/jenkins/${some-name}/Dockerfile ."
you use double quotes so Groovy interpolates all the variables in the string. This includes $FILE so Groovy replaces that with the value of Groovy variable named FILE. You don't have any Groovy variable with that name (but rather bash variable which is different from Groovy) so this gets replaced with an empty string.
To prevent interpolating that particular variable, you need to hint Groovy not to interpolate this particular one, by escaping this $ with \:
sh "cd ${WORKSPACE} && docker build -t ${some-name}\
--build-arg USERNAME=cicd-user \
--build-arg PRIV_KEY_FILE=\$FILE --network=host \
-f software/tools/jenkins/${some-name}/Dockerfile ."
I have 3 different Docker images. I need to build these images from Jenkins file. I have Wildfly, Postgres, Virtuoso Docker images with individual Docker file. As of now, I am using the below command to build these images:
The directory structure is, Docker is the root diretory.
Docker->build->1. wildfly 2. postgres 3. virtuoso
In my jenkins file I have below command to build the image:
stage('Building test images')
{
sh 'docker build -t virtuoso -f $WORKSPACE/build/virtuoso/Dockerfile .'
}
But I am getting error as below:
Step 7/16 : COPY ./install $VIRT_HOME/install
COPY failed: stat /var/lib/docker/tmp/docker-builder636357036/install: no such file or directory
[Pipeline] }
For reference below is my dockerfile:
FROM virtuoso:latest
ENV var1 /opt/virtuoso-opensource
ENV VIRT_db /opt/virtuoso-opensource/var/lib/virtuoso/db
ENV RUN_CONFIG=/opt/virtuoso-opensource/install/config
RUN export PATH=$PATH:/opt/virtuoso-opensource/bin
RUN mkdir $var1/install
COPY ./install $var1/install
WORKDIR $VIRT_db
CMD ["/opt/virtuoso-opensource/bin/init.sh"]
And the workspace is /home/jenkins/Docker and my guess is I am running docker build command from $workspace directory and this command should run from the virtuoso directory.
My question is how build image from Jenkins file?
Thanks in advance.
the easiest solution to solve this would be to enter the proper folder in the script before executing the docker build command.
e.g.:
stage('Building test images') {
steps {
sh '''
cd $WORKSPACE/build/virtuoso
docker build -t virtuoso .
'''
}
}
Below is the answer:
stage('Build images'){
echo "workspace directory is ${workspace}"
dir ("$workspace/build/virtuoso")
{
sh 'docker build -t virtuoso -f $WORKSPACE/build/virtuoso/Dockerfile .'
}
dir ("$workspace/build/wildfly")
{
sh 'docker build -t wildfly -f $WORKSPACE/build/wildfly/Dockerfile .'
}
dir ("$workspace/build/postgres")
{
sh 'docker build -t postgres -f $WORKSPACE/build/postgres/Dockerfile .'
}
}
Thanks for helping me out.
On doing docker build inside Jenkinsfile,
i.e
docker build -f ./Dockerfile -t datastore:1.0.1 .
I am getting error like
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
This is my Jenkinsfile
#!/usr/bin/groovy
node {
checkout scm
// Here are the important data for this build
def DOCKER_REGISTRY = "XXX"
def DATASTORE = "datastore"
def DOCKER_TAG_DATASTORE = "${DOCKER_REGISTRY}/XXX"
def APP_VERSION = "1.0.1"
stage('Build') {
dockerInside('XXX/db-server:1.0.114', '') {
echo "Setting up artifactory location to push docker image ${DATASTORE}:${APP_VERSION}"
sh "docker build -f ./Dockerfile -t ${DATASTORE}:${APP_VERSION} ."
sh "docker tag ${DATASTORE}:${APP_VERSION} ${DOCKER_TAG_DATASTORE}:${APP_VERSION}"
withCredentials([
usernamePassword(
credentialsId: CORE_IZ_USER,
usernameVariable: 'LOG',
passwordVariable: 'PAS'
)]) {
// Doing some upload commands (see artifactory or docker upload commands from Jenkins)
sh "docker push ${DOCKER_TAG_DATASTORE}:${APP_VERSION}"
echo "Push to ${DOCKER_TAG_DATASTORE}:${APP_VERSION}"
}
}
}
stage('Docker image creation') {
echo "Docker image creation"
}
stage('Docker image upload') {
echo "Docker image upload"
}
}
This is my Dockerfile
FROM XXX/rhel:7.5
USER root
RUN yum -y install gcc && yum install -y git && yum install -y docker
# Install Go
RUN curl -O -s https://dl.google.com/go/go1.10.2.linux-amd64.tar.gz
RUN tar -xzf go1.10.2.linux-amd64.tar.gz -C /usr/local
ENV PATH /usr/local/go/bin:$PATH
ENV GOPATH /gopath
ENV GOBIN /usr/local/go/bin
WORKDIR /gopath/src/XXX
RUN mkdir -p /gopath/src/XXX
ADD . /gopath/src/XXX
RUN GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build -tags netgo -installsuffix netgo -o ./db-server /gopath/src/XXX/datastore/main.go
ADD ./db-server /db-server
ENTRYPOINT ["/db-server"]
I need to run docker container in Jenkins so that installed libraries like pycodestyle can be runnable in the following steps.
I successfully built Docker Container (in Dockerfile)
How do I access to the container so that I can use it in the next step? (Please look for >> << code in Build step below)
Thanks
stage('Build') {
// Install python libraries from requirements.txt (Check Dockerfile for more detail)
sh "docker login -u '${DOCKER_USR}' -p '${DOCKER_PSW}' ${DOCKER_REGISTRY}"
sh "docker build \
--tag '${DOCKER_REGISTRY}/${DOCKER_TAG}:latest' \
--build-arg HTTPS_PROXY=${PIP_PROXY} ."
>> sh "docker run -ti ${DOCKER_REGISTRY}/${DOCKER_TAG}:latest sh" <<<
}
}
stage('Linting') {
sh '''
awd=$(pwd)
echo '===== Linting START ====='
for file in $(find . -name '*.py'); do
filename=$(basename $file)
if [[ ${file:(-3)} == ".py" ]] && [[ $filename = *"test"* ]] ; then
echo "perform PEP8 lint (python pylint blah) for $filename"
cd $awd && cd $(dirname "${file}") && pycodestyle "${filename}"
fi
done
echo '===== Linting END ====='
'''
}
You need to mount the workspace of your Jenkins job (containing your python project) as volume (see "docker run -v" option) to your container and then run the "next step" build step inside this container. You can do this by providing a shell script as part of your project's source code, which does the "next step" or write this script in a previous build stage.
It would be something like this:
sh "chmod +x build.sh"
sh "docker run -v $WORKSPACE:/workspace ${DOCKER_REGISTRY}/${DOCKER_TAG}:latest /workspace/build.sh"
build.sh is an executable script, which is part of your project's workspace and performans the "next step".
$WORKSPACE is the folder that is used by your jenkins job (normally /var/jenkins_home/jobs//workspace - it is provided by Jenkins as a build variable.
Please note: This solution requires that the Docker daemon is running on the same host as Jenkins! Otherwise the workspace will not be available to your container.
Another solution would be to run Jenkins as Docker container, so you can share the Jenkins home/workspaces easily with the containers you run within your build jobs, like described here:
Running Jenkins tests in Docker containers build from dockerfile in codebase