Current Setup requires Docker Components (Dockerfile, requirements.txt, Jenkinsfile) to be separate from Source Code. Docker Components resides in a directory whereas the Source Code resides in another directory. Design so far is to build the Docker Image with the Docker Components and run the Docker Image, and the Source Code is inserted into the root directory of the container. Model so far works on the CMD terminal however when translating over to the Jenkins Pipeline, the docker exec command is not recognised on the Jenkins Pipeline
Stages run well on the Jenkins Pipeline until the stage to insert Source Code as a volume. When running $ docker exec -it source-container bash, below is the error log:
C:\WINDOWS\system32\config\systemprofile\AppData\Local\Jenkins\.jenkins\workspace\Source-Code-Pipeline>docker exec -ti source-container bash
the input device is not a TTY. If you are using mintty, try prefixing the command with 'winpty'
When prefixing with 'winpty', pipeline does not execute well too. How can I go about in resolving this issue?
Below is the Jenkinsfile to depict the Pipeline flow:
node {
checkout scm
stage ('Create Docker Registry') {
bat 'docker run -d -p 5000:5000 --restart=always --name registry registry:2'
}
stage ('Build Docker Image') {
def image = docker.build("docker-csv", '.')
}
stage ('Tag and Push Docker Image') {
bat 'docker tag docker-csv localhost:5000/docker-csv'
bat 'docker push localhost:5000/docker-csv'
}
stage ('Pull Docker Image from Local Registry') {
bat 'docker pull localhost:5000/docker-csv'
}
stage ('Insert Source Code as Volume into Container') {
bat 'docker run --name source-container -d -v /c/Users/z0048yrk/Desktop/Source-Code:/root localhost:5000/docker-csv tail -f /dev/null'
bat 'docker exec -it source-container bash'
bat 'cd root'
bat 'python test.py > output.csv'
}
stage ('Copy output.csv into desired directory') {
dir("C:\\Users\\z0048yrk\\Desktop\\LTA\\new-demo") {
bat 'docker cp source-container:/root/output.csv'
}
}
}
Related
can someone please let me know on how to pass insecure registry as an argument when using DIND image as runtime container.
Please find the below sample script which i had been using as docker run time agent.
'''
pipeline {
agent {label 'jenkins-docker-slave'}
stages{
stage('maven version'){
agent {docker { image 'maven:latest'}}
steps{
script{
sh "mvn --version"
}
}
}
stage('docker build'){
agent {
docker {
image 'docker:dind'
args '-v /var/run/docker.sock:/var/run/docker.sock'
}
}
steps{
script{
sh 'docker version'
}
}
}
}
}
'''
Docker version works fine but my motive is to set insecure registry as below during the runtime so that i can use docker login and docker push commands successfully.
{
"insecure-registries": [
"ec2-52-39-183-6.us-west-2.compute.amazonaws.com:8123"
]
}
I want to make the above condition work during the runtime. Can someone please advise on how this can be achived with DIND?
I have found a solution for my query on my own. Below is the solution:
sh 'docker exec --tty $(docker ps -ql) sh -c "mkdir -p /etc/docker"'
sh 'docker exec --tty $(docker ps -ql) sh -c "mkdir -p /root/.docker"'
sh '''
docker exec --tty $(docker ps -ql) sh -c "cat <<EOF > /etc/docker/daemon.json
{
"insecure-registries": [
"ec2-52-36-87-109.us-west-2.compute.amazonaws.com:8123"
]
}"
'''
But the only problem is , the above used port 8123 for the docker repository connector is not opened due to which getting the below error.
Error response from daemon
Is there any way to open ports within docker container to get this worked?
I am following the Jenkins tutorial with some modification.
I run the Jenkins docker container by:
docker run --rm --privileged -u root -p 8080:8080 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$PWD"/vol:/var/jenkins_home \
jenkinsci/blueocean
With my Jenkinsfiles:
stage('Test') {
agent {
docker {
image 'qnib/pytest'
}
}
steps {
sh 'ls' ##### 1
sh 'py.test --junit-xml test-reports/results.xml sources/test_calc.py' ##### 2
}
}
stage('Deliver') {
agent any
environment {
VOLUME = '$(pwd)/sources:/src'
ABS_WS = '/home/myname/vol/workspace'
JOB_WS = "\${PWD##*/}"
IMAGE = 'cdrx/pyinstaller-linux:python2'
}
steps {
dir(path: env.BUILD_ID) {
unstash(name: 'compiled-results')
sh "pwd" ##### 3
sh "ls" ##### 4
sh "docker run -v '${ABS_WS}/${JOB_WS}/sources:/src' ${IMAGE} 'ls'" ##### 5
sh "docker run -v ${ABS_WS}/${JOB_WS}/sources:/src ${IMAGE} 'ls'" ##### 6
sh "docker run -v ${VOLUME} ${IMAGE} 'ls'" ##### 7
}
}
}
The output and my questions for ####1~6:
####1: ls here including the /sources/*.py that docker container(qnib/pytest) can process.
####3: output: /var/jenkins_home/workspace/simple-python-pyinstaller-app/32
####4: ls here also including the /soucres/*.py we need
####5: ls here didn't include /sources/*.py, due to docker volume mounted failed.
I already tried with different solution from here, still not working.
docker run -v '/home/myname/vol/workspace/${PWD##*/}/sources:/src' cdrx/pyinstaller-linux:python2 ls
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
ls
add2vals.spec
build
dist
BUT ####6, similar to ####5 just without Single quotation, nothing output from ls (WHY?):
docker run -v /home/myname/vol/workspace/32/sources:/src cdrx/pyinstaller-linux:python2 ls
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
ls
####7. the output is identical to ####5
docker run
-v /var/jenkins_home/workspace/simple-python-pyinstaller-app/32/sources:/src cdrx/pyinstaller-linux:python2 ls
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
ls
add2vals.spec
build
dist
My questions are:
In Deliver stage, how can I map docker container volume to the host or Jenkins container?
In ####3,4 the path in Jenkins container is /var/jenkins_home/workspace/simple-python-pyinstaller-app/32 , this path including the /sources/*.py; and #####7 we can see /var/jenkins_home/workspace/simple-python-pyinstaller-app/32/sources:/src, I thought it was mounted on the correct path to /src in pyinstaller-linux container.
I am not very clear why in Test stage we don't need to mount any volume when running pytest docker?
And why not Deliver stage going the same way as Test stage? (like ####2)
What is difference between ####6 and ####5 ?
node{
stage('Scm Checkout'){
git credentialsId: 'git-creds', url: 'https://github.com/mouthik/jenkinsfile.git'
}
stage('Build docker image'){
sh 'docker build -t mouthik/my-app:2.0.0 .'
stage('Run docker container'){
sh 'docker run -p 8080:8080 -d -name my-app mouthik/my-app:2.0.0'
}
}
}
You need to install docker on the slave.
Try
sudo dnf install docker-ce
and start the docker service
to build a docker image you need to write Dockerfile
and then docker build https://docs.docker.com/engine/reference/commandline/build/
I have a Jenkinsfile pipeline creating 3 docker images/containers as below:
stage('Build images')
{
echo "workspace directory is ${workspace}"
dir ("$workspace/build/virtuoso")
{
sh 'docker build -t virtuoso -f $WORKSPACE/build/virtuoso/Dockerfile .'
}
dir ("$workspace/build/wildfly")
{
sh 'docker build -t wildfly -f $WORKSPACE/build/wildfly/Dockerfile .'
}
dir ("$workspace/build/postgres")
{
sh 'docker build -t postgres -f $WORKSPACE/build/postgres/Dockerfile .'
}
}
I need to push these 3 images to Nexus repository manager. Can somebody help me on this?
The solution to this is:
stage('Push Docker Images to Nexus Registry'){
sh 'docker login -u user -p password NexusDockerRegistryUrl'
sh 'docker push NexusDockerRegistryUrl/Imagename}'
sh 'docker rmi $(docker images --filter=reference="NexusDockerRegistryUrl/ImageName*" -q)'
sh 'docker logout NexusDockerRegistryUrl'
}
The above stage in Jenkins pipeline would push the docekr image to nexus docker registry and remove existing image.
Thanks
I'm trying to run docker containers in the Jenkins Pipeline.
I have the following in my Jenkinsfile:
stage('test') {
steps {
script {
parallel (
"gatling" : {
sh 'bash ./test-gatling.sh'
},
"python" : {
sh 'bash ./test-python.sh'
})
}
}
}
In the test-gatling.sh I have this:
#!/bin/bash
docker cp RecordedSimulation.scala denvazh/gatling:/RecordedSimulation.scala
docker run -it -m denvazh/gatling /bin/bash
ls
./gatling.sh
The ls command is there just for test, but when it's executed it lists files and folders of my github repository, rather than the files inside the denvazh/gatling container. Why is that? I thought the docker run -it [...] command would open the container so that commands could be run inside it?
================
Also, how do I run a container and just have it running, without executing any commands inside it? (In the Jenkins Pipeline ofc)
I'd like to run: docker run -d -p 8080:8080 -t [my_container] and access it on port 8080. How do I do that...?
If anyone has the same or similar problem, here are the answers:
Use docker exec [name of container] command, and to run any terminal commands inside a container, add /bin/bash -c "[command]"
To be able to access a container/app that is running on any port from a second container, when starting the second container run it with --net=host parameter