using fabric8 docker-maven-plugin in a Jenkins docker container - docker

I am using fabric8's docker-maven-plugin to build and push my docker image. I do something like:
mvn docker:build
in both my dev environment and Jenkins. But here is my problem.
I have Jenkins running in a docker Swarm.
docker service create --name jenkins -p 8080:8080 -p 50000:50000 --replicas=1 --mount type=volume,src=jenkins_home,dst=/var/jenkins_home --mount type=bind,source=/var/run/docker.sock,destination=/var/run/docker.sock jenkins/jenkins
Notice that I have a bind mount from the host's /var/run/docker.sock to the container's /var/run/docker.sock.
I then install the Docker binaries inside the container by running:
apt-get update && apt-get -y install apt-transport-https ca-certificates \
curl gnupg2 software-properties-common && \
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable" && \
apt-get update && \ apt-get -y install docker-ce=18.06.1~ce~3-0~debian
lastly I add the jenkins user to the docker group
useradd -G {docker} jenkins
In reality I will build my own docker image using the jenkins/jenkins image as my root using the above mentioned commands.
I then login to Jenkins, crate my pipeline as follows
node {
def mvnHome
stage('Preparation') { // for display purposes
git branch: 'branch', credentialsId: 'id', url: 'https://url'
mvnHome = tool 'm3'
env.JAVA_HOME="${tool 'java8'}"
env.DOCKER_HOST="unix://var/run/docker.sock"
env.PATH="${env.JAVA_HOME}/bin:${env.PATH}"
}
stage('Build Docker Image') {
dir('rms-donation-manager') {
sh "'${mvnHome}/bin/mvn' clean install docker:build"
}
}
}
When I run the pipeline i get the following error:
+ /var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/m3/bin/mvn clean install docker:build
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building donation-manager 1.0.3-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- docker-maven-plugin:0.28.0:build (default-cli) # donation-manager ---
Apr 07, 2019 3:21:24 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.io.IOException) caught when processing request to {}->unix://127.0.0.1:1: Permission denied
Apr 07, 2019 3:21:24 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->unix://127.0.0.1:1
Apr 07, 2019 3:21:24 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.io.IOException) caught when processing request to {}->unix://127.0.0.1:1: Permission denied
Apr 07, 2019 3:21:24 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->unix://127.0.0.1:1
Apr 07, 2019 3:21:24 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.io.IOException) caught when processing request to {}->unix://127.0.0.1:1: Permission denied
Apr 07, 2019 3:21:24 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->unix://127.0.0.1:1
[ERROR] DOCKER> Cannot create docker access object [Permission denied]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.554 s
[INFO] Finished at: 2019-04-07T03:21:24Z
[INFO] Final Memory: 17M/175M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.28.0:build (default-cli) on project donation-manager: Cannot create docker access object: Permission denied -> [Help 1]
Notice the command that was run by Jenkins
/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/m3/bin/mvn clean install docker:build
Its using the the maven tool 'm3' which i am using in the Jenkins pipeline and running the maven command mvn clean install docker:build and its complaining about permissions
Here is the kicker. I go into the jenkins container
docker exec -it ec4be3dffa62 /bin/bash
Notice that I am not going in as the root user, I am actually the jenkins user. I then go in to my project where the pom.xml file is located and run
/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/m3/bin/mvn clean install docker:build
And it works!!!! The image is built. Why does it work when i am inside the container, and not when I am running it from the jenkins ui. Is the jenkins UI using a different user?

In above mention sequence first you started Jenkins and then you have added jenkins user in docker group. if this is the case, changes is not reflected for the Jenkins server. but when you are executing manually, at that time jenkins added to docker group. To test - create a test job and execute script "id -a". this will show groups for user Jenkins at that point of time. same cmd execute from inside of container. if there is difference, i would suggest you to create a dockerfile with all above installation and useradd and then start Jenkins.

Related

Packer fails on Gitlab-CI with {message:401 Unauthorized}: command not found

I am trying to use packer with GCP in gitlab-ci but every time I get to the packer build it will fail with the following error:
starting remote command: chmod +x /tmp/script_5147.sh; DEBIAN_FRONTEND='noninteractive' PACKER_BUILDER_TYPE='googlecompute' PACKER_BUILD_NAME='my_vm' /tmp/script_5147.sh
==> googlecompute.my_vm: /tmp/script_5147.sh: line 1: {message:401 Unauthorized}: command not found
2022/11/13 13:26:05 [INFO] 0 bytes written for 'stdout'
2022/11/13 13:26:05 packer-plugin-googlecompute_v1.0.16_x5.0_linux_amd64 plugin: 2022/11/13 13:26:05 [ERROR] Remote command exited with '127': chmod +x /tmp/script_5147.sh; DEBIAN_FRONTEND='noninteractive' PACKER_BUILDER_TYPE='googlecompute' PACKER_BUILD_NAME='my_vm' /tmp/script_5147.sh
2022/11/13 13:26:05 packer-plugin-googlecompute_v1.0.16_x5.0_linux_amd64 plugin: 2022/11/13 13:26:05 [INFO] RPC endpoint: Communicator ended with: 127
The script
#!/bin/bash
set -e
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit
fi
apt update
apt install -y curl
curl -fsSL https://deb.nodesource.com/setup_18.x | bash -
apt upgrade -y iptables
# The iptables-persistent must be installed in order to create the /etc/iptables/rules.v4 file
apt install -y nginx libzmq3-dev nodejs ipset iptables-persistent net-tools libre2-dev
npm install -g yarn
rm /etc/nginx/sites-enabled/default
The packer is working locally (also with the image) and I created a VM in GCP all of them are working, only in gitlab it fails.
I created a custom image of packer and it still failed for me on gitlab.
I will try to move it to my own runner but it will take a few days for me.
Would love if someone could help me figure it out.
Ok, I figured out the problem.
I curled some files and the token was not good so when it got to packer it failed because the files were not as expected.
I printing the 401 of the curl.

How to create a Docker image of a Scala Play application using jre-alpine base image

I'd like to take advantage of the jre-alpine Docker base image to shrink down the size of the docker image that I have to push to my Docker registry, but I constantly hit this error:
[info] Step 10/16 : RUN id -u demiourgos728 2> /dev/null || useradd --system --create-home --uid 1001 --gid 0 demiourgos728
[info] ---> Running in 696dcf40530a
[info] /bin/sh: useradd: not found
[info] Removing intermediate container 696dcf40530a
[error] The command '/bin/sh -c id -u demiourgos728 2> /dev/null || useradd --system --create-home --uid 1001 --gid 0 demiourgos728' returned a non-zero code: 127
[error] java.lang.RuntimeException: Nonzero exit value: 127
[error] at com.typesafe.sbt.packager.docker.DockerPlugin$.publishLocalDocker(DockerPlugin.scala:483)
[error] at com.typesafe.sbt.packager.docker.DockerPlugin$.$anonfun$projectSettings$33(DockerPlugin.scala:187)
[error] at com.typesafe.sbt.packager.docker.DockerPlugin$.$anonfun$projectSettings$33$adapted(DockerPlugin.scala:185)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error] at sbt.std.Transform$$anon$4.work(System.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.Execute.work(Execute.scala:278)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] (Docker / publishLocal) Nonzero exit value: 127
Versions involved:
Play 2.7.3
Scala 2.12.8
Docker 19.03.1
sbt 1.2.8
In my build.sbt I added:
enablePlugins(AshScriptPlugin)
dockerBaseImage := "openjdk:jre-alpine"
Following the instructions provided here: https://www.scala-sbt.org/sbt-native-packager/formats/docker.html#busybox-ash-support
The Docker commands that Play tries to execute:
show dockerCommands
[info] Wrote /Users/xxxxx/projects/together/togrx/target/scala-2.12/together-rx_2.12-0.7.0-a.pom
[info] * Cmd(FROM,WrappedArray(openjdk:8-jre-alpine, as, stage0))
[info] * Cmd(WORKDIR,WrappedArray(/opt/docker))
[info] * Cmd(COPY,WrappedArray(opt /opt))
[info] * Cmd(USER,WrappedArray(root))
[info] * ExecCmd(RUN,List(chmod, -R, u=rX,g=rX, /opt/docker))
[info] * ExecCmd(RUN,List(chmod, u+x,g+x, /opt/docker/bin/together-rx))
[info] * DockerStageBreak
[info] * Cmd(FROM,WrappedArray(openjdk:8-jre-alpine))
[info] * Cmd(USER,WrappedArray(root))
[info] * Cmd(RUN,List(id, -u, demiourgos728, 2>, /dev/null, ||, useradd, --system, --create-home, --uid, 1001, --gid, 0, demiourgos728))
[info] * Cmd(WORKDIR,WrappedArray(/opt/docker))
[info] * Cmd(COPY,WrappedArray(--from=stage0 --chown=demiourgos728:root /opt/docker /opt/docker))
[info] * Cmd(EXPOSE,WrappedArray(4300 4301))
[info] * Cmd(USER,WrappedArray(1001))
[info] * ExecCmd(ENTRYPOINT,List(/opt/docker/bin/together-rx))
[info] * ExecCmd(CMD,List())
There is no useradd command in alpine image, use adduser instead, see next:
$ docker run -it openjdk:jre-alpine /bin/sh
Unable to find image 'openjdk:jre-alpine' locally
jre-alpine: Pulling from library/openjdk
[DEPRECATION NOTICE] registry v2 schema1 support will be removed in an upcoming release. Please contact admins of the docker.io registry NOW to avoid future disruption.
8e3ba11ec2a2: Pull complete
311ad0da4533: Pull complete
391a6a6b3651: Pull complete
Digest: sha256:016a7989474f1e685da966631ba6403cb349548621ebb8e4a6205f7c5fa88320
Status: Downloaded newer image for openjdk:jre-alpine
/ # useradd
/bin/sh: useradd: not found
/ # echo $?
127
/ # adduser
BusyBox v1.28.4 (2018-05-30 10:45:57 UTC) multi-call binary.
Usage: adduser [OPTIONS] USER [GROUP]
Create new user, or add USER to GROUP
-h DIR Home directory
-g GECOS GECOS field
-s SHELL Login shell
-G GRP Group
-S Create a system user
-D Don't assign a password
-H Don't create home directory
-u UID User id
-k SKEL Skeleton directory (/etc/skel)
/ #
So, you need to modify build.sbt or something else to make sure use adduser not useradd.
UPDATE:
As you said, the useradd was generated by play framework, then what I suggest is you to customize your own image to add useradd, it's in shadow package, then it will be transparent to play framework, like next:
Dockerfile:
FROM openjdk:jre-alpine
RUN apk add shadow
Build the image in your local docker host machine:
docker build -t myimage .
Finally, in build.sbt:
dockerBaseImage := "myimage"

docker build is not creating an image

I have an sbt project i have written a dockerFile in its root directory here is the Dockerfile
FROM hseeberger/scala-sbt:11.0.2_2.12.8_1.2.8
MAINTAINER name <email#gmail.com>
RUN echo "==> run the project..." && \
echo "==> it will take some time!!!" && \
sbt run
WORKDIR /hazelcastscalaserver
# Define default command.
CMD ["sbt", "run"]
when i do docker build -t test/project .
instead of dcoker creating an image it started to run the command here is expected behviour should be it shpuld build the image then i will run the image through docker run test/project:latest and my project started to run but its not happening where i am mistaken ?
here is the result of docker build
Sending build context to Docker daemon 208.1MB
Step 1/5 : FROM hseeberger/scala-sbt:11.0.2_2.12.8_1.2.8
---> 349a7e4f4029
Step 2/5 : MAINTAINER name <name#gmail.com>
---> Using cache
---> e92083819853
Step 3/5 : RUN echo "==> run the project..." && echo "==> it will take some time!!!" && sbt run
---> Running in 226ce249a353
==> run the project...
==> it will take some time!!!
[info] Updated file /root/project/build.properties: set sbt.version to 1.2.8
[info] Loading project definition from /root/project
[info] Updating ProjectRef(uri("file:/root/project/"), "root-build")...
[info] Done updating.
[info] Set current project to root (in build file:/root/)
[info] Updating ...
[info] Done updating.
[info] Packaging /root/target/scala-2.12/root_2.12-0.1.0-SNAPSHOT.jar ...
[info] Done packaging.
[error] java.lang.RuntimeException: No main class detected.
[error] at scala.sys.package$.error(package.scala:26)
[error] (Compile / bgRun) No main class detected.
[error] Total time: 0 s, completed May 2, 2019, 12:37:43 PM
The command '/bin/sh -c echo "==> run the project..." && echo "==> it will take some time!!!" && sbt run' returned a non-zero code: 1

Openshift container as root user

Hi Iam working on CICD implementation on openshift 3.9. I have a jenkins pod running in openshift. Iam running selenium scripts in jenkins and below is the error which iam getting like missing a package
Running TestSuite
/var/lib/jenkins/jobs/Pipeline/workspace/src/test/resources/chromedriver: error while loading shared libraries: libgconf-2.so.4: cannot open shared object file: No such file or directory
Nov 21, 2018 8:25:36 AM org.openqa.selenium.os.OsProcess checkForError
SEVERE: org.apache.commons.exec.ExecuteException: Process exited with an error: 127 (Exit value: 127)
Tests run: 8, Failures: 1, Errors: 0, Skipped: 7, Time elapsed: 21.9 sec <<< FAILURE! - in TestSuite
BrowserSettings(SecurityCheckList) Time elapsed: 21.273 sec <<< FAILURE!
org.openqa.selenium.WebDriverException: Timed out waiting for driver server to start.
Build info: version: '3.9.1', revision: '63f7b50', time: '2018-02-07T22:25:02.294Z'
System info: host: 'jenkins-1-7zgld', ip: '10.131.0.32', os.name: 'Linux', os.arch: 'i386', os.version: '3.10.0-957.el7.x86_64', java.version: '1.8.0_181'
Driver info: driver.version: ChromeDriver
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at com.google.common.util.concurrent.SimpleTimeLimiter.callWithTimeout(SimpleTimeLimiter.java:148)
at org.openqa.selenium.net.UrlChecker.waitUntilAvailable(UrlChecker.java:75)
at org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:187)
at org.openqa.selenium.remote.service.DriverService.start(DriverService.java:178)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:79)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:601)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:219)
For that i want to install libgconf-2-4 in my jenkins container through the below command
yum install libgconf-2-4
When i try to install the below error is coming in my jenkins container
sh-4.2$ yum install libgconf2-4
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
[Errno 13] Permission denied: '/etc/pki/entitlement-host'
ovl: Error while doing RPMdb copy-up:
[Errno 13] Permission denied: '/var/lib/rpm/.dbenv.lock'
You need to be root to perform this command.
When i goto specified location and try to change the permissions as chmod 777 .dbenv.lock
sh-4.2$ cd /var/lib/rpm/
sh-4.2$ ls -latr
total 19560
-rw-r--r--. 1 root root 0 Aug 9 18:21 .dbenv.lock
it is throwing error as
sh-4.2$ chmod 777 .dbenv.lock
chmod: changing permissions of ‘.dbenv.lock’: Operation not permitted
My question is how to enter into jenkins pod as root user and install the rpm package libgconf-2-4 through yum install libgconf-2-4 in openshift?
It seems you should customize the jenkins images as follows.[0]
Create the Dockerfile.
FROM registry.access.redhat.com/openshift3/jenkins-2-rhel7
USER 0
RUN yum -y install libgconf2-4 && yum clean all -y
USER 1001
Build the image using the Dockerfile.
docker build .
Login the internal registry of OpenShift for pushing image.
docker login -u admin -p docker-registry.default.svc:5000
Retag as OpenShift image format and your tag policy.
docker tag docker-registry.default.svc:5000/openshift/jenkins-2-rhel7-custom
Push the image.
docker push docker-registry.default.svc:5000/openshift/jenkins-2-rhel7-custom
Edit your deploymentConfig
oc edit dc/jenkins
...
containers:
...
image: "openshift/jenkins-2-rhel7-custom"
...
I hope it help you. :^)
[0]General Container Image Guidelines
you can use USER root in your dockerfile that will solve your problem

Running sbt in docker as non-root user

Trying to create a docker image that has sbt installed and can build sbt projects but, when building, will not be running as the root user (this is all in the context of running Jenkins inside docker).
Dockerfile sets up sbt
ENV SBT_VERSION=1.1.6
RUN \
curl -L -o sbt-$SBT_VERSION.deb http://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb && \
apt-get update && \
apt-get install sbt && \
sbt sbtVersion
And if I then run sbt as the root user, all works ok
docker exec -u root myjenkins sbt sbtVersion
produces
[warn] No sbt.version set in project/build.properties, base directory: /
[info] Set current project to root (in build file:/)
[info] 1.1.6
But when I run sbt as the jenkins user, it tries to download sbt 1.1.6 again and eventually fails when it tries to modify an apt system file.
docker exec -u jenkins myjenkins sbt sbtVersion
produces:
Getting org.scala-sbt sbt 1.1.6 (this may take some time)...
downloading https://repo1.maven.org/maven2/org/scala-sbt/sbt/1.1.6/sbt-1.1.6.jar ...
[SUCCESSFUL ] org.scala-sbt#sbt;1.1.6!sbt.jar (68ms)
.
.
.
[warn] No sbt.version set in project/build.properties, base directory: /
[error] java.io.FileNotFoundException: /var/cache/apt/archives/lock (Permission denied)
I understand that all of the "RUN" commands in your Dockerfile are as a root user.
SBT downloading Scala: Check where it is downloading. SBT by default downloads dependencies on ~/.ivy2 (and/or ~/.m2). If you change user, your home also changes, so it will look for dependencies in /home/jenkins/.ivy2, then on .ivy2 (double-check on this), which do not have those dependencies downloaded already, so it tries to download them.
About the var/cache/apt/archives/lock, it is trying to install via SBT via apt with your jenkins user, when you need to be privileged user to use apt. Your app-user should not need to install anything (or anything that requires root access), but rather build an image with all required installs and then use it as a separate user. Also, if apt gives you headaches, you can just install via download into folder, something like:
RUN \
curl -fsL http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz | tar xfz - -C /usr/local && \
ln -s /usr/local/scala-$SCALA_VERSION/bin/* /usr/local/bin/
PS: You may want to run your container always as jenkins user, in that case you can use USER jenkins after you finished installations and do any additional unprivileged operations there.

Resources