Error while running sonarqube from Jenkins as sonar-scanner Permission denied - docker

Below is the error from Jenkins console output:
+ sonar-scanner -Dsonar.login=**** -Dsonar.projectBaseDir=.
/var/lib/jenkins/workspace/Mtr-Pipeline_develop#2#tmp/durable-0080bcff/script.sh: 1: /var/lib/jenkins/workspace/Mtr-Pipeline_develop#2#tmp/durable-0080bcff/script.sh: sonar-scanner: Permission denied
I have setup the token and pasted the key in t-m-sonar-login variable in Jenkins global credentials.But I dont think it should be the keys causing `permission denied error. Can someone provide some pointers to look into the issue.
stage('SonarQube scan') {
agent {
dockerfile { filename 'sonar/Dockerfile' }
}
steps {
withCredentials([string(credentialsId: 't-m-sonar-login', variable: 'SONAR_LOGIN')]) {
script {
unstash 'coverage'
unstash 'testResults'
sh 'sonar-scanner -Dsonar.login="$SONAR_LOGIN" -Dsonar.projectBaseDir=. || true'
}
}
}
}
sonar/Dockerfile:
FROM node:15-buster
################
# Install java #
################
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive \
apt-get -y install openjdk-11-jre-headless && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
############################
# Install SonarQube client #
############################
WORKDIR /root
RUN apt-get install -y curl grep sed unzip
RUN curl --insecure -o ./sonarscanner.zip -L https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.4.0.2170-linux.zip
RUN unzip -q sonarscanner.zip
RUN rm sonarscanner.zip
RUN mv sonar-scanner-4.4.0.2170-linux sonar-scanner
ENV SONAR_RUNNER_HOME=/root/sonar-scanner
ENV PATH $PATH:/root/sonar-scanner/bin
# Include Sonar configuration and project paths
COPY ./sonar/sonar-runner.properties ./sonar-scanner/conf/sonar-scanner.properties
# Ensure Sonar uses the provided Java for musl instead of a borked glibc one
RUN sed -i 's/use_embedded_jre=true/use_embedded_jre=false/g' /root/sonar-scanner/bin/sonar-scanner

Please check whether Java is available on the system where SonarQube Scanner is running.
Another thing you can try is:
Go to SonarQube Scanner Directory -> Go to bin -> chmod +x sonar-scanner

Related

Passing Jenkins Pipeline parameter to a Dockerfile

I'm getting a "Bad substitution" error when trying to pass a pipeline parameter to the Dockerfile.
Jenkins parameter: version
Jenkinsfile:
pipeline {
agent any
stages {
stage('Build in docker container') {
agent { dockerfile true }
steps {
sh 'node -v'
}
}
}
}
Dockerfile:
FROM ubuntu:16.04
WORKDIR /root
# install dependencies
RUN apt-get update
RUN apt-get install curl wget vim nano zip git htop ncdu build-essential chrpath libssl-dev libxft-dev apt-transport-https -y
# install node 10
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash
RUN apt-get install --yes nodejs
#RUN node -v
#RUN npm -v
RUN echo ${params.version}
#ARG VERSION
#RUN echo $VERSION
Jenkins error message:
Jenkins error message
I'm sure the problem is that im new to pipelines/docker. :)
I would be grateful for any help.
issue resolved by adding the ARG variable to the Dockerfile.
This is how the Dockerfile looks like:
FROM ubuntu:16.04
WORKDIR /root
# install dependencies
RUN apt-get update
RUN apt-get install curl wget vim nano zip git htop ncdu build-essential chrpath libssl-dev libxft-dev apt-transport-https -y
# install node 10
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash
RUN apt-get install --yes nodejs
#RUN node -v
#RUN npm -v
ARG version=fisticuff
RUN echo $version
and this is how the Jenkinsfile looks like:
pipeline {
agent any
stages {
stage('Build in docker container') {
agent {
dockerfile {
additionalBuildArgs '--build-arg version="$version"'
}
}
steps {
sh 'node -v'
}
}
}
}
Console output in Jenkins:
Jenkins console output
Much obliged to all of you for giving me the hints. It helped me a lot!
Try running Dockerfile independently first.
Since you are new to docker try one step at a time.

Can you install command-line packages in jib docker image?

I need to install command line tools like jq, curl etc in the docker image created by maven jib plugin. How can I achieve this? Any help would be greatly appreciated. Thanks.
As explained in the other answer, using a base image customized with pre-installed tools that rarely change is a good solution.
Alternatively, you may put curl using Jib's <extraDirectories> feature, which enables adding arbitrary files to the target image. Check the Maven and Gradle docs for more details. As explained in the docs, you will also need to configure <permissions> to set executable bits to curl.
If you prefer, you could even set up your Maven or Gradle builds to download curl and unpack it. Here's an example Jib setup (showing both Maven and Gradle) from the Jib repository.
Adding a reference Dockerfile and you can build your own base image by creating your Dockerfile and then build it.
FROM openjdk:8-jdk-alpine
RUN apk add --no-cache curl tar bash procps
# Downloading and installing Maven
ARG MAVEN_VERSION=3.6.1
ARG USER_HOME_DIR="/root"
ARG SHA=b4880fb7a3d81edd190a029440cdf17f308621af68475a4fe976296e71ff4a4b546dd6d8a58aaafba334d309cc11e638c52808a4b0e818fc0fd544226d952544
ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& echo "Downlaoding maven" \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
\
&& echo "Checking download hash" \
&& echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c - \
\
&& echo "Unziping maven" \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
\
&& echo "Cleaning and setting links" \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
# Downloading and installing Gradle
# 1- Define a constant with the version of gradle you want to install
ARG GRADLE_VERSION=4.0.1
# 2- Define the URL where gradle can be downloaded from
ARG GRADLE_BASE_URL=https://services.gradle.org/distributions
# 3- Define the SHA key to validate the gradle download
# obtained from here https://gradle.org/release-checksums/
ARG GRADLE_SHA=d717e46200d1359893f891dab047fdab98784143ac76861b53c50dbd03b44fd4
# 4- Create the directories, download gradle, validate the download, install it, remove downloaded file and set links
RUN mkdir -p /usr/share/gradle /usr/share/gradle/ref \
&& echo "Downlaoding gradle hash" \
&& curl -fsSL -o /tmp/gradle.zip ${GRADLE_BASE_URL}/gradle-${GRADLE_VERSION}-bin.zip \
\
&& echo "Checking download hash" \
&& echo "${GRADLE_SHA} /tmp/gradle.zip" | sha256sum -c - \
\
&& echo "Unziping gradle" \
&& unzip -d /usr/share/gradle /tmp/gradle.zip \
\
&& echo "Cleaning and setting links" \
&& rm -f /tmp/gradle.zip \
&& ln -s /usr/share/gradle/gradle-${GRADLE_VERSION} /usr/bin/gradle
# 5- Define environmental variables required by gradle
ENV GRADLE_VERSION 4.0.1
ENV GRADLE_HOME /usr/bin/gradle
ENV GRADLE_USER_HOME /cache
ENV PATH $PATH:$GRADLE_HOME/bin
VOLUME $GRADLE_USER_HOME
CMD [""]
Ref:- https://docs.docker.com/engine/reference/builder/
Once your custom image is ready, push it to Registry and then reference it in jib in following manner.
mvn compile jib:build \
-Djib.from.image=customImage

Running dep ensure -vendor-only inside Docker Hangs not able to pull private Repos

My Dockerfile:
FROM golang:1.11.4
RUN apt-get update && apt-get install git bash curl -yqq
ENV ENV test
ENV GIT_TERMINAL_PROMPT=1
ENV GITHUB_TOKEN XXXXXXXXXXXXXXXXXX 
 RUN curl -Ls https://github.com/Masterminds/glide/releases/download/v0.12.3/glide-v0.12.3-linux-amd64.tar.gz | tar xz -C /tmp \
&& mv /tmp/linux-amd64/glide /usr/bin/
RUN curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
RUN mkdir -p $GOPATH/src/github.com/<Myrepo>/
COPY . $GOPATH/src/github.com/<Myrepo>/
WORKDIR $GOPATH/src/github.com/<Myrepo>/
RUN dep ensure -vendor-only
When i am building this docker file it hangs at RUN dep ensure -vendor-only
It fails to pull the dependencies which are private repos
Is there any possiblities to store git credentials inside Docker or any way to build Docker with one or more private repos of GOlang
Use some thing like this
# ensure that the private Github repo is
# accessed using SSH instead of HTTPS
RUN ssh-keyscan github.com > /root/.ssh/known_hosts
RUN echo "$SSH_KEY" > /root/.ssh/id_rsa && chmod 0600 /root/.ssh/id_rsa
RUN echo '[url "ssh://git#github.com/*your_repo*/"]' >> /root/.gitconfig && echo 'insteadOf = https://github.com/*your_repo*/' >> /root/.gitconfig
Refer this to add ssh key to your git repo
Adding .netrc file will pass credentials inside the docker containers and helps to pull more than one private repositories to build dependencies
#vim .netrc
machine github.com
login < your github token >
add those 2 lines and pass your github token
FROM golang:1.11.4
RUN apt-get update && apt-get install git bash curl -yqq
ENV ENV test
ENV GIT_TERMINAL_PROMPT=1
ENV GITHUB_TOKEN XXXXXXXXXXXXXXXXXX 
RUN curl -Ls https://github.com/Masterminds/glide/releases/download/v0.12.3/glide-v0.12.3-linux-amd64.tar.gz | tar xz -C /tmp \
&& mv /tmp/linux-amd64/glide /usr/bin/
RUN curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
RUN mkdir -p $GOPATH/src/github.com/<Myrepo>/
COPY . $GOPATH/src/github.com/<Myrepo>/
COPY .netrc /root/
WORKDIR $GOPATH/src/github.com/<Myrepo>/
RUN dep ensure -vendor-only

Run command in existing Docker using Jenkins-Pipelines

I'm new with Jenkins-Groovy and try to run a command within an existing Docker-Container and before setting some environmental-variables using a Jenkins-Pipeline.
The bash-script used for right now (so just executing it from the command line) looks like that and works:
export LIB_ROOT=/usr/local/LIBS
export TMP_MAC_ADDRESS=b5:17:a3:28:55:ea
sudo docker run --rm -i -v "$LIB_ROOT":/usr/local/LIBS/from-host -v /home/sbuild/Dockerfiles/Sfiles/mnt:/home/sbuild/mount --mac-address="$TMP_MAC_ADDRESS" -t sbuild:current
Afterwards I want to build some of my sources (mounted) inside the Docker-Container using something like:
python3 batchCompile.sh ../mount/src.zip
Right now I've been trying to write it like that in my Jenkins:
node ('linux-slave') {
withEnv(['PATH=/usr/local/LIBS:/usr/local/MATLAB/from-host -v /home/sbuild/Dockerfiles/Sfiles/mnt:/home/sbuild/mount --mac-address=b5:17:a3:28:55:ea']) {
docker.image('sbuild').inside {
sh 'echo $PATH'
sh 'mvn --version'
}
}
sh 'echo $PATH'
}
Yet this just fails with an opaque message:
Running in Durability level: MAX_SURVIVABILITY
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 71: Expected a symbol # line 71, column 25.
docker.image('sbuild:current').inside {
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
I'm not able to figure out what is running wrong.
So I was just trying to get inside the Docker and look what I can do from there. With this little script I was experimenting a little:
script{
docker.image('sbuild:current').inside{
sh 'touch asdf'
sh 'cd /home/sbuild/'
sh 'pwd'
}
Yet by default I'm just working from the Jeninks-Folder and none of these commands are actually called inside the Docker. Also the container doesn't seem to run at any time.
How do I have to write my code to start the Docker I had configured and use commands inside?
There's some documentation outside for creating new Docker containers, but I have difficulties to figure out how to make sense of that error message and how to properly debug.
Edit 1: The Dockerfile:
FROM labs:R2018
# Avoid interaction
ENV DEBIAN_FRONTEND noninteractive
# Set user to root
USER root
# =========== Basic Configuration ======================================================
# Update the system
#RUN apt-get -y update \
# && apt-get install -y sudo build-essential git python python-dev \
# python-setuptools make g++ cmake gfortran ipython swig ant python-numpy \
# python-scipy python-matplotlib cython python-lxml python-nose python-jpype \
# libboost-dev jcc git subversion wget zlib1g-dev pkg-config clang
# Install system libs
# RUN apt-get install sudo
# ========== Install pip for managing python packages ==================================
RUN apt-get install -y python-pip python-lxml && pip install cython
# Install simulix dependencies
RUN apt-get install -y git
RUN apt-get install --assume-yes python
RUN apt-get install --assume-yes cmake
RUN apt-get install --assume-yes mingw-w64
# Add User
#RUN adduser --disabled-password --gecos '' docker
#RUN adduser docker sudo
#RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER build
# Install simulix
WORKDIR /home/sbuild
RUN git clone https://github.com/***.git
RUN mkdir mount
WORKDIR /home/sbuild/Sfiles
RUN pip install -r requirements.txt
When I use Docker with Jenkins Pipeline I do it with the sh step only:
try {
stage('Start Docker') {
sh 'docker-compose up'
}
stage('Build project') {
sh 'docker-compose exec my_service make:build
}
} catch (Error e)
// Maybe do something
} finally {
sh 'docker-compose stop'
}
You want to surround your stages with a try/catch/finally block to always stop the docker containers in case of failure.

npm install fails in Jenkins pipeline

I've created a docker images to be able to run node >= 7.9.0 and monogodb for testing in Jenkins. Some might argue that testing with mongodb is not correct approach but the app uses it extensively and I have some complex updates and deletes so I need it there.
Docker file is under dockerfiles/test/Dockerfile in my github repo. When using the pipeline syntax the docker images is built successfully but I can't do sh 'npm install' or sh 'npm -v' in the steps of the pipeline. The docker images is tested and if I build it locally and run it I can do the npm install there. sh 'node -v' runs successfully in the pipeline and also sh 'ls'.
Here is the pipeline syntax.
pipeline {
agent { dockerfile { dir 'dockerfiles/test' } }
stages {
stage('Build') {
steps {
sh 'npm install'
}
}
}
post {
always {
echo 'I will always say Hello again!'
}
}
}
I get this error: ERROR: script returned exit code -1. I can't see anything wrong here. I've also tested with other node images with the same result. If I run it with a node slave I can do the installation but I do not want to have many different slaves with a lot of setups for integration tests.
And here is the dockerfile
FROM ubuntu:16.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
RUN echo "deb http://repo.mongodb.org/apt/ubuntu $(cat /etc/lsb-release | grep DISTRIB_CODENAME | cut -d= -f2)/mongodb-org/3.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.2.list
RUN apt-get update && apt-get install -y \
curl && \
curl -sL https://deb.nodesource.com/setup_7.x | bash - && \
apt-get install -y nodejs && \
apt-get install -y mongodb-org
RUN mkdir -p /data/db
RUN export LC_ALL=C
RUN groupadd -g 1000 jenkins && useradd -u 1000 jenkins -g jenkins
EXPOSE 27017
CMD ["/usr/bin/mongod"]
Found a workaround to a similar problem.
Problem
Jenkins running a pipeline job
This job is running commands inside a debian slim container
All commands are failing instantly with no error output, only a ERROR: script returned exit code -1
Running the container outside docker and executing the same commands with the same user is working as it should be
Extract from Jenkinfile :
androidImage = docker.build("android")
androidImage.inside('-u root') {
stage('Install'){
sh 'npm install' // is failing with generic error and no output
}
Solution
Found the answer on Jenkins bugtracker : https://issues.jenkins-ci.org/browse/JENKINS-35370 and on Jenkins Docker Pipeline Exit Code -1
My problem was solved by installing the procps package in my debian Dockerfile :
apt-get install -y procps
I replicated your setup as faithfully as I could. I used your Dockerfile and Jenkinsfile, and here's my package.json:
{
"name": "minimal",
"description": "Minimal package.json",
"version": "0.0.1",
"devDependencies": {
"mocha": "*"
}
}
It failed like this for me during npm install:
npm ERR! Error: EACCES: permission denied, mkdir '/home/jenkins'
I updated one line in your Dockerfile to add --create-home:
RUN groupadd -g 1000 jenkins && useradd -u 1000 jenkins -g jenkins --create-home
And the build passed. Kudos to #mkobit for keying in on the issue and linking to the jenkins issue that will make this cleaner in the future.

Resources