I am trying to create a Jenkins pipeline where I need to execute multiline shell commands.
stage ('Test'){
name="myserver"
sh '''
"ssh -o StrictHostKeyChecking=no ${myserver} 'rm -rf temp && mkdir -p temp && mkdir -p real'"
'''
}
But it is always returning error as "command not found". If I run the same with
sh "ssh -o StrictHostKeyChecking=no ${myserver} 'rm -rf temp && mkdir -p temp && mkdir -p real' "
Is there a different way to access variable in multiline shell?
You need to use """ like this:
sh """
"ssh -o StrictHostKeyChecking=no ${myserver} 'rm -rf temp && mkdir -p temp && mkdir -p real'"
"""
Related
I am trying to create docker container with below dockerfile and what i am trying to do is I am trying to build this Dockerfile and What i am looking is, if I'll run container, then it should ask from me that from which user i need to login, I am getting only input like " Enter the Name: " after that It's not going to login with provided username, which i provided to the input.
Please help!
FROM ubuntu:14.04
RUN apt-get -y update && apt-get -y install sudo
ENV VES=14.04
RUN userdel -r Harsh && userdel -r Piyush && userdel -r Sajid
LABEL Name "Harsh Arora"
RUN useradd -ms /bin/bash Harsh && passwd "Harsharora:Harsharora" | chpasswd && adduser Harsh sudo
RUN useradd -m Piyush && passwd "Piyush:Piyush" | chpasswd
RUN useradd -m Sajid && passwd "Sajid:Sajid" | chpasswd
COPY test.sh /tmp/test.sh
ENTRYPOINT ["/tmp/test.sh"]
So, Please find below "test.sh" script for the references.
#!/bin/bash
#This is test script writing for entrypoint example in docker file.
echo -e " Enter The Name: \c" ; read Name
case $Name in
Love)
echo -e " *********************************************************************************************************************"
echo -e "\t\tTHIS CONTAINER RUNNING BY USER $Name"
echo -e " ********************************************************************************************************************* "
;;
Lucky) echo -e " *********************************************************************************************************************"
echo -e "\t\tTHIS CONTAINER RUNNING BY USER $Name"
echo -e " ********************************************************************************************************************* "
;;
Naveen) echo -e " *********************************************************************************************************************"
echo -e "\t\tTHIS CONTAINER RUNNING BY USER $Name"
echo -e " ********************************************************************************************************************* "
;;
*) echo -e " *********************************************************************************************************************"
echo -e "\t\tTHIS CONTAINER RUNNING BY USER $Name"
echo -e " ********************************************************************************************************************* "
;;
esac
$#
su - $Name <<< echo "$Name"
I'm trying to use sshagent plugin to deploy to remote server.
when using below syntax, I'm getting
pipeline {
agent any
stages {
stage('Deploy') {
steps {
sshagent(['nginx-ec2']) {
// some block
sh "ssh -o StrictHostKeyChecking=no ubuntu#<host_ip>"
sh "whoami"
}
}
}
}
}
getting output:
[Pipeline] sh (hide)
+ whoami
jenkins
while I'm expecting to run script at the remote server using provided credentials !!
So, I had to run it this way
pipeline {
agent any
stages {
stage('Deploy') {
steps {
sshagent(['nginx-ec2']) {
// some block
sh "ssh -o StrictHostKeyChecking=no -l ubuntu <host_ip> 'whoami && \
sudo apt update && sudo apt install -y docker.io && \
sudo usermod -aG docker ubuntu && \
source .bashrc && \
docker run -d nginx'"
}
}
}
}
}
Is there any "clean" way to run script on remote server with ubuntu instead of jenkins user ?
Edit:
I understand I need to run it under the ssh command not as separate sh script otherwise, it will run as jenkins and I'm able to do it in the scripted way as below.
That's why I'm asking if there's a better way to write it in the declarative way.
node {
stage('Deploy'){
def dockerRun = "whoami && \
sudo apt update && sudo apt install -y docker.io && \
sudo usermod -aG docker ubuntu && \
source .bashrc && \
docker run -d nginx "
sshagent(['nginx-ec2']) {
sh "ssh -o StrictHostKeyChecking=no ubuntu#<host_ip> '${dockerRun}' "
}
}
}
Thanks,
As noted, you should select a credential which does reference the right remote username, as seen in the SSH Agent Jenkins plugin:
node {
sshagent (credentials: ['deploy-dev']) {
sh 'ssh -o StrictHostKeyChecking=no -l cloudbees 192.168.1.106 uname -a'
}
}
Plus, I would execute only one script which would have the all sequence of commands you want to execute remotely.
Well, so far this is the best way to do so, in-spite of repetition!
pipeline {
agent any
stages {
stage('Deploy') {
steps {
sshagent(['nginx-ec2']) {
// some block
sh "ssh -o StrictHostKeyChecking=no -l ubuntu <remote_ip> 'whoami'"
sh "ssh -o StrictHostKeyChecking=no -l ubuntu <remote_ip> 'sudo apt update && sudo apt install -y docker.io'"
sh "ssh -o StrictHostKeyChecking=no -l ubuntu <remote_ip> 'sudo usermod -aG docker ubuntu'"
sh "ssh -o StrictHostKeyChecking=no -l ubuntu <remote_ip> 'source .bashrc'"
sh "ssh -o StrictHostKeyChecking=no -l ubuntu <remote_ip> 'docker run -d -nginx'"
}
}
}
}
}
I have a need to install awscli and jq library in Hasura Docker Image. I tried to use yum, apt-get or apk commands to install the dependencies, but none of them worked.
Docker Image: https://hub.docker.com/r/hasura/graphql-engine/
how to install these dependencies in Hasura Docker Image? Any help is appreciated.
Dockerfile:
FROM hasura/graphql-engine:latest
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
CMD ["./entrypoint.sh"]
entrypoint.sh:
#!/bin/sh
set -o errexit -o nounset -o pipefail
DB_HOST=${DB_HOST:-postgres}
DB_PORT=${DB_PORT:-5432}
if [ -z "${DB_NAME}" ]; then
echo "Must provide DB_NAME environment variable. Exiting...."
exit 1
fi
if [ -z "${DB_USER}" ]; then
echo "Must provide DB_USER environment variable. Exiting...."
exit 1
fi
if [ -z "${DB_PASSWORD}" ]; then
echo "Must provide DB_PASSWORD environment variable. Exiting...."
exit 1
fi
export HASURA_GRAPHQL_DATABASE_URL=postgres://${DB_USER}:${DB_PASSWORD}#${DB_HOST}:${DB_PORT}/${DB_NAME}
/bin/graphql-engine serve
DB_PASSWORD is encrypted with KMS, so i want to use aws cli to decrypt the password in entrypoint.sh file before setting the Environment Variable: HASURA_GRAPHQL_DATABASE_URL
I was able to customize Hasura Docker Image with the help of Hasura Team support.
Here is the link to github issue: https://github.com/hasura/graphql-engine/issues/2729
Dockerfile:
FROM hasura/graphql-engine:v1.0.0-beta.4 as base
FROM python:3.7-slim-stretch
RUN apt-get -y update \
&& apt-get install -y --no-install-recommends libpq-dev jq \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /usr/share/doc/ \
&& rm -rf /usr/share/man/ \
&& rm -rf /usr/share/locale/ \
&& pip install awscli
# copy hausra binary from base container
COPY --from=base /bin/graphql-engine /bin/graphql-engine
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
CMD ["/entrypoint.sh"]
entrypoint.sh:
#!/bin/bash
set -e
DB_HOST=${DB_HOST:-postgres}
DB_PORT=${DB_PORT:-5432}
AWS_REGION=${AWS_REGION:-us-east-1}
DB_PASSWORD_ENCYPTED=${DB_PASSWORD_ENCYPTED:-false}
if [ -z "${DB_NAME}" ]; then
echo "Must provide DB_NAME environment variable. Exiting...."
exit 1
fi
if [ -z "${DB_USER}" ]; then
echo "Must provide DB_USER environment variable. Exiting...."
exit 1
fi
if [ -z "${DB_PASSWORD}" ]; then
echo "Must provide DB_PASSWORD environment variable. Exiting...."
exit 1
fi
if [ ${DB_PASSWORD_ENCYPTED} == "true" ]
then
echo "loading KMS credentials"
decrypted_value_base64=$( \
aws --region ${AWS_REGION} kms decrypt \
--ciphertext-blob fileb://<(echo "${DB_PASSWORD}" | base64 -d) \
--query Plaintext \
--output text
)
decrypted_value=$(echo $decrypted_value_base64 | base64 -d)
export HASURA_GRAPHQL_DATABASE_URL=postgres://${DB_USER}:${decrypted_value}#${DB_HOST}:${DB_PORT}/${DB_NAME}
else
export HASURA_GRAPHQL_DATABASE_URL=postgres://${DB_USER}:${DB_PASSWORD}#${DB_HOST}:${DB_PORT}/${DB_NAME}
fi
/bin/graphql-engine serve
I tryed to add crontab inside docker image "jenkinsci/blueocean" but after it, jenkins does not start. Where could be the problem?
Many thanks in advance for any help.
<Dockerfile>
FROM jenkinsci/blueocean:1.17.0
USER root
ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.9/supercronic-linux-amd64 \
SUPERCRONIC=supercronic-linux-amd64 \
SUPERCRONIC_SHA1SUM=5ddf8ea26b56d4a7ff6faecdd8966610d5cb9d85
RUN curl -fsSLO "$SUPERCRONIC_URL" \
&& echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \
&& chmod +x "$SUPERCRONIC" \
&& mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
&& ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
ADD crontab /etc/crontab
CMD ["supercronic", "/etc/crontab"]
<crontab>
# Run every minute
*/1 * * * * echo "hello world"
commands:
$docker build -t jenkins_test .
$docker run -it -p 8080:8080 --name=container_jenkins jenkins_test
If use docker inspect jenkinsci/blueocean:1.17.0 you will it's entrypoint is:
"Entrypoint": [
"/sbin/tini",
"--",
"/usr/local/bin/jenkins.sh"
],
So, when start the container it will first execute next script.
/usr/local/bin/jenkins.sh:
#! /bin/bash -e
: "${JENKINS_WAR:="/usr/share/jenkins/jenkins.war"}"
: "${JENKINS_HOME:="/var/jenkins_home"}"
touch "${COPY_REFERENCE_FILE_LOG}" || { echo "Can not write to ${COPY_REFERENCE_FILE_LOG}. Wrong volume permissions?"; exit 1; }
echo "--- Copying files at $(date)" >> "$COPY_REFERENCE_FILE_LOG"
find /usr/share/jenkins/ref/ \( -type f -o -type l \) -exec bash -c '. /usr/local/bin/jenkins-support; for arg; do copy_reference_file "$arg"; done' _ {} +
# if `docker run` first argument start with `--` the user is passing jenkins launcher arguments
if [[ $# -lt 1 ]] || [[ "$1" == "--"* ]]; then
# read JAVA_OPTS and JENKINS_OPTS into arrays to avoid need for eval (and associated vulnerabilities)
java_opts_array=()
while IFS= read -r -d '' item; do
java_opts_array+=( "$item" )
done < <([[ $JAVA_OPTS ]] && xargs printf '%s\0' <<<"$JAVA_OPTS")
readonly agent_port_property='jenkins.model.Jenkins.slaveAgentPort'
if [ -n "${JENKINS_SLAVE_AGENT_PORT:-}" ] && [[ "${JAVA_OPTS:-}" != *"${agent_port_property}"* ]]; then
java_opts_array+=( "-D${agent_port_property}=${JENKINS_SLAVE_AGENT_PORT}" )
fi
if [[ "$DEBUG" ]] ; then
java_opts_array+=( \
'-Xdebug' \
'-Xrunjdwp:server=y,transport=dt_socket,address=5005,suspend=y' \
)
fi
jenkins_opts_array=( )
while IFS= read -r -d '' item; do
jenkins_opts_array+=( "$item" )
done < <([[ $JENKINS_OPTS ]] && xargs printf '%s\0' <<<"$JENKINS_OPTS")
exec java -Duser.home="$JENKINS_HOME" "${java_opts_array[#]}" -jar ${JENKINS_WAR} "${jenkins_opts_array[#]}" "$#"
fi
# As argument is not jenkins, assume user want to run his own process, for example a `bash` shell to explore this image
exec "$#"
From above script, you can see, if you add CMD ["supercronic", "/etc/crontab"] to your own dockerfile, then when your container starts, it equals to execute next:
/usr/local/bin/jenkins.sh "supercronic" "/etc/crontab"
As if [[ $# -lt 1 ]] || [[ "$1" == "--"* ]]; then not match, it will directly execute the exec "$# at the last line, which results in the jenkins start code never execute.
To fix it, you had to use your own docker-entrypoint.sh to override its default entrypoint:
docker-entrypoint.sh:
#!/bin/bash
supercronic /etc/crontab &
/usr/local/bin/jenkins.sh
Dockerfile:
FROM jenkinsci/blueocean:1.17.0
USER root
ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.9/supercronic-linux-amd64 \
SUPERCRONIC=supercronic-linux-amd64 \
SUPERCRONIC_SHA1SUM=5ddf8ea26b56d4a7ff6faecdd8966610d5cb9d85
RUN curl -fsSLO "$SUPERCRONIC_URL" \
&& echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \
&& chmod +x "$SUPERCRONIC" \
&& mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
&& ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
ADD crontab /etc/crontab
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/sbin/tini", "--", "/docker-entrypoint.sh"]
I am running a build in a gradle container with a volume for the cache, but gradle does not make use of the downloaded dependencies in the cache for the subsequent builds.
Here's the dockerfile for the gradle image:
FROM **custom image base**
# Install the Java Development Kit
RUN apk --no-cache add openjdk8=8.131.11-r2
CMD ["gradle"]
ENV GRADLE_HOME /opt/gradle
ENV GRADLE_VERSION 4.6
ARG GRADLE_DOWNLOAD_SHA256=98bd5fd2b30e070517e03c51cbb32beee3e2ee1a84003a5a5d748996d4b1b915
RUN set -o errexit -o nounset \
&& echo "Installing build dependencies" \
&& apk add --no-cache --virtual .build-deps \
ca-certificates \
openssl \
unzip \
\
&& echo "Downloading Gradle" \
&& wget -O gradle.zip "https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-bin.zip" \
\
&& echo "Checking download hash" \
&& echo "${GRADLE_DOWNLOAD_SHA256} *gradle.zip" | sha256sum -c - \
\
&& echo "Installing Gradle" \
&& unzip gradle.zip \
&& rm gradle.zip \
&& mkdir /opt \
&& mv "gradle-${GRADLE_VERSION}" "${GRADLE_HOME}/" \
&& ln -s "${GRADLE_HOME}/bin/gradle" /usr/bin/gradle \
\
&& apk del .build-deps \
\
&& echo "Adding gradle user and group" \
&& addgroup -S -g 1000 gradle \
&& adduser -D -S -G gradle -u 1000 -s /bin/ash gradle \
&& mkdir /home/gradle/.gradle \
&& chown -R gradle:gradle /home/gradle \
\
&& echo "Symlinking root Gradle cache to gradle Gradle cache" \
&& ln -s /home/gradle/.gradle /root/.gradle
# Create Gradle volume
USER gradle
VOLUME "/home/gradle/.gradle"
WORKDIR /home/gradle
RUN set -o errexit -o nounset \
&& echo "Testing Gradle installation" \
&& gradle --version
In the Jenkinsfile I have the build stage declared like this:
stage('Build') {
docker.image('custom-gradle').withRun('-v gradle-cache:/home/gradle/.gradle') { c ->
docker.image('custom-gradle').inside {
sh './scripts/build.py -br ' + branchName
sh 'cp build/libs/JARFILE*.jar build/libs/JARFILE.jar'
}
}
}
The 'gradle-cache' volume is a volume that was created with the docker volume create command.
The python script just runs a gradle command:
gradle clean assemble javaDoc
When I inspect the gradle-cache volume data on the host machine it contains the following files/folders:
4.6 buildOutputCleanup caches daemon native
So the build successfully writes to the cache volume, but appears not to read from it, re-downloading every dependency for every build.
How can I get gradle to use these downloaded dependencies?
UPDATE
stage('Build Bag End') {
docker.image('custom-gradle').inside('-v gradle-cache:/home/gradle/.gradle') {
sh './scripts/build.py -br ' + branchName
sh 'cp build/libs/JARFILE*.jar build/libs/JARFILE.jar'
}
}
So I found the .inside() command also supports parameters but it still won't read from the cache; only write to it.
I believe that Jenkins Docker Plugin always runs containers as the jenkins user. It looks like your gradle image is assuming that it is running as the gradle user. You might try adding the following option:
-e GRADLE_USER_HOME=/home/gradle/.gradle
Though you might run into permission issues with that.