GitHub Self Hosted Runner Can't Find Command - docker

I have the following Dockerfile:
FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -y install curl \
iputils-ping \
apt-transport-https \
tar \
jq \
python && \
curl -sL https://deb.nodesource.com/setup_14.x | bash && \
apt-get install nodejs -yq && \
apt-get clean && apt-get autoremove
RUN npm install -g npm#latest
ARG GH_RUNNER_VERSION="2.283.3"
WORKDIR /actions-runner
RUN curl -o actions.tar.gz --location "https://github.com/actions/runner/releases/download/v${GH_RUNNER_VERSION}/actions-runner-linux-x64-${GH_RUNNER_VERSION}.tar.gz" && \
tar -zxf actions.tar.gz && \
rm -f actions.tar.gz && \
./bin/installdependencies.sh
COPY entrypoint.sh .
RUN chmod +x entrypoint.sh
ENTRYPOINT ["/actions-runner/entrypoint.sh"]
and the following step on the ci:
- name: Create DB
run: npm run dc-up
The output of that step is: npm: command not found.
I added the path using the method the docs suggested, it was done by adding a new step:
- name: add npm to path
run: echo "/usr/bin/npm" >> $GITHUB_PATH
I've checked that node is in the path by printing the path in a separate step inside the CI and the output is:
Run echo "$PATH"
/usr/bin/npm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
I know 100% that NPM is installed into the docker image because when I run it local and only try to interact withit without the ENTRYPOINT then I'm able to print the NPM version and I checked that is it indeed in /usr/bin/npm, but still, inside the steps of the CI it can't find npm for some reason.
And its not only for npm, but for every single installation that I tried to do, I just picked npm for showcase.
Anyone has any idea what can be done?

Related

Errors Installing singularity inside dockerfile

I am trying to run a nextflow pipeline which uses an older version of nextflow (21.04.3) and java version 8. Since I have to use this pipeline on a remote server, therefore I can only use singularity.
As this nextflow pipeline also uses singularity pull calls therefore I need the singularity installed inside the docker image as well. Then, I can convert this image docker image to a singularity image and then I can move it to the remote server.
I am trying to install singularity inside dockerfile but I am getting errors,
This is the dockerfile that I am using,
FROM python:3.8.9-slim
LABEL authors="phil.ewels#scilifelab.se,erik.danielsson#scilifelab.se" \
description="Docker image containing requirements for the nfcore tools"
# Do not pick up python packages from $HOME
ENV PYTHONNUSERSITE=1
# Update pip to latest version
RUN python -m pip install --upgrade pip
# Install dependencies
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
# Install Nextflow dependencies
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y git \
&& apt-get install -y wget
# Create man dir required for Java installation
# and install Java
RUN mkdir -p /usr/share/man/man1 \
&& apt-get install -y openjdk-11-jre \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
# Install Singularity
RUN wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee /etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update
RUN apt-get install -y singularity-container
# Setup ARG for NXF_VER ENV
ARG NXF_VER=""
ENV NXF_VER ${NXF_VER}
# Install Nextflow
RUN wget https://github.com/nextflow- io/nextflow/releases/download/v21.04.3/nextflow | bash \
&& mv nextflow /usr/local/bin \
&& chmod a+rx /usr/local/bin/nextflow
# Add the nf-core source files to the image
COPY . /usr/src/nf_core
WORKDIR /usr/src/nf_core
# Install nf-core
RUN python -m pip install .
# Set up entrypoint and cmd for easy docker usage
CMD [ "." ]
These are the errors I am getting
Step 9/17 : RUN wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee
/etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --
keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update
---> Running in afc3dcbbd1ee
--2022-03-17 17:40:19-- http://neuro.debian.net/lists/xenial.us-ca.full
Resolving neuro.debian.net (neuro.debian.net)... 129.170.233.11
Connecting to neuro.debian.net (neuro.debian.net)|129.170.233.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 262
Saving to: ‘STDOUT’
0K 100% 18.4M=0s
deb http://neurodeb.pirsquared.org data main contrib non-free
#deb-src http://neurodeb.pirsquared.org data main contrib non-free
deb http://neurodeb.pirsquared.org xenial main contrib non-free
#deb-src http://neurodeb.pirsquared.org xenial main contrib non-free
2022-03-17 17:40:19 (18.4 MB/s) - written to stdout [262/262]
/bin/sh: 1: apt-key: not found
The command '/bin/sh -c wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee /etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update'
returned a non-zero code: 127
I there a way to install singularity using a dockerfile ?
Thanks
I made some changes in the dockerfile based on the method to install singularity in linux given here.
The complete dockerfile with which I was able to run successfully nextflow, java and singularity within singularity is given below,
FROM python:3.8.9-slim
LABEL
authors="phil.ewels#scilifelab.se,erik.danielsson#scilifelab.se" \
description="Docker image containing requirements for the nfcore tools"
# Do not pick up python packages from $HOME
ENV PYTHONNUSERSITE=1
# Update pip to latest version
RUN python -m pip install --upgrade pip
# Install dependencies
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
# Install Nextflow dependencies
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y git \
&& apt-get install -y wget
# Create man dir required for Java installation
# and install Java
RUN mkdir -p /usr/share/man/man1 \
&& apt-get install -y openjdk-11-jre \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
# Install Singularity
RUN apt-get update && apt-get install -y \
build-essential \
libssl-dev \
uuid-dev \
libgpgme11-dev \
squashfs-tools \
libseccomp-dev \
wget \
pkg-config \
procps
# Download Go source version 1.16.3, install them and modify the PATH
ENV VERSION=1.16.3
ENV OS=linux
ENV ARCH=amd64
RUN wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
rm go$VERSION.$OS-$ARCH.tar.gz && \
echo 'export PATH=$PATH:/usr/local/go/bin' | tee -a /etc/profile
# Download Singularity from version 3.7.3 (security version)
ENV VERSION=3.7.3
RUN wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
tar -xzf singularity-${VERSION}.tar.gz
# Compile Singularity sources and install it
RUN export PATH=$PATH:/usr/local/go/bin && \
cd singularity && \
./mconfig --without-suid && \
make -C ./builddir && \
make -C ./builddir install
# Setup ARG for NXF_VER ENV
ARG NXF_VER=""
ENV NXF_VER ${NXF_VER}
# Install Nextflow
RUN wget https://github.com/nextflow-io/nextflow/releases/download/v21.04.3/nextflow | bash \
&& mv nextflow /usr/local/bin \
&& chmod a+rx /usr/local/bin/nextflow
# Add the nf-core source files to the image
COPY . /usr/src/nf_core
WORKDIR /usr/src/nf_core
# Install nf-core
RUN python -m pip install .
# Set up entrypoint and cmd for easy docker usage
CMD [ "." ]
The file named requirements.txt used in the above dockerfile is given below,
click
GitPython
jinja2
jsonschema
packaging
prompt_toolkit>=3.0.3
pyyaml
pytest-workflow
questionary>=1.8.0
requests_cache
requests
rich>=10.0.0
tabulate

how to start docker container from gradle?

I am trying to switch from Cmake to gradle. I want to configure gradle to work as follow
$ cd myapp && ls myapp
$ Dockerfile build.gradle src
$ gradle build
Build the docker image from docker file
start container
build the application
The docker image contains complete environment for my app.
FROM debian:stretch
RUN apt-get update -y && apt install -y git \
python3-dev libncurses5-dev libxml2-dev \
libedit-dev swig doxygen graphviz xz-utils ninja-build
RUN echo "deb http://ftp.de.debian.org/debian stretch main" >> /etc/apt/source.list
RUN apt-get update && apt-get install -y openjdk-8-jre openjdk-8-jdk
# Clang 8 as a compiler
RUN apt-get update && apt-get install -y \
xz-utils \
build-essential \
curl \
&& rm -rf /var/lib/apt/lists/* \
&& curl -SL http://releases.llvm.org/8.0.0/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz \
| tar -xJC . && \
mv clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 clang_8.0.0 && \
echo 'export PATH=/clang_8.0.0/bin:$PATH' >> ~/.bashrc && \
echo 'export LD_LIBRARY_PATH=/clang_8.0.0/lib:LD_LIBRARY_PATH' >> ~/.bashrc
#
RUN apt-get update
#install sdkman
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
RUN apt-get -qq -y install curl wget unzip zip
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh"
#install gradle
RUN yes | /bin/bash -l -c 'sdk install gradle 6.1'
PS: This is cpp project
You can build a docker image from Gradle tasks by using **com.bmuschko:gradle-docker-plugin:3.1.0 plugin**
buildscript {
repositories {
jcenter()
mavenCentral()
}
dependencies {
classpath 'com.bmuschko:gradle-docker-plugin:3.1.0'
}
}
apply plugin: 'com.bmuschko.docker-remote-api'
import com.bmuschko.gradle.docker.tasks.image.Dockerfile
import com.bmuschko.gradle.docker.tasks.image.DockerBuildImage
import com.bmuschko.gradle.docker.tasks.image.*
task buildImage(type: DockerBuildImage) {
group = ''
inputDir = file('.')
tag = 'image name:'+tag
}
read the documentation for more details https://bmuschko.github.io/gradle-docker-plugin/
Build an image from Gradle task - ./gradlew taskname
To start container and run the command inside it you can use CMD or ENTRYPOINT and specify the command in Dockerfile
CMD ["start.sh"]
in start.sh you can specify your command to be executed after running the container
Let me slightly clean up that Dockerfile first:
FROM debian:stretch
RUN echo "deb http://ftp.de.debian.org/debian stretch main" >> /etc/apt/source.list
RUN apt-get update -y && apt install -qq -y \
python3-dev libncurses5-dev libxml2-dev \
libedit-dev swig doxygen graphviz xz-utils ninja-build \
openjdk-8-jre openjdk-8-jdk \
xz-utils curl git build-essential wget unzip zip
# Clang 8 as a compiler
RUN curl -SL http://releases.llvm.org/8.0.0/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz \
| tar -xJC . && \
mv clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 clang_8.0.0 && \
echo 'export PATH=/clang_8.0.0/bin:$PATH' >> ~/.bashrc && \
echo 'export LD_LIBRARY_PATH=/clang_8.0.0/lib:LD_LIBRARY_PATH' >> ~/.bashrc
#install sdkman
RUN ln -fs /bin/bash /bin/sh
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh"
RUN yes | /bin/bash -l -c 'sdk install gradle 6.1'
RUN mkdir /src /work
WORKDIR /src
ENTRYPOINT gradle build -p /src
The important bits are at the bottom: it creates a /src directory and executes gradle build there. All that remains for you is to make that directory available when you build.
Assuming you built the container once with docker build -t my-build-container ., you can run it as follows:
docker run -v $(pwd):/src my-build-container
Depending on your build system, this might pollute your source tree with various build artifacts owned by root. If so, consider switching to out-of-tree builds by changing the default working directory to /work instead. All build results will go to /work, and you can extract them from the container afterwards.
Add docker plugin first
buildscript {
dependencies {
classpath("se.transmode.gradle:gradle-docker:1.2")
}
}
create simple task like this in build.gradle file
task buildDocker(type: Docker, dependsOn: build) {
push = false
project.group = 'testProject'
project.archivesBaseName = jar.baseName
applicationName = jar.baseName
dockerfile = file('src/main/docker/Dockerfile')
doFirst {
copy {
from jar
into stageDir
}
}
}

docker run error: Unable to access jarfile

Docker image is built but when I want to run it, it shows this error:
Error: Unable to access jarfile rest-service-1.0.jar
My OS is Ubuntu 18.04.1 LTS and I use docker build -t doc-service & docker run doc-service.
This is my Dockerfile:
FROM ubuntu:16.04
MAINTAINER Frederico Apostolo <frederico.apostolo#blockfactory.com> (#fapostolo)
RUN apt-get update && apt-get -y upgrade
RUN apt-get install -y software-properties-common python-software-properties language-pack-en-base
RUN add-apt-repository ppa:webupd8team/java
RUN apt-get update && apt-get update --fix-missing && apt-get -y --allow-downgrades --allow-remove-essential --allow-change-held-packages upgrade \
&& echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections \
&& apt-get install -y --allow-downgrades --allow-remove-essential --allow-change-held-packages curl vim unzip wget oracle-java8-installer \
&& apt-get clean && rm -rf /var/cache/* /var/lib/apt/lists/*
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle/
run java -version
run echo $JAVA_HOME
#use locate for debug
RUN apt-get update && apt-get install -y locate mlocate && updatedb
#LIBREOFFICE START
RUN apt-get update && apt-get update --fix-missing && apt-get install -y -q libreoffice \
libreoffice-writer ure libreoffice-java-common libreoffice-core libreoffice-common \
fonts-opensymbol hyphen-fr hyphen-de hyphen-en-us hyphen-it hyphen-ru fonts-dejavu \
fonts-dejavu-core fonts-dejavu-extra fonts-noto fonts-dustin fonts-f500 fonts-fanwood \
fonts-freefont-ttf fonts-liberation fonts-lmodern fonts-lyx fonts-sil-gentium \
fonts-texgyre fonts-tlwg-purisa
#LIBREOFFICE END
#font configuration
COPY 00-odt-template-renderer-fontconfig.conf /etc/fonts/conf.d
RUN mkdir /document-service /document-service/fonts /document-service/module /document-service/logs
# local settings
RUN echo "127.0.0.1 http://www.arbs.local http://arbs.local www.arbs.local arbs.local" >> /etc/hosts
# && mkdir /logs/ && echo "dummy" >> /logs/errors.log
#EXPOSE 2115
COPY document-service-java_with_user_arg.sh /
RUN chmod +x /document-service-java_with_user_arg.sh
RUN apt-get update && apt-get -y --no-install-recommends install \
ca-certificates \
curl
RUN gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
RUN curl -o /usr/local/bin/gosu -SL "https://github.com/tianon/gosu/releases/download/1.4/gosu-$(dpkg --print-architecture)" \
&& curl -o /usr/local/bin/gosu.asc -SL "https://github.com/tianon/gosu/releases/download/1.4/gosu-$(dpkg --print-architecture).asc" \
&& gpg --verify /usr/local/bin/gosu.asc \
&& rm /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu
ENV LANG="en_US.UTF-8"
# In case someone loses the Dockerfile
# Needs to be in the end so it doesn't invalidate unaltered cache whenever the file is updated.
RUN rm -rf /etc/Dockerfile
ADD Dockerfile /etc/Dockerfile
ENTRYPOINT ["/document-service-java_with_user_arg.sh"]
this is document-service-java_with_user_arg.sh:
#!/bin/bash
USER_ID=${LOCAL_USER_ID:-9001}
USER_NAME=${LOCAL_USER_NAME:-jetty}
echo "Starting user: $USER_NAME with UID : $USER_ID"
useradd --shell /bin/bash --home-dir /document-service/dockerhome --non-unique --uid $USER_ID $USER_NAME
cd /document-service
/usr/local/bin/gosu $USER_NAME "$#" java -jar rest-service-1.0.jar
Can anyone help me on this?
Based on the comments, you must add the JAR when building the image by defining in your Dockerfile :
COPY rest-service-1.0.jar /document-service/rest-service-1.0.jar
You could also just use :
COPY rest-service-1.0.jar /rest-service-1.0.jar
, and remove cd /document-service in your entrypoint script, as on ubuntu:16.04 images, default working directory is /. My opinion is that setting the working directory in the script is safer, so you should just go for the first solution.
Note that you could also use ADD instead of COPY (as you already did in your Dockerfile), but here only COPY is necessary (read this post if you want more info : What is the difference between the `COPY` and `ADD` commands in a Dockerfile?).
Finally, I suggest you to add the COPY line at the end of your Dockerfile, so that if a new JAR is built, image won't be rebuilt from scratch but from an existing layer, speeding up build time.
it looking error about workdir
you must select workdir for this copy format
try WORKDIR /yourpath/

Jenkins-cli.jar is corrupted when I run docker file in bash

I need to build the dockerfile that downloads jenkins.war and through it jenkins-cli.jar need to be downloaded.
I have conf.xml also to configure it.**
Then I need that image to run in the bash, which needs to run that jar file commands.
Here is the code:
FROM ubuntu:14.04
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y software-properties-common && \
add-apt-repository ppa:webupd8team/java -y && \
apt-get update && \
echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \
apt-get install -f -y oracle-java8-installer && \
apt install -y default-jre curl wget git nano; \
apt-get clean
# Install dependencies
RUN apt-get -y update && \
apt-get -yqq --no-install-recommends install git bzip2 curl unzip && \
apt-get update
ENV JAVA_HOME /usr
ENV PATH $JAVA_HOME/bin:$PATH
# copy jenkins war file to the container
ADD http://mirrors.jenkins.io/war-stable/2.107.1/jenkins.war /opt/jenkins.war
RUN chmod 644 /opt/jenkins.war
ENV JENKINS_HOME /jenkins
# configure the container to run jenkins, mapping container port 8080 to that host port
RUN mkdir /jenkins/
RUN echo 2.107.1 > /jenkins/jenkins.install.UpgradeWizard.state
RUN echo 2.107.1 > /jenkins/jenkins.install.InstallUtil.lastExecVersion
CMD ["nohup","java", "-jar", "/opt/jenkins.war"]
EXPOSE 8080
VOLUME /jenkins
#COPY jenkins-cli.jar /jenkins/jenkins-cli.jar
#jenkins-cli installation
ENV JENKINS_URL "http://192.168.99.100:8080"
RUN curl --insecure http://192.168.99.100:8080/jnlpJars/jenkins-cli.jar \
--output /jenkins/jenkins-cli.jar
CMD ["java","-jar","/jenkins/jenkins-cli.jar","-noCertificateCheck","-noKeyAuth"]
Here is what im getting.
MY ASSUMPTION
Do I need to run along congf.xml?If yes , then HOW?
Do I need to be running jenkins.war instance in background??? HOW?
Thank you in advance
If you see the reference , I can find those comments.
There can only be one CMD instruction in a Dockerfile. If you list more than one CMD then only the last CMD will take effect.
in your dockerfile, there are multiple CMD commands. only the last one will be executed.
If you want to run multiple commands at once. try bash scripts. here is the example
#!/bin/bash
echo "Starting sshd"
exec /usr/sbin/sshd -D &
if [ -z "$1" ];
then
tail -f $HADOOP_INSTALL/logs/*
fi

Running jenkins-cli in bash

HI i need too wright a docker script which downloads jenkins and jenkin-cli.jar from it. but im unable to run jar file from bash. MAy i know why? and solution for it?
Here is my script
FROM ubuntu:14.04
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y software-properties-common && \
add-apt-repository ppa:webupd8team/java -y && \
apt-get update && \
echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \
apt-get install -f -y oracle-java8-installer && \
apt install -y default-jre curl wget git nano; \
apt-get clean
# Install dependencies
RUN apt-get -y update && \
apt-get -yqq --no-install-recommends install git bzip2 curl unzip && \
apt-get update
ENV JAVA_HOME /usr
ENV PATH $JAVA_HOME/bin:$PATH
# copy jenkins war file to the container
ADD http://mirrors.jenkins.io/war-stable/2.107.1/jenkins.war /opt/jenkins.war
RUN chmod 644 /opt/jenkins.war
ENV JENKINS_HOME /jenkins
# configure the container to run jenkins, mapping container port 8080 to that host port
RUN mkdir /jenkins/
RUN echo 2.107.1 > /jenkins/jenkins.install.UpgradeWizard.state
RUN echo 2.107.1 > /jenkins/jenkins.install.InstallUtil.lastExecVersion
ENTRYPOINT ["nohup","java", "-jar", "/opt/jenkins.war"]
EXPOSE 8080
VOLUME /jenkins
#COPY jenkins-cli.jar /jenkins/jenkins-cli.jar
#jenkins-cli installation
ENV JENKINS_URL "http://192.168.99.100:8080/cli"
RUN curl --insecure -OL http://192.168.99.100:8080/jnlpJars/jenkins-cli.jar \
--output jenkins/jenkins-cli.jar
ENTRYPOINT ["java","-jar","/jenkins/jenkins-cli.jar","-noCertificateCheck","-noKeyAuth"]
Error:
MY ASSUMPTION
Enrytpoint is conflicting
If any other problem let me know that and solution for that.

Resources