dockerfile: run tool on several repositories - docker

I have a cli tool that I'm creating an E2E test suite using docker. The basic idea is in the docker container I'll build the local code and then run the tool over several public repositories. The main goal of this is to make it easier to see any missing functionality that my CLI tool may need to add.
I did manage to get it working but it's a bit of a pain to manage due to all the chaining and folder management.
## Build a node application
from node:8.11.1
WORKDIR /app
## Copy all the files
COPY . ./sortier
## Build run and test
RUN cd ./sortier \
&& npm install --unsafe-perm \
&& npm run test \
&& cd .. \
## Run react-redux-typescript-guide/playground test
&& pwd \
&& git clone https://github.com/piotrwitek/react-redux-typescript-guide \
&& cd react-redux-typescript-guide/playground \
&& npm install --unsafe-perm \
&& echo "{ isHelpMode: true }" > .sortierrc \
&& cd ../../sortier \
&& npm run start -- "../react-redux-typescript-guide/playground/src/**/*.ts" \
&& npm run start -- "../react-redux-typescript-guide/playground/src/**/*.tsx" \
&& cd ../react-redux-typescript-guide/playground \
&& npm run build \
&& cd ../.. \
## Run prettier test
&& pwd \
&& git clone https://github.com/prettier/prettier \
&& cd prettier \
&& npm install --unsafe-perm \
&& echo "{ isHelpMode: true }" > .sortierrc \
&& cd .. \
&& npm run start -- "prettier/src/**/*.js" \
&& cd prettier \
&& npm run build \
&& npm run test \
&& cd ..
I was trying to figure out how to use WORKDIR instead to change directories which would clean it up a lot but being able to reference work directories from one another didn't seem to work.
Any advice on how I can clean up this dockerfile?

And of course after I post the question I figure out the answer (do'h)
## Build a node application
from node:8.11.1
## Sortier creation, build and test
WORKDIR /sortier
COPY . .
RUN npm install --unsafe-perm
RUN npm run test
## react-redux-typescript-guide/playground
WORKDIR /react-redux-typescript-guide
RUN git clone https://github.com/piotrwitek/react-redux-typescript-guide .
WORKDIR /react-redux-typescript-guide/playground
RUN npm install --unsafe-perm
RUN echo "{ isHelpMode: true }" > .sortierrc
WORKDIR /sortier
RUN npm run start -- "/react-redux-typescript-guide/playground/src/**/*.ts"
RUN npm run start -- "/react-redux-typescript-guide/playground/src/**/*.tsx"
WORKDIR /react-redux-typescript-guide/playground
RUN npm run build
RUN set CI=true&&npm run test
## prettier
WORKDIR /prettier
RUN git clone https://github.com/prettier/prettier
WORKDIR /prettier
RUN npm install --unsafe-perm
RUN echo "{ isHelpMode: true }" > .sortierrc
WORKDIR /sortier
RUN npm run start -- "/prettier/src/**/*.js"
WORKDIR /prettier
RUN npm run build
RUN npm run test

Related

GitHub Self Hosted Runner Can't Find Command

I have the following Dockerfile:
FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -y install curl \
iputils-ping \
apt-transport-https \
tar \
jq \
python && \
curl -sL https://deb.nodesource.com/setup_14.x | bash && \
apt-get install nodejs -yq && \
apt-get clean && apt-get autoremove
RUN npm install -g npm#latest
ARG GH_RUNNER_VERSION="2.283.3"
WORKDIR /actions-runner
RUN curl -o actions.tar.gz --location "https://github.com/actions/runner/releases/download/v${GH_RUNNER_VERSION}/actions-runner-linux-x64-${GH_RUNNER_VERSION}.tar.gz" && \
tar -zxf actions.tar.gz && \
rm -f actions.tar.gz && \
./bin/installdependencies.sh
COPY entrypoint.sh .
RUN chmod +x entrypoint.sh
ENTRYPOINT ["/actions-runner/entrypoint.sh"]
and the following step on the ci:
- name: Create DB
run: npm run dc-up
The output of that step is: npm: command not found.
I added the path using the method the docs suggested, it was done by adding a new step:
- name: add npm to path
run: echo "/usr/bin/npm" >> $GITHUB_PATH
I've checked that node is in the path by printing the path in a separate step inside the CI and the output is:
Run echo "$PATH"
/usr/bin/npm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
I know 100% that NPM is installed into the docker image because when I run it local and only try to interact withit without the ENTRYPOINT then I'm able to print the NPM version and I checked that is it indeed in /usr/bin/npm, but still, inside the steps of the CI it can't find npm for some reason.
And its not only for npm, but for every single installation that I tried to do, I just picked npm for showcase.
Anyone has any idea what can be done?

Can't clone github public repository into Dockerfile's Run order

Premise · What I want to realize
I'm trying to clone a git public Repository into Dockerfiles Run order, but I'm not going well...
testing environment
MacOS Mojave
10.14.6
Docker
19.03.8
python
3.6.10
bash
3.2.57
What I did
**1. make a Dockerfile **
FROM python:3.6
LABEL maintainer="aaa"
SHELL ["/bin/bash", "-c"]
WORKDIR /usr/local/src/
RUN git clone https://path/to/target_repository.git \
&& chmod -R 755 ./target_repository \
&& cd ./target_repository \
&& pip install -r requirements.txt \
&& mkdir -p ./data/hojin/zip \
&& mv ../13_tokyo_all_20200529.zip ./data/hojin/zip/ \
&& sh scripts/download.sh \
&& pip install IPython seqeval \
&& sh scripts/generate_alias.sh \
&& python tools/dataset_converter.py \
&& python tools/dataset_preprocess.py
EXPOSE 80
CMD ["/sbin/init"]
Problems occurring · Error messages
...
Cloning into 'target-repository'...
chmod: cannot access './target-repository': No such file or directory
...
that's all
I got the errors. What shoud I do?
Could you lend me a hand?
I changed a bit your Dockerfile to test with my repo and it works well.
FROM python:3.6
LABEL maintainer="aaa"
SHELL ["/bin/bash", "-c"]
WORKDIR /usr/local/src/
RUN git clone https://path/to/my_repository.git
RUN chmod -R 755 ./my_repository
RUN cd ./my_repository
You can use RUN commands like me to be more clear and make sure that you type exactly the name of folder.

Conditional npm install ARG in Dockerfile

I am trying to install node module packages from GitHub on the basis of the argument. I have tried but however I do not understand this solution completely. There is no error in the execution but I am sure it does not install the package. I run the container locally before deploy and getting the module not found an error
Dokerfile is like that.Can it be possible to keep each command in a new line?
FROM node:10
ENV HOME /usr/src/reach-live-api
ARG siteval
RUN mkdir -p $HOME
RUN mkdir -p /root/.ssh
WORKDIR $HOME
COPY ./keys/reachlive_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
RUN echo "Host bitbucket.org\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config
COPY . $HOME
RUN npm install --silent --production
RUN if [ "$siteval" = "prod" ]; then \
RUN npm install "reach-live-elasticsearch#git+https://github.com/lalitmohan001/reach-live-elasticsearch-prod.git" \
RUN npm install "reach-live-firebase#git+https://github.com/lalitmohan001/reach-live-firebase-prod.git" \
RUN npm install "reach-live-paypal#git+https://github.com/lalitmohan001/reach-live-paypal-prod.git" \
else \
RUN npm install "reach-live-elasticsearch#git+https://github.com/lalitmohan001/reach-live-elasticsearch.git" \
RUN npm install "reach-live-firebase#git+https://github.com/lalitmohan001/reach-live-firebase.git" \
RUN npm install "reach-live-paypal#git+https://github.com/lalitmohan001/reach-live-paypal.git"; \
fi
RUN npm prune --production
RUN rm -fr .npm
RUN rm -fr /root/.ssh
RUN rm -fr keys
CMD ["npm", "start"]
I use the following command to build
docker build -t gcr.io/reachlive-123/api:25Apr2020 . --build-arg siteval=dev
Help would be much appreciated
turns out the problem was in the
RUN if [ "$siteval" = "prod" ]; then \
RUN npm install "reach-live-elasticsearch#git+https://github.com/lalitmohan001/reach-live-elasticsearch-prod.git" \
RUN npm install "reach-live-firebase#git+https://github.com/lalitmohan001/reach-live-firebase-prod.git" \
RUN npm install "reach-live-paypal#git+https://github.com/lalitmohan001/reach-live-paypal-prod.git" \
else \
RUN npm install "reach-live-elasticsearch#git+https://github.com/lalitmohan001/reach-live-elasticsearch.git" \
RUN npm install "reach-live-firebase#git+https://github.com/lalitmohan001/reach-live-firebase.git" \
RUN npm install "reach-live-paypal#git+https://github.com/lalitmohan001/reach-live-paypal.git"; \
fi
I have changed it to and it works
RUN if [ "$arg" = "prod" ]; then \
npm install reach-live-elasticsearch#git+https://github.com/lalitmohan001/reach-live-elasticsearch-prod.git \
reach-live-firebase#git+https://github.com/lalitmohan001/reach-live-firebase-prod.git \
reach-live-paypal#git+https://github.com/lalitmohan001/reach-live-paypal-prod.git ; \
else \
npm install reach-live-elasticsearch#git+https://github.com/lalitmohan001/reach-live-elasticsearch.git \
reach-live-firebase#git+https://github.com/lalitmohan001/reach-live-firebase.git \
reach-live-paypal#git+https://github.com/lalitmohan001/reach-live-paypal.git; \
fi
Thanks to Pavittar Singh for helping us figure this out!

How to decrease the size of docker image with gRPC in golang supported?

I have some server and clients who are communicating using gRPC/golang. Now I want to containerize my application but the size of the docker image containing goland execution and grpc support is larger(more than 1GB). I would like to decrease the size of the docker image.
The required golang version is 1.9 and higher. Here is the Dockerfile script is given. If there is other way please suggest it.
FROM golang:1.11
RUN apt-get update && \
apt-get -y install git unzip build-essential autoconf libtool
RUN git clone https://github.com/google/protobuf.git && \
cd protobuf && \
./autogen.sh && \
./configure && \
make && \
make install && \
ldconfig && \
make clean && \
cd .. && \
rm -r protobuf
RUN go get google.golang.org/grpc
RUN go get github.com/golang/protobuf/protoc-gen-go
RUN ls -la
WORKDIR /helloworld
COPY . /helloworld
RUN protoc -I helloworld/ helloworld/helloworld.proto --go_out=plugins=grpc:helloworld
CMD ["go", "run", "helloworld/greeter_server/main.go"]
try to make a multistage docker image like this
# Compile stage
FROM golang:1.11 as build-env
RUN apt-get update && \
apt-get -y install git unzip build-essential autoconf libtool
RUN git clone https://github.com/google/protobuf.git && \
cd protobuf && \
./autogen.sh && \
./configure && \
make && \
make install && \
ldconfig && \
make clean && \
cd .. && \
rm -r protobuf
RUN go get google.golang.org/grpc
RUN go get github.com/golang/protobuf/protoc-gen-go
RUN ls -la
WORKDIR /helloworld
COPY . /helloworld
RUN protoc -I helloworld/ helloworld/helloworld.proto --go_out=plugins=grpc:helloworld
RUN go build -o server helloworld/greeter_server/main.go
# Making image
FROM alpine:3.8 AS host
RUN apk add --no-cache \
ca-certificates
COPY --from=build-env /helloworld/server /
# copy any other files you need
WORKDIR /
EXPOSE 8000
CMD ["server"]
You can try to use distroless base images and multi-stage builds. That might help you.

Dockerfile issues

Hi I got a project with a dockerfile, and I am trying to build the dockerfile to run the project in the environment it was created but I seem to get an error at step 5 of the build and when I look at the dockerfile I find the code a bit strange/I dont understand it at that point.
This is the dockerfile:
FROM node:8.10-alpine
ENV NODE_ENV development
# Create app directory
WORKDIR /var/app
# Install Node packages
COPY package.json package.json
RUN apk install git \
&& npm i \
&& apk del .gyp\
&& mv /var/app/node_modules /node_modules \
&& rm -rf /var/cache/apk/* \
&& apk del git
# Bundle app source
COPY . .
#COPY entrypoint.sh entrypoint.sh
# Expose port
EXPOSE 88
#ENTRYPOINT ["./entrypoint.sh"]
CMD ["npm", "run", "dev"]
This is the error i am getting:
Step 5/8 : RUN apk install git && npm i && apk del .gyp && mv /var/app/node_modules /node_modules && rm -rf /var/cache/apk/* && apk del git
---> Running in 251259cdb8a2
apk-tools 2.7.5, compiled for x86_64.
Then I get a bunch of text which resembles what you get if you type -help on something and then at the end i get:
This apk has coffee making abilities.
The command '/bin/sh -c apk install git && npm i && apk del .gyp && mv /var/app/node_modules /node_modules && rm -rf /var/cache/apk/* && apk del git' returned a non-zero code: 1
This seems to be the problematic part:
RUN apk install git \
&& npm i \
&& apk del .gyp\
&& mv /var/app/node_modules /node_modules \
&& rm -rf /var/cache/apk/* \
&& apk del git
Just try adding an additional line before using apk and see if it fixes
RUN echo "ipv6" >> /etc/modules
RUN apk install git \
Reference: link
Note: Breaking the problematic step into multiple steps like
RUN apk install git
RUN npm i
RUN apk del .gyp
RUN mv /var/app/node_modules /node_modules
RUN rm -rf /var/cache/apk/*
RUN apk del git
will help to locate the point of problem more accurately.
You have 2 issues. One is with the command apk del .gyp (which return code is different from 0) , and the other is related to the fact that you do not mount correctly your folder.
# apk del .gyp
# echo $?
1
Besides, there is no such thing as /var/app/node_modules mounted in the container:
# ls /var/app/node_modules
# ls: /var/app/node_modules: No such file or directory
What you would do, is
Make sure you mount correctly /var/app/node_modules in the container
I am not sure what the command apk del .gyp is doing, but you may need to investigate it. It does not seems to work properly.

Resources