I'm trying to build a docker image of a Rocket application. I've followed the rust docker documentation to make an image with a builder, but the resulting container wont launch the application.
Dockerfile
FROM rust:latest as builder
COPY ./id_rsa* /root/.ssh/
RUN chmod 600 /root/.ssh/id_rsa
RUN git config --global user.name "Runner"
RUN git config --global user.email "some#email.com"
RUN ssh-keyscan -t rsa gitlab.com >> ~/.ssh/known_hosts
RUN rustup default nightly
WORKDIR /usr/src/api
COPY ./api .
VOLUME /redis
RUN cargo install --path .
# ---
FROM debian:buster
ENV ROCKET_DATABASES={redis={url="redis://redis:6379/"}}
COPY --from=builder /usr/local/cargo/bin/api /usr/local/bin/api
EXPOSE 8000
CMD api
This is the dockerfile I expect to work, but instead results in a 'hanging' container. To be more percise the container runs with no console output.
I have connected to the resulting container and verified that the executable is being copied over properly and that it runs as expected in the container's shell. I can even get it to launch properly by removing the second stage, but this isn't ideal as it leaves SSH keys in the container.
Working Dockerfile
FROM rust:latest as builder
COPY ./id_rsa* /root/.ssh/
RUN chmod 600 /root/.ssh/id_rsa
RUN git config --global user.name "Runner"
RUN git config --global user.email "some#email.com"
RUN ssh-keyscan -t rsa gitlab.com >> ~/.ssh/known_hosts
RUN rustup default nightly
WORKDIR /usr/src/api
COPY ./api .
VOLUME /redis
RUN cargo install --path .
CMD /usr/local/cargo/bin/api
Related
I just started learning docker. To teach myself, I managed to containerize bandit (a python code scanner) but I'm not able to see the output of the scan before the container destroys itself. How can I copy the output file from inside the container to the host, or otherwise save it?
Right now i'm just using bandit to scan itself basically :)
Dockerfile
FROM python:3-alpine
WORKDIR /
RUN pip install bandit
RUN apk update && apk upgrade
RUN apk add git
RUN git clone https://github.com/PyCQA/bandit.git ./code-to-scan
CMD [ "python -m bandit -r ./code-to-scan -o bandit.txt" ]
You can mount a volume on you host where you can share the output of bandit.
For example, you can run your container with:
docker run -v $(pwd)/output:/tmp/output -t your_awesome_container:latest
And you in your dockerfile:
...
CMD [ "python -m bandit -r ./code-to-scan -o /tmp/bandit.txt" ]
This way the bandit.txt file will be found in the output folder.
Better place the code in your image not in the root directory.
I did some adjustments to your Dockerfile.
FROM python:3-alpine
WORKDIR /usr/myapp
RUN pip install bandit
RUN apk update && apk upgrade
RUN apk add git
RUN git clone https://github.com/PyCQA/bandit.git .
CMD [ "bandit","-r",".","-o","bandit.txt" ]`
This clones git in your WORKDIR.
Note the CMD, it is an array, so just devide all commands and args as in the Dockerfile about.
I put the the Dockerfile in my D:\test directory (Windows).
docker build -t test .
docker run -v D:/test/:/usr/myapp test
It will generate you bandit.txt in the test folder.
After the code is execute the container exits, as there are nothing else to do.
you can also put --rm to remove the container once it finishs.
docker run --rm -v D:/test/:/usr/myapp test
I have a Dockerfile like below:
# syntax=docker/dockerfile:1
FROM continuumio/miniconda3
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
# Create directory to store our application
WORKDIR /app
## The following three commands adapted from Dockerfile snippet at
## https://docs.docker.com/develop/develop-images/build_enhancements/#using-ssh-to-access-private-data-in-builds
# Install ssh client and git
RUN apt-get upgrade && apt-get update && apt-get install openssh-client git -y
# Download public key for gitlab.com
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
# clone my-repo, authenticating using client's ssh-agent.
RUN --mount=type=ssh git clone git#gitlab.com:mycompany/data-products/my-repo.git /app/
# set up python (conda) environment to run application
RUN conda create --name recenv --file conda-linux-64.lock
# run my-package with the conda environment we just created.
CMD ["conda", "run", "-n", "recenv", "python", "-m", "my_package.train" "path/to/gcp/service/account.json"]
This dockerfile builds successfully with docker build . --no-cache --tag my-package --ssh default but fails (as expected) on docker run my-package:latest with:
FileNotFoundError: [Errno 2] No such file or directory: path/to/gcp/service/account.json
So I've gotten the ssh secrets management working so the RUN ...git clone step uses my ssh/rsa creds successfully. But I'm having trouble using my other secret - my gcp service account json file. The difference is I only need the ssh secret in a RUN step but I need my gcp service account secret in my CMD step.
While everything I've read, such as docker docs page on using the --secret flag, tutorials and SO answers I've found, all reference how to pass in a secret to be used in a RUN step but not the CMD step. But I need to pass my GCP service account json file to my CMD step.
I could just COPY the file into my container, but from my reading that's supposedly not a great solution from a security standpoint.
What is the recommended, secure way of passing a secret json file to the CMD step of a docker container?
I'm trying to use a multistep Dockerfile that uses FROM AS, but when I run the Dockerfile in a Jenkins job I get an error
FROM node:8.12.0-alpine AS firstStep
Error parsing reference: "node:8.12.0-alpine AS firstStep" is not a valid repository/tag: invalid reference format
The Dockerfile is this:
FROM node:8.12.0-alpine AS firstStep
WORKDIR /usr/src/app/
# Copy both the package.json and the package-lock.json
COPY package*.json ./
COPY . .
# Deployment container
FROM nginx:1.14.0-alpine
RUN apk add --no-cache bash
RUN apk add --update curl
#set env var for certs
ENV NODE_EXTRA_CA_CERTS /confs/MyPem.pem
# Forward logs to stdout and stderr
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
&& ln -sf /dev/stderr /var/log/nginx/error.log
# Create nginx config dir and copy nginx files for environments into it
RUN mkdir /confs
COPY ./nginxconf/* /confs/
#This copies the Keystore from the workspace and places it at the root of the container
COPY ./MyPem.pem /confs/MyPem.pem
COPY --from=firstStep /usr/src/app/dist /usr/share/nginx/html
COPY ./entrypoint.sh /opt/entrypoint.sh
RUN chmod a+x /opt/entrypoint.sh
ENTRYPOINT ["/opt/entrypoint.sh"]
If you check in dockerhub.com the image node:8.12.0-alpine indeed does not exists, Use for example "node:8.12-alpine" . Also you should use lowercase for "firstStep" so ... "firststep"
Support for multi-stage builds was added in 17.05.0. You can check the current version of the docker client and server with docker version. You'll need to upgrade the docker engine performing the build. Older docker releases are not supported once a new major release is delivered, so you'll want to pick a current stable release. Follow the installation guide from docker for your platform to install from the docker repos, you'll often find that Linux distributions have older versions of docker in their repos.
Docker COPY is not copying over the bash script
FROM alpine:latest
#Install Go and Tini - These remain.
RUN apk add --no-cache go build-base gcc go
RUN apk add --no-cache --update ca-certificates redis git && update-ca-certificates
# Set Env Variables for Go and add Go to Path.
ENV GOPATH /go
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH
RUN go get github.com/rakyll/hey
RUN echo GOLANG VERSION `go version`
COPY ./bench.sh /root/bench.sh
RUN chmod +x /root/bench.sh
ENTRYPOINT /root/bench.sh
Here is the script -
#!/bin/bash
set -e;
echo "entered";
hey;
I try running the above Dockerfile with
$ docker build -t test-bench .
$ docker run -it test-bench
But I get the error
/bin/sh: /root/bench.sh: not found
The file does exist -
$ docker run --rm -it test-bench sh
/ # ls
bin dev etc go home lib media mnt opt proc root run sbin srv sys tmp usr var
/ # cd root
~ # ls
bench.sh
~ #
Is your docker build successful. When I tried to simulate this, found the following error
---> Running in 96468658cebd
go: missing Git command. See https://golang.org/s/gogetcmd
package github.com/rakyll/hey: exec: "git": executable file not found in $PATH
The command '/bin/sh -c go get github.com/rakyll/hey' returned a non-zero code: 1
Try installing git using Dockerfile RUN apk add --no-cache go build-base gcc go git and run again.
The COPY operation here seems to be correct. Make sure it is present in the directory from where docker build is executed.
Okay, the script is using /bin/bash the bash binary is not available in the alpine image. Either it has to be installed or a /bin/sh shell should be used
I'm trying to configure my docker container so it's possible to ssh into it (the container will be run on Azure). I managed to create an image that enables user to ssh into a container created from that image, the Dockerfile looks like that (it's not mine, I found it on the internet):
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
EXPOSE 2222
RUN apt-get update && apt-get install -y openssh-server
RUN mkdir /var/run/sshd
COPY sshd_config /etc/ssh
RUN echo 'root:Docker' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed 's#session\s*required\s*pam_loginuid.so#session optional pam_loginuid.so#g' -i /etc/pam.d/sshd
ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile
CMD ["/usr/sbin/sshd", "-D"]
I'm using mcr.microsoft.com/dotnet/core/sdk:2.2-stretch because it's what I need later on to run the application.
Having the Dockerfile above, I run docker build . -t ssh. I can confirm that it's possible to ssh into a container created from ssh image with following instructions:
docker run -d -p 0.0.0.0:2222:22 --name ssh ssh
ssh root#localhost -p 2222
My application's Dockerfile:
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
COPY ["Application.WebAPI/Application.WebAPI.csproj", "Application.WebAPI/"]
COPY ["Processing.Dependency/Processing.Dependency.csproj", "Processing.Dependency/"]
COPY ["Processing.QueryHandling/Processing.QueryHandling.csproj", "Processing.QueryHandling/"]
COPY ["Model.ViewModels/Model.ViewModels.csproj", "Model.ViewModels/"]
COPY ["Core.Infrastructure/Core.Infrastructure.csproj", "Core.Infrastructure/"]
COPY ["Model.Values/Model.Values.csproj", "Model.Values/"]
COPY ["Sql.Business/Sql.Business.csproj", "Sql.Business/"]
COPY ["Model.Events/Model.Events.csproj", "Model.Events/"]
COPY ["Model.Messages/Model.Messages.csproj", "Model.Messages/"]
COPY ["Model.Commands/Model.Commands.csproj", "Model.Commands/"]
COPY ["Sql.Common/Sql.Common.csproj", "Sql.Common/"]
COPY ["Model.Business/Model.Business.csproj", "Model.Business/"]
COPY ["Processing.MessageBus/Processing.MessageBus.csproj", "Processing.MessageBus/"]
COPY [".Processing.CommandHandling/Processing.CommandHandling.csproj", "Processing.CommandHandling/"]
COPY ["Processing.EventHandling/Processing.EventHandling.csproj", "Processing.EventHandling/"]
COPY ["Sql.System/Sql.System.csproj", "Sql.System/"]
COPY ["Application.Common/Application.Common.csproj", "Application.Common/"]
RUN dotnet restore "Application.WebAPI/Application.WebAPI.csproj"
COPY . .
WORKDIR "/src/Application.WebAPI"
RUN dotnet build "Application.WebAPI.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "Application.WebAPI.csproj" -c Release -o /app
FROM ssh AS final
WORKDIR /app
EXPOSE 80
EXPOSE 443
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "Application.WebApi.dll"]
As you can see I'm using ssh image as a base image in the final stage. Even though I was able to sshe into the container created from ssh image, I'm unable to ssh into a container created from the latter Dockerfile. Here is the docker-compose.yml I'm using in order to ease starting the container:
version: '3.7'
services:
application.webapi:
image: application.webapi
container_name: webapi
ports:
- "0.0.0.0:5000:80"
- "0.0.0.0:2222:22"
build:
context: .
dockerfile: Application.WebAPI/Dockerfile
environment:
- ASPNETCORE_ENVIRONMENT=docker
When I run docker exec -it webapi bashand execute service ssh status, I'm getting [FAIL] sshd is not running ... failed! - but when I do service ssh start and try to ssh into that container, it works. Unfortunately this approach is not acceptable, ssh daemon should launch itself on startup.
I tried using cron and other stuff available on debian but it's a slim version and systemd is not available there - I'm also not fond of installing hundreds of things on slim versions.
Do you have any ideas what could be wrong here?
You have conflicting startup command definitions in your final image. Note that CMD does not simply run a command in your image, it defines the startup command, and has a complex interaction with ENTRYPOINT (in short: if both are present, CMD just supplies extra arguments to ENTRYPOINT).
You can see the table of possibilities in the Dockerfile documentation: https://docs.docker.com/engine/reference/builder/. In addition, there's a bonus complication when you mix and match CMD and ENTRYPOINT in different layers:
Note: If CMD is defined from the base image, setting ENTRYPOINT will reset CMD to an empty value. In this scenario, CMD must be defined in the current image to have a value.
As far as I know, you can't get what you want just by layering images. You will need to create a startup script in your final image that both runs sshd -D and then runs dotnet Application.WebApi.dll.