WORKDIR $HOME cannot normalize nothing - docker

I'm new to docker. I've a very simple Dockerfile but when I try to build an image from this Dockerfile, i get "cannot normalize nothing" error.
FROM alpine:latest
RUN apk add vim
CMD ["bash"]
WORKDIR $HOME
For building the image, I use the following command:
$ docker build -t aamirglb/test:1.0 .
I used Play With Docker (PWD) to test this script and here is the output.
pwd output
If I comment out WORKDIR $HOME line, the build is successful.
Any help in understanding this error will be highly appreciated.
Thanks in advance.

For those that can't reproduce, this error does not happen with buildkit:
$ docker build -t test-workdir -f df.workdir-empty .
[+] Building 0.2s (6/6) FINISHED
=> [internal] load build definition from df.workdir-empty 0.0s
=> => transferring dockerfile: 43B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.0s
=> [1/3] FROM docker.io/library/alpine:latest 0.0s
=> CACHED [2/3] RUN apk add vim 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:cadafef9a77c95b1b32499bcc1beba016ff0a7071710a1c37eb4e5f32e5d1c94 0.0s
=> => naming to docker.io/library/test-workdir 0.0s
But if you build with the classic builder, the error does appear:
$ DOCKER_BUILDKIT=0 docker build -t test-workdir -f df.workdir-empty .
Sending build context to Docker daemon 23.04kB
Step 1/4 : FROM alpine:latest
---> 14119a10abf4
Step 2/4 : RUN apk add vim
---> Running in a286e7f3107a
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/5) Installing xxd (8.2.4708-r0)
(2/5) Installing lua5.3-libs (5.3.6-r0)
(3/5) Installing ncurses-terminfo-base (6.2_p20210612-r0)
(4/5) Installing ncurses-libs (6.2_p20210612-r0)
(5/5) Installing vim (8.2.4708-r0)
Executing busybox-1.33.1-r3.trigger
OK: 26 MiB in 19 packages
Removing intermediate container a286e7f3107a
---> acdd6e1963db
Step 3/4 : CMD ["bash"]
---> Running in 6deb306db96a
Removing intermediate container 6deb306db96a
---> 960f0de2f376
Step 4/4 : WORKDIR $HOME
cannot normalize nothing
And before you get your hopes up, workdir in the buildkit example is not set to the user's home directory:
$ docker inspect test-workdir:latest --format '{{.Config.WorkingDir}}'
/
So first, what does cannot normalize nothing mean? Normalizing is the process of taking a path like /.//some/../path and turning that into /path, cleaning the string of unnecessary content and converting it into a uniform path. That process doesn't work on nothing, it needs a string or value, or at least the classic builder does (buildkit seems to have a sensible default).
Why doesn't $HOME default to /root? Because that variable was never set in docker. The only environment variable defined in there is a path:
$ docker inspect test-workdir:latest --format '{{.Config.Env}}'
[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin]
When you launch a shell like /bin/sh in the container, that shell is what is defining $HOME and various other variables:
$ docker run -it --rm test-workdir /bin/sh
/ # env
HOSTNAME=c68d4370e413
SHLVL=1
HOME=/root
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
Since there was no ENV or ARG to set $HOME, you cannot use it in the WORKDIR step.

Related

dockerfile cannot build: CONDA env create

Hi there I'm new to Docker and Dockerfiles in general,
However, I will need to create one in order to load an application on a server using WDL. With that said, there are few important aspects of this Dockerfile:
requires to create a Conda environment
in there I have to install Snakemake (through Mamba)
finally, I will need to git clone a repository and follow the steps to generate an executable for the application, later invoked by Snakemake
Luckily, it seems most of the pieces are already on dockerhub; correct if I'm wrong based on the script (see below)
# getting ubuntu base image & anaconda3 loaded
2 FROM ubuntu:latest
3 FROM continuumio/anaconda3:2021.05
4 FROM condaforge/mambaforge:latest
5 FROM snakemake/snakemake:stable
6
7 FROM node:alpine
8 RUN apk add --no-cache git
9 RUN apk add --no-cache openssh
10
11 MAINTAINER Name <email>
12
13 WORKDIR /home/xxx/Desktop/Pangenie
14
15 ## ACTUAL PanGenIe INSTALLATION
16 RUN git clone https://github.com/eblerjana/pangenie.git /home/xxx/Desktop/Pangenie
17 # create the environment
18 RUN conda env create -f environment.yml
19 # build the executable
20 RUN conda activate pangenie
21 RUN mkdir build; cd build; cmake .. ; make
First, I think that loading also Mamba and Snakemake would allow me to simply launch the application, as the tools are already set-up by the Dockerfile. Then, I ideally would like to build from the repository the executable, still I get an error at line 18 when I try to create a Conda environment, this is what I get:
[+] Building 1.7s (10/10) FINISHED
[internal] load build definition from Dockerfile
0.1s => => transferring dockerfile: 708B 0.1s => [internal] load .dockerignore 0.1s => => transferring context: 2B 0.1s => [internal] load metadata for docker.io/library/node:alpine 1.4s => [auth] library/node:pull token for registry-1.docker.io 0.0s => [stage-4 1/6] FROM docker.io/library/node:alpine#sha256:1a04e2ec39cc0c3a9657c1d6f8291ea2f5ccadf6ef4521dec946e522833e87ea
0.0s => CACHED [stage-4 2/6] RUN apk add --no-cache git 0.0s => CACHED [stage-4 3/6] RUN apk add --no-cache openssh 0.0s => CACHED [stage-4 4/6] WORKDIR /home/mat/Desktop/Pangenie 0.0s => CACHED [stage-4 5/6] RUN git clone https://github.com/eblerjana/pangenie.git /home/mat/Desktop/Pangenie
0.0s => ERROR [stage-4 6/6] RUN conda env create -f environment.yml 0.1s
[stage-4 6/6] RUN conda env create -f environment.yml:
#10 0.125 /bin/sh: conda: not found executor failed running [/bin/sh -c conda env create -f environment.yml]: exit code: 127
Now, I'm not really experienced as I said, and I spent some time looking for a solution and tried different things, but nothing worked out... if anyone has an idea or even suggesions on how to fix this Dockerfile, please let me know.
Thanks in advance!

Different docker image being created from command line and JetBrains WebStorm

I am creating a docker image. When I create it from WebStorm it creates a different image ( that doesn't run most importantly ) than when I create it with docker build.
Is this something that WebStorm does that I am unaware of given both methods are using the exact same files and Dockerfile yet are creating different images?
FROM cm2network/steamcmd:root
RUN apt update \
&& apt install -y curl \
&& apt install -y nodejs -y npm
WORKDIR /home/steam/steamcmd
VOLUME /home/steam/steamcmd
COPY . .
RUN npm install
USER steam
EXPOSE 3000
CMD ["node", "./src/index.js"]
The error says that the vdf package ( first one required ) isn't installed.
Log from terminal when running docker build -t <tag> .
[+] Building 12.1s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 310B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/cm2network/steamcmd:root 0.0s
=> [1/5] FROM docker.io/cm2network/steamcmd:root 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 7.20kB 0.0s
=> CACHED [2/5] RUN apt update && apt install -y curl && apt install -y nodejs -y npm 0.0s
=> CACHED [3/5] WORKDIR /home/steam/steamcmd 0.0s
=> [4/5] COPY . . 0.1s
=> [5/5] RUN npm install 9.8s
=> exporting to image 2.1s
=> => exporting layers 2.1s
=> => writing image sha256:b1c36641fc4d49cff4d38092879d556c6257a96cc55b4401827cac7c5f72c5c9 0.0s
=> => naming to docker.io/library/steamcmd-terminal 0.0s
Log from WebStorm when using the native docker config:
Deploying 'steamcmd-webstorm Dockerfile: Dockerfile'…
Building image…
Preparing build context archive…
[==================================================>]50/50 files
Done
Sending build context to Docker daemon…
[==================================================>] 18.14kB
Done
Step 1/9 : FROM cm2network/steamcmd:root
---> d490902b909c
Step 2/9 : RUN apt update && apt install -y curl && apt install -y nodejs -y npm
---> Running in 56188b34650b
<Taken out the entire logs here as it's massive but nothing exciting seems to be going on here. Just installing Node and npm>
Step 3/9 : WORKDIR /home/steam/steamcmd
---> Running in 5c9c276586ce
Removing intermediate container 5c9c276586ce
---> c93a98ea80fe
Step 4/9 : VOLUME /home/steam/steamcmd
---> Running in e608a8c467f4
Removing intermediate container e608a8c467f4
---> 03f450c267b2
Step 5/9 : COPY . .
---> bcd9ce25e8a5
Step 6/9 : RUN npm install
---> Running in 16a46251730b
added 93 packages, and audited 94 packages in 8s
38 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
Removing intermediate container 16a46251730b
---> 49bac07990ea
Step 7/9 : USER steam
---> Running in a50475541075
Removing intermediate container a50475541075
---> 9f8b28506b7f
Step 8/9 : EXPOSE 3000
---> Running in 17e53097f7b4
Removing intermediate container 17e53097f7b4
---> 11788721f0c2
Step 9/9 : CMD ["node", "./src/index.js"]
---> Running in 99d37cb33066
Removing intermediate container 99d37cb33066
---> 95ce7c9c21b7
Successfully built 95ce7c9c21b7
Successfully tagged steamcmd-webstorm:latest
'steamcmd-webstorm Dockerfile: Dockerfile' has been deployed successfully.

Docker Go image: starting container process caused: exec: "app": executable file not found in $PATH: unknown

I have been reading a lot of similar issues on different languages, none of them are Go.
I just created a Dockerfile with the instructions I followed on official Docker hub page:
FROM golang:1.17.3
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
This is my folder structure:
users-service
|-> .gitignore
|-> Dockerfile
|-> go.mod
|-> main.go
|-> README.md
If anyone needs to see some code, this is how my main.go looks like:
package main
import "fmt"
func main() {
fmt.Println("Hello, World!")
}
I ran docker build -t users-service .:
$ docker build -t users-service .
[+] Building 5.5s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 154B 0.1s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/golang:1.17.3 3.3s
=> [auth] library/golang:pull token for registry-1.docker.io 0.0s
=> [1/5] FROM docker.io/library/golang:1.17.3#sha256:6556ce40115451e40d6afbc12658567906c9250b0fda250302dffbee9d529987 0.3s
=> [internal] load build context 0.1s
=> => transferring context: 2.05kB 0.0s
=> [2/5] WORKDIR /go/src/app 0.1s
=> [3/5] COPY . . 0.1s
=> [4/5] RUN go get -d -v ./... 0.6s
=> [5/5] RUN go install -v ./... 0.7s
=> exporting to image 0.2s
=> => exporting layers 0.1s
=> => writing image sha256:1f0e97ed123b079f80eb259dh3e34c90a48bf93e8f55629d05044fec8bfcaca6 0.0s
=> => naming to docker.io/library/users-service 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Then I ran docker run users-service but I get that error:
$ docker run users-service
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "app": executable file not found in $PATH: unknown.
I remember I had some troubles with GOPATH environment variable in Visual Studio Code on Windows, maybe it's related... Any sugguestions?
The official Docker documentation has useful instructions for building a Go image: https://docs.docker.com/language/golang/build-images/
In summary, you need to build your Go binary and you need to configure the CMD appropriately, e.g.:
FROM golang:1.17.3
WORKDIR /app
COPY main.go .
COPY go.mod ./
RUN go build -o /my-go-app
CMD ["/my-go-app"]
Build the container:
$ docker build -t users-service .
Run the docker container:
$ docker run --rm -it users-service
Hello, World!
Your "app" executable binary should be available in your $PATH to call globally without any path prefix. Otherwise, you have to supply your full path to your executable like CMD ["/my/app"]
Also, I recommend using an ENTRYPOINT instruction. ENTRYPOINT indicates the direct path to the executable, while CMD indicates arguments supplied to the ENTRYPOINT.
Using combined RUN instructions make your image layers minimal, your overall image size becomes little bit smaller compared to using multiple RUNs.

Check if stage exists in Dockerfile

I have a CI script that builds Dockerfiles. My plan is that unit tests should be run in a test stage in each Dockerfile, for example:
FROM alpine AS build
WORKDIR /app
COPY src .
...
FROM build AS test
RUN mvn clean test
FROM build AS package
COPY --from=build ...
So, for a given Dockerfile, I would like to check if it has a test stage and, if so, run docker build --target test .... If it doesn't have a test stage, I don't want to run docker build (which would fail).
How can I check if a Dockerfile contains a certain stage without actually building it?
I do realize this question has some XY problem vibes to it, so feel free to enlighten me. But I also think the question can be generally useful anyway.
I'm going to shy away from trying to parse the Dockerfile since there are a lot of ways to inject false positives or negatives. E.g.
RUN echo \
FROM base as test
or
FROM base \
as test
So instead, I'm going to favor letting docker do the hard work, and modifying the file to not fail on a missing test. This can be done by adding a test stage to a file even when it already as a test stage. Whether you want to put this at the beginning or end of the Dockerfile depends on whether you are running buildkit:
$ cat df.dup-target
FROM busybox as test
RUN exit 1
FROM busybox as test
RUN exit 0
$ DOCKER_BUILDKIT=0 docker build --target test -f df.dup-target .
Sending build context to Docker daemon 20.99kB
Step 1/2 : FROM busybox as test
---> be5888e67be6
Step 2/2 : RUN exit 1
---> Running in 9f96f42bc6d8
The command '/bin/sh -c exit 1' returned a non-zero code: 1
$ DOCKER_BUILDKIT=1 docker build --target test -f df.dup-target .
[+] Building 0.1s (6/6) FINISHED
=> [internal] load build definition from df.dup-target 0.0s
=> => transferring dockerfile: 114B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 0.0s
=> [test 1/2] FROM docker.io/library/busybox 0.0s
=> CACHED [test 2/2] RUN exit 0 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:8129063cb183c1c1aafaf3eef0c8671e86a54f795092fa7a918145c14da3ec3b 0.0s
Then you could append the always successful test at the beginning or end, passing that modified Dockerfile to stdin for the docker build to process:
$ cat df.simple
FROM busybox as build
RUN exit 0
$ cat - df.simple <<EOF | DOCKER_BUILDKIT=1 docker build --target test -f - .
FROM busybox as test
RUN exit 0
EOF
[+] Building 0.1s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 109B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 0.0s
=> [test 1/2] FROM docker.io/library/busybox 0.0s
=> CACHED [test 2/2] RUN exit 0 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:8129063cb183c1c1aafaf3eef0c8671e86a54f795092fa7a918145c14da3ec3b 0.0s
This is a simple grep invocation:
egrep -i -q '^FROM .* AS test$' Dockerfile
You also might consider running your unit tests outside of Docker, before you start building containers. (Or, if your CI system supports running steps inside containers, use a container to get a language runtime, but not necessarily run the Dockerfile.) You'll still need a Docker-based setup to run larger integration tests, but you can run these on your built production-ready containers.

How do I build a dockerfile if the name of the dockerfile isn't Dockerfile?

I am able a build a Dockerfile like
docker build -t deepak/ruby .
But for a Dockerfile which is not named Dockerfile
# DOCKER-VERSION 0.4.8
FROM deepak/ruby
MAINTAINER Deepak Kannan "deepak#example.com"
RUN ./bin/rails s
let us say it is called Dockerfile.app
which we build with
docker build -t deepak/app Dockerfile.app
then i get the error
Uploading context 0 bytes
Error build: EOF
EOF
Notice there is a dot . at the end of both commands.
docker build -f MyDockerfile .
Or with a tag:
docker build -t mysuperimage -f MyDockerfile .
This Works
docker build -t doronaviguy/helloworld -f SomeDockerFile .
Docker build Documentation
The last parameter to docker build is the build path, when you put . it means this is the path where you will find the Dockerfile. When you change it to Dockerfile.app it will then try and look for Dockerfile.app/Dockerfile, which isn't correct.
I'm not sure if it will still work, but you used to be able to do this.
$ docker build -t deepak/app - < Dockerfile.app
Try that and see if it helps, if not, maybe open a docker issue to add this feature back in, or update the documentation on how to use a Dockerfile with a different name.
More info here: http://docs.docker.io/en/latest/commandline/command/build/
Try dockerfeed. It uses the docker feature to build a context via stdin. I wrote the script to address exactly your problem I was facing myself.
To replace a Dockerfile with a different one you do it like this:
dockerfeed -d Dockerfile.app . | docker build -t deepak/ruby -
And voilà. Dockerfeed is doing the same as docker build. It packs the source with its Dockerfile but lets you swap out the old Dockerfile with the desired one. No files are created in the process, no source is changed. The generated tar archive is piped into docker, which in turn sends it down to the docker daemon.
Update:
This was a valid answer in the old days when there was no -f switch available. With docker version 1.5 this option was introduced. Now you can build provide a different Dockerfile like this:
docker build -f other-Dockerfile .
This should work,
docker build -t <tag-name> -f <file-name> .
You can acheive this also using docker-compose.
In your docker-compose.yml under the build section, you can specify the directory in which you store your dockerfile and its alternate-name as follow :
build:
context: "/path/to/docker/directory"
dockerfile: "dockerfile-alternate-name"
docker-compose
Windows User should try below command:
docker build -f absolute_docker_file_path dot
for example :
docker build -f D:\Code\core-api\API_TEST.Dockerfile .
If you have multiple Dockerfile's, as could be the case in large projects, you could specify the respective fully-qualified Dockerfile to use.
docker build -t swagger_local -f /Users/123456/myrepos/storage/services/gateway/swagger/Dockerfile .
The command when run from /Users/123456/myrepos/storage creates a swagger_local image using the Dockerfile in a child project (services/gateway in my case )
My case is in the /tmp directory and there are a lot of files and docker try with others files althought I pass the -f <Dockerfile.foo>
The most clean and easy (for me) solution is (intead the dockerfeed from above answers):
cat DockerFile.debian.foo | docker build -t debian.foo -
Let assume that you have successfully install docker toolbox including docker-compose and docker-machine. This is my docker-compose.yml
version: '2'
services:
web: ./www/python
ports:
- "5000:5000"
And this is under Dockerfile
FROM python:3.4-alpine
ADD ./www/python /code
WORKDIR /code
RUN pip install -r requirements.txt
CMD ["python", "app.py"]
Make sure those files located precisely under root folder. When you run docker-compose up then you are going to build image for web.
For me it worked once I put the entire path in the file option:
(meta_learning) brandomiranda~ ❯ docker build -f ~/iit-term-synthesis/Dockerfile_arm -t brandojazz/iit-term-synthesis:test_arm ~/iit-term-synthesis/
[+] Building 43.8s (9/28)
=> [internal] load build definition from Dockerfile_arm 0.0s
=> => transferring dockerfile: 2.59kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/continuumio/miniconda3:latest 0.0s
=> [ 1/24] FROM docker.io/continuumio/miniconda3 0.0s
=> CACHED https://api.github.com/repos/IBM/pycoq/git/refs/heads/main 0.0s
=> [ 2/24] RUN apt-get update && apt-get install -y --no-install-recommends ssh git m4 libgmp-dev opam wget ca-ce 14.9s
=> [ 3/24] RUN useradd -m bot 0.2s
=> [ 4/24] WORKDIR /home/bot 0.0s
=> [ 5/24] ADD https://api.github.com/repos/IBM/pycoq/git/refs/heads/main version.json 0.0s
=> [ 6/24] RUN opam init --disable-sandboxing 28.6s
=> => # [ocaml-base-compiler.4.14.0] downloaded from cache at https://opam.ocaml.org/cache
=> => # <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
=> => # ∗ installed base-bigarray.base
=> => # ∗ installed base-threads.base
=> => # ∗ installed base-unix.base
=> => # ∗ installed ocaml-options-vanilla.1
this failed for me:
(meta_learning) brandomiranda~ ❯ docker build -f Dockerfile_arm -t brandojazz/iit-term-synthesis:test_arm ~/iit-term-synthesis/
[+] Building 0.0s (1/2)
=> ERROR [internal] load build definition from Dockerfile_arm 0.0s
=> => transferring dockerfile: 40B 0.0s
------
> [internal] load build definition from Dockerfile_arm:
------
failed to solve with frontend dockerfile.v0: failed to read dockerfile: error from sender: open .Trash: operation not permitted

Resources