Problem statement: I need to pull docker(projectA and projectB) from two different urls based on the arg provided.
ARG url=docker-local.artifactory.com/projectA #By default its for A.
RUN echo ${url}
FROM $url
Ideal Solution:
docker build -t hello . should build docker of project A
docker build --build-arg url="docker-local.artifactory.com/projectB" -t hello . should build docker of project B.
Current Issue:
"base name ($url) should not be blank"
Using the docs for reference, if you want to use ARG before FROM, don't use anything in-between. See this section for details.
This minimal Dockerfile works:
ARG url=docker-local.artifactory.com/projectA
FROM $url
Built using this command with a build arg:
docker build -t from --build-arg url=alpine:3.9 .
[+] Building 0.1s (5/5) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 116B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:3.9 0.0s
=> CACHED [1/1] FROM docker.io/library/alpine:3.9 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:352159a49b502edb1c17a3ad142b320155bd541830000c02093b79f4058a3bd1 0.0s
=> => naming to docker.io/library/from
The docs also show an example if you want to re-use the ARG value after the first FROM command:
ARG url=docker-local.artifactory.com/projectA
FROM $url
ARG url
RUN echo $url
Using the following build file,
ARG VERSION=busybox:latest
FROM $VERSION
ARG VERSION
RUN echo $VERSION
Running with the default value
docker build -t test .
Sending build context to Docker daemon 16.38kB
Step 1/4 : ARG VERSION=busybox:latest
Step 2/4 : FROM $VERSION
latest: Pulling from library/busybox
Running with value changed during build
docker build -t test --build-arg VERSION="ubuntu:20.04" .
Sending build context to Docker daemon 16.38kB
Step 1/4 : ARG VERSION=busybox:latest
Step 2/4 : FROM $VERSION
20.04: Pulling from library/ubuntu
Related
The following commands do not show the output ubuntu1 image:
docker buildx build -f 1.dockerfile -t ubuntu1 .
docker image ls | grep ubuntu1
# no output
1.dockerfile:
FROM ubuntu:latest
RUN echo "my ubuntu"
Plus, I cannot use the image in FROM statements in other docker files (both builds are on my local Windows box):
2.dockerfile:
FROM ubuntu1
RUN echo "my ubuntu 2"
docker buildx build -f 2.dockerfile -t ubuntu2 .
#error:
WARNING: No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 1.8s (4/4) FINISHED
=> [internal] load build definition from 2.dockerfile 0.0s
=> => transferring dockerfile: 84B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> ERROR [internal] load metadata for docker.io/library/ubuntu1:latest 1.8s
=> [auth] library/ubuntu1:pull token for registry-1.docker.io 0.0s
------
> [internal] load metadata for docker.io/library/ubuntu1:latest:
------
2.dockerfile:1
--------------------
1 | >>> FROM ubuntu1:latest
2 | RUN echo "my ubuntu 2"
3 |
--------------------
error: failed to solve: ubuntu1:latest: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed (did you mean ubuntu:latest?)
Any idea what's going on? How can I see what buildx prepared and reference one image in another dockerfile?
Ok found a partial solution, I need to add --output type=docker as per the docs. This puts the docker in the image list. But I still cannot use it in the second docker.
I'm new to docker. I've a very simple Dockerfile but when I try to build an image from this Dockerfile, i get "cannot normalize nothing" error.
FROM alpine:latest
RUN apk add vim
CMD ["bash"]
WORKDIR $HOME
For building the image, I use the following command:
$ docker build -t aamirglb/test:1.0 .
I used Play With Docker (PWD) to test this script and here is the output.
pwd output
If I comment out WORKDIR $HOME line, the build is successful.
Any help in understanding this error will be highly appreciated.
Thanks in advance.
For those that can't reproduce, this error does not happen with buildkit:
$ docker build -t test-workdir -f df.workdir-empty .
[+] Building 0.2s (6/6) FINISHED
=> [internal] load build definition from df.workdir-empty 0.0s
=> => transferring dockerfile: 43B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.0s
=> [1/3] FROM docker.io/library/alpine:latest 0.0s
=> CACHED [2/3] RUN apk add vim 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:cadafef9a77c95b1b32499bcc1beba016ff0a7071710a1c37eb4e5f32e5d1c94 0.0s
=> => naming to docker.io/library/test-workdir 0.0s
But if you build with the classic builder, the error does appear:
$ DOCKER_BUILDKIT=0 docker build -t test-workdir -f df.workdir-empty .
Sending build context to Docker daemon 23.04kB
Step 1/4 : FROM alpine:latest
---> 14119a10abf4
Step 2/4 : RUN apk add vim
---> Running in a286e7f3107a
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/5) Installing xxd (8.2.4708-r0)
(2/5) Installing lua5.3-libs (5.3.6-r0)
(3/5) Installing ncurses-terminfo-base (6.2_p20210612-r0)
(4/5) Installing ncurses-libs (6.2_p20210612-r0)
(5/5) Installing vim (8.2.4708-r0)
Executing busybox-1.33.1-r3.trigger
OK: 26 MiB in 19 packages
Removing intermediate container a286e7f3107a
---> acdd6e1963db
Step 3/4 : CMD ["bash"]
---> Running in 6deb306db96a
Removing intermediate container 6deb306db96a
---> 960f0de2f376
Step 4/4 : WORKDIR $HOME
cannot normalize nothing
And before you get your hopes up, workdir in the buildkit example is not set to the user's home directory:
$ docker inspect test-workdir:latest --format '{{.Config.WorkingDir}}'
/
So first, what does cannot normalize nothing mean? Normalizing is the process of taking a path like /.//some/../path and turning that into /path, cleaning the string of unnecessary content and converting it into a uniform path. That process doesn't work on nothing, it needs a string or value, or at least the classic builder does (buildkit seems to have a sensible default).
Why doesn't $HOME default to /root? Because that variable was never set in docker. The only environment variable defined in there is a path:
$ docker inspect test-workdir:latest --format '{{.Config.Env}}'
[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin]
When you launch a shell like /bin/sh in the container, that shell is what is defining $HOME and various other variables:
$ docker run -it --rm test-workdir /bin/sh
/ # env
HOSTNAME=c68d4370e413
SHLVL=1
HOME=/root
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
Since there was no ENV or ARG to set $HOME, you cannot use it in the WORKDIR step.
I have been reading a lot of similar issues on different languages, none of them are Go.
I just created a Dockerfile with the instructions I followed on official Docker hub page:
FROM golang:1.17.3
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
This is my folder structure:
users-service
|-> .gitignore
|-> Dockerfile
|-> go.mod
|-> main.go
|-> README.md
If anyone needs to see some code, this is how my main.go looks like:
package main
import "fmt"
func main() {
fmt.Println("Hello, World!")
}
I ran docker build -t users-service .:
$ docker build -t users-service .
[+] Building 5.5s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 154B 0.1s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/golang:1.17.3 3.3s
=> [auth] library/golang:pull token for registry-1.docker.io 0.0s
=> [1/5] FROM docker.io/library/golang:1.17.3#sha256:6556ce40115451e40d6afbc12658567906c9250b0fda250302dffbee9d529987 0.3s
=> [internal] load build context 0.1s
=> => transferring context: 2.05kB 0.0s
=> [2/5] WORKDIR /go/src/app 0.1s
=> [3/5] COPY . . 0.1s
=> [4/5] RUN go get -d -v ./... 0.6s
=> [5/5] RUN go install -v ./... 0.7s
=> exporting to image 0.2s
=> => exporting layers 0.1s
=> => writing image sha256:1f0e97ed123b079f80eb259dh3e34c90a48bf93e8f55629d05044fec8bfcaca6 0.0s
=> => naming to docker.io/library/users-service 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Then I ran docker run users-service but I get that error:
$ docker run users-service
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "app": executable file not found in $PATH: unknown.
I remember I had some troubles with GOPATH environment variable in Visual Studio Code on Windows, maybe it's related... Any sugguestions?
The official Docker documentation has useful instructions for building a Go image: https://docs.docker.com/language/golang/build-images/
In summary, you need to build your Go binary and you need to configure the CMD appropriately, e.g.:
FROM golang:1.17.3
WORKDIR /app
COPY main.go .
COPY go.mod ./
RUN go build -o /my-go-app
CMD ["/my-go-app"]
Build the container:
$ docker build -t users-service .
Run the docker container:
$ docker run --rm -it users-service
Hello, World!
Your "app" executable binary should be available in your $PATH to call globally without any path prefix. Otherwise, you have to supply your full path to your executable like CMD ["/my/app"]
Also, I recommend using an ENTRYPOINT instruction. ENTRYPOINT indicates the direct path to the executable, while CMD indicates arguments supplied to the ENTRYPOINT.
Using combined RUN instructions make your image layers minimal, your overall image size becomes little bit smaller compared to using multiple RUNs.
I have a CI script that builds Dockerfiles. My plan is that unit tests should be run in a test stage in each Dockerfile, for example:
FROM alpine AS build
WORKDIR /app
COPY src .
...
FROM build AS test
RUN mvn clean test
FROM build AS package
COPY --from=build ...
So, for a given Dockerfile, I would like to check if it has a test stage and, if so, run docker build --target test .... If it doesn't have a test stage, I don't want to run docker build (which would fail).
How can I check if a Dockerfile contains a certain stage without actually building it?
I do realize this question has some XY problem vibes to it, so feel free to enlighten me. But I also think the question can be generally useful anyway.
I'm going to shy away from trying to parse the Dockerfile since there are a lot of ways to inject false positives or negatives. E.g.
RUN echo \
FROM base as test
or
FROM base \
as test
So instead, I'm going to favor letting docker do the hard work, and modifying the file to not fail on a missing test. This can be done by adding a test stage to a file even when it already as a test stage. Whether you want to put this at the beginning or end of the Dockerfile depends on whether you are running buildkit:
$ cat df.dup-target
FROM busybox as test
RUN exit 1
FROM busybox as test
RUN exit 0
$ DOCKER_BUILDKIT=0 docker build --target test -f df.dup-target .
Sending build context to Docker daemon 20.99kB
Step 1/2 : FROM busybox as test
---> be5888e67be6
Step 2/2 : RUN exit 1
---> Running in 9f96f42bc6d8
The command '/bin/sh -c exit 1' returned a non-zero code: 1
$ DOCKER_BUILDKIT=1 docker build --target test -f df.dup-target .
[+] Building 0.1s (6/6) FINISHED
=> [internal] load build definition from df.dup-target 0.0s
=> => transferring dockerfile: 114B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 0.0s
=> [test 1/2] FROM docker.io/library/busybox 0.0s
=> CACHED [test 2/2] RUN exit 0 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:8129063cb183c1c1aafaf3eef0c8671e86a54f795092fa7a918145c14da3ec3b 0.0s
Then you could append the always successful test at the beginning or end, passing that modified Dockerfile to stdin for the docker build to process:
$ cat df.simple
FROM busybox as build
RUN exit 0
$ cat - df.simple <<EOF | DOCKER_BUILDKIT=1 docker build --target test -f - .
FROM busybox as test
RUN exit 0
EOF
[+] Building 0.1s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 109B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 0.0s
=> [test 1/2] FROM docker.io/library/busybox 0.0s
=> CACHED [test 2/2] RUN exit 0 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:8129063cb183c1c1aafaf3eef0c8671e86a54f795092fa7a918145c14da3ec3b 0.0s
This is a simple grep invocation:
egrep -i -q '^FROM .* AS test$' Dockerfile
You also might consider running your unit tests outside of Docker, before you start building containers. (Or, if your CI system supports running steps inside containers, use a container to get a language runtime, but not necessarily run the Dockerfile.) You'll still need a Docker-based setup to run larger integration tests, but you can run these on your built production-ready containers.
I am able a build a Dockerfile like
docker build -t deepak/ruby .
But for a Dockerfile which is not named Dockerfile
# DOCKER-VERSION 0.4.8
FROM deepak/ruby
MAINTAINER Deepak Kannan "deepak#example.com"
RUN ./bin/rails s
let us say it is called Dockerfile.app
which we build with
docker build -t deepak/app Dockerfile.app
then i get the error
Uploading context 0 bytes
Error build: EOF
EOF
Notice there is a dot . at the end of both commands.
docker build -f MyDockerfile .
Or with a tag:
docker build -t mysuperimage -f MyDockerfile .
This Works
docker build -t doronaviguy/helloworld -f SomeDockerFile .
Docker build Documentation
The last parameter to docker build is the build path, when you put . it means this is the path where you will find the Dockerfile. When you change it to Dockerfile.app it will then try and look for Dockerfile.app/Dockerfile, which isn't correct.
I'm not sure if it will still work, but you used to be able to do this.
$ docker build -t deepak/app - < Dockerfile.app
Try that and see if it helps, if not, maybe open a docker issue to add this feature back in, or update the documentation on how to use a Dockerfile with a different name.
More info here: http://docs.docker.io/en/latest/commandline/command/build/
Try dockerfeed. It uses the docker feature to build a context via stdin. I wrote the script to address exactly your problem I was facing myself.
To replace a Dockerfile with a different one you do it like this:
dockerfeed -d Dockerfile.app . | docker build -t deepak/ruby -
And voilà. Dockerfeed is doing the same as docker build. It packs the source with its Dockerfile but lets you swap out the old Dockerfile with the desired one. No files are created in the process, no source is changed. The generated tar archive is piped into docker, which in turn sends it down to the docker daemon.
Update:
This was a valid answer in the old days when there was no -f switch available. With docker version 1.5 this option was introduced. Now you can build provide a different Dockerfile like this:
docker build -f other-Dockerfile .
This should work,
docker build -t <tag-name> -f <file-name> .
You can acheive this also using docker-compose.
In your docker-compose.yml under the build section, you can specify the directory in which you store your dockerfile and its alternate-name as follow :
build:
context: "/path/to/docker/directory"
dockerfile: "dockerfile-alternate-name"
docker-compose
Windows User should try below command:
docker build -f absolute_docker_file_path dot
for example :
docker build -f D:\Code\core-api\API_TEST.Dockerfile .
If you have multiple Dockerfile's, as could be the case in large projects, you could specify the respective fully-qualified Dockerfile to use.
docker build -t swagger_local -f /Users/123456/myrepos/storage/services/gateway/swagger/Dockerfile .
The command when run from /Users/123456/myrepos/storage creates a swagger_local image using the Dockerfile in a child project (services/gateway in my case )
My case is in the /tmp directory and there are a lot of files and docker try with others files althought I pass the -f <Dockerfile.foo>
The most clean and easy (for me) solution is (intead the dockerfeed from above answers):
cat DockerFile.debian.foo | docker build -t debian.foo -
Let assume that you have successfully install docker toolbox including docker-compose and docker-machine. This is my docker-compose.yml
version: '2'
services:
web: ./www/python
ports:
- "5000:5000"
And this is under Dockerfile
FROM python:3.4-alpine
ADD ./www/python /code
WORKDIR /code
RUN pip install -r requirements.txt
CMD ["python", "app.py"]
Make sure those files located precisely under root folder. When you run docker-compose up then you are going to build image for web.
For me it worked once I put the entire path in the file option:
(meta_learning) brandomiranda~ ❯ docker build -f ~/iit-term-synthesis/Dockerfile_arm -t brandojazz/iit-term-synthesis:test_arm ~/iit-term-synthesis/
[+] Building 43.8s (9/28)
=> [internal] load build definition from Dockerfile_arm 0.0s
=> => transferring dockerfile: 2.59kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/continuumio/miniconda3:latest 0.0s
=> [ 1/24] FROM docker.io/continuumio/miniconda3 0.0s
=> CACHED https://api.github.com/repos/IBM/pycoq/git/refs/heads/main 0.0s
=> [ 2/24] RUN apt-get update && apt-get install -y --no-install-recommends ssh git m4 libgmp-dev opam wget ca-ce 14.9s
=> [ 3/24] RUN useradd -m bot 0.2s
=> [ 4/24] WORKDIR /home/bot 0.0s
=> [ 5/24] ADD https://api.github.com/repos/IBM/pycoq/git/refs/heads/main version.json 0.0s
=> [ 6/24] RUN opam init --disable-sandboxing 28.6s
=> => # [ocaml-base-compiler.4.14.0] downloaded from cache at https://opam.ocaml.org/cache
=> => # <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
=> => # ∗ installed base-bigarray.base
=> => # ∗ installed base-threads.base
=> => # ∗ installed base-unix.base
=> => # ∗ installed ocaml-options-vanilla.1
this failed for me:
(meta_learning) brandomiranda~ ❯ docker build -f Dockerfile_arm -t brandojazz/iit-term-synthesis:test_arm ~/iit-term-synthesis/
[+] Building 0.0s (1/2)
=> ERROR [internal] load build definition from Dockerfile_arm 0.0s
=> => transferring dockerfile: 40B 0.0s
------
> [internal] load build definition from Dockerfile_arm:
------
failed to solve with frontend dockerfile.v0: failed to read dockerfile: error from sender: open .Trash: operation not permitted