WORKDIR doesn't work when running docker-compose run on Windows - docker

It looks like the WORKDIR inside my Dockerfile doesn't work. I'm trying this on Windows 10. The same works perfectly on Mac.
Here are my project files:
docker-compose.yml:
version: '3'
services:
php_under_test:
build: tests/DockerImages/${PHP_VERSION}
volumes:
- .:/opt/project/phpstorm-stubs
test_runner:
build: tests/DockerImages/testRunner
volumes:
- .:/opt/project/phpstorm-stubs
The tests/DockerImages/testRunner folder has only this Dockerfile:
FROM php:8.1-apache
RUN echo 'memory_limit = 1024M' >> /usr/local/etc/php/conf.d/docker-php-memlimit.ini
COPY --from=composer /usr/bin/composer /usr/bin/composer
RUN apt-get update && apt-get -y install git zip unzip
WORKDIR /opt/project/phpstorm-stubs
If I run docker-compose -f docker-compose.yml run test_runner /usr/local/bin/php tests/Tools/generate-stub-map in cmd.exe in the same directory where docker-compose.yml lies, I would get:
C:\Projects\phpstorm-stubs>docker-compose -f docker-compose.yml run test_runner /usr/local/bin/php tests/Tools/generate-stub-map
WARNING: The PHP_VERSION variable is not set. Defaulting to a blank string.
Creating phpstorm-stubs_test_runner_run ... done
Could not open input file: tests/Tools/generate-stub-map
ERROR: 1
However, if I run the same using -w: docker-compose -f docker-compose.yml run -w /opt/project/phpstorm-stubs test_runner /usr/local/bin/php tests/Tools/generate-stub-map - that would work. Output:
C:\Projects\phpstorm-stubs>docker-compose -f docker-compose.yml run -w /opt/project/phpstorm-stubs test_runner /usr/local/bin/php tests/Tools/generate-stub-map
WARNING: The PHP_VERSION variable is not set. Defaulting to a blank string.
Creating phpstorm-stubs_test_runner_run ... done
Parsing "/opt/project/phpstorm-stubs/tests/Tools/../../aerospike/aerospike.php"
Parsing "/opt/project/phpstorm-stubs/tests/Tools/../../aerospike/Bytes.php"
etc...
I can see here that -w is outdated and I should use --project-directory instead, but doesn't work also:
C:\Projects\phpstorm-stubs>docker-compose -f docker-compose.yml --project-directory /opt/project/phpstorm-stubs run test_runner /usr/local/bin/php
tests/Tools/generate-stub-map
WARNING: The PHP_VERSION variable is not set. Defaulting to a blank string.
ERROR: build path C:\opt\project\phpstorm-stubs\tests\DockerImages either does not exist, is not accessible, or is not a valid URL.
I've tried to look for something related at docker/compose github issues but found none.
docker version output:
Client:
Cloud integration: v1.0.24
Version: 20.10.17
API version: 1.41
Go version: go1.17.11
Git commit: 100c701
Built: Mon Jun 6 23:09:02 2022
OS/Arch: windows/amd64
Context: default
Experimental: true
Server: Docker Desktop 4.10.1 (82475)
Engine:
Version: 20.10.17
API version: 1.41 (minimum version 1.12)
Go version: go1.17.11
Git commit: a89b842
Built: Mon Jun 6 23:01:23 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.6
GitCommit: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
runc:
Version: 1.1.2
GitCommit: v1.1.2-0-ga916309
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Any help with this is much appreciated.

Sounds like you built test_runner image once using Dockerfile of previous version without WORKDIR parameter and trying to re-run container using this outdated image.
You have to remove image manually or force rebuild it using docker-compose build (or docker-compose up --build) instead of docker-compose run
Also this link may be helpfull

WORKDIR from dockerfile corresponds to working_dir in compose file https://docs.docker.com/compose/compose-file/#working_dir
Try to use
test_runner:
build: tests/DockerImages/testRunner
working_dir: /opt/project/phpstorm-stubs
volumes:
- .:/opt/project/phpstorm-stubs

You have to understand the different path of your whole project.
Location of your docker file C:\Projects\phpstorm-stubs\
Location of base of your project /opt/project/phpstorm-stubs so the exact whole path to your project base in reality is C:\Projects\phpstorm-stubs\opt\project\phpstorm-stubs and so the files it is trying to find is actually at C:\Projects\phpstorm-stubs\opt\project\phpstorm-stubs\usr\local\bin\php tests\Tools\generate-stub-map
So once you used the -f docker-compose.yml it by default thinks that your whole project is at C:\Projects\phpstorm-stubs\ but when you use it with -w /opt/project/phpstorm-stubs it knows that base of your project path is here so it knows that tests/Tools/generate-stub-map is further ahead of it.
And lastly --project-directory requires the whole path from the base of current drive
To sum up my answer you need to use command
docker-compose -f docker-compose.yml --project-directory /Projects/phpstorm-stubs/opt/project/phpstorm-stubs run test_runner /usr/local/bin/php tests/Tools/generate-stub-map

Related

Docker COPY destination path not found

I am trying to COPY a source file that is locally present to a destination path that is dynamically passed using the docker ARG command.
Example dockerfile is:
$ cat mydockerfile
FROM debian:latest
RUN apt update
ENV app_env='prod'
ARG src_app_dir
ARG dest_app_dir
RUN echo ${src_app_dir}
RUN echo ${dest_app_dir}
RUN mkdir /root/${dest_app_dir}
COPY ${src_app_dir}/file.txt /root/${dest_app_dir}/filenew.txt
WORKDIR /
CMD ["bash"]
I am trying to pass the build arg dest_app_dir="server_app_dir" and expecting the build process creates the container path /root/server_app_dir/
The source folder is already present on my local machine and where the docker-build context is present.
$ ls -d local_app_dir/
local_app_dir/
$ ls local_app_dir/
file.txt
But I am getting the following error for the destination path:
$ docker image build --build-arg src_app_dir="local_app_dir" dest_app_dir="server_app_dir" --tag arg_env:1.0 --file mydockerfile
unable to prepare context: path "dest_app_dir=server_app_dir" not found
Does not it work that way or am I missing the correct concept/usage of Docker build ARG and COPY commands here?
I am using docker-desktop on Windows11.
$ docker version
Client: Docker Engine - Community
Cloud integration: v1.0.23
Version: 20.10.14
API version: 1.41
Go version: go1.16.15
Git commit: a224086
Built: Thu Mar 24 01:48:21 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Desktop
Engine:
Version: 20.10.14
API version: 1.41 (minimum version 1.12)
Go version: go1.16.15
Git commit: 87a90dc
Built: Thu Mar 24 01:46:14 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.5.11
GitCommit: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc:
Version: 1.0.3
GitCommit: v1.0.3-0-gf46b6ba
docker-init:
Version: 0.19.0
GitCommit: de40ad0
You need to specify the build-arg as many times as the arguments
docker image build --build-arg src_app_dir="local_app_dir" --build-arg dest_app_dir="server_app_dir" --tag arg_env:1.0 --file mydockerfile .
Example
EDIT: Forgot to add context. Thanks #BMitch

Docker LABEL to access build image in multi-stage build

During my CI build I extract build artifacts from the build image in a multi-stage Docker build (test reports) by adding a LABEL build step and then creating a container with the labeled image to extract the artifact. This seems to be broken in Docker Desktop for Mac, or I misunderstand how this is supposed to work?
Here is a minimal build to reproduce the issue:
# Dockerfile.one
FROM alpine AS one
WORKDIR /test
RUN touch here
LABEL build=here
# Dockerfile.two
FROM alpine AS one
WORKDIR /test
RUN touch here
LABEL build=here
FROM alpine AS two
COPY --from=one /test/* /test/
RUN touch again
# test.sh
#! /bin/bash -u
# Clean up
trap 'rm -r test & docker rmi $image' EXIT
docker build -f $1 .
docker images --filter 'label=build=here'
image=$(docker images --filter 'label=build=here' -q | head -n 1)
id=$(docker create $image) && docker cp $id:/test/ . && docker rm $id
ls -l test
Running ./test.sh Dockerfile.one (only one stage) produces the expected output of the file here being present in the working directory.
However, running ./test.sh Dockerfile.two (multi-stage) fails, apparently because no images match --filter label=build=here.
What is wrong here?
$ docker version
Client:
Cloud integration: 1.0.17
Version: 20.10.7
API version: 1.41
Go version: go1.16.4
Git commit: f0df350
Built: Wed Jun 2 11:56:22 2021
OS/Arch: darwin/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: b0f5bc3
Built: Wed Jun 2 11:54:58 2021
OS/Arch: linux/amd64
Experimental: true

How to build docker image, using a dockerfile from stdin with shell varaibles

I try to build a docker image with Jenkins, using here docuemnt. Part of the The shell:
# Docker image build.
mkdir -p "$BASE_PATH/.docker"
cd "$BASE_PATH/.docker"
echo "docker version: "
docker version
docker login --username=****** --password ****** ******
docker build -t "******/$DOCKER_IMAGE" -f- . <<EOF
FROM ******
ARG NGINX_CONF_FILE=$NGINX_CONF_FILE
ENV DEPLOY_PATH=$DEPLOY_PATH
ENV NGINX_CONF_DIR=$NGINX_CONF_DIR
RUN mkdir -p \$DEPLOY_PATH \\
&& chmod 777 "\$DEPLOY_PATH"
WORKDIR \$DEPLOY_PATH
ADD customer customer/
ADD mall mall/
ADD marketing marketing/
ADD portal portal/
ADD setup setup/
ADD store store/
ADD work work/
WORKDIR \$NGINX_CONF_DIR
COPY ./\$NGINX_CONF_FILE .
EXPOSE 8000
EOF
When running it with bash, Jenkins complains:
docker version:
Client:
Version: 17.03.1-ce
API version: 1.27
Go version: go1.7.5
Git commit: 0801b25
Built: Tue Mar 28 08:29:28 2017
OS/Arch: linux/amd64
Server:
Version: 17.03.1-ce
API version: 1.27 (minimum version 1.12)
Go version: go1.7.5
Git commit: 0801b25
Built: Tue Mar 28 08:29:28 2017
OS/Arch: linux/amd64
Experimental: false
Login Succeeded
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /var/jenkins_home/workspace/docker-test-scrmv3/.docker/-: no such file or directory
Build step 'Execute shell' marked build as failure
New run name is '#17 ver:0.0.17'
Finished: FAILURE
Looks like docker treat the hyphen (-) as a directory not stdin. Accroding to the offical recommand this should work but dont know how. Any way to fix it?
Docker should be above 17.05. The document not metion it.

copying * files in Docker to directory gives error

I have the following Dockerfile
FROM openjdk:8-jdk-alpine
RUN mkdir -p /webpieces
COPY * /webpieces
WORKDIR "/webpieces"
ENTRYPOINT ["./bin/webpiecesexample"]
When I build like so, I get the following error
Deans-MacBook-Pro:webpiecesexample dean$ docker build -t gcr.io/braided-topic/webpieces2 .
Sending build context to Docker daemon 75.56MB
Step 1/5 : FROM openjdk:8-jdk-alpine
---> a3562aa0b991
Step 2/5 : RUN mkdir -p /webpieces
---> Running in bc615c0cd540
Removing intermediate container bc615c0cd540
---> 69a2f4530c44
Step 3/5 : COPY * /webpieces
When using COPY with more than one source file, the destination must be a directory and end with a /
When I trim the DockerFile and just build with the first two lines and then run with a basic shell to view the directories, I see the webpieces directory there
docker run -it --entrypoint sh gcr.io/braided-topic-266113/webpieces2
I can cd into webpieces and everything. Why is the copy command not working here?
docker version here:
Deans-MacBook-Pro:distributions dean$ docker version
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:22:34 2019
OS/Arch: darwin/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:29:19 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683
When using COPY with more than one source file, the destination must be a directory and end with a /.
Change the COPY line to
COPY * /webpieces/
...it seems to copy the contents of each directory instead of the directories themselves. I would prefer not to name each directory I am moving as we prefer auto-add when we make changes.
Use . instead of * and it'll preserve all the nesting.
COPY . /webpieces/

Version in "./docker-compose.yml" is unsupported

I have the following docker-compose.yml file
services:
containerA:
healthcheck:
test: "/build/docheck"
interval: "10s"
hostname: "containerA"
container_name: "containerA"
build:
dockerfile: "Dockerfile-5.6"
ports:
- "8081:8081"
version: "2.1"
When I try the following command: docker-compose up
it fails due to:
ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0") and place your service definitions under the services key, or omit the version key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/
I have the following setting:
$> docker-compose --version
docker-compose version 1.8.0, build unknown
$> docker --version
Docker version 17.12.0-ce, build c97c6d6
$> uname -r
4.9.0-4-amd64
$> lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.3 (stretch)
Release: 9.3
Codename: stretch
I need to use version 2.1 because I am using the healthcheck. Any ideas why this is complaining?
adding this because i ran into the same error on Ubuntu.
For me, it had nothing to do with my Compose version despite what the error message says; i just had to run with sudo. so
sudo docker-compose up
Compose files Version 2.1 are supported by Docker Compose version 1.9.0+. And you have docker-compose version 1.8.0. To upgrade docker-compose you can run following commands:
If installed via apt-get:
sudo apt-get remove docker-compose
If installed via curl:
sudo rm /usr/local/bin/docker-compose
If installed via pip:
pip uninstall docker-compose
Then find the newest version on the release page at GitHub or by curling the API and extracting the version from the response using grep:
VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | grep -Po '"tag_name": "\K.*\d')
Finally, download to your $PATH-accessible location and set permissions:
DESTINATION=/usr/bin/docker-compose
sudo curl -L https://github.com/docker/compose/releases/download/${VERSION}/docker-compose-$(uname -s)-$(uname -m) -o $DESTINATION
sudo chmod 755 $DESTINATION
Following the link to details about versions, you'll find:
Version 2.1
An upgrade of version 2 that introduces new parameters only available
with Docker Engine version 1.12.0+. Version 2.1 files are supported by
Compose 1.9.0+.
https://docs.docker.com/compose/compose-file/compose-versioning/#version-21
You'll need to upgrade your docker-compose install to support the 2.1 file version. https://docs.docker.com/compose/install/
I was getting this error when I was trying the command docker-compose up with the docker compose version 3. Then I run the command to check the docker version docker version, which result the below output.
PS C:\SVenu\M4Movie\Api\Api> docker version
Client:
Version: 18.03.1-ce
API version: 1.37
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:12:48 2018
OS/Arch: windows/amd64
Experimental: false
Orchestrator: swarm
So my version is 18.03, which is the latest one for now. So I moved to this link to check the compose file version which supports this engine. Then changed my docker compose version to 3.6.
version: '3.6'
Everything was working fine after that.
Installation of a new version of Docker Compose helped me to fix this error. See official documentation.

Resources