Docker volume mounts not working in Azure DevOps Pipeline - docker

Question

Docker volume mounts not working in Azure DevOps Pipeline, please find my code below:
I tried two approaches to run my docker container in the pipeline - please refer below - both returning empty volume - volume mount not happening. I'm not sure what mistake I'm doing here. It would be really appreciated if someone can help me to fix this issue.
I would like to mount run.sh, test.sh and test.txt under /test
In entrypoint.sh - I'm listing all the files inside docker - but it's returning empty - not mounted all these files run.sh, test.sh and test.txt
I'm struggling for the last two days to fix this issue but not getting any resolution - any help would be really appreciated.
This is my folder structure:
my-app/
├─ test/
│ ├─ run.sh
│ ├─ test.sh
│ ├─ test.txt
├─ azure-pipeline.yml
test.sh
#!/bin/bash
rootPath=$1
echo "Root path: $rootPath"
./run.sh $rootPath
run.sh
#!/bin/bash
echo "starting run script"
NAME="testApp"
IMAGE="sample/test-app:1.1"
ROOTPATH=$1
echo "$ROOTPATH"
# Finally run
docker stop $NAME > /dev/null 2>&1
docker rm $NAME > /dev/null 2>&1
docker run --name $NAME -i -v $ROOTPATH:/test -w /test $IMAGE
azure-pipeline.yml (Approach -1)
trigger:
- none
jobs:
- job: test
pool:
name: my-Linux-agents
displayName: Run tests
steps:
- task: Bash#3
displayName: Docker Prune
inputs:
targetType: inline
script: |
docker system prune -f -a
- task: Docker#2
displayName: Docker Login
inputs:
containerRegistry: myRegistry w/ asdf
command: login
- task: Bash#3
displayName: Execute Sample Java
inputs:
targetType: filePath
filePath: 'test/test.sh'
arguments: '$PWD'
workingDirectory: test
azure-pipeline.yml (Approach -2)
trigger:
- none
jobs:
- job: test
pool:
name: my-Linux-agents
displayName: Run tests
steps:
- task: Bash#3
displayName: Docker Prune
inputs:
targetType: inline
script: |
docker system prune -f -a
- task: Docker#2
displayName: Docker Login
inputs:
containerRegistry: myRegistry w/ asdf
command: login
- bash: |
echo "Executing docker run command"
echo $(Build.SourcesDirectory)
echo $PWD
docker run --name testApp -i -v $(Build.SourcesDirectory):/test -w /test sample/test-app:1.1
My Docker Image - files
Dockerfile
FROM alpine:3.12
COPY entrypoint.sh /
RUN echo "hello"
ENTRYPOINT ["/entrypoint.sh"]
entrypoint.sh
#!/bin/sh
echo "START Running Docker"
echo "Listing Files"
ls -la

Answer 1

TL;DR
Docker volume mounts work in Azure DevOps Pipelines (at least on the Microsoft Hosted Agents).
As can be seen below both the approaches described in the OP works with the ubuntu-latest agent pool. If self hosted agents are used in my-Linux-pool the problem is likely with them rather than with the Dockerfile or pipeline config that was shared in the original post.
Please see the full working demo with git repo and Pipeline
EDIT: If the self hosted pipeline agents are run as docker containers, the problems could come from that the inner container references a path that exists only in the outer container, but not on the host. For more details on how to mount volumes when launching a docker container from another docker container, please see the section on Mounting volumes using Docker within a Docker container from the Azure Pipeline docs, or this answer from Mounting docker run in Azure Pipeline job
Test setup
I have modified the azure-pipelines.yaml to create a fully self contained example:
The following changes were made to the azure-pipelines.yml in the OP
Setting the agentpool to ubuntu-latest rather than my-Linux-agents
Changing the Docker login step to a docker build step that actually builds the image as part of the pipeline. (Not necessary, but it makes the pipeline independent of a custom image registry)
Add a step that lists all the files in the repo recursively and their permission (So that we can easily verify that all files in the /test folder are readable by all)
Adding the steps that runs the containers from the two approaches into the same pipeline so that both approaches are demonstrated in the same pipeline
The azure-pipelines.yml now looks like this:
trigger:
- none
jobs:
- job: test
pool:
vmImage: 'ubuntu-latest'
displayName: Run tests
steps:
- task: Docker#2
displayName: Build Docker Image
inputs:
repository: sample/test-app
command: build
Dockerfile: image/Dockerfile
tags: '1.1'
- bash: |
echo $(Build.SourcesDirectory)
ls -lrtR
displayName: List files
- bash: |
echo "Executing docker run command"
docker run --name testApp -i -v $(Build.SourcesDirectory)/test:/test -w /test sample/test-app:1.1
displayName: Run docker inline in pipeline
- task: Bash#3
displayName: Run test.sh
inputs:
targetType: filePath
filePath: 'test/test.sh'
arguments: '$PWD'
workingDirectory: test
The rest of the files looks the same, the full test setup can be found here
Test Results
When running the pipeline the following output is obtained (Full pipeline run can be found here here
List Files
/home/vsts/work/1/s
.:
total 16
drwxr-xr-x 2 vsts docker 4096 Sep 18 16:51 test
drwxr-xr-x 2 vsts docker 4096 Sep 18 16:51 image
-rw-r--r-- 1 vsts docker 849 Sep 18 16:51 azure-pipelines.yml
-rw-r--r-- 1 vsts docker 198 Sep 18 16:51 README.md
./test:
total 8
-rw-r--r-- 1 vsts docker 0 Sep 18 16:51 test.txt
-rwxr-xr-x 1 vsts docker 72 Sep 18 16:51 test.sh
-rwxr-xr-x 1 vsts docker 258 Sep 18 16:51 run.sh
./image:
total 8
-rwxr-xr-x 1 vsts docker 67 Sep 18 16:51 entrypoint.sh
-rwxr-xr-x 1 vsts docker 87 Sep 18 16:51 Dockerfile
Run docker inline in pipeline
Executing docker run command
START Running Docker
Listing Files
total 16
drwxr-xr-x 2 1001 121 4096 Sep 18 16:51 .
drwxr-xr-x 1 root root 4096 Sep 18 16:51 ..
-rwxr-xr-x 1 1001 121 258 Sep 18 16:51 run.sh
-rwxr-xr-x 1 1001 121 72 Sep 18 16:51 test.sh
-rw-r--r-- 1 1001 121 0 Sep 18 16:51 test.txt
Run test.sh
Root path: /home/vsts/work/1/s/test
starting run script
/home/vsts/work/1/s/test
START Running Docker
Listing Files
total 16
drwxr-xr-x 2 1001 121 4096 Sep 18 16:51 .
drwxr-xr-x 1 root root 4096 Sep 18 16:51 ..
-rwxr-xr-x 1 1001 121 258 Sep 18 16:51 run.sh
-rwxr-xr-x 1 1001 121 72 Sep 18 16:51 test.sh
-rw-r--r-- 1 1001 121 0 Sep 18 16:51 test.txt

Answer 2

i believe solution is relatively simple, idea derived from here,
give this a try:
in pipelines
pool:
vmImage: 'ubuntu-latest'
in Dockerfile
VOLUME ["/test"]
RUN ls -la >> my.log
and then check log