Google Cloud Build - Different scopes of Dockerfile and cloudbuild.yaml - docker

Question

I recently asked a question about why I get the error Specified public directory 'dist/browser' does not exist, can't deploy hosting to site PROJECT-ID when I´m trying to deploy to Firebase Hosting in my cloudbuild.yaml. However, since I find the question too bloated with information I tried to break it down.
I created a simple image to visualize what happens when I call gcloud builds submit --config=cloudbuild.yaml. So why can´t I access the directory dist/browser from cloudbuild.yaml even though it is processed after the Dockerfile where the directory dist/browser is created?

Answer 1

Cloud Build is best conceptualized as a series of functions (steps) applied to data in the form of a local file system (often just /workspace as this is a default volume mount added to each step, but you can add other volume mounts) and the Internet.
Output of each function (step) is self-contained unless you explicitly publish data back to one of these two sources (one of the step's volume mounts or the Internet).
In this case docker build consumes local files (not shown in your example) and generates dist/browser in the image that results but this folder is only accessible within that image; nothing is added to e.g. /workspace that you could use in subsequent steps.
In order to use that directory subsequently:
Hack a way to mount the (file system of the) image generated by the step and extract the directory from it (not advised; possible not permitted).
You'd need to run that image as a container and then docker cp files from it back into the Cloud Build's (VM's) file system (perhaps somewhere on /workspace).
Not put the directory in an image in the first place (see below)
Proposal
Instead of docker build'ing an image containing the directory, deconstruct the Dockerfile into a series of Cloud Build steps. This way, the artifacts you want (if written somewhere under one of the step's volume mounts), will be available in subsequent steps:
steps:
- name: gcr.io/cloud-builders/npm
args:
- install
- name: gcr.io/cloud-builders/npm
args:
- run
- build:ssr # Presumably this is where dist/browser is generated?
- name: firebase
args:
- deploy # dist/browser
NOTE Every Cloud Build step has an implicit:
- name: some-step
volumes:
- name: workspace
path: /workspace
Proof
Here's a minimal Cloud Build config that uses a volume called testdir that maps to the Cloud Build VM's /testdir directory.
NOTE The example uses testdir to prove the point. Each Cloud Build step automatically mounts /workspace and this could be used instead.
The config:
Lists the empty /testdir
Creates a file freddie.txt in /testdir
Lists /testdir now containing freddie.txt
options:
# volumes:
# - name: testdir
# path: /testdir
steps:
- name: busybox
volumes:
- name: testdir
path: /testdir
args:
- ash
- -c
- "ls -1a /testdir"
- name: busybox
volumes:
- name: testdir
path: /testdir
args:
- ash
- -c
- 'echo "Hello Freddie" > /testdir/freddie.txt'
- name: busybox
volumes:
- name: testdir
path: /testdir
args:
- ash
- -c
- "ls -1a /testdir"
NOTE Uncommenting volumes under options would remove the need to reproduce the volumes in each step.
The edited output is:
gcloud builds submit \
--config=./cloudbuild.yaml \
--project=${PROJECT}
# Lists (empty) /testdir
Starting Step #0
Step #0: Pulling image: busybox
Step #0: .
Step #0: ..
# Creates /test/freddie.txt
Starting Step #1
Step #1: Already have image: busybox
Finished Step #1
# List /testdir containing freddie.txt
Starting Step #2
Step #2: .
Step #2: ..
Step #2: freddie.txt
Finished Step #2