Google Cloud Build + Google Secret Manager Substitution Problems - docker

Question

we have a repository that needs to go get a private repo. To do this, we are using an SSH key to access the private repo/module.
We are storing this SSH key using Google Secret Manager and passing it to Docker using the build-arg flag. Now, when we do this locally, the Dockerfile builds and runs as intended. This is the command we use for a local build:
export SSH_PRIVATE_KEY="$(gcloud secrets versions access latest --secret=secret-data)" && \
docker build --build-arg SSH_PRIVATE_KEY -t my-image .
However, when we try to move this setup to Google Cloud Build, we run into 403 forbidden errors from Bitbucket, which leads me to believe that the SSH key is either not being read or formatted correctly.
The full 403 error is:
https://api.bitbucket.org/2.0/repositories/my-repo?fields=scm: 403 Forbidden
Step #0 - "Build": server response: Access denied. You must have write or admin access.
What is even stranger is that when I run the Cloud Build local emulator, it works fine using this command: cloud-build-local --config=builder/cloudbuild-prod.yaml --dryrun=false .
I've tried many different formats and methods, so out of desperation I am asking the community for help. What could be the problem?
Here is our cloudbuild.yaml:
steps:
# Get secret
- id: 'Get Secret'
name: gcr.io/cloud-builders/gcloud
entrypoint: 'bash'
args:
- '-c'
- |
gcloud secrets versions access latest --secret=secret-data > /workspace/SSH_PRIVATE_KEY.txt
# Build
- id: 'Build'
name: 'gcr.io/cloud-builders/docker'
entrypoint: 'bash'
args:
- '-c'
- |
export SSH_PRIVATE_KEY=$(cat /workspace/SSH_PRIVATE_KEY.txt) &&
docker build --build-arg SSH_PRIVATE_KEY -t my-image .

Answer 1

With Cloud Build, when you want to get local linux variable, and not the substitution variable, you have to espace the $ with another $. Look at this:
# Build
- id: 'Build'
name: 'gcr.io/cloud-builders/docker'
entrypoint: 'bash'
args:
- '-c'
- |
export SSH_PRIVATE_KEY=$(cat /workspace/SSH_PRIVATE_KEY.txt)
docker build --build-arg $$SSH_PRIVATE_KEY -t my-image .
The SSH_PRIVATE_KEY is prefixed by $$ to say: don't look at the substitution variable, but look at the linux variable.
I also remove the && at the end of the export line. The pipe | means: Run each command in succession, line return limit each command

Answer 2

Thanks for all the help! This one was pretty weird. Turns out it's not an issue with Cloud Build or Secret Manager but the Dockerfile I was using.
Instead of setting GOPRIVATE with the command in the Dockerfile below, I was using a statement like RUN export GOPRIVATE="bitbucket.org/odds".
In case anyone runs into something like this again, here's the full Dockerfile that works.
FROM golang:1.15.1
WORKDIR $GOPATH/src/bitbucket.org/gml/my-srv
ENTRYPOINT ["./my-srv"]
ARG CREDENTIALS
RUN git config \
--system \
url."https://${CREDENTIALS}#bitbucket.org/".insteadOf \
"https://bitbucket.org/"
RUN go env -w GOPRIVATE="bitbucket.org/my-team"
COPY . .
RUN make build