How to automatically shutdown a GCE VM after the docker container finishes? - docker

Question

I have a batch process (packaged in a docker image) that needs to run once a day somewhere in the cloud. Google Compute Engine (GCE) seems like a simple and easy to use option for this with it's container-optimized OS VM. This works really well, except that I cannot find an easy way to automatically shutdown the VM after the docker container finishes. You can specify a startup script but GCE seems to execute it before the container is started, so doing docker wait there does not work. I don't care that it's a docker-optimized VM. One of the other basic Linux OSs (like Debian) is fine as long as it can easily be setup with docker.
Is there an easy way of automatically shutting a Linux VM down after the docker container finishes?

Answer 1

As requested in the question comments, Python code to shutdown a Compute Engine instance.
Note: The Google Compute Engine Metadata server can provide the REPLACE variables in the script. Also, you do not need to wait for the results to be STOPPED, just verify that the script did not fail. You can test this program locally as well.
See this answer for tips on COS container credentials. The program below uses ADC (Application Default Credentials).
from pprint import pprint
import time
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials
credentials = GoogleCredentials.get_application_default()
service = discovery.build('compute', 'v1', credentials=credentials)
# Project ID for this request.
project = 'REPLACE_WITH_PROJECT_ID'
# The name of the zone for this request.
zone = 'REPLACE_WITH_COMPUTE_ENGINE_ZONE'
# Name of the instance resource to start.
instance = 'REPLACE_WITH_COMPUTE_ENGINE_INSTANCE_NAME'
request = service.instances().stop(project=project, zone=zone, instance=instance)
response = request.execute()
pprint(response)
print('Waiting for operation to finish...')
print('Name:', response['name'])
while True:
result = service.zoneOperations().get(
project=project,
zone=zone,
operation=response['name']).execute()
print('status:', result['status'])
if result['status'] == 'DONE':
print("done.")
break;
if 'error' in result:
raise Exception(result['error'])
time.sleep(1)

Answer 2

I was able to avoid shutting down the VM from within the docker container by simply deploying a regular Debian VM on Compute Engine. Steps:
First, create a Debian VM on Compute Engine and pass a startup script that installs docker. Startup script:
#!/bin/bash
# Install docker. Taken from the docker documentation.
# Passed into the gcloud command that creates the VM instance.
apt-get -y remove docker docker-engine docker.io containerd runc
apt-get update
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
$(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-get -y install docker-ce docker-ce-cli containerd.io
echo Done installing docker.
Second, replace that startup script with one that pulls and runs the docker image:
#!/bin/bash
...
# Get authorization to pull from artifact registry
gcloud -q auth configure-docker us-east1-docker.pkg.dev
# Pull the image and run it, then shutdown.
docker pull $image
docker rm $container
docker run -v $vol_ini:$app_ini -v $vol_log:$app_log --name $container $image
shutdown now
Now this VM can be scheduled to run the docker image on a daily basis and automatically shut itself down.