How to run docker image prepared locally in Sagemaker? - docker

Question

I have a train.py and using a docker pushed an image from local to AWS ECR.
But i am getting this error
The primary container for production variant variant-1 did not pass the ping
health check. Please check CloudWatch logs for this endpoint.
Here is the complete Docker file. What am I missing.
FROM python:3.7
RUN python -m pip install sagemaker-training snowflake-connector-python[pandas] \
pandas scikit-learn boto3 numpy joblib sagemaker flask gevent gunicorn
ENV PYTHONUNBUFFERED=TRUE
ENV PYTHONDONTWRITEBYTECODE=TRUE
ENV PATH="/opt/ml:${PATH}"
# Set up the program in the image
COPY pred_demo_sm/train.py /opt/ml/code/train.py
COPY pred_demo_sm/serve /opt/ml/code/serve
COPY pred_demo_sm/output /opt/ml/output
COPY pred_demo_sm/model /opt/ml/model
WORKDIR /opt/ml
ENTRYPOINT [ "python3.7", "/opt/ml/code/train.py"]
Here is the complete .sh file which builds and pushes the image to AWS ECR
#!/usr/bin/env bash
# This script shows how to build the Docker image and push it to ECR to be ready for use
# by SageMaker.
# The argument to this script is the image name. This will be used as the image on the local
# machine and combined with the account and region to form the repository name for ECR.
image=$1
if [ "$image" == "" ]
then
echo "Usage: $0 <image-name>"
exit 1
fi
chmod +x pred_demo_sm/train.py
chmod +x pred_demo_sm/serve
chmod +x pred_demo_sm/model/*
# Get the account number associated with the current IAM credentials
account=$(aws sts get-caller-identity --query Account --output text)
if [ $? -ne 0 ]
then
exit 255
fi
# Get the region defined in the current configuration (default to us-west-2 if none defined)
region=$(aws configure get region)
region=${region:-us-west-2}
fullname="${account}.dkr.ecr.${region}.amazonaws.com/${image}:latest"
# If the repository doesn't exist in ECR, create it.
aws ecr describe-repositories --repository-names "${image}" > /dev/null 2>&1
if [ $? -ne 0 ]
then
aws ecr create-repository --repository-name "${image}" > /dev/null
fi
# Get the login command from ECR and execute it directly
aws ecr get-login-password --region "${region}" | docker login --username AWS --password-stdin "${account}".dkr.ecr."${region}".amazonaws.com
# Build the docker image locally with the image name and then push it to ECR
# with the full name.
docker build -t ${image}
docker tag ${image} ${fullname}
docker push ${fullname}
The training job gets successfully completed in Sagemaker.
But fails while deploying the model in sagemaker.

Answer 1

Within your Dockerfile, could you try replacing the COPY, WORKDIR, and ENTRYPOINT commands with the following.
COPY pred_demo_sm/opt/program
WORKDIR /opt/program
Make sure that your serve file is executable as well.