Error Running TensorFlow Serving from Dockerfile - docker

Question

I am working on a container to run TensorFlow Serving.
Here is my Dockerfile:
FROM tensorflow/serving:latest
WORKDIR /
COPY models.config /models/models.config
COPY models/mnist /models/mnist
Here is my models.config for a simple mnist model:
model_config_list {
config: {
name: "mnist",
base_path: "/models/mnist"
model_platform: "tensorflow"
model_version_policy {
specific {
versions: 1646266834
}
}
version_labels {
key: 'stable'
value: 1646266834
}
}
}
The models directory is setup as follows:
$ ls -Rls models
total 0
0 drwxr-xr-x 3 david staff 96 Mar 2 16:21 mnist
models/mnist:
total 0
0 drwxr-xr-x 6 david staff 192 Mar 2 16:21 1646266834
models/mnist/1646266834:
total 304
0 drwxr-xr-x 2 david staff 64 Mar 2 16:21 assets
32 -rw-r--r-- 1 david staff 15873 Mar 2 16:20 keras_metadata.pb
272 -rw-r--r-- 1 david staff 138167 Mar 2 16:20 saved_model.pb
0 drwxr-xr-x 4 david staff 128 Mar 2 16:21 variables
models/mnist/1646266834/assets:
total 0
models/mnist/1646266834/variables:
total 1424
1416 -rw-r--r-- 1 david staff 722959 Mar 2 16:20 variables.data-00000-of-00001
8 -rw-r--r-- 1 david staff 2262 Mar 2 16:20 variables.index
The problem is that when I build and run my container, I receive an error.
$ docker build -t example.com/example-tf-serving:1.0 .
$ docker run -it -p 8500:8500 -p 8501:8501 --name example-tf-serving --rm example.com/example-tf-serving:1.0
The error is as follows Not found: /models/model:
2022-03-03 00:48:06.242923: I tensorflow_serving/model_servers/server.cc:89] Building single TensorFlow model file config: model_name: model model_base_path: /models/model
2022-03-03 00:48:06.243215: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2022-03-03 00:48:06.243254: I tensorflow_serving/model_servers/server_core.cc:591] (Re-)adding model: model
2022-03-03 00:48:06.243899: E tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:365] FileSystemStoragePathSource encountered a filesystem access error: Could not find base path /models/model for servable model with error Not found: /models/model not found
How do I fix my Dockerfile so that the above command will work?
For this solution, the quick and easy way will not work for me, so I cannot accept it as a solution:
docker run --name=the_name -p 9000:9000 -it -v "/path_to_the_model_in_computer:/path_to_model_in_docker" tensorflow/serving:1.15.0 --model_name=MODEL_NAME --port=9000

Answer 1

https://www.tensorflow.org/tfx/serving/docker
Optional environment variable MODEL_NAME (defaults to model)
Optional environment variable MODEL_BASE_PATH (defaults to /models)
You are using default values of these env variables, so Tensorflow is trying to find model in /models/model. You have different model path in the container, so /models/model not found is correct.
I would say simple configuration of MODEL_NAME env variable should solve the problem:
$ docker run -it -p 8500:8500 -p 8501:8501 \
--name example-tf-serving \
-e MODEL_NAME=mnist \
--rm example.com/example-tf-serving:1.0
For multiple models https://www.tensorflow.org/tfx/serving/serving_config#model_server_configuration
The easiest way to serve a model is to provide the --model_name and --model_base_path flags (or setting the MODEL_NAME environment variable if using Docker). However, if you would like to serve multiple models, or configure options like polling frequency for new versions, you may do so by writing a Model Server config file.
You may provide this configuration file using the --model_config_file flag and instruct Tensorflow Serving to periodically poll for updated versions of this configuration file at the specifed path by setting the --model_config_file_poll_wait_seconds flag.
See docker doc: https://www.tensorflow.org/tfx/serving/docker#passing_additional_arguments
You need to set CMD in the Dockerfile (so you don't need to specify it in run time, because requirement is to use only Dockerfile), e.g.:
FROM tensorflow/serving:latest
WORKDIR /
COPY models.config /models/models.config
COPY models/mnist /models/mnist
CMD ["--model_config_file=/models/models.config"]