Is it possible to upload a pre-trained machine learning model (from my local computer, for which I generated a model.pkl) on databricks, and serve it?
Or is it impossible on Databricks ?
I managed it this way.
with mlflow.start_run(run_name="Your_MlFlow_RunName") as mlflow_run:
model=load_model(<filepath_mod>)
model.load_weights(<filepath_weights>)
model.summary()
mlflow.keras.log_model(model,"YourModelName",keras_module=tf.keras,registered_model_name="YourModelName")
But I did upload a zip file, not pickle to also have access to best weights .
import zipfile
with zipfile.ZipFile("/dbfs/FileStore/shared_uploads/<xxx#yyy>/TF_model/MyModel.zip", 'r') as zip_ref:
zip_ref.extractall("/dbfs/FileStore/shared_uploads/<xxx#yyy>/TF_model/")
Also note first time I did databraicks changed some filenames while unzipping, eg. MyModelWeights.data_0000 to MyModelWeights_data_0000 so had to write a fix for that too. Other times it did not happen though, so no idea why
Related
Hallo can someone tell me in what format my input data has to be. Now I have it in csv format with the first column being the target variable but I always get a Algorithm Error which I think is due to wrong input data format.
trainpath = sess.upload_data(
path='revenue_train.csv', bucket=bucket,
key_prefix='production')
testpath = sess.upload_data(
path='revenue_test.csv', bucket=bucket,
key_prefix='production')
# launch training job, with asynchronous call
sklearn_estimator.fit({'train':trainpath, 'test': testpath}, wait=False)
when you use a custom Docker or framework estimator (like you do) you can use any file format (csv, pdf, mp4, whatever you have in S3). The Sklearn container and estimator are agnostic of the file format ; it is the role of your user-provided Python code in the estimator to know how to read those files.
I have a trained decision tree model file music-recommender.joblib. When I am using Jupyter notebook, I am able to load this trained model successfully and able to do prediction. But the same code I used in pycharm and it is showing error.
In Jupyter Notebook
As you can see, it is predicting correctly based on the trained model
In PyCharm
Does anyone knows why is it this way?
You call the joblib.load() method but do not assign the output to any variable. And in the next step you are trying to use the model variable which is not defined anywhere above and hence the error.
You need to do:
model = joblib.load(...)
We are trying to use text classification example from the TensorFlow Examples (tensorflow/examples/learn/text_classification.py) . It works well with db_pedia data.
Now we are trying to save/restore the model using Saver but we are not getting where to use Saver APIs as code in text_classification.py doesn’t use Session at all and Saver API need session to save/restore.
This example uses tf.estimator.Estimator, which has a special method
export_savedmodel for saving.
In addition, you can specify model_dir in constructor:
Directory to save model parameters, graph and etc. This can also be
used to load checkpoints from the directory into a estimator to
continue training a previously saved model. If None, the model_dir in
config will be used if set. If both are set, they must be same. If
both are None, a temporary directory will be used.
I've followed the TensorFlow for Poets tutorial and replaced the stock flower_photos with a few classes of my own. Now I've got my labels.txt file and my graph.pb saved on my local machine.
Is there a way for me to deploy this pre-trained model to Google Cloud Platform? I've been reading the docs and all I can find are instructions on how to create, train, and deploy models from within their ML Engine. But I don't want to spend money training my model on Google's servers when I only need them to host my model so I can call it for predictions.
Anyone else run into the same problem?
Deploying a locally trained model is a supported use case; the instructions are essentially the same regardless of where you trained it:
To deploy a model version you'll need:
A TensorFlow SavedModel saved on Google Cloud Storage. You can get a
model by:
Following the Cloud ML Engine training steps to train in the
cloud.
Training elsewhere and exporting to a SavedModel.
Unfortunately, TensorFlow for Poets does not show how to export a SavedModel (I've filed a feature request to address that). In the meantime, you can write a "converter" script like the following (you could alternatively do this at the end of training instead of saving out graph.pb and reading it back in):
input_graph = 'graph.pb'
saved_model_dir = 'my_model'
with tf.Graph() as graph:
# Read in the export graph
with tf.gfile.FastGFile(input_graph, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
tf.import_graph_def(graph_def, name='')
# CloudML Engine and early versions of TensorFlow Serving do
# not currently support graphs without variables. Add a
# prosthetic variable.
dummy_var = tf.Variable(0)
# Define SavedModel Signature (inputs and outputs)
in_image = graph.get_tensor_by_name('DecodeJpeg/contents:0')
inputs = {'image_bytes':
tf.saved_model.utils.build_tensor_info(in_image)}
out_classes = graph.get_tensor_by_name('final_result:0')
outputs = {'prediction': tf.saved_model.utils.build_tensor_info(out_classes)}
signature = tf.saved_model.signature_def_utils.build_signature_def(
inputs=inputs,
outputs=outputs,
method_name='tensorflow/serving/predict'
)
# Save out the SavedModel.
b = saved_model_builder.SavedModelBuilder(saved_model_dir)
b.add_meta_graph_and_variables(sess,
[tf.saved_model.tag_constants.SERVING],
signature_def_map={'predict_images': signature})
b.save()
(Untested code based on this codelab and this SO post).
If you want the output to use string labels instead of integer indices, make the following change:
# Loads label file, strips off carriage return
label_lines = [line.rstrip() for line
in tf.gfile.GFile("retrained_labels.txt")]
out_classes = graph.get_tensor_by_name('final_result:0')
out_labels = tf.gather(label_lines, ot_classes)
outputs = {'prediction': tf.saved_model.utils.build_tensor_info(out_labels)}
Partial answer only, unfortunately, but I have been able to accomplish this...but with some ongoing issues that I have not yet resolved. I ported over the trained pb and txt files to my server, installed Tensorflow, and am calling the trained model via HTTP request. It works perfectly...on the first run. Then fails every other time.
tensorflow deployment on openshift, errors with gunicorn and mod_wsgi
Surprised there are not more people out there trying to go after this general issue.
I am using a custom image set to train a neural network using Tensorflow API. After successful training process I get these checkpoint files containing values of different training var. I now want to get an inference model from these checkpoint files, I found this script which does that, which I can then use to generate deepdream images as explained in this tutorial. The problem is when I load my model using:
import tensorflow as tf
model_fn = 'export'
graph = tf.Graph()
sess = tf.InteractiveSession(graph=graph)
with tf.gfile.FastGFile(model_fn, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
t_input = tf.placeholder(np.float32, name='input')
imagenet_mean = 117.0
t_preprocessed = tf.expand_dims(t_input-imagenet_mean, 0)
tf.import_graph_def(graph_def, {'input':t_preprocessed})
I get this error:
graph_def.ParseFromString(f.read())
self.MergeFromString(serialized)
raise message_mod.DecodeError('Unexpected end-group tag.')
google.protobuf.message.DecodeError: Unexpected end-group tag.
The script expect a protocol buffer file, I am not sure the script I am using to generate inference models is giving me proto buffer files or not.
Can someone please suggest what am I doing wrong, or is there a better way to achieve this. I simply want to convert checkpoint files generated by tensor to proto buffer.
Thanks
The link to the script you ran is broken, but in any case the recommended thing is not to try to generate an inference model from a checkpoint, but rather to embed code at the end of your training program that will emit a "SavedModel" export (which is not the same thing as a checkpoint).
Please see [1], and in particular the heading "Building a Saved Model". Note that a Saved Model constitutes multiple files, one of which is indeed a protocol buffer (which directly answers your question I hope); the others are variable files and (optional) asset files.
[1] https://www.tensorflow.org/programmers_guide/saved_model