I'm planning a TFF scheme in which the clients send to the sever data besides the weights, like their hardware information (e.g CPU frequency). To achieve that, I need to call functions of third-party python libraries, like psutils. Is it possible to serialize (using tff.tf_computation) such kind of functions?
If not, what could be a solution to achieve this objective in a scenario where I'm using a remote executor setting through gRPC?
Unfortunately no, this does not work without modification. TFF uses TensorFlow graphs to serialize the computation logic to run on remote machines. TFF does not interpret Python code on the remote machines.
There maybe a solution by creating a TensorFlow custom op. This would mean writing C++ code to retrieve CPU frequency, and then a Python API to add the operation to the TensorFlow graph during computation construction. TensorFlow's guide for Create an op can provide detailed instructions.
Related
Does TFF have any way to save and load optimizer state similar to model weights. For model weights there are ModelWeights.assign_weights_to() and tff.learning.state_with_new_model_weights() functions, Is there a way to save and load optimizer state especially when using server side optimizer other than SGD.
I could not find anything to save and load state of optimizer.
Piggybacking off of Keith's answer: The checkpoint manager is no longer available in Google's federated research repository. It has been upstreamed to TFF (see it on GitHub here).
You can access it either through the tensorflow-federated-nightly pip package, or else by cloning the repository.
The code essentially delegates to tf.saved_model.save, so you could alternatively simply use this.
This should be achievable with TFF's tff.simulation.FileCheckpointManager. In particular, the usage in Google Research's federated repo was originally written to support restarting from checkpoints when using learning rate scheduling and adaptive optimization on the server , an application for which correctly restoring the optimizer state is crucial.
As long as your tff.templates.IterativeProcess returns the appropriate optimizer state, simply using the FileCheckpointManager out of the box should just work.
I was wondering whether I could leverage the modularity drake gives to test Visual SLAM algorithms on realtime data. I would like to create 3 blocks that output acceleration, angular speed, and RGBD data. The blocks should pull information from a real sensor. Another block would process the data and produce the current transform of the camera and a global map. Effectively, I would like to cast my problem into a "Systems" framework so I can easily add filters where I need them.
My question is: Given other people's experience with this library, is Drake the right tool for the job for this usecase? Specifically, can I use this library to process real time information in a production setting?
Visual SLAM is not a use case I've implemented myself, but I believe the Drake Systems framework should be up to the task, depending on what you mean by "realtime".
We definitely ship RGBD data through the framework often.
We haven't made any attempt to support running Drake in hard realtime, but certainly can run at high rates. If you were to hit a performance bottleneck, we tend to be pretty responsive and would welcome PRs.
As for the "production-level", it is certainly our intention for the code / process to be mature enough for that setting, and numerous teams do already.
I would like to set up a prediction task, but the data preprocessing step requires using tools outside of Python's data science ecosystem, though Python has APIs to work with those tools (e.g. a compiled java NLP tool set). I first thought about creating a Docker container to have an environment with those tools available, but a commentator has said that that is not currently supported. Is there perhaps some other way to make such tools available to the Python prediction class needed for AI Platform? I don't really have a clear sense of what's happening on the backend with AI platform, and how much ability a user has to modify or set that up.
Not possible today. Is there any specific use case you are targeting not satisfied today?
Cloud AI platform offers multiple prediction frameworks (TensorFlow, scikit-learn, XGboost, Pytorch, Custom predictions) in multiple versions.
After looking into the requirements you can use the new AI Platform feature custom prediction, https://cloud.google.com/ml-engine/docs/tensorflow/custom-prediction-routine-keras
To deploy a custom prediction routine to serve predictions from your trained model, do the following:
Create a custom predictor to handle requests
Package your predictor and your preprocessing module. Here you can install your custom libraries.
Upload your model artifacts and your custom code to Cloud Storage
Deploy your custom prediction routine to AI Platform
The Watson Machine Learning service provides three options for training deep learning models. The docs list the following:
There are several ways to train models Use one of the following
methods to train your model:
Experiment Builder
Command line interface (CLI)
Python client
I believe these approaches will differ with their (1) maturity and (2) the features they support.
What are the differences in these approaches? To ensure this question meets the quality requirements, can you please provide a objective list of the differences? Providing your answer as a community wiki answer will also allow the answer to be updated over time when the list changes.
If you feel this question is not a good fit for stack overflow, please provide a comment listing why and I will do my best to improve it.
The reasons to use these techniques depends on a user's skillset and how they are fitting the training/monitoring/deploying steps into their workflow:
Command Line Interface (CLI)
The CLI is useful for quick and random access to details about your training runs. It's also useful if you're building a data science workflow using shell scripts.
Python Library
WML's python library allows users to integrate their model training+deployment into a programmatic workflow. It can be used both within notebooks as well as via IDEs. The library has become the most widely used way for executing batch training experiments.
Experiment Builder UI
This is the "easy button" for executing batch training experiments within Watson Studio. It's a quick way to learn the basics of the batch training capabilities in Watson Studio. At present, it's not expected that data scientists would use Experiment Builder as their primary way of starting batch training experiments. Perhaps as Model Builder matures, this could change but the Python library is more flexible for integrating into production workflows.
What's the best way to run machine learning algorithms on Google Cloud Dataflow? I can imagine that using Mahout would be one option given it's Java based.
The answer is probably no, but is there a way to invoke R or Python (that have strong support for algorithms) based scripts to offload ML execution?
-Girish
You can already implement many algorithms in terms of Dataflow transforms.
A class of algorithms that may not be as easy to implement are iterative algorithms, where the pipeline's execution graph depends on the data itself. Simplifying implementation of iterative algorithms is something that we are interested in, and you can expect future improvements and simplifications in this area.
Invoking Python (or any other) executable shouldn't be hard from a Dataflow pipeline. A ParDo can, for example, shell out and start an arbitrary process. You can use, for example, --filesToStage pipeline option to add additional files to the Dataflow worker environment.
There is also http://quickml.org/ (haven't used personally) and Weka. I remember the docs mention that it's possible to launch a new process from within the job, but AFAIK it's not recommended.