Could you please point me to the documentation sample showcasing how to put together pytorch dependencies for training on AzureML?
Few related questions to the scenario of running pytorch training workloads on AzureML:
How can I set cuda version to 10.1?
Could you please point to sample demonstrating how to use “official” pytorch docker https://hub.docker.com/r/pytorch/pytorch (which should have all cuda stuff https://github.com/pytorch/pytorch/blob/master/docker/pytorch/Dockerfile)?.
I’ve found distributed-pytorch-with-horovod.yml in the docs but it does not mention any pytorch dependencies -- am I looking in the right place?
Install Pytorch with CUDA Version 10.1 with the following command on windows
pip3 install torch===1.3.1 torchvision===0.4.2 -f https://download.pytorch.org/whl/torch_stable.htm.
From .yml file:
https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/ml-frameworks/pytorch/deployment/train-hyperparameter-tune-deploy-with-pytorch/train-hyperparameter-tune-deploy-with-pytorch.yml
Please follow the below documents for pytorch on Azure.
https://notebooks.azure.com/pytorch
https://azure.microsoft.com/en-us/blog/pytorch-on-azure-full-support-for-pytorch-1-2/
Related
I am trying to run some old matlab code with octave. Unfortunately this code contains a geotiffread function and I think I should change this function with rasterread (package mapping).
However, when I try to install the mapping package I get this warning:
octave:7> pkg install mapping-1.4.0.tar.gz
configure: WARNING: GDAL library not found. Reading of raster files will be disabled.
For information about changes from previous versions of the mapping package, run 'news mapping'.
I tried to run octave (5.2.0 version) within:
a Debian Buster distribution (snap and flatpak package)
a docker container (MacOS 10.15 host, installed from the mtmiller/octave image).
online with the octave-online service, running this code:
pkg load mapping;
[bands, info] = rasterread ('mexutm250.tiff');
With this output:
octave:3> source("my_script.m")
error: gdalread: reading of raster file with GDAL was disabled during installation
error: called from
rasterread at line 56 column 26
my_script at line 2 column 15
No attempt was successful.
EDIT 2: I know that my octave installations are without GDAL support. I would like to use octave with full mapping package, and GDAL support, without recompile it. There is a way to do it (e.g. update a library path within the docker installation to add the libgdal library)?
If there is no way to add GDAL support without recompile octave, there is a guide to do it with minimal effort?
EDIT 3: I already installed the gdal dependencies:
$ sudo aptitude search gdal |grep ^i
[sudo] password for virtuser:
i gdal-bin - Geospatial Data Abstraction Library - programmi di utilità
i A gdal-data - libreria Geospatial Data Abstraction Library - file di dati
i libgdal-dev - libreria Geospatial Data Abstraction Library - file di sviluppo
i libgdal20 - libreria Geospatial Data Abstraction Library
Thank you.
I got octave with GDAL integration when I installed the octave package from the debian repository. I needed octave 5.2, so I switched to Ubuntu 20.04.
as suggested in one of the comments, checking
>> news mapping
(also at https://octave.sourceforge.io/mapping/NEWS.html)
looking at mapping 1.2.1 where rasterread was introduced, it states:
** New features
Reading GIS raster data: A first go is provided using
functions rasterread.m and rasterinfo.m. Both invoke binary
function gdalread() of which an initial version was provided
by Shashank Khare. rasterread.m and rasterinfo.m can read
and return info on any raster data type that the underlying
GDAL library can read. As such, separate functions for e.g.,
GeoTIFF and ArcGrid etc. are not required.
To make use of these functions the GDAL library must be
present on your system => GDAL is a suggested dependency.
You should be able to install the GDAL library in Debian using your preferred installation method.
Unsure whether or not you'll need to uninstall/reinstall the mapping package afterward, but if an unload/reload doesn't get rid of the message, try that and see if mapping is able to see the library.
Using transfer learning, I trained SSD MobileNetV2 (ssd_mobilenet_v2_coco.config) model in TensorFlow (tensorflow-gpu==1.15.0). After freezing the graph (.pb) using TensorFlow API Python script (export_inference_graph.py), I created a text graph (.pbtxt) using the Python script provided in OpenCV wiki (tf_text_graph_ssd.py).
I used the Python code snippet from the wiki to test inference, but I am getting the following error:
cv2.error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\dnn\src\dnn.cpp:562: error: (-2:Unspecified error) Can't create layer "FeatureExtractor/MobilenetV2/expanded_conv_2/add" of type "AddV2" in function 'cv::dnn::dnn4_v20191202::LayerData::getLayerInstance'
I am using Windows 10, Python 3.6.8, and OpenCV 4.2.0.32. I have tried downgrading OpenCV, but earlier versions give different errors.
However, in Ubuntu 18.04.4, installing OpenCV from source, I do not get any errors. Does anybody know if this is an incompatible layer in binary wheels of OpenCV for Windows? Should I wait until the next release?
I downloaded the bullet3 github repository (Bullet Physics SDK) from https://github.com/bulletphysics/bullet3.git which contains python bindings for OpenAI Gym examples to run Reinforcement Learning Algorithms.
I tried running the kuka_grasp_block_playback.py example present in the following directory: bullet3-master/examples/pybullet/examples/
My system does not support pybullet.GUI and so I went ahead with pybullet.DIRECT.
But, on running it using python kuka_grasp_block_playback.py, its giving me the following error:
Traceback (most recent call last):
File "kuka_grasp_block_playback.py", line 92, in <module>
stepIndex = int(p.readUserDebugParameter(stepIndexId))
pybullet.error: Failed to read parameter.
How to deal with this ?
This is an out-of-date question, but just in case:. the latest version of pybullet contains a similar KUKA grasping implementation as OpenAI Gym environment. Just run 'sudo pip install pybullet' to get it.
This gym environment can be run without GUI without issue. See the quickstart guide at http://pybullet.org for more info about those pybullet Gym environments for Deep Reinforcement Learning.
The readUserDebugParameter only executes when using GUI mode, it fails in DIRECT mode, so just comment it out.
Following several instructions on the web, I could install Opencv3.0 and tensorflow on my Ubuntu 16.04. Each tutorial recommends using virtual environment. Although I agree that, problem is that I just followed the tutorials and created separate environments.
** For minor information, Tensorflow installation was easy, but Opencv3.0 was hard.
I used virtualenv for Tensorflow with the name tf, and virtualenvwrapper for Opencv with the name cv, i.e., I activate tf by $ source ~/project/tf/bin/activate, and cv by $ workon cv.
In this case, what is the best way of using both?
Should I activate both always?
Should I enter one environment and install the other again?
Should I symlink site-package/cv.so to tf environment?
I think cv is now in the python site-package folder. I create tf with --site-package option, but it was before installing cv. I am so confusing. Please help.
Yes, I had the same issue and the sym-link between tensorflow and openCV will not work after making the cv.so inside the tf virtualenv after some struggle I got them both to work in the same environment but I suggest uninstalling openCV and then install it back without using a virtual environment for better results.
Cheers.
I am new to Digital Image Processing and want to do it using OpenCV on Eclipse. I just want to know how can i start doing it and how i can configure opencv and eclipse using CMAKE . Please suggest me some good tutorial.Also please help me with adding opencv include files and library in Eclipse.
I am using Eclipse Juno.
on Windows 7.
Thanks.
Your best bet is to begin with the OpenCV documentation. Their Getting Started tutorial should be your first stop. They have another strategy with a custom FindOpenCV.cmake file as documented here but I would suggest sticking with the strategy outlined in Getting Started.
In terms of eclipse, cmake generates IDE related metadata for you and Kitware does provide an eclipse CDT generator, documented here. Two important things to keep in mind. First cmake actually generates the eclipse metadata which you then import as an existing project. Second, the example they give is intended to work with Unix makefiles:
cmake -G"Eclipse CDT4 - Unix Makefiles" -D CMAKE_BUILD_TYPE=Debug ../certi_src
If you are using windows you'll want to choose an appropriate generator instead of "Unix Makefiles"