caffe powered and GPU enabled Microsoft Azure VM - machine-learning

I'm trying to build a VM for model training in Azure. I found this Data Science Virtual Machine for Linux (Ubuntu) VM which seems to be a suitable candidate.
Unfortunately, when I spun up the VM and installed the caffe prerequisites I wasn't able to run the tests. I'm getting the following error on make runtest (make all and make test were completed without errors):
NVIDIA: no NVIDIA devices found
Cuda number of devices: 0
Setting to use device 0
Current device id: 0
Current device name:
Note: Randomizing tests' orders with a seed of 97204 .
[==========] Running 2041 tests from 267 test cases.
[----------] Global test environment set-up.
[----------] 11 tests from AdaDeltaSolverTest/3, where TypeParam = caffe::GPUDevice<double>
[ RUN ] AdaDeltaSolverTest/3.TestAdaDeltaLeastSquaresUpdateWithHalfMomentum
NVIDIA: no NVIDIA devices found
E0715 02:24:32.097311 59355 common.cpp:114] Cannot create Cublas handle. Cublas won't be available.
NVIDIA: no NVIDIA devices found
E0715 02:24:32.103780 59355 common.cpp:121] Cannot create Curand generator. Curand won't be available.
F0715 02:24:32.103914 59355 test_gradient_based_solver.cpp:80] Check failed: error == cudaSuccess (30 vs. 0) unknown error
*** Check failure stack trace: ***
# 0x7f77a463f5cd google::LogMessage::Fail()
# 0x7f77a4641433 google::LogMessage::SendToLog()
# 0x7f77a463f15b google::LogMessage::Flush()
# 0x7f77a4641e1e google::LogMessageFatal::~LogMessageFatal()
# 0x7115e3 caffe::GradientBasedSolverTest<>::TestLeastSquaresUpdate()
# 0x7122af caffe::AdaDeltaSolverTest_TestAdaDeltaLeastSquaresUpdateWithHalfMomentum_Test<>::TestBody()
# 0x8e6023 testing::internal::HandleExceptionsInMethodIfSupported<>()
# 0x8df63a testing::Test::Run()
# 0x8df788 testing::TestInfo::Run()
# 0x8df865 testing::TestCase::Run()
# 0x8e0b3f testing::internal::UnitTestImpl::RunAllTests()
# 0x8e0e63 testing::UnitTest::Run()
# 0x466ecd main
# 0x7f77a111c830 __libc_start_main
# 0x46e589 _start
# (nil) (unknown)
Makefile:532: recipe for target 'runtest' failed
make: *** [runtest] Aborted (core dumped)
Is it possible to spin up a virtual machine in Azure suitable for GPU enabled machine learning using caffe?
All the details about the VM here

The Data Science Virtual Machine (DSVM) for Ubuntu already has Caffe installed in /opt/caffe. To use it on a GPU, create a VM with a K80 GPU by choosing the one of the NC sizes. (Be sure to choose HDD as the storage type, or the NC sizes will not appear.) Caffe will then be available out of the box.
Also note that PyCaffe is available. At a terminal:
source activate root
And python will then have PyCaffe available.

Related

xen linux + baremetal cortex A53

I am trying to run Linux (buildroot) + baremetal app on Xen.
Xen boots great and buildroot is ok on the first CPU.
My problem is the baremetal application.
The baremetal app is generated by Vitis (my A53 is in the FPGA).
This is the simplest example application : just an hello world and exit.
I am able to boot directly on this app using the jtag but when I try to run the baremetal app in Xen the app just hand and doesn't print anything.
My configurations :
FILE : xen.cfg
name = "test"
kernel = "/opt/baremetal_app.bin"
memory = 8
vcpus = 1
cpus = [1]
iomem = [ "0xff010,1" ] # UART1
Xen command line :
console=dtuart dtuart=serial0 dom0_mem=1G bootscrub=0 maxcpus=1 timer_slop=0
dom0 command line :
console=hvc0 earlycon=xen earlyprintk=xen maxcpus=1 clk_ignore_unused root=/dev/mmcblk0p2
Vitis is configured to include the hypervisor_guest the in binary.
I already tried to change the psu_ddr_0_MEM_0 to 0x40000000, len 100000.
When I run the command "xl create xen.cfg", the system print "Parsing config from xen.cfg" and returns. I see my app test in "xl list".
The uart1 doesn't print anything and the "xl console test" neither.
With "xl top" my app test seems to use 100% of the cpu.
Any idea ?

ElectronJS Dependency Problem with Buildroot

I'm trying to run ElectronJS app to Buildroot-based Operating System for a vehicle instrument. The SBC is Raspberry Pi 4B and target packages are included based on Debian's dependencies list. Then after I run the app, it throws this message:
# DISPLAY=:0 ./my-app --no-sandbox --disable-gpu-sandbox
./my-app: /usr/lib64/libasound.so.2: no version information available (required by ./my-app)
./my-app: /usr/lib64/libasound.so.2: no version information available (required by ./my-app)
/root/my-app: /usr/lib64/libasound.so.2: no version information available (required by /root/my-app)
/root/my-app: /usr/lib64/libasound.so.2: no version information available (required by /root/my-app)
/root/my-app: /usr/lib64/libasound.so.2: no version information available (required by /root/my-app)
/root/my-app: /usr/lib64/libasound.so.2: no version information available (required by /root/my-app)
[1763:0101/075845.947675:ERROR:bus.cc(392)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
Trace/breakpoint trap
# _
Keep in mind that glxgears and glxinfo are working perfectly. The frame rate is around 198 FPS. Thus, GPU supposed to not be blamed.
I am pretty sure something wrong with ALSA libraries or DBUS, perhaps anyone experiencing same problem? Thanks!
Adding { frame: false, fullscreen: true } in the electron BrowserWindow constructor options make it work.

How to use DNN OPENCL on AMD GPU?

On Windows 10, I want to use GPU as DNN backend to save CPU power. It works for Intel GPU, but there is problem on AMD GPU.
After I use setPreferableTarget(DNN_TARGET_OPENCL), the speed become very slow (much slower than DNN_TARGET_CPU).
I checked with task manager and saw all calculation is on CPU actually.
The following is some log.
[ INFO:0] global D:\work\opencv\opencv\modules\core\src\ocl.cpp (891)
cv::ocl::haveOpenCL Initialize OpenCL runtime...
OpenCV(ocl4dnn): consider to specify kernel configuration cache
directory via OPENCV_OCL4DNN_CONFIG_PATH parameter.
[ INFO:0] global D:\work\opencv\opencv\modules\core\src\ocl.cpp (433)
cv::ocl::OpenCLBinaryCacheConfigurator::OpenCLBinaryCacheConfigurator
Successfully initialized OpenCL cache directory:
C:\Users\wangq\AppData\Local\Temp\opencv\4.3\opencl_cache\
[ INFO:0] global D:\work\opencv\opencv\modules\core\src\ocl.cpp (457)
cv::ocl::OpenCLBinaryCacheConfigurator::prepareCacheDirectoryForContext
Preparing OpenCL cache configuration for context:
Advanced_Micro_Devices__Inc_--Baffin--2906_10
OpenCL program build log: dnn/dummy
Status -66: CL_INVALID_COMPILER_OPTIONS
-cl-no-subgroup-ifp -D AMD_DEVICE
Any help would be appreciated.

NVIDIA Jetson Nano with Realsense 435i via Isaac - Camera not found

I posted about this over on the Isaac forums, but listing it here for visibility as well. I am trying to get the Isaac Realsense examples working on a Jetson Nano with my 435i (firmware downgraded to 5.11.15 per the Isaac documentation), but I've been unable to so far. I've got a Nano flashed with Jetpack4.3 and have installed all dependencies on both the desktop and the Nano. The realsense-viewer works fine, so I know the camera is functioning properly and is being detected by the Nano. However, when I run ./apps/samples/realsense_camera/realsense_camera it throws an error:
ERROR engine/alice/components/Codelet.cpp#229: Component 'camera/realsense' of type 'isaac::RealsenseCamera' reported FAILURE:
No device connected, please connect a RealSense device
ERROR engine/alice/backend/event_manager.cpp#42: Stopping node 'camera' because it reached status 'FAILURE'
I've attached the log of this output as well. I get the same error running locally on my desktop, but that's running through WSL so I was willing to write that off. Any suggestions would be greatly appreciated!
0m2020-06-15 17:18:20.620 INFO engine/alice/tools/websight.cpp#166: Loading websight...0m
33m2020-06-15 17:18:20.621 WARN engine/alice/backend/application_json_loader.cpp#174: This application does not have an explicit scheduler configuration. One will be autogenerated to the best of the system's abilities if possible.0m
0m2020-06-15 17:18:20.622 INFO engine/alice/backend/redis_backend.cpp#40: Successfully connected to Redis server.
0m
33m2020-06-15 17:18:20.623 WARN engine/alice/backend/backend.cpp#201: This application does not have an execution group configuration. One will be autogenerated to the best of the systems abilities if possible.0m
33m2020-06-15 17:18:20.623 WARN engine/gems/scheduler/scheduler.cpp#337: No default execution groups specified. Attempting to create scheduler configuration for 4 remaining cores. This may be non optimal for the system and application.0m
0m2020-06-15 17:18:20.623 INFO engine/gems/scheduler/scheduler.cpp#290: Scheduler execution groups are:0m
0m2020-06-15 17:18:20.623 INFO engine/gems/scheduler/scheduler.cpp#299: __BlockerGroup__: Cores = [3], Workers = No0m
0m2020-06-15 17:18:20.623 INFO engine/gems/scheduler/scheduler.cpp#299: __WorkerGroup__: Cores = [0, 1, 2], Workers = Yes0m
0m2020-06-15 17:18:20.660 INFO engine/alice/backend/modules.cpp#226: Loaded module 'packages/realsense/librealsense_module.so': Now has 45 components total0m
0m2020-06-15 17:18:20.679 INFO engine/alice/backend/modules.cpp#226: Loaded module 'packages/rgbd_processing/librgbd_processing_module.so': Now has 51 components total0m
0m2020-06-15 17:18:20.696 INFO engine/alice/backend/modules.cpp#226: Loaded module 'packages/sight/libsight_module.so': Now has 54 components total0m
0m2020-06-15 17:18:20.720 INFO engine/alice/backend/modules.cpp#226: Loaded module 'packages/viewers/libviewers_module.so': Now has 83 components total0m
90m2020-06-15 17:18:20.720 DEBUG engine/alice/application.cpp#348: Loaded 83 components: isaac::RealsenseCamera, isaac::alice::BufferAllocatorReport, isaac::alice::ChannelMonitor, isaac::alice::CheckJetsonPerformanceModel, isaac::alice::CheckOperatingSystem, isaac::alice::Config, isaac::alice::ConfigBridge, isaac::alice::ConfigLoader, isaac::alice::Failsafe, isaac::alice::FailsafeHeartbeat, isaac::alice::InteractiveMarkersBridge, isaac::alice::JsonToProto, isaac::alice::LifecycleReport, isaac::alice::MessageLedger, isaac::alice::MessagePassingReport, isaac::alice::NodeStatistics, isaac::alice::Pose, isaac::alice::Pose2Comparer, isaac::alice::PoseFromFile, isaac::alice::PoseInitializer, isaac::alice::PoseMessageInjector, isaac::alice::PoseToFile, isaac::alice::PoseToMessage, isaac::alice::PoseTree, isaac::alice::PoseTreeJsonBridge, isaac::alice::PoseTreeRelink, isaac::alice::ProtoToJson, isaac::alice::PyCodelet, isaac::alice::Random, isaac::alice::Recorder, isaac::alice::RecorderBridge, isaac::alice::Replay, isaac::alice::ReplayBridge, isaac::alice::Scheduling, isaac::alice::Sight, isaac::alice::SightChannelStatus, isaac::alice::Subgraph, isaac::alice::Subprocess, isaac::alice::TcpPublisher, isaac::alice::TcpSubscriber, isaac::alice::Throttle, isaac::alice::TimeOffset, isaac::alice::TimeSynchronizer, isaac::alice::UdpPublisher, isaac::alice::UdpSubscriber, isaac::map::Map, isaac::map::ObstacleAtlas, isaac::map::OccupancyGridMapLayer, isaac::map::PolygonMapLayer, isaac::map::WaypointMapLayer, isaac::navigation::DistanceMap, isaac::navigation::NavigationMap, isaac::navigation::RangeScanModelClassic, isaac::navigation::RangeScanModelFlatloc, isaac::rgbd_processing::DepthEdges, isaac::rgbd_processing::DepthImageFlattening, isaac::rgbd_processing::DepthImageToPointCloud, isaac::rgbd_processing::DepthNormals, isaac::rgbd_processing::DepthPoints, isaac::rgbd_processing::FreespaceFromDepth, isaac::sight::AliceSight, isaac::sight::SightWidget, isaac::sight::WebsightServer, isaac::viewers::BinaryMapViewer, isaac::viewers::ColorCameraViewer, isaac::viewers::DepthCameraViewer, isaac::viewers::Detections3Viewer, isaac::viewers::DetectionsViewer, isaac::viewers::FiducialsViewer, isaac::viewers::FlatscanViewer, isaac::viewers::GoalViewer, isaac::viewers::ImageKeypointViewer, isaac::viewers::LidarViewer, isaac::viewers::MosaicViewer, isaac::viewers::ObjectViewer, isaac::viewers::OccupancyMapViewer, isaac::viewers::PointCloudViewer, isaac::viewers::PoseTrailViewer, isaac::viewers::SegmentationCameraViewer, isaac::viewers::SegmentationViewer, isaac::viewers::SkeletonViewer, isaac::viewers::TensorViewer, isaac::viewers::TrajectoryListViewer, 0m
33m2020-06-15 17:18:20.723 WARN engine/alice/application.cpp#164: The function Application::findComponentByName is deprecated. Please use `getNodeComponentOrNull` instead. Note that the new method requires a node name instead of a component name. (argument: 'websight/isaac.sight.AliceSight')0m
0m2020-06-15 17:18:20.723 INFO engine/alice/application.cpp#255: Starting application 'realsense_camera' (instance UUID: 'e24992d0-af66-11ea-8bcf-c957460c567e') ...0m
90m2020-06-15 17:18:20.723 DEBUG engine/gems/scheduler/execution_groups.cpp#476: Launching 0 pre-start job(s)0m
90m2020-06-15 17:18:20.723 DEBUG engine/gems/scheduler/execution_groups.cpp#485: Replaying 0 pre-start event(s)0m
90m2020-06-15 17:18:20.723 DEBUG engine/gems/scheduler/execution_groups.cpp#476: Launching 0 pre-start job(s)0m
90m2020-06-15 17:18:20.723 DEBUG engine/gems/scheduler/execution_groups.cpp#485: Replaying 0 pre-start event(s)0m
0m2020-06-15 17:18:20.723 INFO engine/alice/backend/asio_backend.cpp#33: Starting ASIO service0m
0m2020-06-15 17:18:20.727 INFO packages/sight/WebsightServer.cpp#216: Sight webserver is loaded0m
0m2020-06-15 17:18:20.727 INFO packages/sight/WebsightServer.cpp#217: Please open Chrome Browser and navigate to http://<ip address>:30000m
33m2020-06-15 17:18:20.727 WARN engine/alice/backend/codelet_canister.cpp#225: Codelet 'websight/isaac.sight.AliceSight' was not added to scheduler because no tick method is specified.0m
33m2020-06-15 17:18:20.728 WARN engine/alice/components/Codelet.cpp#53: Function deprecated. Set tick_period to the desired tick paramater0m
33m2020-06-15 17:18:20.728 WARN engine/alice/backend/codelet_canister.cpp#225: Codelet '_check_operating_system/isaac.alice.CheckOperatingSystem' was not added to scheduler because no tick method is specified.0m
33m2020-06-15 17:18:20.728 WARN engine/alice/components/Codelet.cpp#53: Function deprecated. Set tick_period to the desired tick paramater0m
33m2020-06-15 17:18:20.730 WARN engine/alice/components/Codelet.cpp#53: Function deprecated. Set tick_period to the desired tick paramater0m
1;31m2020-06-15 17:18:20.741 ERROR engine/alice/components/Codelet.cpp#229: Component 'camera/realsense' of type 'isaac::RealsenseCamera' reported FAILURE:
No device connected, please connect a RealSense device
0m
1;31m2020-06-15 17:18:20.741 ERROR engine/alice/backend/event_manager.cpp#42: Stopping node 'camera' because it reached status 'FAILURE'0m
33m2020-06-15 17:18:20.743 WARN engine/alice/backend/codelet_canister.cpp#225: Codelet 'camera/realsense' was not added to scheduler because no tick method is specified.0m
0m2020-06-15 17:18:21.278 INFO packages/sight/WebsightServer.cpp#113: Server connected / 10m
0m2020-06-15 17:18:30.723 INFO engine/alice/backend/allocator_backend.cpp#57: Optimized memory CPU allocator.0m
0m2020-06-15 17:18:30.724 INFO engine/alice/backend/allocator_backend.cpp#66: Optimized memory CUDA allocator.0m

Intel Edison + Ubilinux + USB to UART = USB Device not created

Dear community members,
We are working in the development of a robot autonomous control system running inside an Intel Edison, which -in turn- runs in a PixHawk automatic pilot module . This system uses ROS (Robot Operating System) and therefore we had to install Ubilinux. Our system requires being connected with a 360-degree laser (RPLIDAR-360) that sends data through a serial channel, so we are trying to get laser's data through USB-to-UART adapter (Silicon lab's CP2102) (the PixHawk only has a USB available for data transfer).
In summary, the laser is connected to the USB2UART adapter, and the adapter is connected to Edison's serial port.
However, we are stuck with some kind of driver issue. When tracing 'dmesg' the device is detected (idVendor=10c4, idProduct=ea60):
[ 917.812195] usb usb2: Product: xHCI Host Controller
[ 917.812214] usb usb2: Manufacturer: Linux 3.10.17-yocto-standard-r2 dwc-xhci
[ 917.812232] usb usb2: SerialNumber: dwc3-host.2
[ 917.812858] xHCI xhci_add_endpoint called for root hub
[ 917.812878] xHCI xhci_check_bandwidth called for root hub
[ 917.813141] hub 2-0:1.0: USB hub found
[ 917.813185] hub 2-0:1.0: 1 port detected
[ 918.128982] usb 1-1: new full-speed USB device number 2 using dwc3-host
[ 918.151786] usb 1-1: New USB device found, idVendor=10c4, idProduct=ea60
[ 918.151818] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 918.151839] usb 1-1: Product: CP2102 USB to UART Bridge Controller
[ 918.151857] usb 1-1: Manufacturer: Silicon Labs
[ 918.151875] usb 1-1: SerialNumber: 0001
With the following details:
T: Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 2 Spd=12 MxCh= 0
D: Ver= 1.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1
P: Vendor=10c4 ProdID=ea60 Rev= 1.00
S: Manufacturer=Silicon Labs
S: Product=CP2102 USB to UART Bridge Controller
S: SerialNumber=0001
C:* #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=100mA
I:* If#= 0 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=00 Prot=00 Driver=(none)
E: Ad=81(I) Atr=02(Bulk) MxPS= 64 Ivl=0ms
E: Ad=01(O) Atr=02(Bulk) MxPS= 64 Ivl=0ms
But no /dev/ttyUSBX device is created, and there are no error messages about missing Drivers or configuration failures.
We are using Ubilinux 3, which reports the following with the 'uname' command:
** 3.10.17-yocto-standard-r2
And the following is the listing of the installed drivers:
usb
/lib/modules/3.10.17-yocto-standard-r2/kernel/drivers/media/usb:
/lib/modules/3.10.17-yocto-standard-r2/kernel/drivers/media/usb/gspca:
/lib/modules/3.10.17-yocto-standard-r2/kernel/drivers/media/usb/uvc:
/lib/modules/3.10.17-yocto-standard-r2/kernel/drivers/usb:
/lib/modules/3.10.17-yocto-standard-r2/kernel/drivers/usb/gadget:
usb_f_acm.ko
/lib/modules/3.10.17-yocto-standard-r2/kernel/drivers/usb/serial:
usb_wwan.ko
Does anyone has had a similar configuration and managed to make work a CP2102 adapter?
As a last resort, we are trying to compile and install the driver by ourselves, using this reference: [url=https://askubuntu.com/questions/941594/installing-cp210x-driver]Installing CP210x Driver?Installing CP210x Driver? - Ask Ubuntu[/url]
The 'make' process crashes due to a lack of headers. We were unable to download the headers through APT, but at the end, we managed to compile the '.ko' module by downloading the headers manually from a Website.
However, after doing:
insmod cp210x.ko
We got:
Insmod: ERROR: could not insert module cp210x.ko: Invalid module format
So, at this point we have several questions... any advice about any of them would be really appreciated:
Does anybody know how to install the Kernel headers in Ubilinux - 3.10.17?
What is the official procedure to upgrade the Kernel in Ubilinux?, we already have ROS working on the board... do you think a kernel upgrade could mess our ROS configuration?
In general, do you have any successful experience in the configuration of a cp210x device?
Thanks in advance,
Héctor
The driver for cp2102 already exist in kernel, no need of extra efforts. Just use the correct udev rule file (not present by default in linux distro). An example is here. With this file /dev/ttyUSBX will be created.

Resources