cuda torch integration - CudaTensor class not found - lua

Probably, I'm not sufficiently familiar with the dependencies stack here, but I've installed a Deep Learning project that uses Torch and CUDA: https://github.com/donglixp/lang2logic.
Running the project (e.g. by issuing: ./pretrain.sh seq2seq jobqueries lstm), I get this error:
THCudaCheck FAIL file=/tmp/luarocks_cutorch-scm-1-1028/cutorch/lib/THC/THCGeneral.c line=66 error=30 : unknown error
package cunn not found!
package cutorch not found!
If cutorch and cunn are installed, your CUDA toolkit may be improperly configured.
Check your CUDA toolkit installation, rebuild cutorch and cunn, and try again.
Falling back on CPU mode
~/torch/install/share/lua/5.1/torch/File.lua:343: unknown Torch class <torch.CudaTensor>
The last error by the way, is issued from this self-explanatory piece of lua code, which checks for the availability of the CudaTensor class:
if not torch.factory(className) then
error(string.format('unknown Torch class <%s>', tostring(className)))
end
After getting this error, I also installed Lua's cutorch and cunn through the following commands, which seemed to have finished well.
luarocks install cutorch
luarocks install cunn
But I still get the very same error.
The only installation quirk I've noticed had been that luarocks install class seemed to do nothing, and I'm not running on Scientific Linux as the original author, but rather on Ubuntu 16.04.
I had installed CUDA itself through: sudo apt-get install nvidia-cuda-toolkit
How would you address this integration error?
Thanks!

I would have preferred leaving only a comment, but I do not have enough reputation for that yet so here goes.
It seems like you have installation issues. The most obvious thing to try would be to uninstall torch entirely via rm -rf ~/torch and reinstall it following your link's instructions (replacing ClassNLLCriterion.cu file with theirs). Usually, following the instruction here should install cutorch and cunn for you but you can always install them via luarocks as you mentioned. This is also why luarocks install class didn't do anything because it was already installed.
In answer to your comment, torch.CudaTensor as the name suggests is a torch class defined in cutorch. It acts just like any other Tensor class from the torch library but it lives on GPU.
More information on how you installed torch could help.
You also of course need a CUDA compatible device in order to use cutorch.

Related

Cannot import opencv after succesfull install on Raspberry Pi

I try to install opencv on my Raspberry Pi with 4GB RAM (Raspberry OS version Linux 5.15.84-v8+ aarch64). Instructions I found on q-engineering. After a 2hr lasting build process the make announced that opencv was succesfully installed. When I try to import it I get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'cv2'
But opencv appears to be succesfully installed. When I run opencv_version I got:
$ opencv_version
4.7.0-dev
and:
/usr/local/bin/opencv_visualisation
/usr/local/bin/opencv_annotation
/usr/local/bin/opencv_version
/usr/local/bin/opencv_model_diagnostics
/usr/local/bin/opencv_waldboost_detector
/usr/local/bin/opencv_interactive-calibration
/usr/local/share/licenses/opencv4
/usr/local/share/opencv4
/usr/local/lib/aarch64-linux-gnu/cmake/opencv4
/usr/local/lib/aarch64-linux-gnu/pkgconfig/opencv4.pc
/usr/local/include/opencv4
/usr/local/include/opencv4/opencv2
/usr/local/include/opencv4/opencv2/opencv.hpp
/usr/local/include/opencv4/opencv2/gapi/opencv_includes.hpp
/usr/local/include/opencv4/opencv2/opencv_modules.hpp
Looking for cv2 or *cv2* yielded nothing except what was already listed above. The date/time of the files conform to the time I ran the install script.
Next I tried opencv-lite according to instructions from the same source. Same result: succesful installation, traces of newly put opencv files but import cv2 still fails.
To complicate matters I use anaconda environments. I got two: base and rpi. I installed with rpi activated. In both no trace of opencv or cv2 and in both cases import fails. Some information where the system fetches python:
$ which python
/home/arnold/bin/anaconda/envs/rpi/bin/python
(rpi) arnold#bottie:~/bin/opencv/opencv/build
There is python in /bin/python and /usr/bin/python I found out with vscode. I tried both and both failed.
My questions:
did installation of opencv really succeed?
if so, is it possible to 'connect' the installation to a running version of python, preferably a conda environment?
if not, what is the best way to proceed?
I try this links. install OpenCV 4 for RPI 4 It worked for me using RPi 4B/8GB 64-bit.
Do update first
Install OpenCV
Check getBuildInformation()
After that if not getting . Then install sudo apt
pyhton3-opencv-contrib
No needed to do cmake. And you ready to go.

Error when installing the pyqt5 package into docker image (Host: mac os m1) [duplicate]

A program I am trying to install requires the installation of PyQt5 5.15.0 , which gives me this error. The odd thing is that the installation works fine for the latest version of PyQt5 (5.15.2), but this program requires 5.15.0 specifically.
Command Output:
Collecting PyQt5==5.15.0
Using cached PyQt5-5.15.0.tar.gz (3.3 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\mshal\appdata\local\programs\python\python39\python.exe' 'C:\Users\mshal\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py' prepare_metadata_for_build_wheel 'C:\Users\mshal\AppData\Local\Temp\tmp41s11ev6'
cwd: C:\Users\mshal\AppData\Local\Temp\pip-install-sfw90hvc\pyqt5_e2cc46859b554da7b84798abae5378ba
Complete output (31 lines):
Traceback (most recent call last):
File "C:\Users\mshal\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py", line 126, in prepare_metadata_for_build_wheel
hook = backend.prepare_metadata_for_build_wheel
AttributeError: module 'sipbuild.api' has no attribute 'prepare_metadata_for_build_wheel'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\mshal\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py", line 280, in <module>
main()
File "C:\Users\mshal\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\Users\mshal\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py", line 130, in prepare_metadata_for_build_wheel
return _get_wheel_metadata_from_wheel(backend, metadata_directory,
File "C:\Users\mshal\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py", line 159, in _get_wheel_metadata_from_wheel
whl_basename = backend.build_wheel(metadata_directory, config_settings)
File "C:\Users\mshal\AppData\Local\Temp\pip-build-env-nnx_yu09\overlay\Lib\site-packages\sipbuild\api.py", line 51, in build_wheel
project = AbstractProject.bootstrap('pep517')
File "C:\Users\mshal\AppData\Local\Temp\pip-build-env-nnx_yu09\overlay\Lib\site-packages\sipbuild\abstract_project.py", line 83, in bootstrap
project.setup(pyproject, tool, tool_description)
File "C:\Users\mshal\AppData\Local\Temp\pip-build-env-nnx_yu09\overlay\Lib\site-packages\sipbuild\project.py", line 479, in setup
self.apply_user_defaults(tool)
File "project.py", line 62, in apply_user_defaults
super().apply_user_defaults(tool)
File "C:\Users\mshal\AppData\Local\Temp\pip-build-env-nnx_yu09\overlay\Lib\site-packages\pyqtbuild\project.py", line 79, in apply_user_defaults
super().apply_user_defaults(tool)
File "C:\Users\mshal\AppData\Local\Temp\pip-build-env-nnx_yu09\overlay\Lib\site-packages\sipbuild\project.py", line 225, in apply_user_defaults
self.builder.apply_user_defaults(tool)
File "C:\Users\mshal\AppData\Local\Temp\pip-build-env-nnx_yu09\overlay\Lib\site-packages\pyqtbuild\builder.py", line 66, in apply_user_defaults
raise PyProjectOptionException('qmake',
sipbuild.pyproject.PyProjectOptionException
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\mshal\appdata\local\programs\python\python39\python.exe' 'C:\Users\mshal\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py' prepare_metadata_for_build_wheel 'C:\Users\mshal\AppData\Local\Temp\tmp41s11ev6' Check the logs for full command output.
I am on the latest version of pip. Any ideas on the root cause of this issue?
What helped me is upgrading pip from 20.2.3 to the latest one (in my case 21.1.1)
For Mac/Homebrew users.
The answer by #the-x is leading in the right direction. On a Mac with QT5 installed via Homebrew the qmake binary just needs to be added to the path. This can be achieved through
export PATH="/opt/homebrew/opt/qt5/bin:$PATH"
(of course depending on where the homebrew files are installed)
Running on arm with python3.6 (ubuntu18 on nvidia Xavier):
sudo apt install qt5-default
For MacOS users.
I am on Apple M1 silicon using Python 3.9.8. What worked for me was #Apaul's comment in the original question section. Install pyqt5-sip prior to pyqt5.
I also have an Intel Mac and on that machine, I do not need to do this.
Checking the binaries that PyQt5 provides in pypi for version 5.15.0 I see that it does not provide the binaries for python3.9 in windows, so pip is trying to compile using the source code which is complicated and can generate several dependency problems (for example you must have Qt 5.15 installed, etc). So my recommendation is to install a more updated version of PyQt5, for example 5.15.2 since if it provides the binaries for python3.9 on windows, in addition to being a wrapper of an LTS version of Qt then it will have solved several bugs.
python -m pip install PyQt5==5.15.2
Another solution is to use python3.8 instead of python3.9 so that you can install pyqt5 5.15.0 from pypi without problems.
Upgrading your pip enables you to install PyQt5. Personally, I had the same issue while installing PyQt6 and I upgraded my pip, and everything installed perfectly. I think both python and pip versions play an important role in installing PyQt so make sure you have later versions.
This is the command I used in Linux:
pip install --upgrade pip
Combining several answers on this question: On an Apple M1 Pro Macbook with macOS Ventura 13.0.1, with Homebrew 3.6.17 and python 3.11.0 the following commands fixed it for me (no sudo):
brew install qt5
export PATH="/opt/homebrew/opt/qt5/bin:$PATH"
python3 -m ensurepip --default-pip
pip3 install pyqt5-sip
pip3 install pyqt5 --config-settings --confirm-license= --verbose
That last step calls qmake to compile all of Qt on your M1 and takes many minutes to complete, be patient and let it finish.
Since qt5-default was not available, I installed qt5-default's dependencies
sudo apt-get install qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools
after that I installed pyqt5 via apt-get first and afterwards via pip
sudo apt-get install pyqt5-dev
pip install pyqt5
now wheel seems to work
side-note:
I am not sure if sudo apt-get install pyqt5-dev is even necessary
The error message thrown here is misleading - it's not an issue with a sipbuild.api attribute. Indeed, in this case program qmake is missing, see last line of the Python traceback. Have a look if it's installed on your system and add it to your PATH variable. Otherwise, install it. On Linux this would be done with
sudo apt-get install qt5-qmake
I had this problem on my M1 Mac using Python 3.9.12 when I was trying to install a library: pip install pixellib.
The first thing I did was: pip install pixellib --verbose to see the whole log, and there I noticed that PyQt5 was waiting for an input. So then I found someone else with that issue, and used pip install pyqt5 --config-settings --confirm-license= --verbose which took some time to compile, but worked!
I could not get any of the above solutions to work but I managed to get it working using python3.9, PyQt5=5.15.2, pip=22.0.2 and sip=6.5.0 by using sudo apt-get install PyQt5. If you need it in a virtual environment, you can manually copy the PyQt5 folder from your default /usr/lib/python3/dist-packages to the site-packages folder in your virtual environment.
To all those that are struggling with Apple M1 installation, here is a working solution, specifically addressing the problem of installing the pixellib library that depends on PyQt5 but you can apply it equally to other libs:
PyQt5 is not supported on Apple M1, it needs qt6: https://www.reddit.com/r/learnpython/comments/o4w1ut/comment/h2jele3/?utm_source=share&utm_medium=web2x&context=3 , https://www.qt.io/product/qt6
this means you need to install PyQt6: python3 -m pip install PyQt6
go to the lib you need, in my case pixellib: https://pypi.org/project/pixellib/#files and
download the wheel file
get the wheel tool: pip install wheel
unpack the wheel wheel unpack pixellib-0.7.1-py3-none-any.whl
Change its dependency of PyQt5 to PyQt6
edit pixellib-0.7.1/pixellib-0.7.1.dist-info/METADATA
pyQt5 => pyQt6
pack it back wheel pack pixellib-0.7.1
install it: pip install pixellib-0.7.1-py3-none-any.whl
test in python: `
# should work
import pixellib
P.S. thanks to Terra and ChaOS for supporting work on the project underlying this report.
I finally managed to make it works on M1/M2 Macbook Pro.
None of these answers worked for me, so I looked at brew to install pyqt.
The following command will install pyqt5 via brew:
brew install pyqt#5
Then it just worked.
This can be resolved by switching to an environment with Python >= 3.8

Error: yq#3 has been disabled because it is not maintained upstream! How to install a disabled brew Formulae?

I'm trying to install yq#3 on my Mac running brew install yq#3 and I get the error:
Error: yq#3 has been disabled because it is not maintained upstream!
I see that it's there on their website at https://formulae.brew.sh/formula/yq#3#default but it doesn't seem to be supported anymore.
I still need to install it since our projects at work are using this specific version.
The only way that I'm thinking about is downloading the source code, building it myself, and adding it to the path but I'm thinking that there might be a simpler solution.
Any suggestion?
Thanks!
From yq github, you can install a binary by running:
wget https://github.com/mikefarah/yq/releases/download/3.4.1/yq_darwin_amd64 -O /usr/local/bin/yq &&\
chmod +x /usr/local/bin/yq
3.4.1 is the latest 3 version, darwin_amd64 is the Mac package (don't worry about having an Intel machine and installing the package that says AMD, the name comes from something about AMD invented the 64-bit instruction set).

Erlang installation giving erlang-mode is broken error

I am trying to install Elixir from the elixir website on Ubuntu 14.04. I get the following error when I execute $ sudo apt-get install esl-erlang. Can anyone help me out here ?
$ sudo apt-get install esl-erlang
Reading package lists... Done
Building dependency tree
Reading state information... Done
esl-erlang is already the newest version.
The following packages were automatically installed and are no longer required:
libgconf2-4 python-requests-whl python-setuptools-whl python-six-whl
python-urllib3-whl python-wheel
Use 'apt-get autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 406 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up erlang-mode (1:21.0.5-1) ...
ERROR: erlang-mode is broken - called emacs-package-install as a new-style add-on, but has no compat file.
Install emacsen-common for emacs23
emacsen-common: Handling install of emacsen flavor emacs23
Wrote /etc/emacs23/site-start.d/00debian-vars.elc
Wrote /usr/share/emacs23/site-lisp/debian-startup.elc
Install emacsen-common for emacs24
emacsen-common: Handling install of emacsen flavor emacs24
Wrote /etc/emacs24/site-start.d/00debian-vars.elc
Wrote /usr/share/emacs24/site-lisp/debian-startup.elc
Install erlang-mode for emacs
Install erlang-mode for emacs23
install/erlang: Handling install for emacsen flavor emacs23
Wrote /usr/share/emacs23/site-lisp/erlang/erlang-edoc.elc
Wrote /usr/share/emacs23/site-lisp/erlang/erlang-eunit.elc
Wrote /usr/share/emacs23/site-lisp/erlang/erlang-flymake.elc
Wrote /usr/share/emacs23/site-lisp/erlang/erlang-skels-old.elc
Wrote /usr/share/emacs23/site-lisp/erlang/erlang-skels.elc
Wrote /usr/share/emacs23/site-lisp/erlang/erlang-start.elc
Wrote /usr/share/emacs23/site-lisp/erlang/erlang.elc
Wrote /usr/share/emacs23/site-lisp/erlang/erlang_appwiz.elc
In toplevel form:
erldoc.el:64:1:Error: Cannot open load file: cl-lib
Wrote /usr/share/emacs23/site-lisp/erlang/path.elc
ERROR: install script from erlang-mode package failed
dpkg: error processing package erlang-mode (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
erlang-mode
E: Sub-process /usr/bin/dpkg returned an error code (1)
Thanks
I don't know what is causing your errors, but you could try ASDF with the Elixir plugin. This allows you to install multiple versions of Elixir and its dependencies, and uses your user environment with shims instead of installing to system directories. This avoids many of the dependency and permission problems that comes with performing a single system-level installation.
It looks like you have both emacs23 and emacs24 installed. The failure is happening when trying to build for emacs23. Unless you really want emacs23 for some reason, remove it and then it should install fine:
sudo apt-get remove emacs23
It is possible you might have to delete some of the other emacs23-* packages by hand.
This problem only affects the erlang-mode package, which adds support for editing Erlang code in Emacs. If you're not using Emacs (or not developing Erlang in it), you can just uninstall that package:
sudo apt remove erlang-mode
As noted in Penguin Brian's answer, this happens when installing erlang-mode on a system using Emacs 23, as erlang-mode only supports Emacs 24 or newer. (Looks like this is a bug in the package: it shouldn't try to compile itself for unsupported Emacs versions.) If you do want to use erlang-mode in Emacs, I'd suggest running Emacs 24 and installing the Erlang mode from MELPA.

Apt-get install package for libmapscript-ruby1.8 not behaving as anticipated

TL;DR: How can I get the right package for libmapscript-ruby1.8 on my system?
Context:
We have found an open source Rails 2.3 app that solves an internal tool problem.
https://github.com/timwaters/mapwarper
Additional instructions
https://github.com/l34marr/mapwarper/blob/master/README#L125
I've some experience with Rails but am just learning about the Ubuntu eco-system and apt-get.
Problem:
The perceived problem is that one of the external libraries (mapscript) is not functioning.
Further detail:
The perceived source of the problem is that apt-get install libmapscript-ruby1.8 does not seem to load a ruby1.8 version of mapscript. Instead it loads to /usr/lib/ruby/vendor_ruby/1.9.1/x86_64-linux/mapscript.so via dependencies of libmapscript-ruby1.8
So when the Rails app links to the installed mapscript.so, it breaks, since the syntax is presumably different between mapscript.so build for 1.8.7 and 1.9.1.
Example error (note 1.9.1 version of mapscript.so is copied into 1.8.7 folder here)
TypeError (wrong argument type swig_runtime_data (expected Struct)):
/home/ubuntu/.rvm/rubies/ruby-1.8.7-p374/lib/ruby/site_ruby/1.8/x86_64-linux/mapscript.so
Contact with the application creator has been largely fruitless so far, since they have not encountered this specific issue.
There is a rubygem but it seems to also be for 1.9.1
https://github.com/sourcepole/ruby_mapscript
Is there some apt-get magic that I am missing? I've just read that something called backports exists but don't know if that is a solution.
I know it is an old threat, but in case someone else got this problem, I solved the problem like this:
(I used this github page: https://github.com/normanb/mapserver/tree/master/mapserver/mapscript)
Install old libgif
1.) apt install unzip libgdal-dev swig libproj-dev proj-data proj-bin
2.) wget "http://launchpadlibrarian.net/90361644/libgif4_4.1.6-9ubuntu1_amd64.deb"
3.) sudo dpkg -i libgif4_4.1.6-9ubuntu1_amd64.deb
4.) wget "http://security.ubuntu.com/ubuntu/pool/main/g/giflib/libgif-dev_4.1.6-9ubuntu1_amd64.deb"
5.) sudo dpkg -i libgif-dev_4.1.6-9ubuntu1_amd64.deb
Install old GD
6.) wget "www.boutell.com/gd/http/gd-2.0.33.tar.gz"
7.) unzip
8.) go to folder
9.) ./configure
10.) make
11.) make install
Install webserver
12.) Download zip from https://github.com/normanb/mapserver/
13.) unzip mapserver-master.zip
14.) Goto folder mapserver-master/mapserver/
15.) ./configure --with-wmsclient --with-proj --with-gdal --with-postgis (choose the options you need for your mapscript)
16.) make
For ruby mapscript: (for others chech out https://github.com/normanb/mapserver/tree/master/mapserver/mapscript)
16.) ruby extconf.rb
17.) make
18.) make install
Please note that there is a bug in set filter and you need to comment the filter like so "\"id = 123\"" (https://github.com/mapserver/mapserver/issues/3983)
Ultimately, it appears that the packages are (for my intent and purposes) broken.
Paired with a Debian guru, who basically installed Mapserver on the system in order to compile and generate the correct mapscript.so
He recommended I get in touch with the various package maintainers and outline the problems that I encountered.

Resources