I am having trouble importing modules from within xonsh script that are installed in to the virtual environment.
I installed the virtual environment using vox
xontrib load vox
vox activate venv
pip install PyPDF2
Here is my minimal example
#!/usr/bin/env xonsh
import PyPDF2
If I run this from within the Python interpreter it works:
$ python
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import PyPDF2
>>>
If I try to run the xonsh script as a script I get the following:
$ ./minimaltest.x
sh
/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import fnmatch, glob, traceback, errno, sys, atexit, locale, imp, stat
Traceback (most recent call last):
File "/usr/bin/xonsh", line 4, in <module>
main()
File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 24019, in main
_failback_to_other_shells(args, err)
File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 23983, in _failback_to_other_shells
raise err
File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 24017, in main
return main_xonsh(args)
File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 24060, in main_xonsh
run_script_with_cache(
File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 3043, in run_script_with_cache
run_compiled_code(ccode, glb, loc, mode)
File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 2948, in run_compiled_code
func(code, glb, loc)
File "./minimaltest.xsh", line 2, in <module>
import PyPDF2
ModuleNotFoundError: No module named 'PyPDF2'
Any thoughts about what might be wrong?
xonsh can only import modules that exist in the same venv that it is installed in. If you install xonsh into the same venv as PyPDF2 it should work.
Related
While trying to import glob in a python venv environment, it is referring to the system package and not the virtual environment even though pandas module is referring to the virtual environment.
I am using python 3.8 and I created a virtual environment using python venv :
cd trial_3
python3 -m venv trial_3_env
On trying to use glob module (which i haven't yet installed in the environment), I can see that it is not throwing any error, but using the glob module from the system packages.
Please find the screenshot showing the same below:
(trial_3_env) anitta#vinjohn:~/Desktop/Study_Data_Engineering/virtualenv_trial/trial_3$ pip freeze
numpy==1.23.4
pyspark==3.3.0
python-dateutil==2.8.2
pytz==2022.6
six==1.16.0
(trial_3_env) anitta#vinjohn:~/Desktop/Study_Data_Engineering/virtualenv_trial/trial_3$ python3
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import glob
>>> glob.__file__
'/usr/lib/python3.8/glob.py'
>>>
I tried checking this behavior with pandas module, but they are working as expected and throw error while importing when I have not preinstalled them in my system.
(trial_3_env) anitta#vinjohn:~/Desktop/Study_Data_Engineering/virtualenv_trial/trial_3$ python3
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'
>>>
Could someone let me know the cause of globs behavior ? and if such scenario can occur for other modules as well.
Thanks in advance!
#ChrisD and #sinoroc answers helped me. standard libraries of venv python interpreter are referenced from the system python interpreter path itself and venv folder doesn't have any python standard libraries stored inside.
Issue:
I am installing snips in docker with step 1 given in To Reproduce section. I am ending with No module named 'distutils.msvccompiler' error. I am running everything on Linux based system. is there alternative way to install snips-nlu in docker?
To Reproduce
Created installSnips.sh (following) file for installing rust, rust setup tool
pip3 install numpy
pip3 install scipy
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
pip3 install setuptools-rust
source ~/.cargo/env
pip3 install snips-nlu
while running installSnips.sh pip3 install snip-nlu enter in following error
Building wheel for scikit-learn (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [26 lines of output]
Partial import of sklearn during the build process.
/tmp/pip-install-h9opcxf9/scikit-learn_fc42ebd0e5804549ad6f611dced79620/setup.py:123: DeprecationWarning:
`numpy.distutils` is deprecated since NumPy 1.23.0, as a result
of the deprecation of `distutils` itself. It will be removed for
Python >= 3.12. For older Python versions it will remain present.
It is recommended to use `setuptools < 60.0` for those Python versions.
For more details, see:
https://numpy.org/devdocs/reference/distutils_status_migration.html
from numpy.distutils.command.build_ext import build_ext # noqa
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-install-h9opcxf9/scikit-learn_fc42ebd0e5804549ad6f611dced79620/setup.py", line 303, in <module>
setup_package()
File "/tmp/pip-install-h9opcxf9/scikit-learn_fc42ebd0e5804549ad6f611dced79620/setup.py", line 295, in setup_package
from numpy.distutils.core import setup
File "/home/drjslab/.local/lib/python3.10/site-packages/numpy/distutils/core.py", line 24, in <module>
from numpy.distutils.command import config, config_compiler, \
File "/home/drjslab/.local/lib/python3.10/site-packages/numpy/distutils/command/config.py", line 19, in <module>
from numpy.distutils.mingw32ccompiler import generate_manifest
File "/home/drjslab/.local/lib/python3.10/site-packages/numpy/distutils/mingw32ccompiler.py", line 28, in <module>
from distutils.msvccompiler import get_build_version as get_build_msvc_version
ModuleNotFoundError: No module named 'distutils.msvccompiler'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for scikit-learn
Running setup.py clean for scikit-learn
Failed to build scikit-learn
Installing collected packages: requests, pyaml, packaging, scikit-learn, deprecation
Attempting uninstall: scikit-learn
Found existing installation: scikit-learn 1.1.2
Uninstalling scikit-learn-1.1.2:
Successfully uninstalled scikit-learn-1.1.2
Running setup.py install for scikit-learn ... error
error: subprocess-exited-with-error
× Running setup.py install for scikit-learn did not run successfully.
│ exit code: 1
╰─> [26 lines of output]
Partial import of sklearn during the build process.
/tmp/pip-install-h9opcxf9/scikit-learn_fc42ebd0e5804549ad6f611dced79620/setup.py:123: DeprecationWarning:
`numpy.distutils` is deprecated since NumPy 1.23.0, as a result
of the deprecation of `distutils` itself. It will be removed for
Python >= 3.12. For older Python versions it will remain present.
It is recommended to use `setuptools < 60.0` for those Python versions.
For more details, see:
https://numpy.org/devdocs/reference/distutils_status_migration.html
from numpy.distutils.command.build_ext import build_ext # noqa
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-install-h9opcxf9/scikit-learn_fc42ebd0e5804549ad6f611dced79620/setup.py", line 303, in <module>
setup_package()
File "/tmp/pip-install-h9opcxf9/scikit-learn_fc42ebd0e5804549ad6f611dced79620/setup.py", line 295, in setup_package
from numpy.distutils.core import setup
File "/home/drjslab/.local/lib/python3.10/site-packages/numpy/distutils/core.py", line 24, in <module>
from numpy.distutils.command import config, config_compiler, \
File "/home/drjslab/.local/lib/python3.10/site-packages/numpy/distutils/command/config.py", line 19, in <module>
from numpy.distutils.mingw32ccompiler import generate_manifest
File "/home/drjslab/.local/lib/python3.10/site-packages/numpy/distutils/mingw32ccompiler.py", line 28, in <module>
from distutils.msvccompiler import get_build_version as get_build_msvc_version
ModuleNotFoundError: No module named 'distutils.msvccompiler'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
Rolling back uninstall of scikit-learn
Moving to /home/drjslab/.local/lib/python3.10/site-packages/scikit_learn-1.1.2.dist-info/
from /home/drjslab/.local/lib/python3.10/site-packages/~cikit_learn-1.1.2.dist-info
Moving to /home/drjslab/.local/lib/python3.10/site-packages/scikit_learn.libs/
from /home/drjslab/.local/lib/python3.10/site-packages/~cikit_learn.libs
Moving to /home/drjslab/.local/lib/python3.10/site-packages/sklearn/
from /home/drjslab/.local/lib/python3.10/site-packages/~klearn
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> scikit-learn
note: This is an issue with the package mentioned above, not pip.
Environment:
Base OS: Ubuntu 20.04
Base Python version: 3.8
snips-nlu version:Latest
Docker OS: Ubuntu 20.04
Docker Python: 3.10.4
It seems a recent update of setuptools broke numpy.distutils which is a dependency of scikit-learn.
https://github.com/pypa/setuptools/pull/3505
A reported possible solution is to install setuptools<65 with pip install --no-use-pep517
I had installed numpy on alpine docker and everything looked good before running numpy:
Traceback (most recent call last):
File "/app/venv/lib/python3.7/site-packages/numpy/core/__init__.py", line 24, in <module>
from . import multiarray
File "/app/venv/lib/python3.7/site-packages/numpy/core/multiarray.py", line 14, in <module>
from . import overrides
File "/app/venv/lib/python3.7/site-packages/numpy/core/overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: Error loading shared library libcblas.so.3: No such file or directory (needed by /app/venv/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so)
I found that licblas.so might be in /usr/lib/ as mentioned here: alpine blas
but it wasn't there.
After installing lapack, the missing file libcblas.so.3 appeared in /usr/lib/:
apk add lapack
I had openblas-dev installed from before but that was not sufficient.
Followed through installation tips, but after typing
$ python underactuated/src/double_pendulum/simulate.py
I get
ImportError: dynamic module does not define init function (init_module_py).
If I run python3 I get
ImportError: cannot import name 'FindResource'.
After typing $ python3 -c 'import pydrake; print(pydrake.__file__)' I get /opt/drake/lib/python3.6/site-packages/pydrake/__init__.py
Edit:
Full output of running python3
Traceback (most recent call last):
File "underactuated/src/double_pendulum/simulate.py", line 5, in <module>
from underactuated import FindResource, PlanarSceneGraphVisualizer
ImportError: cannot import name 'FindResource'
And full output of running python (without3):
Traceback (most recent call last):
File "underactuated/src/double_pendulum/simulate.py", line 1, in <module>
from pydrake.all import (AddMultibodyPlantSceneGraph,
File "/opt/drake/lib/python3.6/site-packages/pydrake/__init__.py", line 32, in <module>
from . import common
File "/opt/drake/lib/python3.6/site-packages/pydrake/common/__init__.py", line 3, in <module>
from ._module_py import *
ImportError: dynamic module does not define init function (init_module_py)
Version of python after typing python -V is Python 2.7.15+
The fact that the print test succeeds suggests that drake is installed correctly.
Please post the full output of running simulate.py with python3.
this is driving me crazy, I already tried a lot of things,e.g. changing locale, but still getting this after workspace init and then doing catkin_make on subdir.
Running on a recent arch dist, ros-indigo, python2.7 and python 3.5 installed
johnny#localhost:~/ros$ catkin_make
Base path: /home/johnny/ros
Source space: /home/johnny/ros/src
Build space: /home/johnny/ros/build
Devel space: /home/johnny/ros/devel
Install space: /home/johnny/ros/install
####
#### Running command: "cmake /home/johnny/ros/src -DCATKIN_DEVEL_PREFIX=/home/johnny/ros/devel -DCMAKE_INSTALL_PREFIX=/home/johnny/ros/install -G Unix Makefiles" in "/home/johnny/ros/build"
####
-- Using CATKIN_DEVEL_PREFIX: /home/johnny/ros/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/indigo
-- This workspace overlays: /opt/ros/indigo
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using default Python package layout
-- Using empy: /usr/lib/python2.7/site-packages/em.py
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/johnny/ros/build/test_results
-- Found gtest: gtests will be built
CMake Warning at /opt/ros/indigo/share/catkin/cmake/test/nosetests.cmake:96 (message):
nosetests not found, Python tests can not be run (try installing package
'python3-nose')
Call Stack (most recent call first):
/opt/ros/indigo/share/catkin/cmake/all.cmake:147 (include)
/opt/ros/indigo/share/catkin/cmake/catkinConfig.cmake:20 (include)
CMakeLists.txt:52 (find_package)
-- catkin 0.6.18
-- BUILD_SHARED_LIBS is on
/opt/ros/indigo/share/catkin/cmake/em/order_packages.cmake.em:23: error: <class 'UnicodeDecodeError'>: 'ascii' codec can't decode byte 0xc3 in position 205: ordinal not in range(128)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/em.py", line 3302, in <module>
if __name__ == '__main__': main()
File "/usr/lib/python2.7/site-packages/em.py", line 3300, in main
invoke(sys.argv[1:])
File "/usr/lib/python2.7/site-packages/em.py", line 3283, in invoke
interpreter.wrap(interpreter.file, (file, name))
File "/usr/lib/python2.7/site-packages/em.py", line 2295, in wrap
self.fail(e)
File "/usr/lib/python2.7/site-packages/em.py", line 2284, in wrap
callable(*args)
File "/usr/lib/python2.7/site-packages/em.py", line 2359, in file
self.safe(scanner, done, locals)
File "/usr/lib/python2.7/site-packages/em.py", line 2401, in safe
self.parse(scanner, locals)
File "/usr/lib/python2.7/site-packages/em.py", line 2421, in parse
token.run(self, locals)
File "/usr/lib/python2.7/site-packages/em.py", line 1425, in run
interpreter.execute(self.code, locals)
File "/usr/lib/python2.7/site-packages/em.py", line 2595, in execute
_exec(statements, self.globals, locals)
File "<string>", line 17, in <module>
File "/usr/lib/python2.7/site-packages/catkin_pkg/topological_order.py", line 111, in topological_order
for path, package in find_packages(space).items():
File "/usr/lib/python2.7/site-packages/catkin_pkg/packages.py", line 83, in find_packages
packages = find_packages_allowing_duplicates(basepath, exclude_paths=exclude_paths, exclude_subspaces=exclude_subspaces, warnings=warnings)
File "/usr/lib/python2.7/site-packages/catkin_pkg/packages.py", line 110, in find_packages_allowing_duplicates
packages[path] = parse_package(os.path.join(basepath, path), warnings=warnings)
File "/usr/lib/python2.7/site-packages/catkin_pkg/package.py", line 370, in parse_package
return parse_package_string(f.read(), filename, warnings=warnings)
File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 205: ordinal not in range(128)
CMake Error at /opt/ros/indigo/share/catkin/cmake/safe_execute_process.cmake:11 (message):
execute_process(/home/johnny/ros/build/catkin_generated/env_cached.sh
"/usr/bin/python" "/usr/lib/python2.7/site-packages/em.py" "--raw-errors"
"-F" "/home/johnny/ros/build/catkin_generated/order_packages.py" "-o"
"/home/johnny/ros/build/catkin_generated/order_packages.cmake"
"/opt/ros/indigo/share/catkin/cmake/em/order_packages.cmake.em") returned
error code 1
Call Stack (most recent call first):
/opt/ros/indigo/share/catkin/cmake/em_expand.cmake:25 (safe_execute_process)
/opt/ros/indigo/share/catkin/cmake/catkin_workspace.cmake:35 (em_expand)
CMakeLists.txt:63 (catkin_workspace)
-- Configuring incomplete, errors occurred!
See also "/home/johnny/ros/build/CMakeFiles/CMakeOutput.log".
See also "/home/johnny/ros/build/CMakeFiles/CMakeError.log".
Invoking "cmake" failed'
Thank you very much!
It looks like an unicode problem. In very brief summary, you have passed something that is being interpreted as a string of bytes to something that needs to decode it into Unicode characters, but the default codec (ascii) is failing.
More on this here.
there's a lot of solutions to this one. the one I suggest is doing this :
yassin#pc:/usr/local/lib/python2.7/site-packages# cat sitecustomize.py
# encoding=utf8
import sys
reload(sys)
sys.setdefaultencoding('utf8')
and to check if that works, you need to do the following :
yassin#pc:~/home# python
Python 2.7.6 (default, Dec 6 2013, 14:49:02)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> reload(sys)
<module 'sys' (built-in)>
>>> sys.getdefaultencoding()
'utf8'
>>>
Hope it helps ! Cheers.
Set the system default locale to en_IN.utf8 should solve the problem.
For example, if you use bash, you can add below lines into your ~/.bashrc:
LANG=en_IN.utf8
export LANG
The catkin toolkit use system default locale to load message or service files.