Ray cluster ModuleNotFoundError - machine-learning

I am new to Ray and wanted to set up a cluster with some dependencies. I first set up a cluster with some dependencies supported out of the box and everything worked fine. Now, I have the following scenario that I simplified to this demonstration. There is a problem with joblib even though I specify it in a runtime env as to be downloaded with pip and then pass it to ray.init.
from ray.util.joblib import register_ray
import joblib
import ray
import numpy as np
# connect to ray
runtime_env={"pip": ["joblib"]}
ray.init(address='auto', _redis_password=..., runtime_env=runtime_env)
#ray.remote(num_cpus=1)
def train_ray(...):
...
register_ray()
with joblib.parallel_backend('ray'):
...
return ...
I get the following error:
from ray.util.joblib import register_ray
File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/util/joblib/__init__.py", line 1, in <module>
from joblib.parallel import register_parallel_backend
ModuleNotFoundError: No module named 'joblib'
command terminated with exit code 1
Thanks for any suggestions.

Related

ModuleNotFoundError: No module named 'pyproj._network'

I am trying to run Python Code on Spyder for scientific computing. I use geometric information from shape files using geopandas. Installing geopandas was already difficult but I finally managed. However now I have another issue, that a module called 'pyproj._network' is not found.
I checked and the module is there and was also added to the path using the PYTHONPATH manager of Spyder.
Now I dont know anymore- any help very much appreciated!
Thanks!
`ModuleNotFoundError: No module named 'pyproj._network'
Entering post mortem debugging...
c:\users\stefano-work\miniconda3\lib\site-packages\pyproj\network.py(10)()
8 import certifi
9
---> 10 from pyproj._network import ( # noqa: F401
11 _set_ca_bundle_path,
12 is_network_enabled,
`

Can't import plotnine

Getting an error when trying to import plotline; I pip installed Unicode, but still seeing the same error. Not sure what else I need to install.
import plotline as p9
ImportError: cannot import name 'unicode'

ImportError during python run

Currently I am executing doamusic python codes from
https://github.com/yump/doamusic.git
I am using python 3.7.2. (Python IDLE) on windows.
My directory tree is
F:\doamusic_project\doamusic\music.py
when I run music.py, I am getting the following error
Traceback (most recent call last):
File "F:\doamusic_project\doamusic\music.py", line 31, in
from . import util
File "..\doamusic__init__.py", line 1, in
from doamusic.music import *
File "..\doamusic\music.py", line 32, in
from . import _music
ImportError: cannot import name '_music' from 'doamusic' (..\doamusic__init__.py)
What is the reason for not importing _music ?.

You are trying to do an import from .pyx file. A .pyx file must be compiled unlike a .py file.
Try to do that before the import of _music:
import pyximport
pyximport.install()
More info Cython

Flask vs Terminal: how do I manage imports so that I can use codes in both?

I have a Flask project with the following directory structure:
-my_project
- flask_code.py
- module
- submodule
- foo.py
- bar.py
- xyz.py
In flask_code.py, I must import both foo and bar. Therefore, flask_code has the following import statements:
"""flask_code.py"""
from module.foo import *
from module.bar import *
In foo.py, I need to import xyz. I also have implemented a main function in foo for testing, which I usually call from command line. If I write the imports of foo like this:
"""foo.py import 1"""
from module.xyz import *
The flask application runs exactly as expected. But, if I try to run foo.py from command line, I get the following error:
ModuleNotFoundError: No module named 'module'
On the other hand, if I change the imports of foo to:
"""foo.py import 2"""
from xyz import *
the ModuleNotFoundError is not thrown anymore when foo.py is called from command line, which I guess it's expected. But, the flask application can't find module xyz on its own.
Both my_project and module folders have a init.py empty file.
I want to manage these imports so that I can both use the flask application and call the scripts from command line for testing. Thanks in advance!

ImportError when importing from Bio.PDB

I want to use Biopython's PDBParser to download PDB files, but this error appears when I try to import it:
Traceback (most recent call last):File "C:\Python27\TAREA 3 FINAL.PY", line 33, in [HTML] from Bio.PDB import * File "C:\Python27\lib\site-packages\Bio\PDB_init_.py", line 15, in [HTML] from PDBParser import PDBParser File "C:\Python27\lib\site-packages\Bio\PDB\PDBParser.py", line 13, in [HTML] import numpy ImportError: No module named numpy
What's going on here?

Biopython requires the numpy module. You need to install numpy in order to use Biopython. You can find the latest version here.

Resources