Using Ubuntu 14.04,
DataFlow Python SDK
Following instructions at [https://github.com/GoogleCloudPlatform/DataflowPythonSDK#status-of-this- release] , after everything is loaded when I try the wordcount example I try get the error "Import by filename is not supported".
I suspect the issue is at line 23 of the wordcount.py example
import google.cloud.dataflow as df
Is there a workaround for this issue?
I have tried the solution posted at Python / ImportError: Import by filename is not supported , but that does not solve the problem.
Since this fails at the first import statement the immediate thing to check is if the Python Dataflow package is installed at all. Th way to do that is by running 'pip freeze'. Here is some output from running this in a virtual environment:
$ pip freeze
... Nothing since it is a clean virtual environment ...
$ pip install https://github.com/GoogleCloudPlatform/DataflowPythonSDK/archive/v0.2.3.tar.gz
... Output from installing packages ...
$ pip freeze
...
python-dataflow==0.2.3
...
Now you can run python and execute 'import google.cloud.dataflow as df' and it should work.
Hopefully this helps!
Related
I'm using taskwarrior with a couple of python hooks installed that get triggered when issuing certain taskwarrior commands. The hooks run fine when I use taskwarrior normally from the command line.
But I'm running into problems when the the hooks are trigged by a hotkey app, Karabiner Elements (I'm on a mac). Karabiner calls a perl script which in turn execute this bash command containing the task command:
/bin/bash -c 'TASKRC=/Users/me/.taskrc /usr/local/bin/task add \'the task\''
Unfortunately, the stack error is cutoff in the Karabiner log. This is as much as I get:
[2021-12-07 06:24:51.797] [error] [console_user_server] shell_command stderr:Traceback (most recent call last): File "/Users/me/.task_work/hooks/on-add-pirate", line 9, in <module> from tasklib import TaskWarrior, Task File "/Users/me/Library/Python/3.8/lib/python/site-packages/tasklib/__init__.py", l...
I'm guessing the python script is choking because it can't figure out where needed libraries are. But I don't see any shell environment variables that I might be able to set. I have python3 installed with brew and the tasklib library installed with pip3 (I believe).
Here's the hook script:
#!/usr/bin/env python3
import glob
from types import ModuleType
from importlib.machinery import SourceFileLoader, ModuleSpec
from importlib.util import module_from_spec
import os
from tasklib import TaskWarrior, Task
<-- snip -->
task = Task.from_input()
for hook in find_hooks('pirate_add'):
hook(task)
print(task.export_data())
And here's the __init__.py script mentioned in the error:
from .backends import TaskWarrior
from .task import Task
from .serializing import local_zone
__version__ = '2.2.1'
Ok, problem was bash was defaulting to older version of python3.
The fix is here: https://stackoverflow.com/a/70267561/1641112
I'm new to data science and wanted to do a little tutorial, which requires airflow, among other things. I installed it on windows using git bash in VS Code. I tried running it but it had a problem not being able to load the sqlite3 import
command (module not found error). I figured out that if I added the directory of sqlite3.py to the path, it would run, but now it gives me a similar error: pwd module not found from daemon.py
File "C:\ProgramData\Anaconda3\lib\site-packages\daemon\daemon.py", line 18, in <module>
import pwd
ModuleNotFoundError: No module named 'pwd'
Strange to me that it can't find pwd. Obviously pwd works in both git bash and powershell natively. It seems like a basic universal command. I'd love to learn more about what's going on. I don't want to have to end up adding 100 things to path just to get this program to run. I'd love any insights anyone can provide.
PS I'm using Anaconda.
it's seems to be the side effects of Spawning new Python Daemons .
You likely can fix this by downgrading the Python-Daemon :
pip install python-daemon==2.1.2
I have a project with venv that works just fine, but when we need to run roslaunch rosbridge_server rosbridge_websocket.launch it will result in the error below unless we deactivate venv.
process[rosbridge_websocket-2]: started with pid [10182]
process[rosapi-3]: started with pid [10183]
Failed to load Python extension for LZ4 support. LZ4 compression will not be available.
Failed to load Python extension for LZ4 support. LZ4 compression will not be available.
registered capabilities (classes):
-
-
-
-
-
-
-
-
Traceback (most recent call last):
File "/opt/ros/kinetic/lib/rosbridge_server/rosbridge_websocket", line 44, in
from rosbridge_server import RosbridgeWebSocket, ClientManager
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbridge_server/init.py", line 4, in
from .udp_handler import RosbridgeUdpSocket,RosbridgeUdpFactory
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbridge_server/udp_handler.py", line 6, in
from twisted.internet.protocol import DatagramProtocol,Factory
ImportError: No module named 'twisted'
I am assuming this is because there's no module named twisted installed in the virtual environment but it is installed on my computer. Following this hypothesis, this means that making roslaunch rosbridge_server rosbridge_websocket.launch work would require to install all the dependencies that it needs in venv? How would one go to do that?
This is a bit late, but what version of python are you targeting with venv? I suspect it's a python 3.x, and if so, you have to setup for it.
As you suspect, you're missing packages, this should be a good start (from here), adjusting the commands to suit your venv.
sudo apt-get install python3-pip python3-yaml
sudo pip3 install rospkg catkin_pkg
For more details, see the new ros wiki pages: http://wiki.ros.org/UsingPython3/IdentifyDependencies
just make sure the version of python you are using in venv and the environment where those packages are installed is same, if thats not the case, then you probably need to correct the python paths.
Mostly when you are running something in venv, the interpreter will only look inside that virtual environment, so when i face that issue, i just stopped using venv for ros and exported the pythonpaths or any ros module if they are installed for python3. distribution.
I'm using docker to run an NLP system that uses nltk, languagetool etc...
When I use docker-compose build --build-arg env=dev I get the warning message:
/usr/local/lib/python3.6/runpy.py:125: RuntimeWarning:
'nltk.downloader' found in sys.modules after import of package 'nltk',
but prior to execution of 'nltk.downloader'; this may result in
unpredictable behaviour warn(RuntimeWarning(msg))
Then when I use docker-compose up I get these errors when trying to run my system:
Please help me figure out how to fix this!
In your Dockerfile:
RUN python -c "import nltk;nltk.download('your_library')"
After pip install nltk
the skeleton code of what I do is
from IPython import parallel
.....
rcAll = parallel.Client()
all_engines = rcAll[:]
lbvAll = rcAll.load_balanced_view()
....
for anInpt in allInpt:
lbvAll.apply(mputil.doAll, anInpt)
lbvAll.wait()
lbvAll.get_result()
....
for ijk in range(len(list(lbvAll.results.values()))):
out = list(lbvAll.results.values())[ijk]
ionS = out[0]
However, all that out ever contains is import error messages.
This worked before but but something must have changed between IPython 2.1 and 2.2. At least, that is my guess.
Check the output of:
cat /usr/local/lib/python2.7/dist-packages/*.pth
Delete the following path if it exits inside the "catted" folder:
/usr/lib/python2.7/dist-packages
usr/lib/python2.7/dist-packages being on the front of sys.path means that there is an easy-install.pth file with this path, which should be removed. It is caused by a bug in setuptools.
If that doesn't work, simply upgrading some of your tools might fix the problem.
pip install --upgrade ipython
pip install --upgrade setuptools pip
I found the problem. I started the ipcluster in a shell with a different PYTHONPATH than the one I was running the notebook in. That simple but I took a while. I apologize for the noise.