Ipython notebook, how to set the correct path to kernel - path

When Running ipyhton notebook on Windows 7 64bit and launching notebook with python 2 kernel I get an error:
Traceback (most recent call last):
File "C:\Users\USER1\Anaconda2\lib\site-packages\notebook\base\handlers.py", line 436, in wrapper
result = yield gen.maybe_future(method(self, *args, **kwargs))
File "C:\Users\USER1\Anaconda2\lib\site-packages\notebook\services\sessions\handlers.py", line 56, in post
model = sm.create_session(path=path, kernel_name=kernel_name)
File "C:\Users\USER1\Anaconda2\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 66, in create_session
kernel_name=kernel_name)
File "C:\Users\USER1\Anaconda2\lib\site-packages\notebook\services\kernels\kernelmanager.py", line 84, in start_kernel
**kwargs)
File "C:\Users\USER1\Anaconda2\lib\site-packages\jupyter_client\multikernelmanager.py", line 109, in start_kernel
km.start_kernel(**kwargs)
File "C:\Users\USER1\Anaconda2\lib\site-packages\jupyter_client\manager.py", line 244, in start_kernel
**kw)
File "C:\Users\USER1\Anaconda2\lib\site-packages\jupyter_client\manager.py", line 190, in _launch_kernel
return launch_kernel(kernel_cmd, **kw)
File "C:\Users\USER1\Anaconda2\lib\site-packages\jupyter_client\launcher.py", line 115, in launch_kernel
proc = Popen(cmd, **kwargs)
File "C:\Users\USER1\Anaconda2\lib\subprocess.py", line 710, in __init__
errread, errwrite)
File "C:\Users\USER1\Anaconda2\lib\subprocess.py", line 958, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
I have investigated further and I have added following print lines before proc = Popen(cmd, **kwargs) inside launcher.py file
print cmd
print kwargs
Now I see that proc = Popen(cmd, **kwargs) is called with cmd=
['C:\\Users\\USER1\\Anaconda2_32bit\\python.exe', '-m', 'ipykernel', '-f', '
C:\\Users\\USER1\\AppData\\Roaming\\jupyter\\runtime\\kernel-a3f46334-4491-4
fef-aeb1-6772b8392954.json']
this is a problem because my python.exe is not in
C:\\Users\\USER1\\Anaconda2_32bit\\python.exe
but in
C:\\Users\\USER1\\Anaconda2\\python.exe
However I have checked paths in Computer/Advanced system settings/Advanced/Enviroment variables and \\Anaconda2_32bit\\ is never specified there.
Thus I suspect that the false path is specified somewhere else. Where could this be and how can I fix it?
Also I have previously had an installation of Anaconda in \\Anaconda2_32bit\\ but I have uninstalled it.

The ipython has kernels registered in special configuration files
I have run the command:
ipython kernelspec list
the output was:
Available kernels:
python2 C:\ProgramData\jupyter\kernels\python2
I have looked into C:\ProgramData\jupyter\kernels\python2\kernel.json file and there was a wrong path set for python2. I have fixed the path and it works now.

Related

PartitionedDataSet not found when Kedro pipeline is run in Docker

I have multiple text files in an S3 bucket which I read and process. So, I defined PartitionedDataSet in Kedro datacatalog which looks like this:
raw_data:
type: PartitionedDataSet
path: s3://reads/raw
dataset: pandas.CSVDataSet
load_args:
sep: "\t"
comment: "#"
In addition, I implemented this solution to get all secrets from credentials file via environment variables including AWS secret keys.
When I run things locally using kedro run everything works just fine, but when I build Docker image (using kedro-docker) and run pipeline in Docker environement with kedro docker run and by providing all enviornement variables using --docker-args option I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/kedro", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/kedro/framework/cli/cli.py", line 724, in main
cli_collection()
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/kedro/kedro_cli.py", line 230, in run
pipeline_name=pipeline,
File "/usr/local/lib/python3.7/site-packages/kedro/framework/context/context.py", line 767, in run
raise exc
File "/usr/local/lib/python3.7/site-packages/kedro/framework/context/context.py", line 759, in run
run_result = runner.run(filtered_pipeline, catalog, run_id)
File "/usr/local/lib/python3.7/site-packages/kedro/runner/runner.py", line 101, in run
self._run(pipeline, catalog, run_id)
File "/usr/local/lib/python3.7/site-packages/kedro/runner/sequential_runner.py", line 90, in _run
run_node(node, catalog, self._is_async, run_id)
File "/usr/local/lib/python3.7/site-packages/kedro/runner/runner.py", line 213, in run_node
node = _run_node_sequential(node, catalog, run_id)
File "/usr/local/lib/python3.7/site-packages/kedro/runner/runner.py", line 221, in _run_node_sequential
inputs = {name: catalog.load(name) for name in node.inputs}
File "/usr/local/lib/python3.7/site-packages/kedro/runner/runner.py", line 221, in <dictcomp>
inputs = {name: catalog.load(name) for name in node.inputs}
File "/usr/local/lib/python3.7/site-packages/kedro/io/data_catalog.py", line 392, in load
result = func()
File "/usr/local/lib/python3.7/site-packages/kedro/io/core.py", line 213, in load
return self._load()
File "/usr/local/lib/python3.7/site-packages/kedro/io/partitioned_data_set.py", line 240, in _load
raise DataSetError("No partitions found in `{}`".format(self._path))
kedro.io.core.DataSetError: No partitions found in `s3://reads/raw`
Note: Pipeline works just fine in Docker environment, if I move files to some local directory, define PartitionedDataSet and build Docker image and provide environment variables through --docker-args
The solution (at least in my case) was to provide AWS_DEFAULT_REGION env variable in the kedro docker run command.

Error in making python2.7 file executable with official py2app instructions on a hello word program

OSX 10.14 (Mojave)
python 2.7
py2app version = most recent as of Feb 11th 2019
I follow the official instructions found here:
https://py2app.readthedocs.io/en/latest/tutorial.html
I'm trying to get it to make a standalone for a 'hello world' program.
Everything works until the last step, which is 'python setup.py py2app'. I get the following error:
"Traceback (most recent call last):
File "setup.py", line 18, in
setup_requires=['py2app'],"
I did not modify the contents of the setup.py file, so they are as follows"
#
from setuptools import setup
APP = ['appp.py']
DATA_FILES = []
OPTIONS = {}
setup(
app=APP,
data_files=DATA_FILES,
options={'py2app': OPTIONS},
setup_requires=['py2app'],
)
#
Note that prior to this last step, I did indeed run 'rm -rf build dist' as instructed.
The error that I get is:
Traceback (most recent call last):
File "setup.py", line 18, in
setup_requires=['py2app'],
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app/build_app.py", line 838, in run
self._run()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app/build_app.py", line 1053, in _run
self.run_normal()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app/build_app.py", line 1150, in run_normal
self.process_recipes(mf, filters, flatpackages, loader_files)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app/build_app.py", line 1007, in process_recipes
rval = check(self, mf)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app/recipes/six.py", line 113, in check
mf.import_hook(nm, m)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/modulegraph/modulegraph.py", line 1082, in import_hook
q, tail = self._find_head_package(parent, name, level)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/modulegraph/modulegraph.py", line 1182, in _find_head_package
raise ImportError("No module named " + qname)
ImportError: No module named gdbm

JupyterHub - oauth_client_id not found

I am using Azure to run python notebook using Jupyterhub. After spinning up the VM, I was able to access the notebooks just by using my username and password (just like ssh). However, one day later when I switched to another network (I am not claiming that the network might have been a problem) I am unable to access the link. It gives me The site can't be reached error.
So I tried rerunning the process again, and since then I have been struggling to make it run again. I have searched for similar issues on GitHub, but they aren't helpful either.
After the kill the process using kill pid command, I tried running the jupyterhub through this command:
/anaconda/envs/py35/bin/python /anaconda/envs/py35/bin/jupyterhub-singleuser --port=50387 --notebook-dir="~/notebooks" --config=/etc/jupyterhub/jupyterhub_config.py
And it gives me the error:
JUPYTERHUB_API_TOKEN env is required to run jupyterhub-singleuser. Did you launch it manually?
So I searched through github issues similar to this. I tried generating token manually using:
jupyterhub token username
And I added that token to JUPYTERHUB_API_TOKEN via export JUPYTERHUB_API_TOKEN=token. I also added token:username to c.Authenticator.tokens in jupyterhub_config.py. Now I get this error:
Traceback (most recent call last):
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 528, in get
value = obj._trait_values[self.name]
KeyError: 'oauth_client_id'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda/envs/py35/bin/jupyterhub-singleuser", line 6, in <module>
main()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 455, in main
return SingleUserNotebookApp.launch_instance(argv)
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyter_core/application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "<decorator-gen-7>", line 2, in initialize
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/notebook/notebookapp.py", line 1296, in initialize
self.init_webapp()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 393, in init_webapp
self.init_hub_auth()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 388, in init_hub_auth
if not self.hub_auth.oauth_client_id:
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 556, in __get__
return self.get(obj, cls)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 535, in get
value = self._validate(obj, dynamic_default())
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 593, in _validate
value = self._cross_validate(obj, value)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 599, in _cross_validate
value = obj._trait_validators[self.name](obj, proposal)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 907, in __call__
return self.func(*args, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/services/auth.py", line 439, in _ensure_not_empty
raise ValueError("%s cannot be empty." % proposal.trait.name)
ValueError: oauth_client_id cannot be empty.
I am not sure where I went wrong in this process. Anybody familiar with this issue?
Try running jupyterhub instead of jupyterhub-singleuser
For your specific use case, the command would be as follows:
sudo /anaconda/envs/py35/bin/python /anaconda/envs/py35/bin/jupyterhub --port=50387 --notebook-dir="~/notebooks" --config=/etc/jupyterhub/jupyterhub_config.py
Make sure that jupyterhub is installed (correctly) in the path you mentioned.

Dart VM fails to build on windows

I am trying to buiil Dart-VM on windows, I follow the steps as described here
https://github.com/dart-lang/sdk/wiki/Building
When I run the build.py command as below:
.\tools\build.py --mode release --arch x64 create_sdk
I get the following error:
gn gen --check in out\ReleaseX64
Traceback (most recent call last):
File "D:\ops\dart\sdk\tools\gn.py", line 436, in <module>
sys.exit(main(sys.argv))
File "D:\ops\dart\sdk\tools\gn.py", line 423, in main
results = pool.map(run_command, commands, chunksize=1)
File "C:\app\Python27\lib\multiprocessing\pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "C:\app\Python27\lib\multiprocessing\pool.py", line 567, in get
raise self._value
WindowsError: [Error 2] System cannot find file。
Tried to run GN, but it failed. Try running it manually:
$ python D:\ops\dart\sdk\tools\gn.py -m release -a x64 --os host -v
Traceback (most recent call last):
File "D:\ops\dart\sdk\tools\build.py", line 658, in <module>
sys.exit(Main())
File "D:\ops\dart\sdk\tools\build.py", line 651, in Main
mode, arch, cross_build) != 0:
File "D:\ops\dart\sdk\tools\build.py", line 491, in BuildOneConfig
args = BuildNinjaCommand(options, target, target_os, mode, arch)
File "D:\ops\dart\sdk\tools\build.py", line 473, in BuildNinjaCommand
if UseGoma(out_dir):
File "D:\ops\dart\sdk\tools\build.py", line 431, in UseGoma
return 'use_goma = true' in open(args_gn, 'r').read()
IOError: [Errno 2] No such file or directory: 'out\\ReleaseX64\\args.gn'
It seems missing the args.gn file in out\ReleaseX64 folder. but I cannot find args.gn in Dart source folders. Is it generated during building process? whether I do wrong steps led to no such file generated?

PyLaTeX Tutorial Generating Error: [WinError 2]

I am trying to run a tutorial for PyLaTeX in Python 3.5. I am using Anaconda / Spyder. When it runs the command "generate_pdf()", [WinError 2] is generated. Below is the code (it is note quoting properly) and the error. It may be easier simply to look at the link.
def fill_document(doc):
with doc.create(Section('A section')):
doc.append('Some regular text and some ')
doc.append(italic('italic text. '))
with doc.create(Subsection('A subsection')):
doc.append('Also some crazy characters: $&#{}')
if __name__ == '__main__':
# Basic document
doc = Document('basic')
fill_document(doc)
doc.generate_pdf(filepath = r'C:\Users\James\Documents\Important Files\Python Scripts\PyLaTeX',clean_tex=False)
Error:
Traceback (most recent call last):
File "<ipython-input-14-0158fa80c4f1>", line 1, in <module>
runfile('C:/Users/James/Documents/Important Files/Python Scripts/PyLaTeX/basic_tutorial.py',
wdir='C:/Users/James/Documents/Important Files/Python
Scripts/PyLaTeX')
File
"C:\Users\James\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py",
line 714, in runfile
execfile(filename, namespace)
File
"C:\Users\James\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py",
line 89, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/James/Documents/Important Files/Python
Scripts/PyLaTeX/basic_tutorial.py", line 21, in <module>
doc.generate_pdf(filepath = r'C:\Users\James\Documents\Important Files\Python Scripts\PyLaTeX',clean_tex=False)
File
"C:\Users\James\Anaconda3\lib\site-packages\pylatex\document.py", line
203, in generate_pdf
raise(os_error)
File
"C:\Users\James\Anaconda3\lib\site-packages\pylatex\document.py", line
157, in generate_pdf
stderr=subprocess.STDOUT)
File "C:\Users\James\Anaconda3\lib\subprocess.py", line 629, in
check_output
**kwargs).stdout
File "C:\Users\James\Anaconda3\lib\subprocess.py", line 696, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\James\Anaconda3\lib\subprocess.py", line 950, in
__init__
restore_signals, start_new_session)
File "C:\Users\James\Anaconda3\lib\subprocess.py", line 1220, in
_execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file
specified
Thanks in advance.
I also had the exact same problem, but there was a .log file present in the directory. So I checked out the log, and the last few lines said that a package called lastpage.sty was not found (I'm on Linux), so I just downloaded all the latex packages again using apt.
So I think, you should read the log file too, and if there IS something like that, then go to ctan (all packages are found there) and install the required package for Windows.
Hope it helped.
Also try putting the document.generate_tex() before the document.generate_pdf(filepath="path/to/file", clean_tex=False)

Resources