I've started dask-scheduler on windows
Now I attempt to run dask-worker <ip>:<port> in ec2 instance.
I've been thrown at the following error:
distributed.nanny - INFO - Start Nanny at: 'tcp://10.34.33.12:36525'
distributed.diskutils - INFO - Found stale lock file and directory '/dask-worker-space/worker-v_5Vmm', purging
distributed.nanny - ERROR - Failed to start worker
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/distributed/nanny.py", line 541, in run
yield worker._start(*worker_start_args)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 1099, in run
value = future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 260, in result
raise_exc_info(self._exc_info)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 315, in wrapper
yielded = next(result)
File "/usr/lib/python2.7/site-packages/distributed/worker.py", line 425, in _start
self.start_services(listen_host)
File "/usr/lib/python2.7/site-packages/distributed/worker.py", line 368, in start_services
self.services[k] = v(self, io_loop=self.loop, **kwargs)
File "/usr/lib/python2.7/site-packages/distributed/bokeh/worker.py", line 634, in __init__
main = Application(FunctionHandler(partial(main_doc, worker, extra)))
File "/usr/lib/python2.7/site-packages/bokeh/application/handlers/function.py", line 11, in __init__
_check_callback(func, ('doc',))
File "/usr/lib/python2.7/site-packages/bokeh/util/callback_manager.py", line 12, in _check_callback
sig = signature(callback)
File "/usr/lib/python2.7/site-packages/bokeh/util/future.py", line 85, in signature
for name in func.keywords.keys():
AttributeError: 'NoneType' object has no attribute 'keys'
distributed.nanny - INFO - Closing Nanny at 'tcp://10.34.33.12:36525'
distributed.dask_worker - INFO - End worker
Can you tell me what is happening?
Is it even possible for dask to connect to a machine for making a cluster with different os?
Related
I'm new to docker and I'm trying to follow this simple "getting started" tutorial https://docs.docker.com/compose/gettingstarted/ using a newly first time installation of docker (for Windows 10) dowloaded from here: https://hub.docker.com/editions/community/docker-ce-desktop-windows/.
At step 4 of this tutorial i get this error:
PS D:\composetest> docker-compose up
Building web
Traceback (most recent call last):
File "site-packages\docker\credentials\store.py", line 80, in _execute
File "subprocess.py", line 395, in check_output
File "subprocess.py", line 487, in run
subprocess.CalledProcessError: Command '['C:\\Program Files\\Docker\\Docker\\resources\\bin\\docker-credential-desktop.EXE', 'list']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker-compose", line 6, in <module>
File "compose\cli\main.py", line 72, in main
File "compose\cli\main.py", line 128, in perform_command
File "compose\cli\main.py", line 1078, in up
File "compose\cli\main.py", line 1074, in up
File "compose\project.py", line 548, in up
File "compose\service.py", line 367, in ensure_image_exists
File "compose\service.py", line 1106, in build
File "site-packages\docker\api\build.py", line 261, in build
File "site-packages\docker\api\build.py", line 308, in _set_auth_headers
File "site-packages\docker\auth.py", line 302, in get_all_credentials
File "site-packages\docker\credentials\store.py", line 71, in list
File "site-packages\docker\credentials\store.py", line 93, in _execute
docker.credentials.errors.StoreError: Credentials store docker-credential-desktop exited with "error listing credentials - err: exit status 1, out: `Impossibile trovare elemento.`".
[13284] Failed to execute script docker-compose
EDIT: The accepted solution in docker-compose unable to start, unfortunately, did not work.
What is going wrong?
I have the same issue...
Try:
$nano ~/.docker/config.json
In this file change credsStore to credStore
Now run your docker-compose. If its not works try sudo
try to add to the environment path :
C:\Program Files\Docker\Docker\resources\bin
If you are using WSL2 run this command
sudo ln -s /mnt/c/Program\ Files/Docker/Docker/resources/bin/docker-credential-desktop.exe /usr/bin/docker-credential-desktop.exe
I have a standard cluster setup on kubernetes using the dask docker images but not using the dask helm charts. I tried running an existing script on the cluster but doesn’t seem to run. It keeps throwing errors.
The cluster details: 1 notebook, 1 scheduler, 1 worker & 1 shared vol.
I read up on some of the threads on KilledWorkers so I looked into the logs but couldn't figure it out.
distributed.worker - ERROR - None Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/distributed/worker.py", line 814, in handle_scheduler comm, every_cycle=[self.ensure_communicating, self.ensure_computing] File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 748, in run yielded = self.gen.send(value) File "/opt/conda/lib/python3.7/site-packages/distributed/core.py", line 457, in handle_stream msgs = yield comm.read() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 742, in run yielded = self.gen.throw(*exc_info) # type: ignore File "/opt/conda/lib/python3.7/site-packages/distributed/comm/tcp.py", line 218, in read frames, deserialize=self.deserialize, deserializers=deserializers File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 209, in wrapper yielded = next(result) File "/opt/conda/lib/python3.7/site-packages/distributed/comm/utils.py", line 85, in from_frames res = _from_frames() File "/opt/conda/lib/python3.7/site-packages/distributed/comm/utils.py", line 71, in _from_frames frames, deserialize=deserialize, deserializers=deserializers File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/core.py", line 126, in loads value = _deserialize(head, fs, deserializers=deserializers) File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 189, in deserialize dumps, loads, wants_context = families[name] KeyError: None
I got same problem and found solution.
In Dask 2.3 distributed serialization changed a bit. Your client is probably higher than 2.3.0 and scheduler and workers aren't. Try to upgrade your cluster so either everything is higher than 2.3.0 or lower than that.
I export database json doing commands explained here:
http://kiwitcms.org/blog/atodorov/2018/07/30/how-to-backup-docker-volumes-for-kiwi-tcms/
I'm running latest version of Kiwi.
docker exec -it kiwi_web /bin/bash -c '/Kiwi/manage.py sqlflush | /Kiwi/manage.py dbshell'
2.cat database.json | docker exec -i kiwi_web /Kiwi/manage.py loaddata --format json -
and I get this error:
Traceback (most recent call last):
File "/venv/lib/python3.6/site-packages/django/db/models/options.py", line 564, in get_field
return self.fields_map[field_name]
KeyError: 'description'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/venv/lib/python3.6/site-packages/django/core/serializers/json.py", line 69, in Deserializer
yield from PythonDeserializer(objects, **options)
File "/venv/lib/python3.6/site-packages/django/core/serializers/python.py", line 116, in Deserializer
field = Model._meta.get_field(field_name)
File "/venv/lib/python3.6/site-packages/django/db/models/options.py", line 566, in get_field
sh-4.2$ cat database.json | ./manage.py loaddata --format json -
Traceback (most recent call last):
File "/venv/lib/python3.6/site-packages/django/db/models/options.py", line 564, in get_field
return self.fields_map[field_name]
KeyError: 'description'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/venv/lib/python3.6/site-packages/django/core/serializers/json.py", line 69, in Deserializer
yield from PythonDeserializer(objects, **options)
File "/venv/lib/python3.6/site-packages/django/core/serializers/python.py", line 116, in Deserializer
field = Model._meta.get_field(field_name)
File "/venv/lib/python3.6/site-packages/django/db/models/options.py", line 566, in get_field
raise FieldDoesNotExist("%s has no field named '%s'" % (self.object_name, field_name))
django.core.exceptions.FieldDoesNotExist: Classification has no field named 'description'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "./manage.py", line 12, in <module>
execute_from_command_line(sys.argv)
File "/venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/venv/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(*args, **cmd_options)
File "/venv/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(*args, **options)
File "/venv/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 72, in handle
self.loaddata(fixture_labels)
File "/venv/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 113, in loaddata
self.load_label(fixture_label)
File "/venv/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 168, in load_label
for obj in objects:
File "/venv/lib/python3.6/site-packages/django/core/serializers/json.py", line 73, in Deserializer
raise DeserializationError() from exc
django.core.serializers.base.DeserializationError: Problem installing fixture '-':
In which version of Kiwi TCMS did you make your backup ?
It looks like backup is from an older version because 6.5 ships with migrations that remove Build.description and Classification.description fields!
I am not sure it is technically possible to handle this gracefully. Please file an issue on GitHub so we can investigate in more details and link back to this SO thread.
A work around for you will be to launch not the latest version of Kiwi TCMS but the version in which you made the backup. Then restore your data, upgrade to the latest version, run the migrations which will change DB schema and then backup again.
If you do not keep around your older docker image you will have to build it from source.
I am using Azure to run python notebook using Jupyterhub. After spinning up the VM, I was able to access the notebooks just by using my username and password (just like ssh). However, one day later when I switched to another network (I am not claiming that the network might have been a problem) I am unable to access the link. It gives me The site can't be reached error.
So I tried rerunning the process again, and since then I have been struggling to make it run again. I have searched for similar issues on GitHub, but they aren't helpful either.
After the kill the process using kill pid command, I tried running the jupyterhub through this command:
/anaconda/envs/py35/bin/python /anaconda/envs/py35/bin/jupyterhub-singleuser --port=50387 --notebook-dir="~/notebooks" --config=/etc/jupyterhub/jupyterhub_config.py
And it gives me the error:
JUPYTERHUB_API_TOKEN env is required to run jupyterhub-singleuser. Did you launch it manually?
So I searched through github issues similar to this. I tried generating token manually using:
jupyterhub token username
And I added that token to JUPYTERHUB_API_TOKEN via export JUPYTERHUB_API_TOKEN=token. I also added token:username to c.Authenticator.tokens in jupyterhub_config.py. Now I get this error:
Traceback (most recent call last):
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 528, in get
value = obj._trait_values[self.name]
KeyError: 'oauth_client_id'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda/envs/py35/bin/jupyterhub-singleuser", line 6, in <module>
main()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 455, in main
return SingleUserNotebookApp.launch_instance(argv)
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyter_core/application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "<decorator-gen-7>", line 2, in initialize
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/notebook/notebookapp.py", line 1296, in initialize
self.init_webapp()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 393, in init_webapp
self.init_hub_auth()
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 388, in init_hub_auth
if not self.hub_auth.oauth_client_id:
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 556, in __get__
return self.get(obj, cls)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 535, in get
value = self._validate(obj, dynamic_default())
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 593, in _validate
value = self._cross_validate(obj, value)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 599, in _cross_validate
value = obj._trait_validators[self.name](obj, proposal)
File "/anaconda/envs/py35/lib/python3.5/site-packages/traitlets/traitlets.py", line 907, in __call__
return self.func(*args, **kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/services/auth.py", line 439, in _ensure_not_empty
raise ValueError("%s cannot be empty." % proposal.trait.name)
ValueError: oauth_client_id cannot be empty.
I am not sure where I went wrong in this process. Anybody familiar with this issue?
Try running jupyterhub instead of jupyterhub-singleuser
For your specific use case, the command would be as follows:
sudo /anaconda/envs/py35/bin/python /anaconda/envs/py35/bin/jupyterhub --port=50387 --notebook-dir="~/notebooks" --config=/etc/jupyterhub/jupyterhub_config.py
Make sure that jupyterhub is installed (correctly) in the path you mentioned.
I am trying to buiil Dart-VM on windows, I follow the steps as described here
https://github.com/dart-lang/sdk/wiki/Building
When I run the build.py command as below:
.\tools\build.py --mode release --arch x64 create_sdk
I get the following error:
gn gen --check in out\ReleaseX64
Traceback (most recent call last):
File "D:\ops\dart\sdk\tools\gn.py", line 436, in <module>
sys.exit(main(sys.argv))
File "D:\ops\dart\sdk\tools\gn.py", line 423, in main
results = pool.map(run_command, commands, chunksize=1)
File "C:\app\Python27\lib\multiprocessing\pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "C:\app\Python27\lib\multiprocessing\pool.py", line 567, in get
raise self._value
WindowsError: [Error 2] System cannot find file。
Tried to run GN, but it failed. Try running it manually:
$ python D:\ops\dart\sdk\tools\gn.py -m release -a x64 --os host -v
Traceback (most recent call last):
File "D:\ops\dart\sdk\tools\build.py", line 658, in <module>
sys.exit(Main())
File "D:\ops\dart\sdk\tools\build.py", line 651, in Main
mode, arch, cross_build) != 0:
File "D:\ops\dart\sdk\tools\build.py", line 491, in BuildOneConfig
args = BuildNinjaCommand(options, target, target_os, mode, arch)
File "D:\ops\dart\sdk\tools\build.py", line 473, in BuildNinjaCommand
if UseGoma(out_dir):
File "D:\ops\dart\sdk\tools\build.py", line 431, in UseGoma
return 'use_goma = true' in open(args_gn, 'r').read()
IOError: [Errno 2] No such file or directory: 'out\\ReleaseX64\\args.gn'
It seems missing the args.gn file in out\ReleaseX64 folder. but I cannot find args.gn in Dart source folders. Is it generated during building process? whether I do wrong steps led to no such file generated?