I am trying to setup KSQLDB setup in my ubuntu 18.04. I am following the steps:
https://docs.ksqldb.io/en/latest/operate-and-deploy/installation/installing/
I am doing step by step how it saying.
But when I run docker-compose up I get this error:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker/api/client.py", line 214, in _retrieve_server_version
File "docker/api/daemon.py", line 181, in version
File "docker/utils/decorators.py", line 46, in inner
File "docker/api/client.py", line 237, in _get
File "requests/sessions.py", line 543, in get
File "requests/sessions.py", line 530, in request
File "requests/sessions.py", line 643, in send
File "requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', PermissionError(13, 'Permission denied'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker-compose", line 3, in <module>
File "compose/cli/main.py", line 80, in main
File "compose/cli/main.py", line 189, in perform_command
File "compose/cli/command.py", line 70, in project_from_options
File "compose/cli/command.py", line 153, in get_project
File "compose/cli/docker_client.py", line 43, in get_client
File "compose/cli/docker_client.py", line 170, in docker_client
File "docker/api/client.py", line 197, in __init__
File "docker/api/client.py", line 222, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))
[7677] Failed to execute script docker-compose
This is probably related to not having your linux user added to the docker group.
By default, you need to run Docker-related commands with sudo permissions.
sudo docker-compose up
To add your user to the group, do this:
Create the group
sudo groupadd docker
Add your current user to the docker group
sudo usermod -aG docker $USER
Reload group permissions
newgrp docker
Related
I have Google Cloud SDK docker configured and running on my windows machine after following this.
https://hub.docker.com/r/google/cloud-sdk/
I'm trying to run this command to list a s3 bucket
docker run --rm -ti --volumes-from gcloud-config google/cloud-sdk gsutil ls s3://bucketname
Authentication fails due to not setting the AWS keys. I presume from .boto file not having aws_access_key_id and aws_secret_access_key set. I can't seem to figure out how to set those variables.
I tried to run this to generate a .boto file but the bucket was shared with me and I don't have access keys.
docker run --rm -ti --volumes-from gcloud-config google/cloud-sdk gsutil config -a
Am I missing something or is there any other way to set these AWS credentials? Maybe with gcloud config set?
Here is the error log
ERROR 1202 03:16:07.326810 utils.py] Caught exception reading instance data
Traceback (most recent call last):
File "/usr/lib/python3.7/urllib/request.py", line 1324, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/usr/lib/python3.7/http/client.py", line 1260, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1306, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1255, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1030, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 970, in send
self.connect()
File "/usr/lib/python3.7/http/client.py", line 942, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/lib/python3.7/socket.py", line 727, in create_connection
raise err
File "/usr/lib/python3.7/socket.py", line 716, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/utils.py", line 220, in retry_url
r = opener.open(req, timeout=timeout)
File "/usr/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 1352, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.7/urllib/request.py", line 1326, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 111] Connection refused>
ERROR 1202 03:16:07.328018 utils.py] Unable to read instance data, giving up
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gsutil", line 21, in <module>
gsutil.RunMain()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gsutil.py", line 122, in RunMain
sys.exit(gslib.__main__.main())
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 444, in main
user_project=user_project)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 783, in _RunNamedCommandAndHandleExceptions
_HandleUnknownFailure(e)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 640, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 412, in RunNamedCommand
return_code = command_inst.RunCommand()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/ls.py", line 683, in RunCommand
listing_helper.ExpandUrlAndPrint(storage_url))
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/ls_helper.py", line 372, in ExpandUrlAndPrint
print_initial_newline=False)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/ls_helper.py", line 449, in _RecurseExpandUrlAndPrint
bucket_listing_fields=self.bucket_listing_fields):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/wildcard_iterator.py", line 540, in IterAll
expand_top_level_buckets=expand_top_level_buckets):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/wildcard_iterator.py", line 174, in __iter__
fields=bucket_listing_fields):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/boto_translation.py", line 447, in ListObjects
headers=headers)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 166, in list_bucket
bucket = self.get_bucket(headers=headers)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 181, in get_bucket
conn = self.connect()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 117, in connect
**connection_args)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/s3/connection.py", line 205, in __init__
validate_certs=validate_certs, profile_name=profile_name)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py", line 573, in __init__
host, config, self.provider, self._required_auth_capability())
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/auth.py", line 1032, in get_auth_handler
'Check your credentials' % (len(names), str(names)))
boto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['S3HmacAuthV4Handler'] Check your credentials
I edited the .boto file in the legacy config and got this to work.
docker restart gcloud-config
docker exec -u 0 -it <container-id-here>/bin/bash
apt-get install nano
nano root/.config/gcloud/legacy_credentials/***/.boto
add under [Credentials]
aws_access_key_id = ***
aws_secret_access_key = ***
enjoy
docker run --rm -ti --volumes-from gcloud-config google/cloud-sdk gsutil ls s3://bucketname
I am trying to compose a Docker file with a YML file as below.
version: '3.6'
services:
postgres:
image: postgres:12
restart: always
volumes:
- db_data:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: postgrespassword
graphql-engine:
image: hasura/graphql-engine:v2.0.3
ports:
- "8080:8080"
depends_on:
- "postgres"
restart: always
environment:
HASURA_GRAPHQL_METADATA_DATABASE_URL: postgres://postgres:postgrespassword#postgres:5432/postgres
## enable the console served by server
HASURA_GRAPHQL_ENABLE_CONSOLE: "true" # set to "false" to disable console
## enable debugging mode. It is recommended to disable this in production
HASURA_GRAPHQL_DEV_MODE: "true"
HASURA_GRAPHQL_ENABLED_LOG_TYPES: startup, http-log, webhook-log, websocket-log, query-log
## uncomment next line to set an admin secret
# HASURA_GRAPHQL_ADMIN_SECRET: myadminsecretkey
volumes:
db_data:
But am getting the following error:
hasura-ais docker-compose up -d
Traceback (most recent call last):
File "site-packages/urllib3/connectionpool.py", line 677, in urlopen
File "site-packages/urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1252, in request
File "http/client.py", line 1298, in _send_request
File "http/client.py", line 1247, in endheaders
File "http/client.py", line 1026, in _send_output
File "http/client.py", line 966, in send
File "site-packages/docker/transport/unixconn.py", line 43, in connect
FileNotFoundError: [Errno 2] No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "site-packages/requests/adapters.py", line 449, in send
File "site-packages/urllib3/connectionpool.py", line 727, in urlopen
File "site-packages/urllib3/util/retry.py", line 403, in increment
File "site-packages/urllib3/packages/six.py", line 734, in reraise
File "site-packages/urllib3/connectionpool.py", line 677, in urlopen
File "site-packages/urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1252, in request
File "http/client.py", line 1298, in _send_request
File "http/client.py", line 1247, in endheaders
File "http/client.py", line 1026, in _send_output
File "http/client.py", line 966, in send
File "site-packages/docker/transport/unixconn.py", line 43, in connect
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "site-packages/docker/api/client.py", line 205, in _retrieve_server_version
File "site-packages/docker/api/daemon.py", line 181, in version
File "site-packages/docker/utils/decorators.py", line 46, in inner
File "site-packages/docker/api/client.py", line 228, in _get
File "site-packages/requests/sessions.py", line 543, in get
File "site-packages/requests/sessions.py", line 530, in request
File "site-packages/requests/sessions.py", line 643, in send
File "site-packages/requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker-compose", line 3, in <module>
File "compose/cli/main.py", line 67, in main
File "compose/cli/main.py", line 123, in perform_command
File "compose/cli/command.py", line 69, in project_from_options
File "compose/cli/command.py", line 132, in get_project
File "compose/cli/docker_client.py", line 43, in get_client
File "compose/cli/docker_client.py", line 170, in docker_client
File "site-packages/docker/api/client.py", line 188, in __init__
File "site-packages/docker/api/client.py", line 213, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
[18722] Failed to execute script docker-compose
Ran into the same issue. For some reason, opening and closing Docker Desktop (I'm on Mac) fixed it. Not sure why.
In case of snap-installed docker in a GNU/Linux machine try running:
docker --version
Then if the error is:
cannot update snap namespace: cannot create symlink in "/etc/docker": existing file in the way
snap-update-ns failed with code 1
Run:
sudo mv /etc/docker /etc/docker_backup
sudo snap refresh docker
Then the error wont occur any more.
I had the same issue while using volumes. I created the folders inside the mounted volume and that fixed it.
Inside my docker, I had a folder like this:
/data/logs
/data/files
and was mounting my local empty /data folder. It was fixed after creating logs and files folders inside my local /data.
I am getting the below error while running docker-compose up in windows server 2016.
docker compose version is docker-compose version 1.27.4, build 40524192
enter image description here
Traceback (most recent call last):
File "site-packages\docker\api\client.py", line 205, in _retrieve_server_version
File "site-packages\docker\api\daemon.py", line 181, in version
File "site-packages\docker\utils\decorators.py", line 46, in inner
File "site-packages\docker\api\client.py", line 228, in _get
File "site-packages\requests\sessions.py", line 543, in get
File "site-packages\requests\sessions.py", line 530, in request
File "site-packages\requests\sessions.py", line 643, in send
File "site-packages\requests\adapters.py", line 449, in send
File "site-packages\urllib3\connectionpool.py", line 677, in urlopen
File "site-packages\urllib3\connectionpool.py", line 392, in _make_request
File "http\client.py", line 1244, in request
File "http\client.py", line 1290, in _send_request
File "http\client.py", line 1239, in endheaders
File "http\client.py", line 1026, in _send_output
File "http\client.py", line 966, in send
File "site-packages\docker\transport\npipeconn.py", line 32, in connect
File "site-packages\docker\transport\npipesocket.py", line 23, in wrapped
File "site-packages\docker\transport\npipesocket.py", line 72, in connect
File "site-packages\docker\transport\npipesocket.py", line 59, in connect
pywintypes.error: (2, 'CreateFile', 'The system cannot find the file specified.')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker-compose", line 3, in <module>
File "compose\cli\main.py", line 67, in main
File "compose\cli\main.py", line 123, in perform_command
File "compose\cli\command.py", line 69, in project_from_options
File "compose\cli\command.py", line 132, in get_project
File "compose\cli\docker_client.py", line 43, in get_client
File "compose\cli\docker_client.py", line 170, in docker_client
File "site-packages\docker\api\client.py", line 188, in __init__
File "site-packages\docker\api\client.py", line 213, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: (2, 'CreateFile', 'The system cannot find the file specified.')
[4332] Failed to execute script docker-compose
Thanks in Advance
I just ran into the exact same symptoms. Turns out Docker wasn't running, and starting Docker solved the problem.
Your docker is probably not running on your machine. Just open Docker Desktop on your computer then go the Containers section to see if its running. If it is running, switch it off then go back to your project and run "docker-compose up --build "
To preface this, I am running docker in a Ubuntu 20.04 VM on virtualbox.
I created a simple shell script to kill any process running on port 9042, then start my docker-compose file. Here is the script in question:
#!/bin/bash
# Check for and kill any processes running on port 9042
sudo kill -9 $(sudo lsof -t -i:9042)
# start docker-compose
docker-compose -f ./docker/docker-compose.yml up
Since running that, however, it has made my docker installation completely unresponsive to any sort of interaction. Any docker commands will hang indefinitely until cancelled with Ctrl+C, and any other system commands that use docker (such as sudo service docker start) will also hang indefinitely.
If I try to run dockerd, it fails with the message failed to start daemon: pid file found, ensure docker is not running or delete /var/run/docker.pid. As my system reports that docker is not running, I go ahead and delete var/run/docker.pid. If I then try to run dockerd again, I get a different error message: failed to start daemon: error while opening volume store metadata database: timeout.
At this stage, some of the docker commands start working again. docker version and docker help both work, but it is still reported that the docker daemon is not running. Attempting to run docker-compose up on a docker-compose file produces this output:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.8/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 950, in send
self.connect()
File "/home/david/.local/lib/python3.8/site-packages/docker/transport/unixconn.py", line 43, in connect
sock.connect(self.unix_socket)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
retries = retries.increment(
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 702, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.8/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 950, in send
self.connect()
File "/home/david/.local/lib/python3.8/site-packages/docker/transport/unixconn.py", line 43, in connect
sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/david/.local/lib/python3.8/site-packages/docker/api/client.py", line 205, in _retrieve_server_version
return self.version(api_version=False)["ApiVersion"]
File "/home/david/.local/lib/python3.8/site-packages/docker/api/daemon.py", line 181, in version
return self._result(self._get(url), json=True)
File "/home/david/.local/lib/python3.8/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/home/david/.local/lib/python3.8/site-packages/docker/api/client.py", line 228, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 546, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/david/.local/bin/docker-compose", line 8, in <module>
sys.exit(main())
File "/home/david/.local/lib/python3.8/site-packages/compose/cli/main.py", line 67, in main
command()
File "/home/david/.local/lib/python3.8/site-packages/compose/cli/main.py", line 123, in perform_command
project = project_from_options('.', options)
File "/home/david/.local/lib/python3.8/site-packages/compose/cli/command.py", line 60, in project_from_options
return get_project(
File "/home/david/.local/lib/python3.8/site-packages/compose/cli/command.py", line 131, in get_project
client = get_client(
File "/home/david/.local/lib/python3.8/site-packages/compose/cli/docker_client.py", line 41, in get_client
client = docker_client(
File "/home/david/.local/lib/python3.8/site-packages/compose/cli/docker_client.py", line 170, in docker_client
client = APIClient(**kwargs)
File "/home/david/.local/lib/python3.8/site-packages/docker/api/client.py", line 188, in __init__
self._version = self._retrieve_server_version()
File "/home/david/.local/lib/python3.8/site-packages/docker/api/client.py", line 212, in _retrieve_server_version
raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
Other system commands such as sudo service docker start still hang indefinitely until killed.
I have tried every single solution in this thread (Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running?) and this one (Docker commands do not respond anymore), but none of them work.
Does anyone know what could be the issue here?
EDIT: A few more points -
The docker.pid file reappears again when I restart my VM
Restarting my VM does not do anything to remedy the problem
Executing commands as the root user likewise doesn't do anything
Trying to reinstall docker using sudo apt-get install --reinstall docker-ce also hangs at the stage Preparing to unpack .../docker-ce_5%3a20.10.0~1.1.beta1-0~ubuntu-focal_amd64.deb ...
I know this is super late, but I found an answer from another similar question I asked.
Docker containers are stored in the default location at /var/lib/docker/ on Linux. I was able to identify the container that was causing the issue and deleted the actual container files. I then used the CLI to remove all other traces of the container, and docker was able to start running as normal.
Obviously doing this is risky, so make sure you take adequate steps to back up your machine first.
I am pretty amateur to docker. I was running my application on docker successfully from many days.
Now when I run the command docker-compose up -d it gives me the error as below
raceback (most recent call last):
File "docker-compose", line 3, in <module>
File "compose/cli/main.py", line 88, in main
File "compose/cli/main.py", line 140, in perform_command
File "compose/cli/main.py", line 900, in up
File "compose/project.py", line 385, in up
File "compose/project.py", line 590, in warn_for_swarm_mode
File "site-packages/docker/api/daemon.py", line 73, in info
File "site-packages/docker/utils/decorators.py", line 47, in inner
File "site-packages/docker/api/client.py", line 179, in _get
File "site-packages/requests/sessions.py", line 488, in get
File "site-packages/requests/sessions.py", line 466, in request
File "site-packages/requests/sessions.py", line 641, in
merge_environment_settings
File "site-packages/requests/utils.py", line 605, in get_environ_proxies
File "site-packages/requests/utils.py", line 589, in should_bypass_proxies
File "urllib.py", line 1510, in proxy_bypass
File "urllib.py", line 1484, in proxy_bypass_macosx_sysconf
ValueError: negative shift count
Failed to execute script docker-compose
I searched this error ValueError: negative shift count a lot but nothing useful found.