I am using the following docker builder to cross-compile an Alpine-based docker image:
Name: builder_name
Driver: docker-container
Nodes:
Name: builder_name0
Endpoint: unix:///var/run/docker.sock
Status: running
Buildkit: v0.10.5
Platforms: linux/arm/v7*, linux/amd64*, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386
Now, when running the command
$ docker buildx build --rm --platform linux/arm/v7 -t container_name:latest --file ./Dockerfile --load ..
the builder fails to install redis-py when running
RUN pip3 install redis
Here are the error messages:
------
> [stage-1 8/37] RUN pip3 install redis:
#0 15.00 ERROR: Exception:
#0 15.00 Traceback (most recent call last):
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
#0 15.00 status = run_func(*args)
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
#0 15.00 return func(self, options, args)
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 287, in run
#0 15.00 session = self.get_default_session(options)
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 75, in get_default_session
#0 15.00 self._session = self.enter_context(self._build_session(options))
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 89, in _build_session
#0 15.00 session = PipSession(
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_internal/network/session.py", line 282, in __init__
#0 15.00 self.headers["User-Agent"] = user_agent()
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_internal/network/session.py", line 122, in user_agent
#0 15.00 linux_distribution = distro.name(), distro.version(), distro.codename()
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_vendor/distro/distro.py", line 345, in version
#0 15.00 return _distro.version(pretty, best)
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_vendor/distro/distro.py", line 874, in version
#0 15.00 self.uname_attr("release"),
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_vendor/distro/distro.py", line 1059, in uname_attr
#0 15.00 return self._uname_info.get(attribute, "")
#0 15.00 File "/usr/lib/python3.10/functools.py", line 981, in __get__
#0 15.00 val = self.func(instance)
#0 15.00 File "/usr/lib/python3.10/site-packages/pip/_vendor/distro/distro.py", line 1173, in _uname_info
#0 15.00 stdout = subprocess.check_output(cmd, stderr=subprocess.DEVNULL)
#0 15.00 File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
#0 15.00 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
#0 15.00 File "/usr/lib/python3.10/subprocess.py", line 524, in run
#0 15.00 raise CalledProcessError(retcode, process.args,
#0 15.00 subprocess.CalledProcessError: Command '('uname', '-rs')' returned non-zero exit status 1.
#0 15.01 Traceback (most recent call last):
#0 15.01 File "/usr/bin/pip3", line 33, in <module>
#0 15.01 sys.exit(load_entry_point('pip==22.1.1', 'console_scripts', 'pip3')())
#0 15.01 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/main.py", line 70, in main
#0 15.01 return command.main(cmd_args)
#0 15.01 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 101, in main
#0 15.01 return self._main(args)
#0 15.01 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
#0 15.02 self.handle_pip_version_check(options)
#0 15.02 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 144, in handle_pip_version_check
#0 15.02 session = self._build_session(
#0 15.02 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 89, in _build_session
#0 15.02 session = PipSession(
#0 15.02 File "/usr/lib/python3.10/site-packages/pip/_internal/network/session.py", line 282, in __init__
#0 15.02 self.headers["User-Agent"] = user_agent()
#0 15.02 File "/usr/lib/python3.10/site-packages/pip/_internal/network/session.py", line 122, in user_agent
#0 15.02 linux_distribution = distro.name(), distro.version(), distro.codename()
#0 15.02 File "/usr/lib/python3.10/site-packages/pip/_vendor/distro/distro.py", line 345, in version
#0 15.02 return _distro.version(pretty, best)
#0 15.02 File "/usr/lib/python3.10/site-packages/pip/_vendor/distro/distro.py", line 874, in version
#0 15.02 self.uname_attr("release"),
#0 15.02 File "/usr/lib/python3.10/site-packages/pip/_vendor/distro/distro.py", line 1059, in uname_attr
#0 15.03 return self._uname_info.get(attribute, "")
#0 15.03 File "/usr/lib/python3.10/functools.py", line 981, in __get__
#0 15.03 val = self.func(instance)
#0 15.03 File "/usr/lib/python3.10/site-packages/pip/_vendor/distro/distro.py", line 1173, in _uname_info
#0 15.04 stdout = subprocess.check_output(cmd, stderr=subprocess.DEVNULL)
#0 15.04 File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
#0 15.04 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
#0 15.04 File "/usr/lib/python3.10/subprocess.py", line 524, in run
#0 15.04 raise CalledProcessError(retcode, process.args,
#0 15.04 subprocess.CalledProcessError: Command '('uname', '-rs')' returned non-zero exit status 1.
------
Has anyone come across an issue like this? I tried searching everywhere for cases when uname -rs would return a non-zero status but could not find anything at all.
In case you did not solve this issue, see
https://github.com/nicolabs/nicobot/issues/60
The last comment solved the issue for me:
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
Related
I am trying to trigger jenkins build with GitHub Action using following workflow file
name: Trigger Jenkins Build [ Backend ]
on:
pull_request:
branches: [ master ]
types: [ closed ]
env:
JENKINS_HOST: "http://jenkins.example.net:8080/"
JENKINS_JOB: "job/Backend/"
JENKINS_SECURITY_GROUP_ID: "sg-#####"
jobs:
trigger:
name: Triggering Jenkins Build
runs-on: ubuntu-latest
steps:
- name: Add public IP to AWS security group
uses: sohelamin/aws-security-group-add-ip-action#master
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: 'us-west-2'
aws-security-group-id: ${{ env.JENKINS_SECURITY_GROUP_ID }}
port: '8080'
to-port: ''
protocol: 'tcp'
description: 'GitHub Action'
- name: Triggering Jenkins Build [ Backend ]
if: github.event.pull_request.merged == true
uses: GoldenspearLLC/build-jenkins-job#master
with:
jenkins-url: ${{ env.JENKINS_HOST }}
user: ${{ secrets.JENKINS_USER }}
jenkins-token: ${{ secrets.JENKINS_USER_TOKEN }}
job-path: ${{ env.JENKINS_JOB }}
job-params: "{'FRESH_BUILD':'True', 'BUILD_BRANCH':'master', 'DEPLOY_DEV':'True', 'DEPLOY_QA':'False, 'DEPLOY_STAGING':'False, 'DEPLOY_PRODUCTION':'False'}"
But Github Action fails for Triggering Jenkins Build [ Backend ] with following error
Run GoldenspearLLC/build-jenkins-job#master
with:
jenkins-url: http://jenkins.example.net:8080/
user: ***
jenkins-token: ***
job-path: job/Backend/
job-params: {'FRESH_BUILD':'True', 'BUILD_BRANCH':'master', 'DEPLOY_DEV':'True', 'DEPLOY_QA':'False, 'DEPLOY_STAGING':'False, 'DEPLOY_PRODUCTION':'False'}
env:
JENKINS_HOST: http://jenkins.example.net:8080/
JENKINS_JOB: job/Backend/
JENKINS_SECURITY_GROUP_ID: sg-####
/usr/bin/docker run --name a33c1e344d347fc1c49778e11b0ded4abedc6_771244 --label 8a33c1 --workdir /***/workspace --rm -e JENKINS_HOST -e JENKINS_JOB -e JENKINS_SECURITY_GROUP_ID -e INPUT_JENKINS-URL -e INPUT_USER -e INPUT_JENKINS-TOKEN -e INPUT_JOB-PATH -e INPUT_JOB-PARAMS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_***_home":"/***/home" -v "/home/runner/work/_temp/_***_workflow":"/***/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/***/file_commands" -v "/home/runner/work/Backend/Backend":"/***/workspace" 8a33c1:e344d347fc1c49778e11b0ded4abedc6 "http://jenkins.example.net:8080/" "***" "***" "job/Backend/" "{'FRESH_BUILD':'True', 'BUILD_BRANCH':'master', 'DEPLOY_DEV':'True', 'DEPLOY_QA':'False, 'DEPLOY_STAGING':'False, 'DEPLOY_PRODUCTION':'False'}"
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.8/http/client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.8/http/client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.8/http/client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.8/http/client.py", line 1007, in _send_output
self.send(msg)
File "/usr/local/lib/python3.8/http/client.py", line 947, in send
self.connect()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 187, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f773f7fc5b0>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 446, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='http', port=80): Max retries exceeded with url: /jenkins.example.net:8080/crumbIssuer/api/json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f773f7fc5b0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/entrypoint.py", line 29, in <module>
user = server.get_whoami()
File "/usr/local/lib/python3.8/site-packages/jenkins/__init__.py", line 793, in get_whoami
response = self.jenkins_open(requests.Request(
File "/usr/local/lib/python3.8/site-packages/jenkins/__init__.py", line 557, in jenkins_open
return self.jenkins_request(req, add_crumb, resolve_auth).text
File "/usr/local/lib/python3.8/site-packages/jenkins/__init__.py", line 573, in jenkins_request
self.maybe_add_crumb(req)
File "/usr/local/lib/python3.8/site-packages/jenkins/__init__.py", line 370, in maybe_add_crumb
response = self.jenkins_open(requests.Request(
File "/usr/local/lib/python3.8/site-packages/jenkins/__init__.py", line 557, in jenkins_open
return self.jenkins_request(req, add_crumb, resolve_auth).text
File "/usr/local/lib/python3.8/site-packages/jenkins/__init__.py", line 576, in jenkins_request
self._request(req))
File "/usr/local/lib/python3.8/site-packages/jenkins/__init__.py", line 550, in _request
return self._session.send(r, **_settings)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='http', port=80): Max retries exceeded with url: /jenkins.example.net:8080/crumbIssuer/api/json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f773f7fc5b0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
During the build running I can see step sohelamin/aws-security-group-add-ip-action#master adds IP to security group and removes at the end.
Why it tries to connect on port=80 instead of port=8080
What I am missing here ?
The error shows that you are wrong in Jenkin domain: http://jenkins.example.net:8080/
Replace it with your correct domain
Error message: "socket.gaierror: [Errno -2] Name or service not known" mean your program cannot make a connection to this domain because it does not exist or DNS can not resolve
checking action source reveals that it constructs URL merely by glueing 'http://' to it [1]
try remove 'http://' of your action parameters, e.g. run with
JENKINS_HOST: jenkins.example.net:8080/
instead of
JENKINS_HOST: http://jenkins.example.net:8080/
[1] - https://github.com/GoldenspearLLC/build-jenkins-job/blob/master/entrypoint.py#L28
UPDATE: error stack is easily reproduced, so I guess, url format is the real cause.
import requests
sess = requests.Session()
r = sess.prepare_request(requests.Request('GET', 'https://https://www.google.com/'))
sess.send(r)
gives :
Traceback (most recent call last):
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/local/Cellar/python#3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 953, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x10e1ce340>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='http', port=443): Max retries exceeded with url: //www.google.com/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x10e1ce340>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/Users/m.shonichev/src/venv/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='http', port=443): Max retries exceeded with url: //www.google.com/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x10e1ce340>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
Given this Dockerfile:
FROM python:3.8-slim-buster
EXPOSE 5000
ADD . /data/sample.csv
# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1
# Install pip requirements
COPY requirements.txt .
RUN python -m pip install -r requirements.txt
WORKDIR /app
COPY . /app
I can run this flask app on my local machine, no errors. Pandas grabs the data from the csv and displays it in a return statement from the function (below):
from flask import Flask
import pandas as pd
app = Flask(__name__)
df = pd.read_csv("data\sample.csv")
lst_one = df['col1'].to_list()
lst_two = df['col2'].to_list()
lst_three = df['col3'].to_list()
lang = "World!"
#app.route('/')
def hello_world():
return f'Hello, {lang} {lst_one}, {lst_three}'
However, after creating Dockerfile and requirements.txt and building image, when running the container, this error persists. (Notice how the error message has the file path slashes going in the opposite direction from those in my code???)
[2021-04-30 01:34:04 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2021-04-30 01:34:04 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2021-04-30 01:34:04 +0000] [1] [INFO] Using worker: sync
[2021-04-30 01:34:04 +0000] [8] [INFO] Booting worker with pid: 8
[2021-04-30 01:34:05 +0000] [8] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 358, in import_app
mod = importlib.import_module(module)
File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/app/app.py", line 5, in <module>
df = pd.read_csv("data\sample.csv")
File "/usr/local/lib/python3.8/site-packages/pandas/io/parsers.py", line 610, in read_csv
return _read(filepath_or_buffer, kwds)
File "/usr/local/lib/python3.8/site-packages/pandas/io/parsers.py", line 462, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/usr/local/lib/python3.8/site-packages/pandas/io/parsers.py", line 819, in __init__
self._engine = self._make_engine(self.engine)
File "/usr/local/lib/python3.8/site-packages/pandas/io/parsers.py", line 1050, in _make_engine
return mapping[engine](self.f, **self.options) # type: ignore[call-arg]
File "/usr/local/lib/python3.8/site-packages/pandas/io/parsers.py", line 1867, in __init__
self._open_handles(src, kwds)
File "/usr/local/lib/python3.8/site-packages/pandas/io/parsers.py", line 1362, in _open_handles
self.handles = get_handle(
File "/usr/local/lib/python3.8/site-packages/pandas/io/common.py", line 642, in get_handle
handle = open(
***FileNotFoundError: [Errno 2] No such file or directory: 'data\\sample.csv'***
[2021-04-30 01:34:05 +0000] [8] [INFO] Worker exiting (pid: 8)
[2021-04-30 01:34:05 +0000] [1] [INFO] Shutting down: Master
[2021-04-30 01:34:05 +0000] [1] [INFO] Reason: Worker failed to boot
Tried a different path, an absolute path, everything and Docker just cannot recognize that this directory/file exists. The actual app I need to containerize has to be able to access data outside the container, eventually, but as a first step, I cannot even get Docker to recognize this data WITHIN the app. So, all you Docker experts, please step up and let me know what presumably simple thing I am obviously missing here. Thank you so much.
I'm trying to deploy a docker image to a remote server using docker-compose but I'm getting authentication issue:
Command:
docker-compose -H "ssh://root#host_ip" --tlscacert "~/.ssh/private_key" ps
Output:
sudo docker-compose -H "ssh://root#jenkins.evercam.io" --tlscacert "~/.ssh/id_jenkins" -f evercam_camera_server/docker-compose.yml ps
Traceback (most recent call last):
File "bin/docker-compose", line 3, in <module>
File "compose/cli/main.py", line 67, in main
File "compose/cli/main.py", line 123, in perform_command
File "compose/cli/command.py", line 69, in project_from_options
File "compose/cli/command.py", line 132, in get_project
File "compose/cli/docker_client.py", line 43, in get_client
File "compose/cli/docker_client.py", line 170, in docker_client
File "docker/api/client.py", line 164, in __init__
File "docker/transport/sshconn.py", line 113, in __init__
File "docker/transport/sshconn.py", line 121, in _connect
File "paramiko/client.py", line 446, in connect
File "paramiko/client.py", line 764, in _auth
File "paramiko/client.py", line 740, in _auth
File "paramiko/transport.py", line 1580, in auth_publickey
File "paramiko/auth_handler.py", line 250, in wait_for_response
paramiko.ssh_exception.AuthenticationException: Authentication failed.
[28861] Failed to execute script docker-compose
docker-compose version 1.27.4, build 40524192
docker version 20.10.5, build 55c4c88
ubuntu version 18.04
I tried with a recent version of docker compose which is 1.28.5 it's able to access the given host, but strangely it's asking the password five times!
I successfully installed docker and nvidia-docker on ubuntu 18.04
I pull this image from NVIDIA's GPU cloud
https://ngc.nvidia.com/catalog/containers/nvidia:caffe
and ran it with this command
nvidia-docker run -it --rm -v /home/stefan/Dropbox:/data -p 8888:8888 nvcr.io/nvidia/caffe:19.03-py2 sh
The container gives me a shell prompt and it seems to work, for example
# nvidia-smi
results in
Sat Mar 30 21:03:30 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.39 Driver Version: 418.39 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 105... On | 00000000:01:00.0 On | N/A |
| 20% 30C P8 N/A / 75W | 441MiB / 4038MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
It sees my wimpy gpu. I try to run jupyter with this command
#jupyter-notebook
but I get
[I 21:05:18.088 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "</usr/local/lib/python2.7/dist-packages/decorator.pyc:decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python2.7/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/notebook/notebookapp.py", line 1628, in initialize
self.init_webapp()
File "/usr/local/lib/python2.7/dist-packages/notebook/notebookapp.py", line 1407, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python2.7/dist-packages/tornado/tcpserver.py", line 143, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python2.7/dist-packages/tornado/netutil.py", line 168, in bind_sockets
sock.bind(sockaddr)
File "/usr/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 99] Cannot assign requested address
I know jupyter is installed in the container because when I type
#jupyter --version
I get
4.4.0
Typing
# jupyter
gives
usage: jupyter [-h] [--version] [--config-dir] [--data-dir] [--runtime-dir]
[--paths] [--json]
[subcommand]
jupyter: error: one of the arguments --version subcommand --config-dir --data-dir --runtime-dir --paths is required
I have several notebooks in the host directory I attached to the container
# ls
NBA.ipynb exponents.ipynb hello_deep_learning-master
but nothing seems to work
# jupyter NBA.ipynb
Error executing Jupyter command 'NBA.ipynb': [Errno 2] No such file or directory
# jupyter notebook NBA.ipynb
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "</usr/local/lib/python2.7/dist-packages/decorator.pyc:decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python2.7/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/notebook/notebookapp.py", line 1628, in initialize
self.init_webapp()
File "/usr/local/lib/python2.7/dist-packages/notebook/notebookapp.py", line 1407, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python2.7/dist-packages/tornado/tcpserver.py", line 143, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python2.7/dist-packages/tornado/netutil.py", line 168, in bind_sockets
sock.bind(sockaddr)
File "/usr/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 99] Cannot assign requested address
# jupyter-notebook NBA.ipynb
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "</usr/local/lib/python2.7/dist-packages/decorator.pyc:decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python2.7/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/notebook/notebookapp.py", line 1628, in initialize
self.init_webapp()
File "/usr/local/lib/python2.7/dist-packages/notebook/notebookapp.py", line 1407, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python2.7/dist-packages/tornado/tcpserver.py", line 143, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python2.7/dist-packages/tornado/netutil.py", line 168, in bind_sockets
sock.bind(sockaddr)
File "/usr/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 99] Cannot assign requested address
I think it's a syntax issue because this works
docker run -it --rm -v ~/Dropbox:/tf/notebooks -p 8888:8888 tensorflow/tensorflow:latest-py3-jupyter
It starts the jupyter server in the container and in a browser I can open a notebook at 127.0.0.1 which shows a directory where I can see a folder named 'notebooks' which contains my Dropbox contents. Just as expected since I mounted my dropbox folder as a volume in the command above.
But if I type this
docker run -it --rm -v ~/Dropbox:/tf/notebooks -p 8888:8888 tensorflow/tensorflow:latest-py3-jupyter sh
I am in a shell but cannot start jupyter. I get the same error as I was before with the nvcr.io/nvidia/caffe image. How can I start jupyter AFTER I am in the running docker container shell?
I think I figured it out. At the shell prompt of the container , I type
jupyter notebook --ip=0.0.0.0 --allow-root
I'll leave this here in case any other noob like me has a similar problem. (Unless the moderator feels it should be edited or nuked)
I am trying to push docker image to a remote docker registry via Jenkins and ansible playbook.
my ansible is:
---
- hosts: localhost
remote_user: root
vars:
git_branch: "{{ GIT_BRANCH|default('development') }}"
tasks:
- name: Log into Docker registry
docker_login:
registry_url: https://registry.....si
username: user
password: ....
- name: Remove images
docker_image:
state: absent
name: registry.....si/project/restAPI-{{ git_branch }}
- name: Create Rest API django image
docker_image:
path: .
name: registry.....si/project/restAPI-{{ git_branch }}
push: yes
the first 2 tasks execute normally, but the last one give me an error :
TASK [Create Rest API django image] ********************************************
task path: /var/jenkins_home/workspace/BSH-Rest-API/restAPI/build_docker.yml:18
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/core/cloud/docker/docker_image.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: jenkins
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1543405086.47-86277873769632 `" && echo ansible-tmp-1543405086.47-86277873769632="` echo ~/.ansible/tmp/ansible-tmp-1543405086.47-86277873769632 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpuIkFE1 TO /var/jenkins_home/.ansible/tmp/ansible-tmp-1543405086.47-86277873769632/docker_image.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /var/jenkins_home/.ansible/tmp/ansible-tmp-1543405086.47-86277873769632/ /var/jenkins_home/.ansible/tmp/ansible-tmp-1543405086.47-86277873769632/docker_image.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /var/jenkins_home/.ansible/tmp/ansible-tmp-1543405086.47-86277873769632/docker_image.py; rm -rf "/var/jenkins_home/.ansible/tmp/ansible-tmp-1543405086.47-86277873769632/" > /dev/null 2>&1 && sleep 0'
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
File "/tmp/ansible_OWiZI2/ansible_module_docker_image.py", line 593, in <module>
main()
File "/tmp/ansible_OWiZI2/ansible_module_docker_image.py", line 585, in main
ImageManager(client, results)
File "/tmp/ansible_OWiZI2/ansible_module_docker_image.py", line 291, in __init__
self.present()
File "/tmp/ansible_OWiZI2/ansible_module_docker_image.py", line 319, in present
self.results['image'] = self.build_image()
File "/tmp/ansible_OWiZI2/ansible_module_docker_image.py", line 514, in build_image
for line in self.client.build(**params):
File "/usr/local/lib/python2.7/dist-packages/docker/api/build.py", line 105, in build
timeout=timeout,
File "/usr/local/lib/python2.7/dist-packages/docker/utils/decorators.py", line 47, in inner
return f(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/docker/client.py", line 135, in _post
return self.post(url, **self._set_request_timeout(kwargs))
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 581, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))
fatal: [localhost]: FAILED! => {
"changed": false,
"failed": true,
"invocation": {
"module_name": "docker_image"
},
"module_stderr": "Traceback (most recent call last):\n File \"/tmp/ansible_OWiZI2/ansible_module_docker_image.py\", line 593, in <module>\n main()\n File \"/tmp/ansible_OWiZI2/ansible_module_docker_image.py\", line 585, in main\n ImageManager(client, results)\n File \"/tmp/ansible_OWiZI2/ansible_module_docker_image.py\", line 291, in __init__\n self.present()\n File \"/tmp/ansible_OWiZI2/ansible_module_docker_image.py\", line 319, in present\n self.results['image'] = self.build_image()\n File \"/tmp/ansible_OWiZI2/ansible_module_docker_image.py\", line 514, in build_image\n for line in self.client.build(**params):\n File \"/usr/local/lib/python2.7/dist-packages/docker/api/build.py\", line 105, in build\n timeout=timeout,\n File \"/usr/local/lib/python2.7/dist-packages/docker/utils/decorators.py\", line 47, in inner\n return f(self, *args, **kwargs)\n File \"/usr/local/lib/python2.7/dist-packages/docker/client.py\", line 135, in _post\n return self.post(url, **self._set_request_timeout(kwargs))\n File \"/usr/local/lib/python2.7/dist-packages/requests/sessions.py\", line 581, in post\n return self.request('POST', url, data=data, json=json, **kwargs)\n File \"/usr/local/lib/python2.7/dist-packages/requests/sessions.py\", line 533, in request\n resp = self.send(prep, **send_kwargs)\n File \"/usr/local/lib/python2.7/dist-packages/requests/sessions.py\", line 646, in send\n r = adapter.send(request, **kwargs)\n File \"/usr/local/lib/python2.7/dist-packages/requests/adapters.py\", line 498, in send\n raise ConnectionError(err, request=request)\nrequests.exceptions.ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))\n",
"module_stdout": "",
"msg": "MODULE FAILURE"
}
What I am doing wrong?
I almost graduate from this one, but is someone has same problem, I found the problem.
the reason is that inside name parameter there are some upper case letters.
I change
name: registry.....si/project/restAPI-{{ git_branch }}
to
name: registry.....si/project/restapi-{{ git_branch }}
and now it works.