I referred to the following site. After completing each setting, I wrote gspread_simple.py which shown the site.
https://qiita.com/164kondo/items/eec4d1d8fd7648217935
As a result, the following error was output (I replaced a folder name under C:\Users with ********).
Traceback (most recent call last):
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 696, in urlopen
self._prepare_proxy(conn)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 964, in _prepare_proxy
conn.connect()
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 359, in connect
conn = self._connect_tls_proxy(hostname, conn)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 496, in _connect_tls_proxy
return ssl_wrap_socket(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 432, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 474, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1040, in _create
self.do_handshake()
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1122)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen
return self.urlopen(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen
return self.urlopen(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen
return self.urlopen(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\util\retry.py", line 573, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='oauth2.googleapis.com', port=443): Max retries exceeded with url: /token (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1122)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\google\auth\transport\requests.py", line 182, in __call__
response = self.session.request(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='oauth2.googleapis.com', port=443): Max retries exceeded with url: /token (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1122)')))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\********\samp\module1.py", line 18, in <module>
ws = connect_gspread(jsonf,spread_sheet_key)
File "C:\Users\********\samp\module1.py", line 12, in connect_gspread
worksheet = gc.open_by_key(SPREADSHEET_KEY).sheet1
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\gspread\models.py", line 123, in sheet1
return self.get_worksheet(0)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\gspread\models.py", line 283, in get_worksheet
sheet_data = self.fetch_sheet_metadata()
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\gspread\models.py", line 265, in fetch_sheet_metadata
r = self.client.request('get', url, params=params)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\gspread\client.py", line 61, in request
response = getattr(self.session, method)(
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\google\auth\transport\requests.py", line 460, in request
self.credentials.before_request(auth_request, method, url, request_headers)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\google\auth\credentials.py", line 133, in before_request
self.refresh(request)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\google\oauth2\service_account.py", line 361, in refresh
access_token, expiry, _ = _client.jwt_grant(request, self._token_uri, assertion)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\google\oauth2\_client.py", line 153, in jwt_grant
response_data = _token_endpoint_request(request, token_uri, body)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\google\oauth2\_client.py", line 105, in _token_endpoint_request
response = request(method="POST", url=token_uri, headers=headers, body=body)
File "C:\Users\********\AppData\Local\Programs\Python\Python39\lib\site-packages\google\auth\transport\requests.py", line 188, in __call__
six.raise_from(new_exc, caught_exc)
File "<string>", line 3, in raise_from
google.auth.exceptions.TransportError: HTTPSConnectionPool(host='oauth2.googleapis.com', port=443): Max retries exceeded with url: /token (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1122)')))
I referred to the following site and so on.
https://qiita.com/vmmhypervisor/items/c8ef0ad87318656facb2
OS: Windows10
Python: 3.9
My Python is not 2.7 series.
As a test, I tried the command.
pip uninstall -y certifi && pip install certifi==2015.04.28
But the output was the same.
And I tried the command.
pip install -U certifi
But the output was the same.
Related
Description of the problem
The error will occur if the num_workers > 0 , But when I set num_workers = 0 , the error disappeared, though, this will slow down the trainning speed. I think the multiprocessing really matters here .How can I solve this problem?
env
docker python3.8 Pytorch 1.11.0+cu113
error output
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/multiprocessing/resource_sharer.py", line 149, in _serve
send(conn, destination_pid)
File "/opt/conda/lib/python3.8/multiprocessing/resource_sharer.py", line 50, in send
reduction.send_handle(conn, new_fd, pid)
File "/opt/conda/lib/python3.8/multiprocessing/reduction.py", line 184, in send_handle
sendfds(s, [handle])
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/multiprocessing/reduction.py", line 149, in sendfds
File "save_disp.py", line 85, in <module>
sock.sendmsg([msg], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, fds)])
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/multiprocessing/resource_sharer.py", line 151, in _serve
test()
File "save_disp.py", line 55, in test
close()
for batch_idx, sample in enumerate(TestImgLoader):
File "/opt/conda/lib/python3.8/multiprocessing/resource_sharer.py", line 52, in close
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 530, in __next__
os.close(new_fd)
OSError: [Errno 9] Bad file descriptor
data = self._next_data()
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1207, in _next_data
idx, data = self._get_data()
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1173, in _get_data
success, data = self._try_get_data()
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1011, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "/opt/conda/lib/python3.8/multiprocessing/queues.py", line 116, in get
return _ForkingPickler.loads(res)
File "/opt/conda/lib/python3.8/site-packages/torch/multiprocessing/reductions.py", line 295, in rebuild_storage_fd
fd = df.detach()
File "/opt/conda/lib/python3.8/multiprocessing/resource_sharer.py", line 58, in detach
return reduction.recv_handle(conn)
File "/opt/conda/lib/python3.8/multiprocessing/reduction.py", line 189, in recv_handle
return recvfds(s, 1)[0]
File "/opt/conda/lib/python3.8/multiprocessing/reduction.py", line 159, in recvfds
raise EOFError
EOFError
I want to use rdpcap to open a traffic capture.
cap = rdpcap("Chall_1.pcapng")
but i receive the following error and I don't know how to solve it.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/scapy/utils.py", line 979, in __call__
i.__init__(filename, fdesc, magic)
File "/usr/lib/python3/dist-packages/scapy/utils.py", line 1124, in __init__
RawPcapReader.__init__(self, filename, fdesc, magic)
File "/usr/lib/python3/dist-packages/scapy/utils.py", line 1035, in __init__
raise Scapy_Exception(
scapy.error.Scapy_Exception: Not a pcap capture file (bad magic: b'\n\r\r\n')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/valentin/Desktop/Tema3/ctf1.py", line 29, in <module>
cap = rdpcap("Chall_1.pcapng")
File "/usr/lib/python3/dist-packages/scapy/utils.py", line 950, in rdpcap
with PcapReader(filename) as fdesc:
File "/usr/lib/python3/dist-packages/scapy/utils.py", line 985, in __call__
i.__init__(filename, fdesc, magic)
File "/usr/lib/python3/dist-packages/scapy/utils.py", line 1320, in __init__
RawPcapNgReader.__init__(self, filename, fdesc, magic)
File "/usr/lib/python3/dist-packages/scapy/utils.py", line 1209, in __init__
self.f.read(blocklen - 24)
MemoryError
The problem was that I didn't have enoght RAM on my virtual machine.
I infer from this Scapy pull request that the intent is that rdpcap() be able to open both pcap and pcapng files. If that's not working, then it's presumably a Scapy bug; please report it on the Scapy issue list.
Unable to install Ipex using docker centos image
I pulled this docker image: docker pull sysstacks/dlrs-pytorch-centos
Tried to run on my linux machine with this command: docker run -it sysstacks/dlrs-pytorch-centos bash
I was trying to install Ipex using centos docker image (image name: sysstacks/dlrs-pytorch-centos) unfortunately i got this error.
after setting env
enter image description here
[root#0d96884d3a05 /]# python -m pip install torch_ipex==1.9.0 -f https://software.intel.com/ipex-whl-stable
Looking in links: https://software.intel.com/ipex-whl-stable
ERROR: Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
status = self.run(options, args)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 204, in wrapper
return func(self, options, args)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 319, in run
reqs, check_supported_wheels=not options.target_dir
File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 128, in resolve
requirements, max_rounds=try_to_avoid_resolution_too_deep
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 473, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 341, in resolve
name, crit = self._merge_into_criterion(r, parent=None)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _merge_into_criterion
if not criterion.candidates:
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/structs.py", line 139, in __bool__
return bool(self._sequence)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in __bool__
return any(self)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 129, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 30, in _iter_built
for version, func in infos:
File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 272, in iter_index_candidate_infos
hashes=hashes,
File "/usr/local/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 879, in find_best_candidate
candidates = self.find_all_candidates(project_name)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 824, in find_all_candidates
page_candidates = list(page_candidates_it)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/index/sources.py", line 134, in page_candidates
yield from self._candidates_from_page(self._link)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 783, in process_project_url
html_page = self._link_collector.fetch_page(project_url)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 512, in fetch_page
return _get_html_page(location, session=self.session)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 422, in _get_html_page
resp = _get_html_response(url, session=session)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 137, in _get_html_response
"Cache-Control": "max-age=0",
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/pip/_internal/network/session.py", line 449, in request
return super().request(method, url, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/cachecontrol/adapter.py", line 53, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/connectionpool.py", line 696, in urlopen
self._prepare_proxy(conn)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/connectionpool.py", line 964, in _prepare_proxy
conn.connect()
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/connection.py", line 359, in connect
conn = self._connect_tls_proxy(hostname, conn)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/connection.py", line 506, in _connect_tls_proxy
ssl_context=ssl_context,
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 432, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 474, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock)
File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket
_context=self, _session=session)
File "/usr/lib64/python3.6/ssl.py", line 732, in __init__
raise ValueError("check_hostname requires server_hostname")
NOTE:
I have tried all these methods shown below but none of them worked
Why requests raise this exception "check_hostname requires server_hostname"?
I think the issue is with your proxy, can you please try below commands in your docker container:
export HTTP_PROXY=http://x.x.x.x
export https_proxy=http://x.x.x.x
export HTTPS_PROXY=http://x.x.x.x
export http_proxy=http://x.x.x.x
Change x with your proxy ip
I am trying to run a simple python script within a docker run command scheduled with Airflow.
I have followed the instructions here Airflow init.
My .env file:
AIRFLOW_UID=1000
AIRFLOW_GID=0
And the docker-compose.yaml is the default one docker-compose.yaml
My dag is configured as followed:
""" this is an example dag """
from datetime import timedelta
from airflow import DAG
from airflow.operators.docker_operator import DockerOperator
from airflow.utils.dates import days_ago
from docker.types import Mount
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'email': ['info#foo.com'],
'email_on_failure': True,
'email_on_retry': False,
'retries': 10,
'retry_delay': timedelta(minutes=5),
}
with DAG(
'msg_europe_etl',
default_args=default_args,
description='Process MSG_EUROPE ETL',
schedule_interval=timedelta(minutes=15),
start_date=days_ago(0),
tags=['satellite_data'],
) as dag:
download_and_store = DockerOperator(
task_id='download_and_store',
image='satellite_image:latest',
auto_remove=True,
api_version='1.41',
network_mode="overlay",
mounts=[Mount(source='/home/archive_1/archive/satellite_data',
target='/app/data'),
Mount(source='/home/dlassahn/projects/forecast-system/meteoIntelligence-satellite',
target='/app')],
command="python3 src/scripts.py download_satellite_images "
"{{ (execution_date - macros.timedelta(hours=4)).strftime('%Y-%m-%d %H:%M') }} "
"'msg_europe' ",
)
download_and_store
But I guess the error from the Airflow log comes from missing access to the socket:
[2021-08-03 16:09:19,968] {taskinstance.py:1501} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.6/http/client.py", line 1287, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1333, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1282, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1042, in _send_output
self.send(msg)
File "/usr/local/lib/python3.6/http/client.py", line 980, in send
self.connect()
File "/home/airflow/.local/lib/python3.6/site-packages/docker/transport/unixconn.py", line 43, in connect
sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 410, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback(tb)
File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.6/http/client.py", line 1287, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1333, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1282, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1042, in _send_output
self.send(msg)
File "/usr/local/lib/python3.6/http/client.py", line 980, in send
self.connect()
File "/home/airflow/.local/lib/python3.6/site-packages/docker/transport/unixconn.py", line 43, in connect
sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1157, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1331, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1361, in _execute_task
result = task_copy.execute(context=context)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/docker/operators/docker.py", line 300, in execute
if self.force_pull or not self.cli.images(name=self.image):
File "/home/airflow/.local/lib/python3.6/site-packages/docker/api/image.py", line 98, in images
res = self._result(self._get(self._url("/images/json"), params=params),
File "/home/airflow/.local/lib/python3.6/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/docker/api/client.py", line 237, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File "/home/airflow/.local/lib/python3.6/site-packages/requests/sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
My Airflow Version: 2.1.2
Output of grep /etc/group -e "docker": docker:x:998:username
You need to map docker socket to inside the Docker container in order to make Docker Operator works. Basically you need to get "docker-in-docker" setup.
Look at the option 1) here: https://devopscube.com/run-docker-in-docker/ - it is probably what you want (you just need to add appropriate mount in the docker-compose file)
I am having issues running the following script for a longer period of time:
I use ampy to execute the script on the ESP:
sudo ampy --port /dev/ttyUSB0 run photoresistor.py
photoresistor.py:
#!/usr/bin/env python3
import machine
import network
from time import sleep
from urllib.urequest import urlopen
import json
wifiap = network.WLAN(network.AP_IF)
wifiap.active(False)
routercon = network.WLAN(network.STA_IF)
routercon.active(True)
routercon.ifconfig(('10.0.0.128','255.255.255.0','10.0.0.138','10.0.0.138'))
routercon.connect('mywifi', '123')
while not routercon.isconnected():
pass
posturl=('http://10.0.0.156:23102/rest/v2/send')
adc = machine.ADC(0)
gc.enable()
while True:
value = adc.read()
if value < 200:
message = {'username': 'test', 'message': value, 'chatid': 'test', 'password': 'test', 'notifyself': 'false'}
r = urlopen(posturl, data=json.dumps(message).encode())
r.close()
gc.collect()
sleep(1)
It works as expected in the beginning but after some time I get the following stacktrace:
Traceback (most recent call last):
File "/usr/local/bin/ampy", line 11, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/ampy/cli.py", line 337, in run
output = board_files.run(local_file, not no_output)
File "/usr/local/lib/python3.6/dist-packages/ampy/files.py", line 303, in run
out = self._pyboard.execfile(filename)
File "/usr/local/lib/python3.6/dist-packages/ampy/pyboard.py", line 273, in execfile
return self.exec_(pyfile)
File "/usr/local/lib/python3.6/dist-packages/ampy/pyboard.py", line 267, in exec_
raise PyboardError('exception', ret, ret_err)
ampy.pyboard.PyboardError: ('exception', b'', b'Traceback (most recent call last):\r\n File "<stdin>", line 28, in <module>\r\n File "urequests.py", line 152, in post\r\n File "urequests.py", line 89, in request\r\nOSError: [Errno 103] ECONNABORTED\r\n')
No idea what to do.
I tried to play around with the garbage collection but it didn't help.
I suspect that the board doesn't clean up sockets properly.
If the board is sending post requests quickly in the loop (every second for 1 minute) and let it sit afterwards for a short period of time it fails quickly with above ECONNABORTED.
If the board sends post requests more slowly (like 2 in a minute) it takes way longer for it to fail. To conclude: I suspect the OS does not properly clean up resources and still has active connections after r.close() or I am overseeing something in the code.
I am not sure what else I can do to make sure these sockets are closed.
EDIT:
I found out it fails on connect (https://github.com/micropython/micropython-lib/blob/master/urllib.urequest/urllib/urequest.py):
line 28:
s.connect(ai[-1])
however routercon.isconnected() returns true:
>>> routercon.isconnected()
True
>>>
How can it be that altough there is a active connection I am unable to send an http post request?
EDIT2:
When this happens sometimes I also can't post to another endpoint e.g. the test server with the same webservice
>>> r = urlopen(posturl, data=json.dumps(message).encode())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "urllib/urequest.py", line 28, in urlopen
OSError: [Errno 103] ECONNABORTED
>>> r = urlopen("http://10.0.0.8:23102/rest/v2/send", data=json.dumps(message).encode())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "urllib/urequest.py", line 28, in urlopen
OSError: [Errno 103] ECONNABORTED
>>>
Interestingly a http get to google works:
>>> r = urlopen("http://www.google.com")
>>>
If I let it sit idle for some time http post start to work again.
Could it be that the OS is performing a cleanup in the background?
I faced the same problem. Restarting your api endpoint device will solve the issue.