Connecting to external networks from inside minikube VM behind proxy in docker container - docker

Question

I have an active kubernetes cluster inside Minikube VM (using VirtualBox as driver), so for deploying new containers I am able to download the images as this connection is already laid out using istio service, now if I ssh into my minikube VM first of all I am not able to wget https content but http contents are connected after setting proxies and no_proxies but if I want to access any link from inside of my containers, say simple pod with python image and urllib library and I want to connect from inside this pod and then print the contents from any link (eg.http://python.org) I am not able to do so, all I am getting is no route to host error in logs which points to some problem with the connection due to proxies.
def basic():
import urllib.request
print("inside basic funtion")
with urllib.request.urlopen('http://python.org/') as response:
html = response.read()
print(html)
this is the python code I am running from inside my container as a pipeline component.
Most recent error I got-
Traceback (most recent call last):
File "/usr/local/lib/python3.7/urllib/request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/usr/local/lib/python3.7/http/client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1016, in _send_output
self.send(msg)
File "/usr/local/lib/python3.7/http/client.py", line 956, in send
self.connect()
File "/usr/local/lib/python3.7/http/client.py", line 928, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/local/lib/python3.7/socket.py", line 727, in create_connection
raise err
File "/usr/local/lib/python3.7/socket.py", line 716, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 11, in <module>
File "<string>", line 3, in basic
File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/local/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.7/urllib/request.py", line 1345, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/local/lib/python3.7/urllib/request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 110] Operation timed out>
I have started minikube as-
minikube start --cpus 6 --memory 12288 --disk-size=80g --extra-config=apiserver.service-account-issuer=api --extra-config=apiserver.service-account-signing-key-file=/var/lib/minikube/certs/apiserver.key --extra-config=apiserver.service-account-api-audiences=api --kubernetes-version v1.14.0
after setting the env variables as well.
Update:
I created a different container just to check the curl from inside the component as- (I am using kfp libraries for creating containers)
def curl_op(text):
return dsl.ContainerOp(
name='curl',
image='tutum/curl',
command=['sh', '-c'],
arguments=['curl -x http://<proxy-server>:<proxy-port> "$0"', text]
)
so using the above argument I am able to connect to external links, which again makes it certain that i need to create the containers with proxies set.

Answer 1

So for running the above python code I mentioned as pipeline component.
I added the environment variables using the os library and this individual piece was able to connect to external networks.
Updated python code-
def basic():
import urllib.request
import os
proxy = 'http://proxy-path:port'
os.environ['http_proxy'] = proxy
os.environ['HTTP_PROXY'] = proxy
os.environ['https_proxy'] = proxy
os.environ['HTTPS_PROXY'] = proxy
print("inside basic funtion")
with urllib.request.urlopen('http://python.org/') as response:
html = response.read()
print(html)
And if the docker image is created from scratch without taking help of pipeline library function then we need to just add the env details into our dockerfile the usual way after the base image call-
ENV HTTP_PROXY http://proxy-path:port
ENV HTTPS_PROXY http://proxy-path:port