I am trying to run following command in my dockerfile for downloading pandas package
RUN curl -x proxy.temp.com:8080 -U myid_9076:pwd123* -L -O https://files.pythonhosted.org/packages/b4/31/bbd2c915aad67c7cb572b7c6ca8f645fcb112064ef6774436d4f65acd5a1/pandas-0.20.3-cp27-cp27m-manylinux1_x86_64.whl
But I get following error while building docker-image using same dockerfile:
---> Running in c8ddcdcdb155
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0curl: (7) Failed to connect to proxy.temp.com port 8080: No route to host
Same command runs and downloads package on bare metal node on which docker is installed. But while building docker-image it gives above error. What is the issue, how can I download this package required for my image?
Also tried setting environment variables in my dockerfile before downloading pandas package as below,
ENV http_proxy 'http://myid_9076:pwd123*#proxy.temp.com:8080/'
ENV https_proxy 'http://myid_9076:pwd123*#proxy.temp.com:8080/'
but same error is seen in this case too.
It was network conflict issue, my network was conflicting with docker network. So I have set parameter "bip" in /etc/docker/daemon.json as shown below
{
"bip" : "12.12.0.1/24"
}
Got the answer from link Unable to access local network IP from docker container
Related
I had tried following the advice here, specifically:
run docker system prune, which freed about 6GB
increased the Disk image size on docker desktop preferences to 64 GB (43 GB used)
but am still seeing this when running skaffold: exiting dev mode because first build failed: couldn't build "user/orders": docker build: Error response from daemon: Error processing tar file(exit status 1): write /tsconfig.json: no space left on device. Another run of skaffold gave me this on another occasion:
exiting dev mode because first build failed: couldn't build "exiting dev mode because first build failed: couldn't build "user/orders": unable to stream build output: failed to create rwlayer: mkdir /var/lib/docker/overlay2/7c6618702ad15fe0fa7d4655109aa6326fb4f954df00d2621f62d66d7b328ed9/diff: no space left on device
Also, when running docker system df, I see this:
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 10 0 28.86GB 28.86GB (100%)
Containers 0 0 0B 0B
Local Volumes 30 0 15.62GB 15.62GB (100%)
Build Cache 0 0 0B 0B
I also have about 200GB of physical hard drive space available.
I'm hoping I don't have to manually run rm * as proposed here, which was for a linux distro.
if you're running on Mac and have 200GB free, will increasing help you?
I am experiencing a network issue with docker that I haven't seen before. Could it be related to my ubuntu network conf? or docker setup?
Sending build context to Docker daemon 36.86kB
Step 1/2 : FROM centos:centos7
---> 5e35e350aded
Step 2/2 : RUN curl https://google.com/
---> Running in d65fe6ad9d57
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to 2a00:1450:4007:808::200e: Cannot assign requested address
regards
The problem was on the network I was using provided by my cellphone (an Android hotspot on a Xiaomi redmi7). I still cannot figure out the real cause of this misfunctionality.
I'm quite new to Docker and Apache-Superset and trying to run a container (using docker) from the container image. Loaded the .tar file with
docker load --input ./inc_superset.tar
Which went as expected, tried running the container from this image file with
docker run --cidfile ./cid.txt <IMAGE_ID>
This starts my container but is has unhealthy status; upon inspecting the container ( with docker inspect) I get a huge JSON, below is a snippet of the log received (can post the entire log on request).
"Health": {
"Status": "unhealthy",
"FailingStreak": 5,
"Log": [
{
"Start": "2019-01-22T19:59:00.8036984+05:30",
"End": "2019-01-22T19:59:01.5698797+05:30",
"ExitCode": 7,
"Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8088: Connection refused\n"
},
...
...
{
"Start": "2019-01-22T20:01:02.589517677+05:30",
"End": "2019-01-22T20:01:02.794486003+05:30",
"ExitCode": 7,
"Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8088: Connection refused\n"
}
]
}
Am I making any mistake? Am I missing something? Any troubleshooting help on this requested.
Thanks
The problem was the webserver within the superset container doesn't run by default as of the configuration available on apache.org as of 7 September 2019.
Solved it as follows:
#Go into the container
docker-compose exec superset bash
#Start the webserver that is exposed on all interfaces so that we can access it from docker host
superset run -p 8088 --host 0.0.0.0
I was facing the same issue. Ran via docker-compose using the instructions on https://superset.incubator.apache.org/installation.html but get no response from localhost:8088.
Docker inspect State.Health.Status = "unhealthy" and log show several entries with curl: (7) Failed to connect to localhost port 8088: Connection refused\n"
docker ps shows that the container is exposed on 0.0.0.0:8088
I have a test which starts a Docker container, performs the verification (which is talking to the Apache httpd in the Docker container), and then stops the Docker container.
When I run this test locally, this test runs just fine. But when it runs on hosted VSTS, thus a hosted build agent, it cannot connect to the Apache httpd in the Docker container.
This is the .vsts-ci.yml file:
queue: Hosted Linux Preview
steps:
- script: |
./test.sh
This is the test.sh shell script to reproduce the problem:
#!/bin/bash
set -e
set -o pipefail
function tearDown {
docker stop test-apache
docker rm test-apache
}
trap tearDown EXIT
docker run -d --name test-apache -p 8083:80 httpd
sleep 10
curl -D - http://localhost:8083/
When I run this test locally, the output that I get is:
$ ./test.sh
469d50447ebc01775d94e8bed65b8310f4d9c7689ad41b2da8111fd57f27cb38
HTTP/1.1 200 OK
Date: Tue, 04 Sep 2018 12:00:17 GMT
Server: Apache/2.4.34 (Unix)
Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
ETag: "2d-432a5e4a73a80"
Accept-Ranges: bytes
Content-Length: 45
Content-Type: text/html
<html><body><h1>It works!</h1></body></html>
test-apache
test-apache
This output is exactly as I expect.
But when I run this test on VSTS, the output that I get is (irrelevant parts replaced with …).
2018-09-04T12:01:23.7909911Z ##[section]Starting: CmdLine
2018-09-04T12:01:23.8044456Z ==============================================================================
2018-09-04T12:01:23.8061703Z Task : Command Line
2018-09-04T12:01:23.8077837Z Description : Run a command line script using cmd.exe on Windows and bash on macOS and Linux.
2018-09-04T12:01:23.8095370Z Version : 2.136.0
2018-09-04T12:01:23.8111699Z Author : Microsoft Corporation
2018-09-04T12:01:23.8128664Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkID=613735)
2018-09-04T12:01:23.8146694Z ==============================================================================
2018-09-04T12:01:26.3345330Z Generating script.
2018-09-04T12:01:26.3392080Z Script contents:
2018-09-04T12:01:26.3409635Z ./test.sh
2018-09-04T12:01:26.3574923Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/02476800-8a7e-4e22-8715-c3f706e3679f.sh
2018-09-04T12:01:27.7054918Z Unable to find image 'httpd:latest' locally
2018-09-04T12:01:30.5555851Z latest: Pulling from library/httpd
2018-09-04T12:01:31.4312351Z d660b1f15b9b: Pulling fs layer
[…]
2018-09-04T12:01:49.1468474Z e86a7f31d4e7506d34e3b854c2a55646eaa4dcc731edc711af2cc934c44da2f9
2018-09-04T12:02:00.2563446Z % Total % Received % Xferd Average Speed Time Time Time Current
2018-09-04T12:02:00.2583211Z Dload Upload Total Spent Left Speed
2018-09-04T12:02:00.2595905Z
2018-09-04T12:02:00.2613320Z 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
2018-09-04T12:02:00.7027822Z test-apache
2018-09-04T12:02:00.7642313Z test-apache
2018-09-04T12:02:00.7826541Z ##[error]Bash exited with code '7'.
2018-09-04T12:02:00.7989841Z ##[section]Finishing: CmdLine
The key thing is this:
curl: (7) Failed to connect to localhost port 8083: Connection refused
10 seconds should be enough for apache to start.
Why can curl not communicate with Apache on its port 8083?
P.S.:
I know that a hard-coded port like this is rubbish and that I should use an ephemeral port instead. I wanted to get it running first wirth a hard-coded port, because that's simpler than using an ephemeral port, and then switch to an ephemeral port as soon as the hard-coded port works. And in case the hard-coded port doesn't work because the port is unavailable, the error should look different, in that case, docker run should fail because the port can't be allocated.
Update:
Just to be sure, I've rerun the test with sleep 100 instead of sleep 10. The results are unchanged, curl cannot connect to localhost port 8083.
Update 2:
When extending the script to execute docker logs, docker logs shows that Apache is running as expected.
When extending the script to execute docker ps, it shows the following output:
2018-09-05T00:02:24.1310783Z CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2018-09-05T00:02:24.1336263Z 3f59aa014216 httpd "httpd-foreground" About a minute ago Up About a minute 0.0.0.0:8083->80/tcp test-apache
2018-09-05T00:02:24.1357782Z 850bda64f847 microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard "/home/vsts/agents/2…" 2 minutes ago Up 2 minutes musing_booth
The problem is that the VSTS build agent runs in a Docker container. When the Docker container for Apache is started, it runs on the same level as the VSTS build agent Docker container, not nested inside the VSTS build agent Docker container.
There are two possible solutions:
Replacing localhost with the ip address of the docker host, keeping the port number 8083
Replacing localhost with the ip address of the docker container, changing the host port number 8083 to the container port number 80.
Access via the Docker Host
In this case, the solution is to replace localhost with the ip address of the docker host. The following shell snippet can do that:
host=localhost
if grep '^1:name=systemd:/docker/' /proc/1/cgroup
then
apt-get update
apt-get install net-tools
host=$(route -n | grep '^0.0.0.0' | sed -e 's/^0.0.0.0\s*//' -e 's/ .*//')
fi
curl -D - http://$host:8083/
The if grep '^1:name=systemd:/docker/' /proc/1/cgroup inspects whether the script is running inside a Docker container. If so, it installs net-tools to get access to the route command, and then parses the default gw from the route command to get the ip address of the host. Note that this only works if the container's network default gw actually is the host.
Direct Access to the Docker Container
After launching the docker container, its ip addresses can be obtained with the following command:
docker container inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}' <container-id>
Replace <container-id> with your container id or name.
So, in this case, it would be (assuming that the first ip address is okay):
ips=($(docker container inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}' nuance-apache))
host=${ips[0]}
curl http://$host/
Hi running docker: Engine: 17.12.0-ce on Mac 10.12.6
I'm using the build_image feature.
FROM openjdk:8-jdk-alpine AS BUILD_IMAGE
ENV APP_ENVIROMENT=dev
ENV APP_RUNTIME=docker
ENV BUILD_HOME=/build
COPY . $BUILD_HOME
WORKDIR $BUILD_HOME
RUN wget http://xxx.xxx.xxx.xxx:8081
Getting the following error. I'm pretty sure the server is reachable from my laptop/wifi and VPN
...
removing intermediate container e5519d15a106
---> b3a3dd24343f
Step 7/16 : RUN wget http://xxx.xxx.xxx.xxx:8081
---> Running in 49fdabef0923
Connecting to xxx.xxx.xxx.xxx:8081 (xxx.xxx.xxx.xxx:8081)
wget: can't connect to remote host (xxx.xxx.xxx.xxx): Host is unreachable
Seems like a DNS issue my /etc/resolv.conf is perfectly fine and I'm pretty sure the machine is reachable. Also looks like some machines on my network are reachable while other aren't.
I can test with
RUN ping xxx.xxx.xxx.xxx
Results:
PING xxxxxx.mydomain.com (xxx.xxx.xxx.xxx): 56 data bytes
--- xxxxxx.mydomain.com ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss
The command '/bin/sh -c ping -c 4 xxxxxx.mydomain.com' returned a non-zero code: 1
Update. Added names to be clear.
FROM openjdk:8-jdk-alpine AS BUILD_IMAGE
ENV APP_ENVIROMENT=dev
ENV APP_RUNTIME=docker
ENV BUILD_HOME=/build
COPY . $BUILD_HOME
WORKDIR $BUILD_HOME
RUN wget http://myserver.domain.net:8081
The result of docker build:
---> 76c263fb4490
Step 7/15 : RUN wget http://myserver.domain.net:8081
---> Running in ef2b13c77b91
Connecting to myserver.domain.net:8081 (10.0.0.1:8081) <-- I.P redacted to protect the inocent. :p
wget: can't connect to remote host (10.0.0.1): Host is unreachable
The command '/bin/sh -c wget http://myserver.domain.net:8081' returned a non-zero code: 1
From my console...
wget http://myserver.domain.net:8081
--2018-02-15 10:36:44-- http://myserver.domain.net:8081/
Resolving myserver.domain.net... 10.0.0.1
Connecting to myserver.domain.net|10.0.0.1|:8081... connected.
HTTP request sent, awaiting response... 200 OK
Clearly you can see docker is resolving the I.P and that my laptop can connect.
Problem solved as per: https://success.docker.com/article/How_do_I_configure_the_default_bridge_(docker0)_network_for_Docker_Engine_to_a_different_subnet
The docker bridge was conflicting with my network. We need to make an entry in daemon.json
{
"bip": "172.26.0.1/16"
}
You can use any value that is suitable for you.