No response from Docker service - docker

I tried following the tutorial here
https://docs.docker.com/get-started/part3/.
First issue I ran into was when I called docker swarm init. It also asked for docker swarm init --advertise-addr with one of two possible IPv6 IPs.
I tried initializing the swarm on both and then starting the service. The service starts succesfully, but I can't get any response when accessing Localhost:4000. It just loads forever.
I have tried rebuilding the image, creating the swarm on both IPs, checking the logs (there was nothing there), but I kind of run out of ideas. If it helps, the computer has dual operating system, might affect the networking in ways I an unable to figure out.
How can I receive a response on my request?

The issue I was facing was a connection between google chrome and docker swarm, documented better here
https://forums.docker.com/t/google-chrome-and-localhost-in-swarm-mode/32229/9.
There is no apparent solution

Related

All docker stack are restarting automatically

I have a multi-services environment that is hosted with docker swarm. There are multiple stacks that are created. All the docker containers which are running have an inbuild Spring Boot application. The issue is coming that all my stacks get restarted on their own. Now I know that in compose file I have mentioned that restart_policy as on failure. Hence it auto restarted. The issue comes that when services are restarted, I get errors from a particular service and this breaks everything.
I am not able to figure out what actually happens.
I did quite a lot of research and found out about these things.
Docker daemon is not restarted. I double-checked this with the uptime of the docker daemon.
I checked the docker service ps <Service_ID> and there I can see service showing shutdown and starting. No other information.
I checked the docker service logs <Service_ID> but no error in there too.
I checked for resource crunch. I can assure you that there was quite a good resource available at the host as well as each container level.
Can someone help where exactly to find logs for this even? Any other thoughts on this?
My host is actually a VM hosted on VMWare Vcenter.
After a lot of research and going through all docker logs, I could not find the solution. Later on, I discovered that there was a memory snapshot taken for backup every 24 hours.
Here is what I observe:
Whenever we take a snapshot, all docker services running on the host restart automatically. There will be no errors in that but they will just restart gracefully.
I found some questions already having this problem with VMware snapshots.
As far as I know, when we take a snapshot, it points to a different memory location and saves the previous one. I am not able to find why it's happening but yes Root cause of the problem was this. If anyone is a VMWare snapshots expert, please let us know.

Rsyslog can't start inside of a docker container

I've got a docker container running a service, and I need that service to send logs to rsyslog. It's an ubuntu image running a set of services in the container. However, the rsyslog service cannot start inside this container. I cannot determine why.
Running service rsyslog start (this image uses upstart, not systemd) returns only the output start: Job failed to start. There is no further information provided, even when I use --verbose.
Furthermore, there are no error logs from this failed startup process. Because rsyslog is the service that can't start, it's obviously not running, so nothing is getting logged. I'm not finding anything relevant in Upstart's logs either: /var/log/upstart/ only contains the logs of a few things that successfully started, as well as dmesg.log which simply contains dmesg: klogctl failed: Operation not permitted. which from what I can tell is because of a docker limitation that cannot really be fixed. And it's unknown if this is even related to the issue.
Here's the interesting bit: I have the exact same container running on a different host, and it's not suffering from this issue. Rsyslog is able to start and run in the container just fine on that host. So obviously the cause is some difference between the hosts. But I don't know where to begin with that: There are LOTS of differences between the hosts (the working one is my local windows system, the failing one is a virtual machine running in a cloud environment), so I wouldn't know where to even begin about which differences could cause this issue and which ones couldn't.
I've exhausted everything that I know to check. My only option left is to come to stackoverflow and ask for any ideas.
Two questions here, really:
Is there any way to get more information out of the failure to start? start itself is a binary file, not a script, so I can't open it up and edit it. I'm reliant solely on the output of that command, and it's not logging anything anywhere useful.
What could possibly be different between these two hosts that could cause this issue? Are there any smoking guns or obvious candidates to check?
Regarding the container itself, unfortunately it's a container provided by a third party that I'm simply modifying. I can't really change anything fundamental about the container, such as the fact that it's entrypoint is /sbin/init (which is a very bad practice for docker containers, and is the root cause of all of my troubles). This is also causing some issues with the docker logging driver, which is why I'm stuck using syslog as the logging solution instead.

How can I debug Docker name resolution?

I have a very peculiar problem regarding Docker and how containers resolve the IP address of other containers using their container names.
My machine runs many different containers interacting with each other, making the problem difficult to reproduce in a minimal example as would be preferred if possible.
The problem I am encountering is that whenever I add a new container/service/docker-network (by running docker-compose up) all of a sudden, my container for reverse proxying HTTP(S) traffic starts resolving the IP addresses of containers incorrectly.
I can verify that the Name --> IP resolutions are incorrect by checking the logs of where the reverse proxy is trying to connect and then running docker network inspect on the container I expect it to resolve. These are now not the same, resulting in errors all around.
Whenever I stop the new service that triggered the errors, the name resolution is back to normal and all works as expected again. I have encountered this same problem when starting multiple different images, so the problem is most likely not in the new container.
Since I can not provide you with a simple minimally reproducible example (I would if I could), how can I further debug the docker name resolution in an attempt at gathering more information about this problem?
Any advice is appriceated. I know this is not the optimal Stack Overflow question, but I can't do much more without more information about my problem.
I hope you understand.

Access to internal infrastructure from Kubernetes

If I run Docker (Docker for Desktop, 2.0.0.3 on Windows 10), then access to internal infrastructure and containers is fine. I can easily do
docker pull internal.registry:5005/container:latest
But ones I enable Kubernetes there, I completely lose an access to internal infrastructure and [Errno 113] Host is unreachable in Kubernetes itself or connect: no route to host from Docker appears.
I have tried several ways, including switching of NAT from DockerNAT to Default Switch. That one doesn't work without restart and restart changes it back to DockerNAT, so, no luck here. This option also seems not to work.
let's start from the basics form the official documentation:
Please make sure you meet all the prerequisites and all other instructions were met.
Also you can use this guide. It has more info with details pointing to what might have gone wrong in your case.
If the above won't help, there are few other things to consider:
In case you are using a virtual machine, make sure that the IP you are referring to is the one of the docker-engines’ host and not the one on which the client is running.
Try to add tmpnginx in docker-compose.
Try to delete the pki directory in C:\programdata\DockerDesktop (first stop Docker, delete the dir and than start Docker). The directory will be recreated and k8s-app=kube-dns labels should work fine.
Please let me know if that helped.

Not able to access to Docker container through bound public ip

I am trying to use docker containers on Bluemix but it looks like I am having troubles tried again this morning but seems it still does not work.
I have followed these steps:
I have released all public ip issuing the cf ic ip release command
I have created a new container from the etherpad image (following the tutorial Tutorial), requesting and binding a new public ip from the Bluemix GUI.
Bluemix assigned 134.168.1.49 IP and bound it to the container.
I expect the application to respond to http://134.168.1.49:9080/ but it hangs and responds me back with a connection timeout.
Running a container from the same image locally works perfectly.
Any idea, suggestion?
There is a known issue with the IBM Containers service where there's a delay with the inbound network access being available after containers start. It can take up to five minutes for this to be available.
Are you able to successfully ping the bound IP address?
Note: The IBM Containers service suffered a major incident yesterday which affected operations. If you were trying to use it during this time, it may be related to that.
We recently experienced some connectivity issues in our US-South datacenter. I would suggest redeploying your container with an IP address again today and determine if you have further success.
I have worked with Bluemix support, that was able to create a new image, start it up and access it successfully with my exact configuration. At this time, it appears there is something wrong with the networking for the tenant space where my containers are running. Bluemix team is investigating.
Thank you all for the support.

Resources