I am trying to access a named postgres server from inside a docker container. I can access the server via it's IP address, but not its name. I've tried --net=host and -p ServerName:5432:5432 options on the docker run command.
I will demonstrate the issue:
# on the host
$ ping ServerName
# This works
$ ping 10.1.1.25
# Works
# Then enter container with:
$ winpty docker exec -it containerName bash
$ ping 10.1.1.25
# Works
$ ping ServerName
# Does NOT work
I would guess that I need to give docker some kind of mapping from the hosts knowlegde of the network to the container. I presume that would be through the network functionality, but I can't find any instructions that I understand.
And before anyone suggests it, the postgres instance cannot be moved, including being moved into a docker container of it's own.
Output of docker ps is:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
15c7903f7ccd imageName "tail -f /dev/null" 47 minutes ago Up 47 minutes 0.0.0.0:5432->5432/tcp, 0.0.0.0:8000->8888/tcp containerName
Solved: I needed to use the --dns option in docker run command. Then, inside the container, I needed to use the fully qualified name i.e. serverName.companyName.com instead of just serverName.
Thanks to #Dupinder Singh for pointing me to some useful DNS articles (see comments on the question).
Related
Steps to reproduce:
Download and run postgres:9.6.24:
docker run --name my_container --restart=always -d -p 127.0.0.1:5432:5432 -e POSTGRES_PASSWORD=pgmypass postgres:9.6.24
Here result:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
879883bfc84a postgres:9.6.24 "docker-entrypoint.s…" 26 seconds ago Up 25 seconds 127.0.0.1:5432->5432/tcp my_container
OK.
Open file inside container /var/lib/postgresql/data/pg_hba.conf
docker exec -it my_container bash
root#879883bfc84a:/# cat /var/lib/postgresql/data/pg_hba.conf
IPv4 local connections:
host all all 127.0.0.1/32 trust
Replace file /var/lib/postgresql/data/pg_hba.conf inside container by my file. Copy and overwrite my file from host to container:
tar --overwrite -c pg_hba.conf | docker exec -i my_container /bin/tar -C /var/lib/postgresql/data/ -x
Make sure the file has been modified. Go inside container and open changed file
docker exec -it my_container bash
root#879883bfc84a:/# cat /var/lib/postgresql/data/pg_hba.conf
IPv4 local connections:
host all all 0.0.0.0/0 trust
As you can see the content of file was changed.
Create new image from container
docker commit my_container
See result:
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
<none> <none> ee57ad4bc6b4 3 seconds ago 200MB
postgres 9.6.24 027ccf656dc1 12 months ago 200MB
Now tag my new image
docker tag ee57ad4bc6b4 my_new_image:1.0.0
See reult:
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
my_new_image 1.0.0 ee57ad4bc6b4 About a minute ago 200MB
postgres 9.6.24 027ccf656dc1 12 months ago 200MB
OK.
Stop and delete old continer:
docker stop my_continer
docker rm my_container
See result:
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
As you can see not exit any container. OK.
Create new continer from new image
docker run --name my_new_container_test --restart=always -d -p 127.0.0.1:5432:5432 -e POSTGRES_PASSWORD=pg1210 my_new_image:1.0.0
See result:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3a965dbbd991 my_new_image:1.0.0 "docker-entrypoint.s…" 7 seconds ago Up 6 seconds 127.0.0.1:5432->5432/tcp my_new_container
Open file inside container /var/lib/postgresql/data/pg_hba.conf
docker exec -it my_new_container bash
root#879883bfc84a:/# cat /var/lib/postgresql/data/pg_hba.conf
IPv4 local connections:
host all all 127.0.0.1/32 trust
As you can see my change in files are lost. The content of file is original. Not my changes.
P.S. This problem is only with file pg_hba.config. E.g if I created in the container the folder and file: /Downaloads/myfile.txt then this file not lost in the my container "my_new_container".
Editing files inside container with docker exec, in general, will in fact cause you to lose work. You mention docker commit but that's almost never a best practice. (If this was successful, but then you discovered PostgreSQL 9.6.24 exactly had some critical bug and you must upgrade, could you recreate the exact some image?)
In the case of the postgres image, the files in /var/lib/postgresql/data are always stored in a Docker volume or mount point. In your case you didn't use a docker run -v option, but the image is configured to create an anonymous volume in that directory. The volume is not included in docker commit, which is why you're not seeing it on the rebuilt container. (Also see docker postgres with initial data is not persisted over commits.)
For editing a configuration file, the easiest thing to do is to store the data on the host system. Create a directory to hold it, and extract the configuration file from the image. (Since the data directory is created by the image's startup script, you need a slightly longer path to get it out.)
mkdir pgdata
docker run -d --name pgtmp postgres:9.6.24
docker cp pgtmp:/var/lib/postgresql/data/pg_hba.conf ./pgdata
docker stop pgtmp
docker rm pgtmp
$EDITOR pgdata/pg_hba.conf
Now when you run the container, provide this data directory as a bind mount. That will inject the configuration file, but also cause the database data to persist over container exits.
docker run -v "$PWD/pgdata:/var/lib/postgresql/data" -u $(id -u) ... postgres:9.6.24
Note that this sequence doesn't use docker exec or "go inside" containers at all, and you haven't created an image without corresponding source. Everything is run with commands from the host. If you do need to reset the database data, in this setup, it's just files, and you can rm -rf pgdata, maybe saving the modified configuration file along the way.
(If I'm reading this configuration change correctly, you're trying to globally disable passwords and instead allow trust authentication for all inbound connections. That's not usually a good idea, especially since username/password authentication is standard in every database library I've encountered. You probably still want the volume to persist data, but I might not make this change to pg_hba.conf.)
Docker Container is a readyonly entity, which means if you will create a file into the container, remove it and re-create it (The container), the file is not supposed to be there.
what you want to do is one of two things,
Map your container to a local directory (volume)
Create a docker file based on the postgres image, and generate this modifications in a script, that your dockerfile reads.
docker volume usages
Dockerfile Reference
I am following this blog on how to connect to a docker instance: https://phoenixnap.com/kb/how-to-ssh-into-docker-container. It mentions using docker attach <name>
Trying this on my ec2 instance gives us:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
849844c1e3a5 6501862...us-east-618356524 "docker-entrypoint.s…" About a minute ago Up About a minute 0.0.0.0:32788->8401/tcp ecs-prod-clia-lab-5-Applicationprodclia-lab-8c88d2e0bc83cfb1230
Now let's try to `docker attach <instance-name>
$ docker attach ecs-prod-clia-lab-5-Applicationprodclia-lab-8c88d2e0bc83cfb1230
Error: No such container: ecs-prod-clia-lab-5-Applicationprodclia-lab-8c88d2e0bc83cfb1230
So that actually does not work? What is the correct way to do this?
To get a shell in a running container, do this:
$ docker exec -it <container-id> /bin/sh
The attach sub-command gives you access to a running containers stdout. That's not what you want here
However, if your conainer is meant to provide SSH as a service, you'll need to run it in such a way that it's exposed on the host, on some available port (like 2222).
The you'd simply "SSH in" like this:
$ ssh 127.0.0.1 -p 2222
I try to run a Shiny app on a remote server (here DigitalOcean) using Docker.
First, I created a package for my app as a .tar.gz file. Then:
Create the following Dockerfile:
FROM thinkr/rfull
COPY myapp_*.tar.gz /myapp.tar.gz
RUN R -e "install.packages('myapp.tar.gz', repos = NULL, type = 'source')"
COPY Rprofile.site /usr/local/lib/R/etc
EXPOSE 3838
CMD ["R", "-e myapp::run()"]
Create the following Rprofile.site
local({
options(shiny.port = 3838, shiny.host = "0.0.0.0")
})
Then I build the image using
docker build -t myapp .
I push the image to DockerHub using
docker tag myapp myrepo/myapp:latest
docker push myrepo/myapp
I connect to my droplet on DigitalOcean
eval $(docker-machine env mydroplet)
I create a container from my image on Dockerhub
docker run -d -p 3838:3838 myrepo/myapp
So far it seems to work fine. No message error and I got expected messages when I run docker logs mycontainer
The problem is that I do not know how to actually access the running container. When I connect to the droplet IP, I got nothing (This site can’t be reached). If use
docker inspect --format '{{ .NetworkSettings.IPAddress }}' mycontainer
I got an IP, it seems to be a local one ('172.17.0.2').
When I run docker ps here is what I got
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6d4195XXXX myrepo/myapp "R '-e myapp::ru…" 10 days ago Up 10 days 0.0.0.0:3838->3838/tcp, 8787/tcp determined_brown
So the question is: how can I run my dockerized shiny app on my droplet IP address?
Check if you have added the firewall rule to allow connections to 3838 port.
https://www.digitalocean.com/docs/networking/firewalls/resources/troubleshooting/
First, you need to publish the port, which is what you already do.
Second, you need to access the IP address of the host machine where there port is published.
The easiest way is probably to check the output of docker-machine env mydroplet and use this IP, together with your published port.
How I can add new nameserver in /etc/resolv.conf (dockerfile)?
On my dockerfile I use:
FROM ubuntu:14.04
RUN echo "nameserver 10.111.122.1" >> /etc/resolv.conf
On my test I use:
docker run --rm 746cb98d6c9b echo cat /etc/resolv.conf
I didn't get my change (the new nameserver)... So I try adding mannualy with
docker run --rm 746cb98d6c9b echo "nameserver 10.111.122.1" >> /etc/resolv.conf
and I get
zsh: permission denied: /etc/resolv.conf
How I can change permission of this file OR use a root user OR use a chmod in docker files ? My real task is to add and dns server for my build of this dockerfile.
I'm using a linux mint.
I'm get a correct result with a ping test on docker run command (with --dns)
So, one of the ways you can add new DNS information to your container's build process is by adding some startup options to your Docker daemon. The documentation for that process reveals that the option you'll use is --dns. The location of your configuration file depends on your specific distro. On my Linux Mint machine, the file is in /etc/default/docker. On Linux Mint, look for the DOCKER_OPTS= line, and add the appropriate --dns=x.x.x.x entries to that line.
For example, if you want to use Google's DNS, you should change that line to look like this:
DOCKER_OPTS="--dns=8.8.4.4 --dns=8.8.8.8"
Additionally, in the absense of --dns or --dns-search startup options, Docker will use the /etc/resolv.conf of the host it's running on instead.
The DNS configuration of a Docker container may be adjusted during the creation of the container and does not need to be hard-coded in the Docker image itself.
Passing a single DNS server to the container works by providing the --dns parameter:
$ docker run --rm --dns=8.8.8.8 <image>
You're free to provide more than one DNS server and you can also define other DNS related options like the DNS search name or common DNS options:
$ docker run --rm --dns=8.8.8.8 --dns=8.8.4.4 --dns-search=your.search.domain --dns-opt=timeout:50 <image>
If you pass cat /etc/resolv.conf as command to your container, you can easily verify that the passed DNS configuration options made it into the container's DNS configuration:
$ docker run --rm --dns=8.8.4.4 --dns=8.8.8.8 --dns-search=your.domain.name --dns-opt=timeout:50 alpine cat /etc/resolv.conf
search your.domain.name
nameserver 8.8.4.4
nameserver 8.8.8.8
options timeout:50
Please also refer to the docker run configuration which can be found at https://docs.docker.com/engine/reference/commandline/run/
What are the ways get the docker host's hostname from inside a container running on that host besides using environment variables? I know I can pass the hostname as an environment variable to the container at container creation time. I'm wondering how I can look it up at run time.
foo.example.com (docker host)
bar (docker container)
Is there a way for container bar running in docker host foo.example.com to get "foo.example.com"?
Edit to add use case:
The container will create an SRV record for service discovery of the form
_service._proto.name. TTL class SRV priority weight port target.
-----------------------------------------------------------------
_bar._http.example.com 60 IN SRV 5000 5000 20003 foo.example.com.
where 20003 is a dynamically allocated port on the docker host for a service listening on some fixed port in bar (docker handles the mapping from host port to container port).
My container will run a health check to make sure it has successfully created that SRV record as there will be many other bar containers on other docker hosts that also create their own SRV records.
_service._proto.name. TTL class SRV priority weight port target.
-----------------------------------------------------------------
_bar._http.example.com 60 IN SRV 5000 5000 20003 foo.example.com. <--
_bar._http.example.com 60 IN SRV 5000 5000 20003 foo2.example.com.
_bar._http.example.com 60 IN SRV 5000 5000 20003 foo3.example.com.
The health check will loop through the SRV records looking for the first one above and thus needs to know its hostname.
aside
I'm using Helios and just found out it adds an env var for me from which I can get the hostname. But I was just curious in case I was using docker without Helios.
You can easily pass it as an environment variable
docker run .. -e HOST_HOSTNAME=`hostname` ..
using
-e HOST_HOSTNAME=`hostname`
will call the hostname and use it's return as an environment variable called HOST_HOSTNAME, of course you can customize the key as you like.
note that this works on bash shell, if you using a different shell you might need to see the alternative for "backtick", for example a fish shell alternative would be
docker run .. -e HOST_HOSTNAME=(hostname) ..
I'm adding this because it's not mentioned in any of the other answers. You can give a container a specific hostname at runtime with the -h directive.
docker run -h=my.docker.container.example.com ubuntu:latest
You can use backticks (or whatever equivalent your shell uses) to get the output of hosthame into the -h argument.
docker run -h=`hostname` ubuntu:latest
There is a caveat, the value of hostname will be taken from the host you run the command from, so if you want the hostname of a virtual machine that's running your docker container then using hostname as an argument may not be correct if you are using the host machine to execute docker commands on the virtual machine.
You can pass in the hostname as an environment variable. You could also mount /etc so you can cat /etc/hostname. But I agree with Vitaly, this isn't the intended use case for containers IMO.
Another option that worked for me was to bind the network namespace of the host to the docker.
By adding:
docker run --net host
You can pass it as an environment variable like this. Generally Node is the host that it is running in. The hostname is defaulted to the host name of the node when it is created.
docker service create -e 'FOO={{.Node.Hostname}}' nginx
Then you can do docker ps to get the process ID and look at the env
$ docker exec -it c81640b6d1f1 env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=c81640b6d1f1
TERM=xterm
FOO=docker-desktop
NGINX_VERSION=1.17.4
NJS_VERSION=0.3.5
PKG_RELEASE=1~buster
HOME=/root
An example of usage would be with metricbeats so you know which node is having system issues which I put in https://github.com/trajano/elk-swarm:
metricbeat:
image: docker.elastic.co/beats/metricbeat:7.4.0
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro
- /proc:/hostfs/proc:ro
- /:/hostfs:ro
user: root
hostname: "{{.Node.Hostname}}"
command:
- -E
- |
metricbeat.modules=[
{
module:docker,
hosts:[unix:///var/run/docker.sock],
period:10s,
enabled:true
}
]
- -E
- processors={1:{add_docker_metadata:{host:unix:///var/run/docker.sock}}}
- -E
- output.elasticsearch.enabled=false
- -E
- output.logstash.enabled=true
- -E
- output.logstash.hosts=["logstash:5044"]
deploy:
mode: global
I think the reason that I have the same issue is a bug in the latest Docker for Mac beta, but buried in the comments there I was able to find a solution that worked for me & my team. We're using this for local development, where we need our containerized services to talk to a monolith as we work to replace it. This is probably not a production-viable solution.
On the host machine, alias a known available IP address to the loopback interface:
$ sudo ifconfig lo0 alias 10.200.10.1/24
Then add that IP with a hostname to your docker config. In my case, I'm using docker-compose, so I added this to my docker-compose.yml:
extra_hosts:
# configure your host to alias 10.200.10.1 to the loopback interface:
# sudo ifconfig lo0 alias 10.200.10.1/24
- "relevant_hostname:10.200.10.1"
I then verified that the desired host service (a web server) was available from inside the container by attaching to a bash session, and using wget to request a page from the host's web server:
$ docker exec -it container_name /bin/bash
$ wget relevant_hostname/index.html
$ cat index.html
OK, this isn't the hostname (as OP was asking), but this will resolve to your docker host from inside your container for connectivity purposes.
host.docker.internal
I was redirected here when googling for this.
HOSTIP=`ip -4 addr show scope global dev eth0 | grep inet | awk '{print $2}' | cut -d / -f 1 | sed -n 1p`
docker run --add-host=myhost:${HOSTIP} --rm -it debian
Now you can access the host under the alias "myhost"
The first line won't run on cygwin, but you can figure out some other way to obtain the local IP address using ipconfig.
you can run:
docker run --network="host"
for sending the value of the machine host to the container.
I ran
docker info | grep Name: | xargs | cut -d' ' -f2
inside my container.
I know it's an old question, but I needed this solution too, and I acme with another solution.
I used an entrypoint.sh to execute the following line, and define a variable with the actual hostname for that instance:
HOST=`hostname --fqdn`
Then, I used it across my entrypoint script:
echo "Value: $HOST"
Hope this helps