I am trying to do this lab and type in the following command:
sudo docker run -it --name bdu_spark2 -P -p 4040:4040 -p 4041:4041 -p 8080:8080 -p 8081:8081 bigdatauniversity/spark2:latest /etc/bootstrap.sh -bash
But I get the following error. Is there a conflict between port 8080 of docker using it and other software trying to use it? I have restarted docker and made sure no other containers are running. Thanks for all the input.
Error response from daemon: Cannot start container 3c62472fe5f8481e5ee957550078f06106b45fc6bffe25669272e2ea924b5f36: failed to create endpoint bdu_spark2 on network bridge: Error starting userland proxy: listen tcp 0.0.0.0:8080: bind: address already in use
This is usually caused because another container is using 8080 port on your docker host.
You can see your running containers by running: $ sudo docker ps
Either stop the other container, or choose a different host port to map your container's 8080 to.
In my case doing this with MySQL, I didn't realize it was because I already had a native MySQL running on that port.
docker run --name db --detach --env="MYSQL_ROOT_PASSWORD=123" --publish=3306:3306 mysql:latest
I did a netstat --all --numeric --program --inet --timers | grep 3306 and noticed it gave me 1418/mysqld. Then I did a ps aux | grep mysql and noticed that was the same process number started by /usr/sbin/mysqld which was my local MySQL instance on my host, nothing to do with containers.
Double check nothing is using those ports, especially 8080, which is very common for stand alone web servers, like those that ship with IDEs.
If you are using that port, you can use the --publish option to specify the host port to be different but still use the same port on the container. i.e. --publish=8081:8080, hostport:containerport.
Related
I am on windows 10 ent
Running command: docker container run -d -p 8080:80 --name mysql -e MYSQL_RANDOM_ROOT_PASSWORD=true mysql
But I am getting this error:
docker: Error response from daemon: driver failed programming external connectivity on endpoint mysql(969f8eac66c92e42a4f19f6f28eec72c6802fea1eabed48dfb382c6a35cbb2ce)Error starting userland proxy: Bind for 0.0.0.0:8080: unexpected error Permission denied.
Need help.
This error is often caused because the port you specified is already in use. Sometimes it is because the current user does not have administrative rights.
If you do not specifically require port 80, try port 8000 or 8080.
docker container run -d -p 8080:8000 --name mysql -e MYSQL_RANDOM_ROOT_PASSWORD=true mysql
If that doesn't fix it, try executing the command in sudo as some ports are system protected and require a user with admin privileges.
In all such cases where you are not sure which port is free on the host machine, you can try using -P option while running your images and then use docker port to see it's bind with which port.
#>docker container run -d -P --name mysql -e MYSQL_RANDOM_ROOT_PASSWORD=true mysql
#>docker port mysql
3306/tcp -> 0.0.0.0:32768
docker#default:~$
After this you know which port is free then you can select that one and use your usual command.
#>docker container run -d -p 32768:80 --name mysql -e MYSQL_RANDOM_ROOT_PASSWORD=true mysql
However I think, instead of 80 - you must expose 3306 port - since the default mysql doesn't provide a web-interface.
Adminer (formerly phpMinAdmin) is a different application that does provide mysql server embedded with it.
I've installed docker in a VM which is publicy available on internet. I've installed mongodb in a docker container in the VM.Mongodb is listening on 27017 port.
I've installed using the following steps
docker run -p 27017:27017 --name da-mongo -v ~/mongo-data:/data/db -d mongo
The port from container is redirected to the host using the -p flag. But the port 27017 is exposed on the internet. I don't want it to happen.
Is there any way to fix it?
Well, if you want it available for certain hosts then you need a firewall. But, if all you need is it working on localhost (your VM machine), then you don't need to expose/bind the port with the host. I suggest you to run the container without the -p option, then, run the following command:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' your_container_id_or_name
After that, it will display an IP, it is the IP of the container you've just ran (Yes, docker uses somewhat an internal virtual network connecting your containers and your host machine between them).
After that, you can connect to it using the IP and port combination, something like:
172.17.0.2:27017
When you publish the port, you can select which host interface to publish on:
docker run -p 127.0.0.1:27017:27017 --name da-mongo \
-v ~/mongo-data:/data/db -d mongo
That will publish the container port 27017 to host interface 127.0.0.1 port 27017. You can only add the interface to the host port, the container itself must still bind to 0.0.0.0.
I am beginner and just started using docker, before posting here I google a lot but a lot of mixed confusing result.
I started docker with this command
docker run -itd --name dockWeb2 -v /var/www/wordpress/ -p 80:80 atozchevara/rpi-apache-php5
hoping I would be able to directly mount wordpress installation onto container , as by default it picks internal path of container /var/www/index.php, to override it I used -v flag. but it doesn't work.
I tried using multiple ports by passing -p arguments again for each port but that too gives error
docker run -itd --name dockWeb3 -v /var/www/wordpress/ -p 80:80 -p 22:22 atozchevara/rpi-apache-php5
66a959e4e99af8122705913005fcae12e2e8a5203da7b77ff1717751314fca28
docker: Error response from daemon: driver failed programming external connectivity on endpoint dockWeb3 (eb42a619a8c79961d35d59e0d8930a92541a20132525055afb3b0d2d87483e7f): Bind for 0.0.0.0:80 failed: port is already allocated.
otherwise Could have uploaded my wordpress using ssh to container's /var/www/ location.
For the first issue if you want to mount a volume from the host you need to use Bind mount a volume
docker run -itd --name dockWeb2 -v your_project_path:/var/www/wordpress/ 0.0.0.0:80 failed: port is already allocated. atozchevara/rpi-apache-php5
For the post using -p 80:80 you are publishing container port 80 to the host port 80, and if the host port is already in use you got an error 0.0.0.0:80 failed: port is already allocated. try to use a different port -p 9090:80.
To make development easier for a project, I've put a couple of services it depends on in docker containers. This makes 'localhost' in the project's config mean something different when it is passed to one of the containers.
edit
To be clear, I'm trying to forward one of the container's ports to the host so when a process running in the container tries to access localhost:5432, it connects to the host's port 5432.
endedit
I'm currently using
HOST_IP=`ip route | grep default | awk '{ printf "%s",$3 }'`
cat /etc/hosts | sed "s/127.0.0.1/$HOST_IP/" > /tmp/etc_hosts
cp /tmp/etc_hosts /etc/hosts
to redirect anything targeting 'localhost' to the container's host. It works in this situation, but I'd prefer to find a way to do this only for the needed port as I expect it won't work in other situations.
Here's what I came up with to do that, but it's not working; when a connection in the container is to localhost:5432, it tries to connect to the container's 5432 instead of the host's:
# --- These are the things that should make redirecting port 5432 to the host machine
# work, provided the container is run in privileged mode.
sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv4.conf.all.route_localnet=1
iptables -t nat -A PREROUTING -p tcp --dport 5432 -j DNAT --to 172.19.0.1:5432
iptables -A FORWARD -d 172.19.0.1 -p tcp --dport 5432 -j ACCEPT
iptables -t nat -A POSTROUTING -j MASQUERADE
If I understand well, for development, you'd want localhost to resolve to a specific container, including when it's called from another container.
Host forwarding
Rewriting your hosts file is, as you mentioned it, not a good idea, since many services can experiment issues if you design localhost as being something different than, well... your local host.
But you can consider a few solutions.
Docker Toolbox
If running docker with Docker Toolbox, or by yourself on a virtual machine with Virtual Box, the intermediate virtual machine is visible, so localhost will represent it. You'll have to run the container, exposing this port, and then to set up a port forwarding in Virtualbox. If I use Wordpress as an example:
docker run -p 80:80 --name website -d wordpress
Virtual Box -> your docker VM (usually called default) -> Network -> Adapter 1 -> port forwarding -> create a mapping from host 8080 to guest 80
It will make Wordpress available at http://localhost:8080. Please note that under MacOS, the kernel restrains non-privileged port forwarding (ports under 1024).
This port forwarding can be created in command line, if you want to put it in a script:
VBoxManage modifyvm "default" --natpf1 "app,tcp,,8080,,80"
Docker for Windows/Docker for Mac
If running docker through Docker for Windows/Docker for Mac (or directly under Linux), rather than Docker Toolbox, you can run the container using the -p parameter, as specified by Scott's post, and your service will be available on localhost at this port (because the intermediate virtual machine is transparent, or no VM under Linux):
docker run -p 5432:5432 --name myapp -d myimage will make myapp available at localhost:5432.
socat (or iptables)
You can run socat on your host this way to forward communication on a specific port to your container:
socat TCP-LISTEN:5432,fork,reuseaddr,user=node,group=node,mode=777 TCP:172.19.0.1:5432 &
(where 172.19.0.1 is your container IP)
Container forwarding
--network
Your containers have their own hosts file, that you can see by issuing such a command:
docker run ubuntu cat /etc/hosts
You can add entries to hosts with the --add-host parameter:
docker run --add-host domain:1.2.3.4 --add-host domain2:5.6.7.8 ubuntu cat /etc/hosts
However this solution will be useless for localhost, because it won't remove the previous localhost associations. What you're looking for (and what is cleaner) is the parameter --network=host which allows the container to share the network interfaces of the host:
docker run --network=host ubuntu
This way, your container will be able to call the other containers services on localhost using their port.
The right way
Of course, the right way to achieve what you want would be to link your containers together and use their link names rather than localhost.
docker run -d --name mariadb -e MYSQL_ROOT_PASSWORD=password mariadb
docker run -d --name="wordpress" -p 8080:80 -e WORDPRESS_DB_PASSWORD=password --link mariadb:mysql wordpress
In this case, the Wordpress container will have a mysql entry in its hosts file, pointing to the mariadb container IP address. To see it, open a bash session in the Wordpress container and see by yourself.
docker exec -ti wordpress bash
#cat /etc/hosts
Show us how you are launching your container
port mapping can happen in your docker run command : -p hostport:containerport
as in
docker run -p 5432:5432 --name mycontainer -d myimage
Ok, I am pretty new to Docker world. So this might be a very basic question.
I have a container running in Docker, which is running RabbitMQ. Let's say the name of this container is "Rabbit-container".
RabbitMQ container was started with this command:
docker run -d -t -i --name rmq -p 5672:5672 rabbitmq:3-management
Python script command with 2 args:
python ~/Documents/myscripts/migrate_data.py amqp://rabbit:5672/ ~/Documents/queue/
Now, I am running a Python script from my host machine, which is creating some messages. I want to send these messages to my "Rabbit-container". Hence I want to connect to this container from my host machine (Mac OSX).
Is this even possible? If yes, how?
Please let me know if more details are needed.
So, I solved it by simply mapping the RMQ listening port to host OS:
docker run -d -t -i --name rmq -p 15672:15672 -p 5672:5672 rabbitmq:3-management
I previously had only -p 15672:15672 in my command. This is mapping the Admin UI from Docker container to my host OS. I added -p 5672:5672, which mapped RabbitMQ listening port from Docker container to host OS.
If you're running this container in your local OSX system then you should find your default docker-machine ip address by running:
docker-machine ip default
Then you can change your python script to point to that address and mapped port on <your_docker_machine_ip>:5672.
That happens because docker runs in a virtualization engine on OSX and Windows, so when you map a port to the host, you're actually mapping it to the virtual machine.
You'd need to run the container with port 5672 exposed, perhaps 15672 as well if you want WebUI, and 5671 if you use SSL, or any other port for which you add tcp listener in rabbitmq.
It would be also easier if you had a specific IP and a host name for the rabbitmq container. To do this, you'd need to create your own docker network
docker network create --subnet=172.18.0.0/16 mynet123
After that start the container like so
docker run -d --net mynet123--ip 172.18.0.11 --hostname rmq1 --name rmq_container_name -p 15673:15672 rabbitmq:3-management
note that with rabbitmq:3-management image the port 5672 is (well, was when I used it) already exposed so no need to do that. --name is for container name, and --hostname obviously for host name.
So now, from your host you can connect to rmq1 rabbitmq server.
You said that you have never used docker-machine before, so i assume you are using the Docker Beta for Mac (you should see the docker-icon in the menu bar at the top).
Your docker run command for rabbit is correct. If you now want to connect to rabbit, you have two options:
Wrap your python script in a new container and link it to rabbit:
docker run -it --rm --name migration --link rmq:rabbit -v ~/Documents/myscripts:/app -w /app python:3 python migrate_data.py
Note that we have to link rmq:rabbit, because you name your container rmq but use rabbit in the script.
Execute your python script on your host machine and use localhost:5672
python ~/Documents/myscripts/migrate_data.py amqp://localhost:5672/ ~/Documents/queue/