Why is are my published ports not working? - docker

I've created a docker image containing a rust application that responds to get requests on port 8000. The application itself is a basic example using the rocket library (https://rocket.rs/) it looks like this
#![feature(proc_macro_hygiene, decl_macro)]
#[macro_use] extern crate rocket;
#[get("/")]
fn index() -> &'static str {
"Hello, world!"
}
fn main() {
rocket::ignite().mount("/", routes![index]).launch();
}
I have compiled this and called it server
I then created a Docker file to host it
FROM ubuntu:16.04
RUN apt-get update; apt-get install -y curl
COPY server /root/
EXPOSE 8000
CMD ["/root/server"]
I build the docker image with
$ docker build -t port_test and run it with $ docker run -p 8000:8000 port_test
At this point it all looks good
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3befe0c272f7 port_test "/root/server" 7 minutes ago Up 7 minutes 0.0.0.0:8000->8000/tcp festive_wilson
If I run curl within the container it works fine
$ docker exec -it 3befe0c272f7 curl -s localhost:8000
Hello, world!
However I can't do the same from the host
$ curl localhost:8000
curl: (56) Recv failure: Connection reset by peer

David Maze was correct. The problem was that the process was binding to localhost in the container. I added a Rocket.toml file with the following entries
[global]
address = "0.0.0.0"
[development]
address = "0.0.0.0"
and now it works fine.
Thanks David.

Rocket have different standard configuration, please try staging or prod to be able to do what you want, source.
ROCKET_ENV=staging cargo run
See also:
Why can I not access this Rust simple server from the Internet?

Related

Unable to connect local scala app with Hbase running in docker

I have spent the entire day trying to figure out why my sscala app running on windows is unable to make a successful connection with hbase running in a docker container.
I can shell into the container and run the hbase shell, create tables etc
Also I can port forward to localhost:16010 and see the Hbase UI. Some additional details of the setup as follows.
Env:
Scala app: Windows (host)
Hbase: docker container
Docker container details
FROM openjdk:8
ENV HBASE_VERSION=2.4.12
RUN apt-get update
RUN apt-get install -y netcat
RUN mkdir -p /var/hbase && \
cd /opt && \
wget -q https://archive.apache.org/dist/hbase/${HBASE_VERSION}/hbase-${HBASE_VERSION}-bin.tar.gz && \
tar xzf hbase-${HBASE_VERSION}-bin.tar.gz
WORKDIR /opt/hbase-${HBASE_VERSION}
COPY hbase-site.xml conf
CMD ./bin/start-hbase.sh && tail -F logs/hbase*.log
hbase-site.xml -> hbase.cluster.distributed & hbase.unsafe.stream.capability.enforce set to false
The hbase container is up and running and accesible. Also confirmed zookeeper is reachable within the container as well as from the host using echo ruok | nc localhost 2181; echo
Running container as follows:
docker run -it -p 2181:2181 -p 2888:2888 -p 3888:3888 -p 16010:16010 -p 16000:16000 -p 16020:16020 -p 16030:16030 -p 8080:8080 -h hbb hbase-1
Scala app
val conf : Configuration = HBaseConfiguration.create()
conf.set("hbase.zookeeper.quorum", "hbb")
conf.set("hbase.zookeeper.property.clientPort", "2181")
conf.set("hbase.master", "hbb")
conf.set("hbase.cluster.distributed","false")
// conf.set("hbase.client.pause", "1000")
// conf.set("hbase.client.retries.number", "2")
// conf.set("zookeeper.recovery.retry", "1")
val connection = ConnectionFactory.createConnection(conf)
The part of the stack trace
1043 [ReadOnlyZKClient-hbb:2181#0x30b6ffe0] DEBUG org.apache.zookeeper.ClientCnxn - zookeeper.disableAutoWatchReset is false
3947 [ReadOnlyZKClient-hbb:2181#0x30b6ffe0-SendThread(hbb:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server hbb:2181. Will not attempt to authenticate using SASL (unknown error)
3949 [ReadOnlyZKClient-hbb:2181#0x30b6ffe0-SendThread(hbb:2181)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server hbb:2181, unexpected error, closing socket connection and attempting reconnect
java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:100)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:620)
at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1021)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1064)
3953 [ReadOnlyZKClient-hbb:2181#0x30b6ffe0-SendThread(hbb:2181)] DEBUG org.apache.zookeeper.ClientCnxnSocketNIO - Ignoring exception during shutdown input
java.net.SocketException: Socket is not connected
at sun.nio.ch.Net.translateToSocketException(Net.java:122)
at sun.nio.ch.Net.translateException(Net.java:156)
at sun.nio.ch.Net.translateException(Net.java:162)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:401)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:200)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1250)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1174)
I have tried changing the hbase.zookeeper.quorum & master props on the client side to localhost / 127.0.0.1 as well as changing the etc/hosts file with the container id. No luck yet
Would greatly appreciate some guidance on this :)

Want to convert OS script to simple docker container

I'm in the process of cleaning up many years of hackery and I'm hoping to get some help. I am wanting to convert a shell script I wrote that uses netcat to connect to a Mochad (X10 interface) container and then grabs the output and formats it and passes that along to my MQTT server. This has worked great for years but now I'd like to make it into a simple Docker container.
Here's the simple script I want to convert (mochadtomqtt.sh):
#!/bin/sh
nc 192.168.0.25 1099 | awk ' /HouseUnit:/ && /Func:/ { system("mosquitto_pub -h 192.168.0.25 -q 1 -t /X10/"$6" -m "$8) } '
and this is the Dockerfile I've tried:
FROM ubuntu:14.04
RUN apt-get update && apt-get install -y mosquitto-clients && apt-get install -y mosquitto
CMD ["/x10/mochadtomqtt.sh", ""]
and finally, the portion from my docker-compose:
x102mqtt:
container_name: x102mqtt
image: 'x102mqtt:latest'
restart: unless-stopped
depends_on:
- mqtt
volumes:
- /sharedfolders/appdata/x102mqtt:/x10
network_mode: host
I've built the image and deployed the container but I do not get any sort of output. No errors no nothing. I know the script is being run as I've added an echo "hello world" and that did show up repeatedly. Can someone tell me how I might debug this or if they can spot my obvious errors (outside that I have no experience at doing this).
Many thanks!
EDIT to add more information:
This is what the output from nc looks like:
04/04 15:47:17 Rx RF HouseUnit: B3 Func: On
Answering #The Fool:
I copied most of this from other files. Noted on the command options being blank, I can remove that. Not sure about the restarting as this is intended to be a one shot call. What causes the restarts?
EDIT
I changed the script to ensure that nc, mosquitto_pub and awk were indeed available. They are. I put the full path to these executables in the commandline. I also added an echo to see if the command was running or if it had ended. It ended.
Here's the script modified:
#!/bin/sh
which nc
which mosquitto_pub
which awk
/bin/nc 192.168.0.25 1099 | /usr/bin/awk ' /HouseUnit:/ && /Func:/ { system("/usr/bin/mosquitto_pub -h 192.168.0.25 -q 1 -t /X10/"$6" -m "$8) } '
echo "End"
and resulting output:
/bin/nc
/usr/bin/mosquitto_pub
/usr/bin/awk
End
I'm not sure what that strange square character comes from. I also did a ping previously to the IP I'm trying to connect to and it did ping fine within the container.
Not sure why nc is exiting instead of staying connected. I get no errors at all.

wget over proxy within Dockerfile doesn't work

i have an server that have access internet only via proxy.
So im using this docker-compose command:
docker-compose build --build-arg HTTP_PROXY=http://myproxy.server:3128 --build-arg HTTPS_PROXY=http://myproxy.server:3128 web
It starts to download alpine-os Image, executes commands like apk add, fetch from internet, which works.
Except downloading wget don't works:
wget -O libiconv.tar.gz "https://ftp.gnu.org/pub/gnu/libiconv/libiconv-$LIBICONV_VERSION.tar.gz
Error:
Connecting to ftp.gnu.org (208.118.235.20:443)
wget: can't connect to remote host (208.118.235.20): Operation timed out
I have tried many variants like adding .wgetrc file within export https_proxy="" or adding parameter to wget command:
wget -Y on
No one of them works..
PS: wget with e option doesn't exists:
wget: unrecognized option: e
If you have connectivity with your host, just add to your docker-compose file:
- network_mode: host
And you'll have inside the container the same interfaces, ports etc than in the host, without having to modify resolv.conf

Understanding the difference in sequence of ENTRYPOINT/CMD between Dockerfile and docker run

Docker noob here...
I am trying to build and run an IBM DataPower container from a Dockerfile, but it doesn't seem to work the same as when just running docker run and passing the same parameters in the terminal.
This works (docker run)
docker run -it \
-v $PWD/config:/drouter/config \
-e DATAPOWER_ACCEPT_LICENSE=true \
-e DATAPOWER_INTERACTIVE=true \
-e DATAPOWER_WORKER_THREADS=4 \
-p 9090:9090 \
--name mydatapower \
ibmcom/datapower
... the key part being that it mounts the ./config folder and the custom configuration is picked up by datapower running in the container.
This doesn't (Dockerfile)
Dockerfile:
FROM ibmcom/datapower
ENV DATAPOWER_ACCEPT_LICENSE=true
ENV DATAPOWER_INTERACTIVE=true
ENV DATAPOWER_WORKER_THREADS=4
EXPOSE 9090
COPY config/auto-startup.cfg /drouter/config/auto-startup.cfg
Build:
docker build -t local/datapower .
Run:
docker run -it \
-p 9090:9090 \
--name mydatapower local/datapower
The problem is that DataPower doesn't pick up the auto-startup.cfg file, so the additional config options doesn't get used. I know the source file path is correct because if I misspell the file name docker throws an error.
I have a theory that it might be running the inherited ENTRYPOINT or CMD before the config file is available. I don't know how to test or prove this. I don't know what the ENTRYPOINT or CMD is because the inherited image is not open source and I can't figure out how to find it.
Does that seem likely?
UPDATE:
The content of the auto-startup.cfg is:
top; co
ssh
web-mgmt
admin enabled
port 9090
exit
It simply enables the DataPower WebGUI.
The output when running it in the commandline with:
docker run -it -v $PWD/config:/drouter/config -v $PWD/local:/drouter/local -e DATAPOWER_ACCEPT_LICENSE=true -e DATAPOWER_INTERACTIVE=true -e DATAPOWER_WORKER_THREADS=4 -p 9091:9090 --name myconfigureddatapower ibmcom/datapower`
...contains this:
20170908T121729.015Z [0x8100006e][system][notice] : Executing startup configuration.
20170908T121729.970Z [0x00350014][mgmt][notice] web-mgmt(WebGUI-Settings): tid(303): Operational state up
...but with Dockerfile it doesn't. That's why I think the config files may be copied into place too late.
I've tried adding CMD ["/bin/drouter"] to the end of my Dockerfile to no avail.
I have tested your Dockerfile and it seems to be working. My auto-startup.cfg file is copied in the proper location and when I launch the container it's reading the file.
I get this output:
[root#ip-172-30-2-164 tmp]# docker run -ti -p 9090:9090 test
20170908T123728.818Z [0x8040006b][system][notice] logging target(default-log): Logging started.
20170908T123729.067Z [0x804000fe][system][notice] : Container instance UUID: 36bcca0e-6139-4694-91b0-2b7b66c3a498, Cores: 4, vCPUs: 4, CPU model: Intel(R) Xeon(R) CPU E5-2676 v3 # 2.40GHz, Memory: 16049.1MB, Platform: docker, OS: dpos, Edition: developers-limited, Up time: 0 minutes
20170908T123729.071Z [0x8040001c][system][notice] : DataPower IDG is on-line.
20170908T123729.071Z [0x8100006f][system][notice] : Executing default startup configuration.
20170908T123729.416Z [0x8100006d][system][notice] : Executing system configuration.
20170908T123729.417Z [0x8100006b][mgmt][notice] domain(default): tid(8143): Domain operational state is up.
708f98be1390
Unauthorized access prohibited.
20170908T123731.239Z [0x806000dd][system][notice] cert-monitor(Certificate Monitor): tid(399): Enabling Certificate Monitor to scan once every 1 days for soon to expire certificates
20170908T123731.552Z [0x8100006e][system][notice] : Executing startup configuration.
20170908T123732.436Z [0x8100003b][mgmt][notice] domain(default): Domain configured successfully.
20170908T123732.449Z [0x00350014][mgmt][notice] web-mgmt(WebGUI-Settings): tid(303): Operational state up
login:
To check that your file has been copied to the container you can run docker run -ti local/datapower sh to enter the container and then check the content of /drouter/config/.
Your base image command is: CMD ["/bin/drouter"] you can check it running docker history ibmcom/datapower.
UPDATE:
The drouter user in the container must be able to read the auto-startup.cfg file. You have 2 options:
set your local auto-startup.cfg with the proper permissions (chmod 644 config/autostart.cfg).
or add these line in the Dockerfile so drouter can read the file:
USER root
RUN chown drouter /drouter/config/auto-startup.cfg
USER drouter

How to test the container or image after docker build?

I have the following Dockerfile
############################################################
# Purpose : Dockerize Django App to be used in AWS EC2
# Django : 1.8.1
# OS : Ubuntu 14.04
# WebServer : nginx
# Database : Postgres inside RDS
# Python : 2.7
# VERSION : 0.1
############################################################
from ubuntu:14.04
maintainer Kim Stacks, kimcity#gmail.com
# make sure package repository is up to date
run echo "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) main universe" > /etc/apt/sources.list
run apt-get update
# install python
# install nginx
Inside my VM, I did the following:
docker build -t ubuntu1404/djangoapp .
It is successful.
What do I do to run the docker image?
Where is the image or container?
I have already tried running
docker run ubuntu1404/djangoapp
Nothing happens.
What I see when I run docker images
root#vagrant-ubuntu-trusty-64:/var/virtual/Apps/DockerFiles/Django27InUbuntu# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
ubuntu1404/djangoapp latest cfb161605c8e 10 minutes ago 198.3 MB
ubuntu 14.04 07f8e8c5e660 10 days ago 188.3 MB
hello-world latest 91c95931e552 3 weeks ago 910 B
When I run docker ps, nothing shows up
You have to give a command your container will have to process.
Example : sh
you could try :
docker run -ti yourimage sh
(-ti is used to keep a terminal open)
If you want to launch a daemon (like a server), you will have to enter something like :
docker run -d yourimage daemontolaunch
Use docker help run for more options.
You also can set a default behaviour with CMD instruction in your Dockerfile so you won't have to give this command to your container each time you want to run it.
EDIT - about container removing :
Containers and images are different.
A container is an instance of an image.
You can run several containers from the same image.
The container automatically stops when the process it runs terminates.
But the container isn't deleted (just stopped, so you can restart it).
But if you want to remove it (removing a container doesn't remove the image) you have two ways to do :
automatically removing it at the end of the process by adding --rm option to docker run.
Manually removing it by using the docker rm command and giving it the container ID or its name (a container has to be stopped before being removed, use docker stop for this).
A usefull command :
Use docker ps to list containers. -q to display only the container IDs, -a to display even stopped containers.
More here.
EDIT 2:
This could also help you to discover docker if you didn't try it.
How to test the container or image after docker build?
In order to test you can add write a bash script which will do the job https://blog.brazdeikis.io/posts/docker-image-tests
Btw, from the post, I see that it does not match the question from the title.
So, Added a link for the souls who arrived here based on the title...
Download the latest shaded dist from https://github.com/dgroup/docker-unittests/releases:
wget https://github.com/dgroup/docker-unittests/releases/download/s1.1.1/docker-unittests-app-1.1.1.jar
De fine an *.yml file with tests.
version: 1.1
setup:
- apt-get update
- apt-get install -y tree
tests:
- assume: java version is 1.9, Debian build
cmd: java -version
output:
contains:
- openjdk version "9.0.1"
- build 9.0.1+11-Debian
- assume: curl version is 7.xxx
cmd: curl --version
output:
startsWith: curl 7.
matches:
- "^curl\\s7.*\\n.*\\nProtocols.+ftps.+https.+telnet.*\\n.*\\n$"
contains:
- AsynchDNS IDN IPv6 Largefile GSS-API
- assume: Setup section installed `tree`
cmd: tree --version
output:
contains: ["Steve Baker", "Florian Sesser"]
Run tests for image
java -jar docker-unittests.jar -f image-tests.yml -i openjdk:9.0.1-11
https://i.stack.imgur.com/DSv72.png

Resources