gocql + docker "not found" - docker

Question

I have a docker compose file set up like this:
version: "3"
services:
web:
image: myApp
container_name: web
environment:
CASSANDRA_HOSTS: "db"
ports:
- "8080:8080"
restart: unless-stopped
depends_on:
- cassandra
links:
- cassandra:db
cassandra:
image: cassandra
restart: unless-stopped
ports:
- "9042:9042"
expose:
- "9042"
- "7000"
- "7001"
- "9160"
volumes:
- "/private/cassandra_docker/:/var/lib/cassandra"
and when start the services up with docker-compose up -d 'web' logs:
2018/11/04 13:01:33 gocql: unable to create session: control: unable to connect to initial hosts: dial tcp 172.19.0.2:9042: connect: connection refused
2018/11/04 13:01:39 gocql: unable to dial control conn 172.19.0.2: dial tcp 172.19.0.2:9042: connect: connection refused
2018/11/04 13:01:39 could not open session with cassandra
2018/11/04 13:01:39 gocql: unable to create session: control: unable to connect to initial hosts: dial tcp 172.19.0.2:9042: connect: connection refused
2018/11/04 13:01:53 gocql: Session.handleNodeUp: 172.19.0.2:9042
2018/11/04 13:01:53 not found
2018/11/04 13:02:19 gocql: Session.handleNodeUp: 172.19.0.2:9042
2018/11/04 13:02:19 not found
when i then use docker inspect docker_cassandra_1 i see it having this ip address - so resolution works fine:
"Networks": {
"docker_default": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"101ae9d068ce",
"cassandra"
],
"NetworkID": "9b416bdf21c152a7d751c264b9f42799534800f08b17ada6d90ee92d62c390bc",
"EndpointID": "bc2fe3cc3fb9967e0e056d24a98b97afbc856d54f7695f8e7cfb2583a397285a",
"Gateway": "172.19.0.1",
"IPAddress": "172.19.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:13:00:02"
}
}
and in the logs of cassandra i see that the node knows what it's ip address is, and it is listening on the correct port too:
10:06:37,709 StorageService.java:2289 - Node /172.19.0.2 state jump to
NORMAL cassandra_1
INFO [main] 2018-11-04 10:06:37,737 Gossiper.java:1692 - Waiting for gossip to settle... cassandra_1
INFO [main] 2018-11-04 10:06:45,745 Gossiper.java:1723 - No gossip
backlog; proceeding cassandra_1
INFO [main] 2018-11-04
10:06:46,379 NativeTransportService.java:70 - Netty using native Epoll
event loop cassandra_1 | INFO [main] 2018-11-04 10:06:46,496
...
Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...
when docker inspect web i see that it has no IP Address and is stuck restarting (and cannot be stopped except by force removing the image:
"Networks": {
"docker_default": {
"IPAMConfig": null,
"Links": [
"docker_cassandra_1_9415bd9be111:cassandra_1_9415bd9be111",
"docker_cassandra_1_9415bd9be111:db",
"docker_cassandra_1_9415bd9be111:docker_cassandra_1_9415bd9be111"
],
"Aliases": [
"82f1f884ee5f",
"web"
],
"NetworkID": "9b416bdf21c152a7d751c264b9f42799534800f08b17ada6d90ee92d62c390bc",
"EndpointID": "",
"Gateway": "",
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "",
"DriverOpts": null
}
}
docker ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
82f1f884ee5f myApp "/bin/bash -c '/runA…" 6 minutes ago Restarting (1) 5 minutes ago web
b3713664e748 cassandra "docker-entrypoint.s…" 6 minutes ago Up 6 minutes 7000-7001/tcp, 7199/tcp, 9160/tcp, 0.0.0.0:9042->9042/tcp docker_cassandra_1
I also checked the system.local table in cassandra - all addresses are the correct one (the one docker dns resolves to)
what is the problem here?
EDIT.:
failed to mention: docker exec -it docker_cassandra_1 cqlsh lets me connect to the database - so it is obviously up
EDIT2:
i did some further testing and installed nmap on the myApp container (and changed it to run /bin/bash on startup instead of my entrypoint script)
this is run on the "web" container
Starting Nmap 7.60 ( https://nmap.org ) at 2018-11-04 14:13 UTC
Nmap scan report for db (172.19.0.2)
Host is up (0.000061s latency).
rDNS record for 172.19.0.2: docker_cassandra_1.docker_default
PORT STATE SERVICE
9042/tcp open unknown
MAC Address: 02:42:AC:13:00:02 (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 0.47 seconds
root#028dcea32e53:/run.sh
2018/11/04 14:14:01 gocql: Session.handleNodeUp: 172.19.0.2:9042
2018/11/04 14:14:01 not found
docker exec -it docker_cassandra_1 /bin/bash
lets me connect via the external IP
csqlsh 172.19.0.2
Connected to Test Cluster at 172.19.0.2:9042.
[cqlsh 5.0.1 | Cassandra 3.11.3 | CQL spec 3.4.4 | Native protocol v4]
This is the code I connect with:
cluster := gocql.NewCluster("db")
cluster.Keyspace = "myApp"
cluster.ProtoVersion = 4
cluster.Consistency = gocql.One
var err error
DataSource.Client, err = cluster.CreateSession()
edit:
it also does not work when i start my app on the hosts and try to connect to 127.0.0.1, while port 9042 is exposed via -p 9042:9042

Answer 1

Open your terminal and first verify the port is already is use or not using the following command
sudo lsof -i :9042
if It is already used the you will get a list containing the process ID. You can kill the operation by using the following command
sudo kill <process ID>
After this check all the actively running docker containers by using the following command
docker ps -a
Now we can remove all the docker containers using the following command
docker rm -f $(docker ps -aq)
After all these operation use the docker compose up command that you used in your project.
docker-compose up -d
Hope it will work!