I have a Spring Boot 2.x project using Mongo. I am running this via Docker (using compose locally) and Kubernetes. I am trying to connect my service to a Mongo server. This is confusing to me, but for development I am using a local instance of Mongo, but deployed in GCP I have named mongo services.
here is my application.properties file:
#mongodb
spring.data.mongodb.uri= mongodb://mongo-serviceone:27017/serviceone
#logging
logging.level.org.springframework.data=trace
logging.level.=trace
And my Docker-compose:
version: '3'
# Define the services/containers to be run
services:
service: #name of your service
build: ./ # specify the directory of the Dockerfile
ports:
- "3009:3009" #specify ports forwarding
links:
- mongo-serviceone # link this service to the database service
volumes:
- .:/usr/src/app
depends_on:
- mongo-serviceone
mongo-serviceone: # name of the service
image: mongo
volumes:
- ./data:/data/db
ports:
- "27017:27017"
When I try docker-compose up . I get the following error:
mongo-serviceone_1 | 2018-08-22T13:50:33.454+0000 I NETWORK
[initandlisten] waiting for connections on port 27017 service_1
| 2018-08-22 13:50:33.526 INFO 1 --- [localhost:27017]
org.mongodb.driver.cluster : Exception in monitor thread
while connecting to server localhost:27017 service_1
| service_1 | com.mongodb.MongoSocketOpenException:
Exception opening socket service_1 | at
com.mongodb.connection.SocketStream.open(SocketStream.java:62)
~[mongodb-driver-core-3.6.3.jar!/:na]
running docker ps shows me:
692ebb72cf30 serviceone_service "java -Djava.securit…" About an hour ago Up 9 minutes 0.0.0.0:3009->3009/tcp, 8080/tcp serviceone_service_1
6cd55ae7bb77 mongo "docker-entrypoint.s…" About an hour ago Up 9 minutes 0.0.0.0:27017->27017/tcp serviceone_mongo-serviceone_1
While I am trying to connect to a local mongo, I thought that by using the name "mongo-serviceone"
Hard to tell what the exact issue is, but maybe this is just an issue because of the space " " after "spring.data.mongodb.uri=" and before "mongodb://mongo-serviceone:27017/serviceone"?
If not, maybe exec into the "service" container and try to ping the mongodb with: ping mongo-serviceone:27017
Let me know the output of this, so I can help you analyze and fix this issue.
Alternatively, you could switch from using docker compose to a Kubernetes native dev tool, as you are planning to run your application on Kubernetes anyways. Here is a list of possible tools:
Allow hot reloading:
DevSpace: https://github.com/covexo/devspace
ksync: https://github.com/vapor-ware/ksync
Pure CI/CD tools for dev:
Skaffold: https://github.com/GoogleContainerTools/skaffold
Draft: https://github.com/Azure/draft
For most of them, you will only need minikube or a dev namespace inside your existing cluster on GCP.
Looks like another application was running on port 27017 on your localhost Similar reported issue
quick way to check on linux/mac:
telnet 127.0.01 27017
check logs files:
docker logs serviceone_service
Related
I want to establish a connection using the draft server/client pattern of ZeroMQ across different docker containers. For that, I have a zmq.server that is running in docker_container_server. In addition, I have a zmq.client which runs in docker_container_client. Both containers are based on different docker images. Despite using the network_mode: host option with docker compose, this does not work. If I do the same within one container, it just works fine.
The docker-compose files are the following.
docker-compose-client.yaml:
services:
client:
privileged: true
image: client-image
container_name: client
command: tail -F anything
network_mode: host
environment:
- "ROS_MASTER_URI=http://127.0.0.1:11311"
- "ROS_HOSTNAME=127.0.0.1"
build:
context: ./client
network: host
dockerfile: Dockerfile
volumes:
- /var/run/docker.sock:/var/run/docker.sock
docker-compose-server.yaml:
services:
server:
image: server-image
privileged: true
container_name: server
network_mode: host
command: tail -F anything
volumes:
- /var/run/docker.sock:/var/run/docker.sock
build:
context: .
network: host
dockerfile: Dockerfile.server
The server application uses pyzmq as follows:
import zmq
class ServerExample():
def __init__(self):
self.stop = False
self.ctx = zmq.Context.instance()
self.url = 'tcp://127.0.0.1:5555'
self.server = self.ctx.socket(zmq.SERVER)
self.server.bind(self.url)
try:
while not self.stop:
msg = self.server.recv(copy=False) # stucks in this line
print(msg.routing_id)
if __name__ == '__main__':
server = ServerExample()
The client uses cppzmq as follows: (I also tried a pyzmq-client. This also works within a container but not across different containers.)
#include <zmq.hpp>
class ClientExample
{
private:
zmq::socket_t socket;
zmq::context_t ctx;
public:
ClientExample()
{
socket = zmq::socket_t(ctx, zmq::socket_type::client);
socket_.connect("tcp://127.0.0.1:5555");
char[] some_data;
socket_.send(zmq::buffer(some_data)); // not received by server
}
};
The containers are started by docker compose up server and docker compose up client.
When I do everything in one container and have a look on netstat -t | grep 5555, I get the same result in the container and on the host: (In this case, there are two clients and one server)
tcp 0 2856 localhost:56674 localhost:5555 ESTABLISHED
tcp 0 2856 localhost:5555 localhost:47666 ESTABLISHED
tcp 0 0 localhost:5555 localhost:56674 ESTABLISHED
That is why I think that the docker compose option network_mode: host is properly working in my case.
When I start the docker_container_server and docker_container_client, the execution stucks in the line where the server should receive a message. netstat -t | grep 5555 ouputs both on the host and in the containers just nothing.
The versions I use:
Ubuntu 20.04
Docker Compose version v2.12.2
Docker Engine Version 20.10.21
libzmq v4.3.4
cppzmq v4.9.0
pyzmq v25.0.0b1
I searched a lot through the internet, read the documentations of docker, docker compose, zmq etc. I came to the conclusion, that it should work with the network_mode: host option and I think I confirmed that the network_mode: host option works as it should. The code itself should not contain any mistakes, as it works fine within one container. But I didn't figure out, why the communication doesn't work across containers.
I am trying to run Gitlab Docker image on mac Big Sur locally following the steps from documentation: https://docs.gitlab.com/ee/install/docker.html . And always cannot see locally https://gitlab.example.com. I tried both Gitlab EE and CE, and different versions of images, including latest. I also tried to use usual Docker basic run and docker-compose. I also updated Docker Desktop to latest version 4.10.1. I refined the logs and for all cases I have same error in Gitally log below:
~/gitlab/logs/gitaly/current:
{"level":"warning","msg":"[core] [Channel #2 SubChannel #4] grpc: addrConn.createTransport failed to connect to {\n \"Addr\": \"/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.0\",\n \"ServerName\": \"/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.0\",\n \"Attributes\": null,\n \"BalancerAttributes\": null,\n \"Type\": 0,\n \"Metadata\": null\n}. Err: connection error: desc = \"transport: Error while dialing dial unix /var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.0: connect: no such file or directory\"","pid":344,"system":"system","time":"2022-07-26T09:57:38.226Z"}
{"level":"warning","msg":"[core] [Channel #3 SubChannel #5] grpc: addrConn.createTransport failed to connect to {\n \"Addr\": \"/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.1\",\n \"ServerName\": \"/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.1\",\n \"Attributes\": null,\n \"BalancerAttributes\": null,\n \"Type\": 0,\n \"Metadata\": null\n}. Err: connection error: desc = \"transport: Error while dialing dial unix /var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.1: connect: no such file or directory\"","pid":344,"system":"system","time":"2022-07-26T09:57:38.228Z"}
{"level":"warning","msg":"spawned","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","344","/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.0"],"supervisor.name":"gitaly-ruby.0","supervisor.pid":400,"time":"2022-07-26T09:57:38.228Z"}
{"address":"/var/opt/gitlab/gitaly/gitaly.socket","level":"info","msg":"listening at unix address","time":"2022-07-26T09:57:38.234Z"}
{"level":"warning","msg":"spawned","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","344","/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.1"],"supervisor.name":"gitaly-ruby.1","supervisor.pid":401,"time":"2022-07-26T09:57:38.234Z"}
But I am not sure that this is the case - also checked other logs and seems to me no errors in other logs - but of course I will be happy to provide more logs if you want.
This is my 'docker ps -a' status, docker version: 4.10.1, is run with docker-compose:
web:
image: 'gitlab/gitlab-ce:latest'
container_name: 'gitlab'
restart: unless-stopped
hostname: 'gitlab.example.com'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://gitlab.example.com'
ports:
- '80:80'
- '443:443'
- '22:22'
volumes:
- '$GITLAB_HOME/config:/etc/gitlab'
- '$GITLAB_HOME/logs:/var/log/gitlab'
- '$GITLAB_HOME/data:/var/opt/gitlab'
list of docker active containers says container is healthy - but actually I see nothing in browser via https://gitlab.example.com.
sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b0fb6252c7f3 gitlab/gitlab-ce:latest "/assets/wrapper" 3 days ago Up 3 minutes (healthy) 0.0.0.0:22->22/tcp, 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp gitlab
I also tried to add docker host IP to my hosts file - but no success:
192.168.31.182 host.docker.internal
192.168.31.182 gateway.docker.internal
192.168.31.182 gitlab.example.com
I still see nothing in browser by https://gitlab.example.com
Adding more info to #sytech "That's just a placeholder URL... Did you replace gitlab.example.com with your actual GitLab instance URL?"
You either need to add an /etc/hosts, setting to route gitlab.example.com to localhost or modify that to be localhost
hostname: 'localhost'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'localhost'
These settings are telling the nginx proxy what url should be forwarded to gitlab. That's useful when you're running multiple sites from one server. If you are just running it locally, you should just use localhost.
I also tried to add docker host IP to my hosts file - but no success:
You are forwarding the ports from the docker ip. Just use localhost. It probably didn't work because the docker IP you're looking at is for a different docker network.
So, I created two docker's images and I want to connect one to which other with the docker composer. The first image is Cassandra 3.11.11 (from the official hub docker) and the other I created by myself with the tomcat version 9.0.54 and my application spring boot.
I ran the docker-compose.ylm below to connect the two container, where cassandra:latest is the cassandra's image and centos7-tomcat9-myapp is my app web's image.
version: '3'
services:
casandra:
image: cassandra:latest
myapp:
image: centos7-tomcat9-myapp
depends_on:
- casandra
environment:
- CASSANDRA_HOST=cassandra
I ran the command line to start the app web's image : docker run -it --rm --name fe3c2f120e01 -p 8888:8080 centos7-tomcat9-app .
In the console log the spring boot show me the error below. It happened, because the myapp's container could not connect to the Cassandra's container.
2021-10-15 15:12:14.240 WARN 1 --- [ s0-admin-1]
c.d.o.d.i.c.control.ControlConnection : [s0] Error connecting to
Node(endPoint=127.0.0.1:9042, hostId=null, hashCode=47889c49), trying
next node (ConnectionInitException: [s0|control|connecting...]
Protocol initialization request, step 1 (OPTIONS): failed to send
request (io.netty.channel.StacklessClosedChannelException))
What am I doing wrong?
EDIT
This is the nodetool status about the cassandra's image:
[root#GDBDEV04 cassandradb]# docker exec 552d359d177e nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 172.18.0.3 84.76 KiB 16 100.0% 685b6e0a-13c2-4d41-ba99-f3b0fa94477c rack1
EDIT 2
I need to connect the Cassandra's DB image with the web application image. It is different to connect microservices. I tried to change the 127.0.0.0 (inside the cassandra.yaml) to 0.0.0.0 (only to test) and the error persist. I think something missing in my docker-compose.yml for sure. However, I did not know what.
Finally I found the error. In my case, I need to fixed the docker-compose.yml file adding the Cassandra and Tomcat's ports. And in my application.properties (spring boot config file), I changed the cluster's name.
Docker-compose.yml:
version: '3'
services:
cassandra:
image: cassandra:latest
ports:
- "9044:9042"
myapp:
image: centos7-tomcat9-myapp
ports:
-"8086:8080"
depends_on:
- cassandra
environment:
- CASSANDRA_HOST=cassandra
Application.config :
# CASSANDRA (CassandraProperties)
cassandra.cluster = Test Cluster
cassandra.contactpoints=${CASSANDRA_HOST}
This question help me to resolve my problem: Accessing docker container mysql databases
This is steps I have done
prisma init
I set postgresql for database in my local(not exist).
It created 3 files, datamodel.graphql, docker-compose.yml, prisma.yml
docker-compose up -d
I confirmed it running successfully
But if I call prisma deploy, it shows me error
Could not connect to server at http://localhost:4466. Please check if your server is running.
All I have done is standard operation described in manual and there is no customization in
https://www.prisma.io/docs/tutorials/deploy-prisma-servers/local-(docker)-meemaesh3k
And this is docker-compose.yml
version: '3'
services:
prisma:
image: prismagraphql/prisma:1.11
restart: always
ports:
- "4466:4466"
environment:
PRISMA_CONFIG: |
port: 4466
# uncomment the next line and provide the env var PRISMA_MANAGEMENT_API_SECRET=my-secret to activate cluster security
# managementApiSecret: my-secret
databases:
default:
connector: postgres
host: localhost
port: '5432'
database: databasename
schema: public
user: postgres
password: root
migrations: true
What am I missing?
I found this solution to the same problem i was facing
docker-machine ip default
Use this address and replace the "localhost" with the IP with the above command to look something like this in prisma.yml file
endpoint: http://1xx.1xx.xx.xxx:4466
The answer is referred from this Github Link
The documentation mentions:
docker ps
You should see output similar to this:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b799c529e73 prismagraphql/prisma:1.7 "/bin/sh -c /app/sta…" 17 hours ago Up 7 hours 0.0.0.0:4466->4466/tcp myapp_prisma_1
757dfba212f7 mysql:5.7 "docker-entrypoint.s…" 17 hours ago
(Here shown with mysql, but valid with postgresql too)
The point is: there should be two containers running, not one.
Check docker-compose logs to see why the second one (database) did not start.
instead of docker-compose up -d
USE:
docker-compose up
and keep the window running which will keep localhost:4466 alive.
Note : If u want to connect to connect to the database created in docker, you need to map the port in the following way:
docker run --name <ENTER_NAME> -e POSTGRES_PASSWORD=<ENTER_PASSWORD> -d -p 5433:5432 postgres
In the above case PORT(5433) = HOST_PORT and PORT(5432) = CONTAINER_PORT
I havs migrated my Rails app (local dev machine) to Docker-Compose. All is working except the Worker Rails instance (batch) cannot connect to Redis.
Completed 500 Internal Server Error in 40ms (ActiveRecord: 2.3ms)
Redis::CannotConnectError (Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED)):
In my docker-compose.yml
redis:
image: redis
ports:
- "6379:6379"
batch:
build: .
command: bundle exec rake environment resque:work QUEUE=*
volumes:
- .:/app
links:
- db
- redis
environment:
- REDIS_URL=redis://redis:6379
I think the Redis instance is available via the IP of the Docker host.
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
default * virtualbox Running tcp://192.168.99.100:2376 v1.10.0
Accessing via 0.0.0.0 doesn't work
$ curl 0.0.0.0:6379
curl: (7) Failed to connect to 0.0.0.0 port 6379: Connection refused
Accessing via the docker-machine IP I think works:
$ curl http://192.168.99.100:6379
-ERR wrong number of arguments for 'get' command
-ERR unknown command 'Host:'
EDIT
After installing redis-cli in the batch instance, I was able to hit the redis server using the 'redis' hostname. I think the problem is possibly in the Rails configuration itself.
Facepalm!!!
The docker containers were communicating just fine, the problem was I hadn't told Resque (the app using Redis) where to find it. Thank you to "The Real Bill" for pointing out I should be using docker-cli.
For anyone else using Docker and Resque, you need this in your config/initializers/resque.rb file:
Resque.redis = Redis.new(host: 'redis', port: 6379)
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
If you run
docker-compose run --rm batch env | grep REDIS
you will get the env variables that your container has (the link line in the compose will auto-generate some).
Then all you need to do is look for one along the lines of _REDIS_1_PORT... and use the correct one. I have never had luck connecting my rails to another service in any other way. But luckily these env variables are always generated on start so they will be up to date even if the container IP happens to change between startups.
You should use the hostname redis to connect to the service, although you may need to wait for redis to start.