I am trying to create an example to create two WEB API 's and containerize them and to communicate between them.
I would like to see the side car design pattern, I have found an example in Github that I am trying to run.
https://github.com/cesaroll/dotnet-sidecar
In the above example, HelloAPI makes a call to HelloSideCar API which is a different project.
In the HelloAPI a call is made to another API in another project. I am trying to run in local using Docker Compose.
When I try to hit the API from HelloAPI(localhost:8080/FromSidecar) project to SideCarAPI, I see a 404 error, request is not going to another container
Below Is my Docker Compose
# docker-compose up -d
# docker-compose stop
# docker-compose rm -f
version: '3.8'
services:
hello-sidecar-api:
image: hello-sidecar-api:latest
container_name: hello-sidecar-api
ports:
- "8180:8080"
hello-api:
image: helloapi:latest
container_name: hello-api
environment:
- SIDERCAR_URL=http://localhost:8180/
depends_on:
- hello-sidecar-api
ports:
- "8080:8080"
You can either change SIDECAR_URL to http://hello-sidecar-api:8080/ or place both APIs in the same container. Note that I also changed the port from 8180 to 8080, because 8180 is a port mapped on the host, but inside your Docker network your API is accessible by other containers on 8080.
Your containers are separate network entities with their own IPs, so when you call http://localhost:8180 from the inside of a container, you're not calling the host, but the same container from which the request originates. (assuming you're not using host network driver).
What you are trying to do here resembles the behavior of pods in Kubernetes (where sidecar term is widely used). In Kubernetes you could put these two containers in one pod and then they could call each other on localhost
Related
I have 8 frontends apps, 12 backends servers. Frontends are Vue.js or AngularJS, backedends are ASP.NET Core 3.1, and SQL server and Redis and Other services
all services are similar configs for Docker containers, except loggings, ports and so on. they all work in same named network mynetwork
abcservice:
image: ${DOCKER_REGISTRY-}abcservice
container_name: abcServer
hostname: abcservice
build:
context: .
dockerfile: abcService/Dockerfile
networks:
- mynetwork
but I have to use http://host.docker.internal:{portnumer}, so all containers can work well together. How can I force all apps work together on http://localhost:{portnumer}?
let's say a simple ASP.NET core app, if started it WITHOUT docker, it can access SQL Server(run in docker) and Redis(run in docker) with domain http://localhost:port, but once if start it with docker, I have to access the app via domain http://host.docker.internal:port, otherwise it cannot reach SQL and Redis. because inside containers, localhost means the container itself, I need something config to let container reach other containers with localhost and specified ports.
Appreciate.
Option 1: Environment variables
You can either use ports for all services, and use environment variables in a .env file to switch between hostnames. The .env file works out-of-the-box with Docker Compose, see docs.
Using ports:
ports:
- 6379:6379
Sample .env file:
REDIS_HOST=redis
REDIS_PORT=6379
Option 2: using network_mode host
Another option is apply host network settings instead using network_mode on each service. That should apply host network settings to a service, instead of running in isolation.
network_mode: host
I have a bit of a problem with connecting the dots.
I managed to dockerized our legacy app and our newer app, but now I need to make them to talk to one another via API call.
Projects:
Project1 = using project1_appnet (bridge driver)
Project2 = using project2_appnet (bridge driver)
Project3 = using project3_appnet (bridge driver)
On my local, I have these 3 projects on 3 separates folders. Each project will have their own app, db and cache services.
This is the docker-compose.yml for one of the project. (They have nearly all the same docker-compose.yml only with different image and volume path)
version: '3'
services:
app:
build: ./docker/app
image: 'cms/app:latest'
networks:
- appnet
volumes:
- './:/var/www/html:cached'
ports:
- '${APP_PORT}:80'
working_dir: /var/www/html
cache:
image: 'redis:alpine'
networks:
- appnet
volumes:
- 'cachedata:/data'
db:
image: 'mysql:5.7'
environment:
MYSQL_ROOT_PASSWORD: '${DB_ROOT_PASSWORD}'
MYSQL_DATABASE: '${DB_DATABASE}'
MYSQL_USER: '${DB_USER}'
MYSQL_PASSWORD: '${DB_PASSWORD}'
ports:
- '${DB_PORT}:3306'
networks:
- appnet
volumes:
- 'dbdata:/var/lib/mysql'
networks:
appnet:
driver: bridge
volumes:
dbdata:
driver: local
cachedata:
driver: local
Question:
How can I make them be able to talk to one another via API call? (On my local for development and for prod environment)
On production, the setting will be a bit different, they will be in different machines but still in the same VPC or even through public network. What is the setting for that?
Note:
I have been looking at link but apparently it is deprecated for v3 or not really recommended
Tried curl from project1 container to project2 container, by doing:
root#bc3afb31a5f1:/var/www/html# curl localhost:8050/login
curl: (7) Failed to connect to localhost port 8050: Connection refused
If your final setup will be that each service will be running on a physically different system, there aren't really any choices. One system can't directly access the Docker network on another system; the only way service 1 will be able to reach service 2 is via its host's DNS name (or IP address) and the published port. Since this will be different in different environments, I'd suggest making that value a configured environment variable.
environment:
SERVICE_2_URL: 'http://service-2-host.example.com/' # default port 80
Once you've settled on that, you can use the same setup for a single-host deployment, mostly. If your developer systems use Docker for Mac or Docker for Windows you should be able to use a special Docker hostname to reach the other service
environment:
SERVICE_2_URL: 'http://host.docker.internal:8082/'
(If you use Linux on the desktop you will have to know some IP address for the host; not localhost because that means "this container", and not the docker0 interface address because that will be on a specific network, but something like the host's eth0 address.)
Your other option is to "borrow" the other Docker Compose network as an external network. There is some trickiness if all of your Docker Compose setups have the same names; from some experimentation it seems like the Docker-internal DNS will always resolve to your own Docker Compose file first, and you have to know something like the Compose-assigned container name (which isn't hard to reconstruct and is stable) to reach the other service.
version: '3'
networks:
app2:
external:
name: app2_appnet
services:
app:
networks:
- appnet
- app2_appnet
environment:
SERVICE_2_URL: 'http://app2_app_1/' # using the service-internal port
MYSQL_HOST: db # in this docker-compose.yml
(I would suggest using the Docker Compose default network over declaring your own; that will mostly let you delete all of the networks: blocks in the file without any ill effect, but in this specific case you will need to declare networks: [default, app2_default] to connect to both.)
You may also consider a multi-host container solution when you're starting to look at this. Kubernetes is kind of heavy-weight, but it will run containers on any node in the cluster (you don't specifically have to worry about placement) and it provides both namespaces and automatic DNS resolution for you; you can just set SERVICE_2_URL: 'http://app.app2/' to point at the other namespace without worrying about these networking details.
If you run this docker compose locally; given app and db are on the same network - appnet - app should be able to talk to db using localhost:${DB_PORT}.
In production, if app and db are on different machines; app would probably need to talk to database using ip or domain name.
Considering that you are using different machines for the different docker deployments you good put them behind a regular webserver (Apache2, Nginx) and then route the traffic from the specific domain to $APP_PORT using a simple vhost. I prefer to do that instead of directly exposing the container to the network. This way you would also be able to host multiple applications on the same machine ( if you like to ). So I suggest you should not try to connect docker networks but "regular " ones.
Was playing around with inspect and cURL. I think I found the solution.
Locally:
In my local, I inspected the container and view the NetworkSettings.Network.<network name>.Gateway which is 172.25.0.1
Then I get the the exposed port which is 8050
Then I did a curl inside the app1 container curl 172.25.0.1:8050/login to check whether app1 can do a http request to app2 container. OR docker exec -it project1_app_1 curl 172.25.0.1:8050/login
Vice versa, I did curl 172.25.0.1:80 for app2 -> app 1 OR docker exec -it project2_app_1 curl 172.25.0.1:80
The only issue is that, the Gateway value changes when we restart via docker-compose up -d
Production likewise:
I am not that pro with networking and stuff. My estimate for production would be:
Do curl app2-domain.com which is pointed to the app by the webserver as they are in their own machine (even with a load balancer).
To start, I am more familiar running Docker through Portainer than I am with doing it through the console.
What I'm Doing:
Currently, I'm running Mopidy through a container, which is being accessed by other machines through the default Mopidy port. In another container, I am running a Slack bot using the Limbo repo as a base. Both of them are running on Alpine Linux.
What I Need:
What I want to do is for my Slack bot to be able to call MPC commands, such as muting the volume, etc. This is where I am stuck. What is the best way for this to work
What I've tried:
I could ssh into the other container to send a command, but it doesn't make sense to do this since they're both running on the same server machine.
The best way to connect a bunch of containers is to define a service stack using docker-compose.yml file and launch all of them using docker-compose up. This way all the containers will be connected via single user-defined bridge network which will make all their ports accessible to each other without you explicitly publishing them. It will also allow the containers to discover each other by the service name via DNS-resolution.
Example of docker-compose.yml:
version: "3"
services:
service1:
image: image1
ports:
# the following only necessary to access port from host machine
- "host_port:container_port"
service2:
image: image2
In the above example any application in the service2 container can reach some port on service1 just by using service2:port address.
I created two Dockerfiles to run frontend and backend in a web application. When I run docker-compose.yaml file, web application front-end is opened of web browser. But I cannot login to the system. I think there is a problem with connecting those containers. Following is my docker-compose.yaml file. What can I do to resolve this problem ?
version: '2'
services:
gulp:
build: './ui'
ports:
- "3000:4000"
python:
build: '.'
ports:
- "5000:5000"
You need to use --links to enable communication between containers and you should use their DNS network alias like http://python:5000
Containers within a docker-compose file are part of one network by default. And one container can access other container using their host name.
Host name can be defined in docker-compose file using hostname. And if hostname is not defined, then the service name is considered the hostname.
Internally, docker containers can talk to each other by referring to each other at their hostname. Like in your case, gulp can access python at http://python:5000 and that would be possible even if you did not declare ports. This all is happening all because it is internal to the docker system.
From outside, if you want to connect to any of the services, then you can define ports, as you did and then access those services at the defined port number.
I assume that there is a way to link via one or a combination of the following: links, external_links and networking.
Any ideas? I have come up empty handed so far.
Here is an example snippet of a Docker-compose which is started from within a separate Ubuntu docker
version: '2'
services:
web:
build: .
depends_on:
- redis
redis:
image: redis
I want to be able to connect to the redis port from the Docker that launched the docker-compose.
I do not want to bind the ports on the host as it means I won't be able to start multiple docker-compose from the same model.
-- context --
I am attempting to run a docker-compose from within a Jenkins maven build Docker so that I can run tests. But I cannot for the life of me get the original Docker to access exposed ports on the docker-compose
Reference the machines by hostname, v2 automatically connects the nodes by hostname on a private network by default. You'll be able to ping "web" and "redis" from within each container. If you want to access the machines from your host, include a "ports" definition for each service in your yml.
The v1 links were removed from the v2 compose syntax since they are now implicit. From the docker compose file documentation
links with environment variables: As documented in the environment variables reference, environment variables created by links have been
deprecated for some time. In the new Docker network system, they have
been removed. You should either connect directly to the appropriate
hostname or set the relevant environment variable yourself...