I have running a DDEV-Environment for Magento2, locally on my Mac OSX (Ventura)
https://ddev.readthedocs.io/en/stable/users/quickstart/#magento-2
For testing purpose I included Nifi per docker-compose.yaml inside my ddev project .ddev/docker-compose.nifi.yaml
Below you can see the docker-compose, which is really minimal at this point. Nifi works like expected, because I can login etc, although it is not persistent yet, but thats a different problem
version: '3'
services:
nifi:
image: apache/nifi:latest
container_name: ddev-${DDEV_SITENAME}-nifi
ports:
# HTTP
- "8080:8080"
# HTTPS
- "8443:8443"
volumes:
# - ./nifi/database_repository:/opt/nifi/nifi-current/database_repository
# - ./nifi/flowfile_repository:/opt/nifi/nifi-current/flowfile_repository
# - ./nifi/content_repository:/opt/nifi/nifi-current/content_repository
# - ./nifi/provenance_repository:/opt/nifi/nifi-current/provenance_repository
# - ./nifi/state:/opt/nifi/nifi-current/state
# - ./nifi/logs:/opt/nifi/nifi-current/logs
# - ./nifi/conf/login-identity-providers.xml:/opt/nifi/nifi-current/conf/login-identity-providers.xml
- ".:/mnt/ddev_config"
All I want to do is sending a POST-requst from Nifi to my Magento2 module.
I tried several IPs now, which I got from docker inspect ddev-ddev-magento2-web but I always receive "Connection refused"
My output from docker network ls:
NETWORK ID NAME DRIVER SCOPE
95bea4031396 bridge bridge local
692b58ca294e ddev-ddev-magento2_default bridge local
46be47991abe ddev_default bridge local
7e19ae1626f1 host host local
f8f4f1aeef04 nifi_docker_default bridge local
dbdba30546d7 nifi_docker_mynetwork bridge local
ca12e667b773 none null local
My Magento2-Module is working properly, because sending requests from Postmanto it works fine
You don't want most of what you have. Please remove the ports statement, which you shouldn't need at all; if you need anything, you'll need an expose. But I doubt you need that in this case?
You'll want to look at the docs:
Additional services and add-ons
Additional services with docker-compose
Then create a .ddev/docker-compose.nifi.yaml with something like
services:
nifi:
image: apache/nifi:latest
container_name: ddev-${DDEV_SITENAME}-nifi
container_name: "ddev-${DDEV_SITENAME}-someservice"
labels:
com.ddev.site-name: ${DDEV_SITENAME}
com.ddev.approot: ${DDEV_APPROOT}
expose:
- "8080"
environment:
- VIRTUAL_HOST=$DDEV_HOSTNAME
- HTTP_EXPOSE=8080:8080
- HTTPS_EXPOSE=9999:8080
volumes:
- ".:/mnt/ddev_config"
The name of the "web" container from inside your nifi container will be "web", curl http://web:8080, assuming that you have nifi on port 8080.
I don't know what you're trying to accomplish, but this may get you started. Feel free to come over to the DDEV Discord channel for more interactive help.
Related
how can we run docker commands inside container with docker-compose?
Simply I want to get IP of some other network container.
I am running three container va-server, db and api-server. All the containers are in same docker-network
Here I am providing docker-compose file below:
version: "2.3"
services:
va-server:
container_name: va_server
image: nitinroxx/facesense:amd64_2022.11.28 #facesense:alpha
runtime: nvidia
restart: always
mem_limit: 4G
networks:
- perimeter-network
db:
container_name: mongodb
image: mongo:latest
ports:
- "27017:27017"
restart: always
volumes:
- ./facesense_db:/data/db
command: [--auth]
networks:
- perimeter-network
api-server:
container_name: api_server
image: nitinroxx/facesense:api_amd64_2022.11.28
ports:
- "80:80"
- "465:465"
restart: always
networks:
- perimeter-network
networks:
perimeter-network:
driver: bridge
ipam:
config:
- gateway: 10.16.239.1
subnet: 10.16.239.0/24
I have install docker inside the container which giving me below permission error:
docker.errors.dockerexception: error while fetching server api version: ('connection aborted.', permissionerror(13, 'permission denied')
...inside [a] container [...] I want to get IP of some other network container....
Docker provides an internal DNS service that can resolve container names to their Docker-internal IP addresses. From one of the containers you show, you could look up a host name like db to get the container's IP address; but in practice, this is a totally normal DNS name and all but the lowest-level networking interfaces can use those directly.
This does require that all of the containers involved be on the same Docker network. Normally Compose sets this up automatically for you; in the file you show I might delete the networks: blocks and container_name: overrides in the name of simplicity. Also see Networking in Compose in the Docker documentation.
In short:
You can probably use the Compose service names va-server, db, and api-server as host names without specifically knowing their IP addresses.
This probably means you never need to know the container IP addresses at all (they're usually unusable from outside Docker).
If you do need an IP address from inside a container, a DNS lookup can find it.
You can't usually run docker commands from inside containers. You can't do this safely without making it possible for the container to take over the whole host. There are usually better patterns that don't tie you to the Docker stack specifically.
I'm trying to dockerize my angular + express application. I have a docker-compose file that creates the two containers, and I am able to hit the containers from my host machine(using my browser), but I'll just get a "ERR_NAME_NOT_RESOLVED" whenever I try to hit the backend from http requests made by my frontend.
I've looked up the issue, and it seems like most suggest that the service name and container port should be enough to hit the the other container when they're on the same network. I've tried to hit "http://express:8000/user?user=030f0e70-9a8f-11e9-b5d1-f5cb6c0f3616" which I think should work given what I've seen from other places, but regardless, I get the same error.
My docker-compose file looks like
version: '3' # specify docker-compose version
# Define the services/containers to be run
services:
angular: # name of the first service
build: ./ # specify the directory of the Dockerfile
ports:
- "4200:80" # specify port forewarding
links:
- "express"
depends_on:
- "express"
express: #name of the second service
build: # specify the directory of the Dockerfile
context: ./
dockerfile: dockerfile.be
ports:
- "8000:8000" #specify ports forewarding
expose:
- "8000"
Ideally, I'd like my frontend to be able to hit the other container with a set endpoint, so I could deploy the application with minimal changes. I'd appreciate any advice. I feel like I'm missing something really simple, but after a few hours of tinkering, I still haven't caught it.
Thanks!
In fact your traffic is as next:
User browser request page from angular container, then all pages will rendered to user's browser.
The front javascript code using angular HttpClient to fetch the data from express container.
At that time, although docker-compose setup a customized network for you which afford auto-dns to resolve angular & express, but this dns just works among containers, not effect for host.
But, your augular HttpClient which call http://express was happened on user's browser which not in container, so the auto-dns defintly not work for you to resolve the name express.
For you, if you just want to open browser from your docker host, you can use localhost, but if you also want the user from other machine to visit your web, you had to use the ip of your dockerhost.
Then, in angular HttpClient you need to use something like http://your_dockerhost_ip:8000 to visit express container.
If interested, you can visit this to see User-defined bridges provide automatic DNS resolution between containers.
A few things:
By using expose, you are making the container's published ports only available to linked/networked services. This is one reason why you are unable to access it locally.
Instead of hitting http://express:8000/ you should try to hit http://localhost:8000. The service is being published to your localhost system and is not being served by anything by default (e.g., IIS, NGINX).
Add a custom defined network in your compose file instead of using links. This is now the main way to network containers together:
version: '3' # specify docker-compose version
services:
angular: # name of the first service
build: ./ # specify the directory of the Dockerfile
ports:
- "4200:80" # maps port 4200 on localhost to 80 in container
network:
- mynetwork
depends_on:
- "express"
express: # name of the second service
build: # specify the directory of the Dockerfile
context: ./
dockerfile: dockerfile.be
ports:
- "8000:8000" # maps port 8000 on localhost to 8000 in container
networks:
- mynetwork
networks:
mynetwork:
2: https://docs.docker.com/compose/compose-file/#expose
**sample docker-compose.yaml**
version: '3.5'
services:
angular:
image: "angular-alpine:0.0.1"
container_name: angular
tty: true
stdin_open: true
networks:
app_net:
ipv4_address: 172.16.238.05
depends_on:
- express
express:
image: "express:0.0.1"
container_name: express
tty: true
stdin_open: true
networks:
app_net:
ipv4_address: 172.16.238.10
networks:
app_net:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.16.238.0/24
>"1. Angular will run on 172.16.238.05:4200 and express will run on 172.16.238.10:some-port.
2. modify your config.ts or parameter.ts or anyfile where you configure express url into 172.16.238.10:some-port. now your angular will connect to express."
Here is my v3.5 docker-compose.yml definition file. It has an analytics network (using an alias of the same name), and where both included services connect to said network to communicate with one another. This works.
However, I want these services (ports) exposed to the HOST machine, as well. There's a way to do that by defining an additional network and/or specifying additional ports: entries within the services themselves, but I can't figure out exactly how because the documentation is very confusing and version-specific (moving targets).
Without destroying the below (because it works internally), what additions do I make (and where) to expose both services to the HOST machine as well?
Thank you!
version: '3.5'
networks:
analytics:
name: analytics
driver: bridge
# ===========================================
# Service: Zookeeper
# ===========================================
zookeeper:
image: 'wurstmeister/zookeeper:latest'
container_name: analytics-ZooKeeper
networks:
- analytics
ports:
- "2181:2181"
volumes:
- ./data.d/zookeeper.d:/opt/zookeeper-3.4.9/data
# ===========================================
# ===========================================
# Service: Kafka
# ===========================================
kafka:
build:
context: ./kafka.d
dockerfile: Dockerfile
image: nmvega/kafka:latest
networks:
- analytics
ports:
- 9092-9094:9092 # For one to three Kafka brokers.
environment:
#KAFKA_ADVERTISED_HOST_NAME: vps00 # Docker host Name. <--- BEFORE
KAFKA_ADVERTISED_HOST_NAME: 192.168.0.180 # Docker host IP. <--- AFTER
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./data.d/kafka.d:/kafka
depends_on:
- zookeeper
# ===========================================
EDIT:
Upon further investigation, the above configuration, as originally posted, is correct with the small modification from the name of the Docker Host to the IP of the Docker host (as prescribed by the readme for the image that I'm using). Accidentally using the name didn't matter until I attempted to access the service from the Host.
Hopefully this example will be valuable to others wanting to see one.
Thank you to the commenters below.
I'd like my web Docker container to access Redis on 127.0.0.1:6379 from within the web container. I've setup my Docker Compose file as the following. I get ECONNREFUSED though:
version: "3"
services:
web:
build: .
ports:
- 8080:8080
command: ["test"]
links:
- redis:127.0.0.1
redis:
image: redis:alpine
ports:
- 6379
Any ideas?
The short answer to this is "don't". Docker containers each get their own loopback interface, 127.0.0.1, that is separate from the host loopback and from that of other containers. You can't redefine 127.0.0.1, and if you could, that would almost certainly break other things.
There is a technically possible way to do it by either running all containers directly on the host, with:
network_mode: "host"
However, that removes the docker network isolation that you'll want with containers.
You can also attach one container to the network of another container (so they have the same loopback interface) with:
docker run --net container:$container_id ...
but I'm not sure if there's a syntax to do this in docker-compose and it's not available in swarm mode since containers may run on different nodes. The main use I've had for this syntax is attach network debugging tools like nicolaka/netshoot.
What you should do instead is make the location of the redis database a configuration parameter to your webapp container. Pass the location in as an environment variable, config file, or command line parameter. If the web app can't support this directly, update the configuration with an entrypoint script that runs before you start your web app. This would change your compose yml file to look like:
version: "3"
services:
web:
# you should include an image name
image: your_webapp_image_name
build: .
ports:
- 8080:8080
command: ["test"]
environment:
- REDIS_URL=redis:6379
# no need to link, it's deprecated, use dns and the network docker creates
#links:
# - redis:127.0.0.1
redis:
image: redis:alpine
# no need to publish the port if you don't need external access
#ports:
# - 6379
I'm running a mongo instance with docker-compose and traefik.
myapp-mongo:
build: ../images/myapp-mongo
restart: always
ports:
- "27017:27017"
labels:
- "traefik.ports=27017,27018"
- "traefik.backend=myapp-mongo"
- "traefik.frontend.rule=Host:myapp-mongo.docker.localhost"
networks:
- development
environment:
- MONGO_USER=${MONGO_USER}
- MONGO_PASSWD=${MONGO_PASSWD}
- MONGO_AUTHDB=${MONGO_AUTHDB}
Mongo is running fine and I can connect using 127.0.0.1 from my Mac.
The problem is that I can't connect using hostname myapp-mongo.docker.localhost. It only works using IP 127.0.0.1.
Trying to ping the IP 127.0.0.1 responds ok, but trying to ping the hostname doesn't work.
I've already added 127.0.0.1 proxy.docker.localhost into /etc/hosts to get traefik working.
All other web apps has hostnames working fine like eg myapp.docker.localhost. This problem is only happening with this mongodb container.
Probably because Træfik is HTTP proxy and so will only support HTTP/HTTPS connections.
I believe #bpatel is right (see comment I left on his answer with link to github conversation) Traefik at the time of writing only supports HTTP/HTTPS.
Solution using native docker networks
However, you can get around this issue! Since you are using docker, you can work around by using the container name in your code (assuming mongo and your mongo accessing code are both running in containers on a shared docker network. This will be the case if the containers are spun up with docker-compose). Run the following to see if your containers are linked up correctly:
run docker ps to get your container names running (under the NAMES column)
run docker network ls to see your network names
run docker network inspect <target_network_name> to verify your containers from step 1 are on the same network.
I run docker-compose from three separate compose files, so you should be able to cover most cases from the following (apologies for any syntax errors, the following are stripped down code examples):
Entire docker-compose file that that starts up traefik (under directory name 'proxy')
version: '2'
services:
traefik:
image: traefik
command: --web --docker --docker.domain=docker.localhost --logLevel=DEBUG
networks:
- webgateway
ports:
- "80:80"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /dev/null:/traefik.toml
networks:
webgateway:
driver: bridge
snippet from my docker-compose file that spins up mongo
version: '2'
services:
database:
image: mongo
ports:
- "27017:27017"
networks:
- web
networks:
web:
external:
name: proxy_webgateway
snippet from docker-compose that has mongo accessing code
version: '2'
services:
topicOntologyBuilder:
image: topic-ontology-builder
labels:
- "traefik.backend=topicOntologyBuilder"
- "traefik.port=80"
- "traefik.frontend.rule=Host:topic-ontology.docker.localhost"
networks:
- web
volumes:
- ./:/home
networks:
web:
external:
name: proxy_webgateway
Connection in Code
Not certain what language you're using, this is what the following js code looked like for me to connect to mongo (inside that 'topicOntologyBuilder` container, while using traefik as the proxy (again, this works because we're making the most of docker networks):
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect('mongodb://<MONGO_CONTAINER_NAME>/<DB_NAME>', function(err, db) {
//insert code here to interact with mongo
})
Why this works
This works because docker does some clever DNS stuff within the containers so that each container knows the IP of other containers, by looking it up in their DNS entry, by the container names
Extra intel
If your containers are on separate computers/vm's, you'll probably want to play around with a service discovery tool (Consul plays well with Traefik) or do something fancy with a docker network overlay which is specific for containers in a cluster.
If using raw docker networks, you can assign container aliases (this doesn't work with Traefik though, or at least it didn't a couple months back).