I am trying to connect to datadog from my fastapi backend. I am currently trying to do this on localhost using a docker-compose file to let both my datadog-agent and my backend-container run in the same network.
Here is a minimal example
dd-minimal
- docker-compose.yml
- backend-client
- Dockerfile
- app
- main.py
docker-compose.yml
version: "3.7"
networks:
my_network:
services:
datadog:
image: datadog/agent:latest
environment:
DD_API_KEY: <my-api-key>
DD_APM_ENABLED: 'true'
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /proc/:/host/proc/:ro
- /sys/fs/cgroup:/host/sys/fs/cgroup:ro
ports:
- "8126:8126/tcp"
networks:
- my_network
backend-web-client:
image: gql-backend-api
build:
dockerfile: Dockerfile
context: ./backend-client
environment:
DD_TRACE_ANALYTICS_ENABLED: 'true'
DD_AGENT_HOST: 172.21.0.2
ports:
- "5555:8080"
networks:
- my_network
depends_on:
- datadog
Dockerfile
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8-slim
COPY ./app /app
RUN pip install ddtrace==0.41.0
CMD exec ddtrace-run gunicorn --bind :8080 --workers 1 --threads 8 --timeout 0 main:api -k uvicorn.workers.UvicornWorker
main.py
import os
import uvicorn
from fastapi import FastAPI
api = FastAPI()
if __name__ == "__main__":
uvicorn.run(api, host="127.0.0.1", port=int(os.environ.get("PORT", 8080)))
I run docker-compose up and then check the ip of my dd-container with
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' dd-minimal_datadog_1
and update it in the compose file.
When I then again run docker-compose up, I get the following error
- DATADOG TRACER DIAGNOSTIC - Agent not reachable. Exception raised: [Errno 111] Connection refused.
Any help would be very appreciated
You may need to set the environment variable DD_APM_NON_LOCAL_TRAFFIC=true in your datadog agent container.
Ref: https://docs.datadoghq.com/agent/docker/apm/?tab=linux#docker-apm-agent-environment-variables
Related
I have 2 application run with a different network and it uses separate docker-compose.yaml. So I trying to call an request from app A to app B, but it not works.
docker exec -it app_a_running curl http://localhost:8012/user/1
So I got an error
cURL error 7: Failed to connect to localhost port 8012
docker-compose-app-a.yaml
version: "3"
services:
app:
build: go/
restart: always
ports:
- 8011:8011
volumes:
- ../src/app:/go/src/app
working_dir: /go/src/app
container_name: app-a
command: sleep 72000
networks:
- app-a-network
networks:
app-a-network:
docker-compose-app-b.yaml
version: "3"
services:
app:
build: go/
restart: always
ports:
- 8012:8012
volumes:
- ../src/app:/go/src/app
working_dir: /go/src/app
container_name: app-b
command: sleep 72000
networks:
- app-b-network
networks:
app-b-network:
Questions:
Is it possible to do this?
If the first question is possible, Please suggest me :)
You can use curl on docker containers. The reason why your curl command didn't work is probably that you did not publish your docker container's port. For example, try:
docker run -d -p 8080:8080 tomcat
instead of
docker run -d tomcat
This will forward the port 8080 of your machine to the port 8080 of your container.
If you have a shell to your container, you can use the service name or the container's name to curl a container on your Docker network, provided your target exists with the same network.
I've been trying to figure out why I cannot containers using "localhost:3000" from host. I've tried installing Docker via Homebrew, as well as the Docker for Mac installer. I believe I have the docker-compose file configured correctly.
Here is the output from docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------------------------------------------
ecm-datacontroller_db_1 docker-entrypoint.sh postgres Up 0.0.0.0:5432->5432/tcp
ecm-datacontroller_kafka_1 supervisord -n Up 0.0.0.0:2181->2181/tcp, 0.0.0.0:9092->9092/tcp
ecm-datacontroller_redis_1 docker-entrypoint.sh redis ... Up 0.0.0.0:6379->6379/tcp
ecm-datacontroller_web_1 npm start Up 0.0.0.0:3000->3000/tcp
Here is my docker-compose.yml
version: '2'
services:
web:
ports:
- "3000:3000"
build: .
command: npm start
env_file: .env
depends_on:
- db
- redis
- kafka
volumes:
- .:/app/user
db:
image: postgres:latest
ports:
- "5432:5432"
redis:
image: redis:alpine
ports:
- "6379:6379"
kafka:
image: heroku/kafka
ports:
- "2181:2181"
- "9092:9092"
I cannot access any ports that are exposed by docker-compose with curl localhost:3000 I get the following result from that
curl: (52) Empty reply from server
I should be getting {"hello":"world"}.
Dockerfile:
FROM heroku/heroku:16-build
# Which version of node?
ENV NODE_ENGINE 10.15.0
# Locate our binaries
ENV PATH /app/heroku/node/bin/:/app/user/node_modules/.bin:$PATH
# Create some needed directories
RUN mkdir -p /app/heroku/node /app/.profile.d
WORKDIR /app/user
# Install node
RUN curl -s https://s3pository.heroku.com/node/v$NODE_ENGINE/node-v$NODE_ENGINE-linux-x64.tar.gz | tar --strip-components=1 -xz -C /app/heroku/node
# Export the node path in .profile.d
RUN echo "export PATH=\"/app/heroku/node/bin:/app/user/node_modules/.bin:\$PATH\"" > /app/.profile.d/nodejs.sh
ADD package.json /app/user/
RUN /app/heroku/node/bin/npm install
ADD . /app/user/
EXPOSE 3000
Anyone have any ideas?
Ultimately, I ended up having a service that was listening on 127.0.0.1 instead of 0.0.0.0. Updating this resolved the connectivity issue I was having.
I've been using localstack to develop a service against locally. I've just been running their docker image via docker run --rm -p 4567-4583:4567-4583 -p 8080:8080 localstack/localstack
And then I manually run a small script to set up my S3 buckets, SQS queues, etc.
Now, I'd like to make this easier for others so I thought I'd just add a Dockerfile and docker-compose.yml file. Unfortunately, when I try to get this up and running, using docker-compose up I get an error that the command from my setup script can't connect to the localstack services.
make_bucket failed: s3://localbucket Could not connect to the endpoint URL: "http://localhost:4572/localbucket"
Dockerfile:
FROM localstack/localstack
#since this is just local dev set up, localstack doesn't require
anything specific here.
ENV AWS_DEFAULT_REGION='[useast1]'
ENV AWS_ACCESS_KEY_ID='[lloyd]'
ENV AWS_SECRET_ACCESS_KEY='[christmas]'
COPY bin/localSetup.sh /localSetup.sh
COPY fixtures/notifications.json /notifications.json
RUN ["chmod", "+x", "/localSetup.sh"]
RUN pip install awscli
# expose service & web dashboard ports
EXPOSE 4567-4582 8080
ENTRYPOINT ["/localSetup.sh"]
docker-compose.yml
version: '3'
services:
localstack:
build: .
ports:
- "8080:8080"
- "4567-4582:4567-4582"
localSetup.sh
#!/bin/bash
aws --endpoint-url=http://localhost:4572 s3 mb s3://localbucket
#additional similar calls but left off for brevity
I've tried switching localhost to 127.0.0.1 in my script commands, but I wind up with the same error. I'm probably missing something silly here.
There is another way to create your custom AWS resources when localstack freshly starts up. Since you already have a bash script for your resources, you can simply volume mount your script to /docker-entrypoint-initaws.d/.
So my docker-compose file would be:
localstack:
image: localstack/localstack:latest
container_name: localstack_aws
ports:
- '4566:4566'
volumes:
- './localSetup.sh:/etc/localstack/init/ready.d/init-aws.sh'
Also, I would prefer awslocal over aws --endpoint in the bash script, as it leverages the credentials work and endpoint for you.
try adding hostname to the docker-compose file and editing your entrypoint file to reflect that hostname.
docker-compose.yml
version: '3'
services:
localstack:
build: .
hostname: localstack
ports:
- "8080:8080"
- "4567-4582:4567-4582"
localSetup.sh
#!/bin/bash
aws --endpoint-url=http://localstack:4572 s3 mb s3://localbucket
This was my docker-compose-dev.yaml I used for testing out an app that was using localstack. I used the command docker-compose -f docker-compose-dev.yaml up, I also used the same localSetup.sh you used.
version: '3'
services:
localstack:
image: localstack/localstack
hostname: localstack
ports:
- "4567-4584:4567-4584"
- "${PORT_WEB_UI-8082}:${PORT_WEB_UI-8082}"
environment:
- SERVICES=s3
- DEBUG=1
- DATA_DIR=${DATA_DIR- }
- PORT_WEB_UI=${PORT_WEB_UI- }
- DOCKER_HOST=unix:///var/run/docker.sock
volumes:
- "${TMPDIR:-/tmp/localstack}:/tmp/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
networks:
- backend
sample-app:
image: "sample-app/sample-app:latest"
networks:
- backend
links:
- localstack
depends_on:
- "localstack"
networks:
backend:
driver: 'bridge'
First, I downloaded the drone image:
go get github.com/drone/drone/cmd/...
Second, I built the image for arm:
GOARM=7 go build -o release/drone-server github.com/drone/drone/cmd/drone-server
After that, I built the image for docker:
docker -f ./go-workspace/src/github.com/drone/drone/Dockerfile build -t drone/drone .
The docker file looks like so:
# docker build --rm -t drone/drone .
FROM drone/ca-certs
EXPOSE 8000 9000 80 443
ENV DATABASE_DRIVER=sqlite3
ENV DATABASE_CONFIG=/var/lib/drone/drone.sqlite
ENV GODEBUG=netdns=go
ENV XDG_CACHE_HOME /var/lib/drone
ADD release/drone-server /bin/
ENTRYPOINT ["/bin/drone-server"]
That's my docker-compose.yml:
version: '2'
services:
drone-server:
image: drone/drone:latest
ports:
- 8000:8000
- 9000:9000
volumes:
- /var/lib/drone:/var/lib/drone
restart: always
env_file:
- /etc/drone/server.env
drone-agent:
image: drone/agent:linux-arm
command: agent
depends_on:
- drone-server
volumes:
- /var/run/docker.sock:/var/run/docker.sock
restart: always
env_file:
- /etc/drone/agent.env
The agent.env file:
DRONE_SECRET=xxx
DRONE_SERVER=[server-hostname]:9000
DOCKER_ARCH=arm
The server.env file:
# Service settings
DRONE_SECRET=xxx
DRONE_HOST=https://[server-hostname]/drone
DRONE_OPEN=false
DRONE_GOGS=true
DRONE_GOGS_URL=https://[server-hostname]/git
DRONE_GOGS_PRIVATE_MODE=true
However, when running docker-compose -f /etc/drone/docker-compose.yml up, I get the following error:
drone-server_1 | standard_init_linux.go:190: exec user process caused "no such file or directory"
And the drone-server exits with code 1.
I configured Apache to reach drone trough a proxy as described here: http://readme.drone.io/0.5/install/setup/apache/
Any help is appreciated.
docker-compose.yml
version: '2'
services:
redis:
image: redis
ports:
- "6379"
myapp:
build: ./myapp
ports:
- "80:80"
links:
- redis
depends_on:
- redis
Dockerfile of myapp
FROM <BASE_IMAGE>
EXPOSE 80
CMD ["curl", "http://redis"]
When I run docker-compose up -d --build and then docker run prefix_myapp_1, I'm getting the 'Host is not resolved' error after 4-5 seconds.
If I change the Dockerfile, say, as follows:
FROM <BASE_IMAGE>
EXPOSE 80
CMD sleep 1000000000
then do docker exec -it prefix_myapp_1 bash, and try running curl http://redis, the host is resolved successfully.
Where the problem lies?