docker-compose need to get properties by service name , espesely ports - docker

I try to get the ports from the services I defined in the compose_file.yml
as i need to to extract the ports docker generate me.
no option to inject them to docker .
version: '3.3'
services:
oracleTest:
image: ora_image
container_name: "test1"
ports:
- 15211:1521
mysqlTest:
image: mysql:8.0
container_name: "test2"
restart: always
ports:
- 33061:3306
what I found is that I can get the ports by container name , this is not what I need
I can get the services with :
docker-compose ps --services
also can cat the mapped port with docker command again not what I need
docker port mysql8.0 | cut -d':' -f 2
what I need is something like this pseudo code:
docker-compose -f compose_file.yml port oracleTest
or any think like this ... any idea?

If you want ports to be access explicitly anyway, then you might want to inject the port into docker-compose.yml file vie environment variables.
Say you have .env file (default for docker-compose to look up) as follow
oraclePort=15211
mysqlPort=33061
you can then edit your docker-compose.yml to accept variable as
version: '3.3'
services:
oracleTest:
image: ora_image
container_name: "test1"
ports:
-$oraclePort:1521 #<-- notice the variable
mysqlTest:
image: mysql:8.0
container_name: "test2"
restart: always
ports:
-$mysqlPort:3306 #<-- notice the variable
hope this is what you need.

Since docker-compose.yml is a regular YAML file, you can use yq as YAML shell parser.
kislyuk/yq: Command-line YAML and XML processor - jq wrapper for YAML/XML documents
Command-line YAML and XML processor - jq wrapper for YAML/XML documents https://kislyuk.github.io/yq/
install yq and jq
pip install yq
pip install jq
Parse ports from docker-compose.yml
cat docker-compose.yml | yq '.services|.[].ports'
[
"15211:1521"
]
[
"33061:3306"
]
Get ports of oracleTest
what i need is something like this pseudo code:
docker-compose -f compose_file.yml port oracleTest
To get ports of specific service, use:
cat docker-compose.yml | yq '.services.oracleTest.ports'
[
"15211:1521"
]

Related

docker-compose config invalid output when replacing env variables

I'm trying to generate multiple instances of the same git repository with a generic docker-compose.yml file and multiple .env files.
For this somewhere in the code I generate a temporary folder which contains:
.env:
APP_PORT="3000"
APP_NAME="app-name"
REPO_NAME="repo-name"
docker-compose.yml
version: '3.6'
services:
web-app:
image: golang:alpine
environment:
- APP_PORT
- APP_NAME
- REDIS_HOST=db-app
ports:
- ${APP_PORT}:${APP_PORT}
volumes:
- /opt/docker/repositories/${REPO_NAME}:/app
command: sh -c "cd /app && go run ./"
db-app:
image: redis:alpine
then running docker-compose config in this directory gives me the following output :
services:
db-app:
image: redis:alpine
web-app:
command: sh -c "cd /app && go run ./"
environment:
APP_NAME: app-name
APP_PORT: '3000'
REDIS_HOST: db-app
image: golang:alpine
ports:
- published: 3000
target: 3000
volumes:
- /opt/docker/repositories/repo-name:/app:rw
version: '3.6'
This did not only interpolate env variables, it also changed some fields such as ports with published and target, and a :rw at the end of my volume.
This is all done in Go, and when I try to unmarshal the output into a Go struct with yaml fields, it is not recognized as a valid docker-compose file because of the ports field (which is supposed to be an array of strings).
How can I make it so docker-compose config only replaces the ${APP_PORT} with its value and not add these extra unwanted fields ?
Reading the source code, I found this in the config types:
def legacy_repr(self):
return normalize_port_dict(self.repr())
Which is the representation you need. So I searched for legacy_repr in the source code and found this:
if 'ports' in service_dict:
service_dict['ports'] = [
p.legacy_repr() if p.external_ip or version < VERSION else p
for p in service_dict['ports']
]
So apparently, to trigger the use of the legacy representation, you either need to have an external IP address or need to do something with the version. I tried to downgrade the docker-compose.yaml file version but it didn't change anything (maybe it's the docker-compose CLI's version instead).
Reading the spec of the docker-compose config file, in the ports section, you can specify the IP address in the short syntax:
[HOST:]CONTAINER[/PROTOCOL] where:
HOST is [IP:](port | range)
CONTAINER is port | range
PROTOCOL to restrict port to specified protocol. tcp and udp values are defined by the specification, Compose implementations MAY offer support for platform-specific protocol names.
So a solution is to replace ${APP_PORT}:${APP_PORT} by:
0.0.0.0:${APP_PORT}:${APP_PORT}
By setting the external IP address to 0.0.0.0 you are not restricting anything and you force the use of the legacy representation.

Reference Dockerfile from docker-compose.yml?

I have a web service that uses a MySQL as its backing store. I want to Dockerize this service as well as its MySQL DB. For the service I have the following Dockerfile:
FROM openjdk:8-jdk-alpine as cce
COPY build/libs/my-service.jar my-service.jar
EXPOSE 9200
ENTRYPOINT [ \
"java", \
"-Ddb.hostAndPort=my-service-db:3306", \
"-Ddb.name=my_service_db_local", \
"-Ddb.username=my-service-user", \
"-Ddb.password=abc123", \
"-jar", \
"my-service.jar" \
]
If I understand the Docker ecosystem correctly, it sounds like I can write a Docker Compose file to spin up the MySQL container instance as well as the web service container instance. So I have a docker-compose.yml file started that looks like so:
version: "3.7"
services:
my-service-db:
image: mysql:8
container_name: my-service-db
command: --default-authentication-plugin=mysql_native_password
restart: always
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: r00tdud3
MYSQL_DATABASE: my_service_db_local
MYSQL_USER: my-service-user
MYSQL_PASSWORD: abc123
volumes:
- ./my-service-db-data:/var/lib/mysql
my-service:
??? how to specify local Dockerfile here ???
depends_on:
- my-service-db
From my own tinkering I'm confident the MySQL container is configured the way I want it and correctly. Now I'm just trying to figure out how to tell docker-compose.yml file (which will live in the root directory of my project, right alongside the Dockerfile) to use the service's Dockerfile, and that it depends on the MySQL container first being started/running. I think I'm close but I'm having a hard time crossing the finish line.
Can anyone help me configure docker-compose.yml to create a service called my-service that uses the local Dockerfile for its config?
You can create an image from the dockerfile:
docker build - < Dockerfile
Then you should tag you image with a proper name.
After creating the image reference it in the docker-compose.yml file:
my-service:
image: ${image_name}
Another option is to simply write:
my-service:
build: .

Hosting docker swarm from one docker-compose but with nodes var

I'm looking to be able to run swarm from same docker-compose file which uses env variables. Currently I only achieved that all nodes are replicating Leaders env. Is it possible to let each node start from its own local env var?
My docker-compose
version: '3.1'
networks:
base:
services:
test:
container_name: ${Name}
restart: always
image: ubuntu:latest
environment:
- Name=${Name}
command: sh -c "echo $Name && sleep 30"
networks:
- base
use env_file option
https://docs.docker.com/compose/environment-variables/
# .env, file
Name=<your_name>
# <your_name>.env, file
TEST_ENV=stackoverflow
# docker-compose.yaml, file
version: '3.1'
services:
test:
container_name: ${Name}
restart: always
image: ubuntu:latest
env_file:
- ${Name}.env
command: sh -c "set | grep TEST_ENV && sleep 30"
docker logs <your_name>
# TEST_ENV='stackoverflow'
You can set env_files with different names in different containers.
for example
# docker-compose.yaml, file
version: '3.1'
services:
test1:
container_name: test1
restart: always
image: ubuntu:latest
env_file:
- first.env
command: sh -c "set | grep TEST_FIRST_ENV && sleep 30"
test2:
container_name: test2
restart: always
image: ubuntu:latest
env_file:
- second.env
command: sh -c "set | grep TEST_SECOND_ENV && sleep 30"
Environment variables referenced in the docker-compose.yml file are not resolved on the leader even, they are resolved on whatever jump box you are deploying too the swarm from.
If you want to reference the env vars from the host system, from the command, or entrypoint, iirc you can escape the reference to "$$Name", but this will only make the env variable available to the entrypoint or command script which are evaluated on the host, not to values like the container_name.
Given your specific use case, perhaps service creation templates are what you are looking for: They let you inject per service instance values into hostname, mount and env.
version: '3.8'
services:
test:
env:
MY_HOSTNAME: "{{.Node.Hostname}}"
...
See Create Service Using Templates for the full list of supported values.

Change Swagger port in docker-compose

I am unable to change the port that Swagger uses in docker compose. It works fine with regular docker, I simply set the -p argument on the run command. It seems that I should just need to set the ports field in the docker-compose file. But no matter what I try it just runs on 8080.
I am using the latest versions of docker and docker-compose. The docker image is called swaggerapi/swagger-ui. I have attempted setting the ports field for the container. Also tried setting the url variable in the swagger definition file. Tried changing the expose port. I tried with the docker-compose run command which lets you start an individual service and has the -p argument. Still nothing.
Ideally I should use this to build and run:
sudo docker-compose up --build --force-recreate
My compose file:
version: '3'
services:
swagger:
build: swagger
network_mode: "host"
ports:
- "8081:8080"
env_file: .env
environment:
- SWAGGER_JSON=/swagger.json
volumes:
data:
driver: "local"
And the docker file for the swagger service:
FROM swaggerapi/swagger-ui
EXPOSE 8081
COPY swagger.json /swagger.json
ENV SWAGGER_JSON "/swagger.json"
No matter what I do it wont change ports.
Just change the port in your docker-compose file
swagger:
build: swagger
network_mode: "host"
ports:
- "8081:"**Port which you want to expose**"
env_file: .env
environment:
- SWAGGER_JSON=/swagger.json

Docker-compose setup script to set environment variables from within container

I have a docker-compose file that will startup 6 different microservices. The way our docker-repository is setup prevents the use of the 'latest' tag so I am looking for a way to run a script before docker-compose pulls the microservice images, which will set environment variables in the scope of the docker-compose.yml file.
version: '3'
services:
#Service 1
svc1:
image: some-snapshot.docker.privaterepo.com/some-service:${LATEST_SVC_TAG}
container_name: service1
ports:
- "8080:8080"
#Service 2
svc2:
image: some-snapshot.docker.privaterepo.com/some-service2:${LATEST_SVC2_TAG}
container_name: service2
ports:
- "8081:8081"
I'm not sure you really need "a script". You can just run something like:
LATEST_SVC_TAG=1.1 LATEST_SVC2_TAG=2.5 docker-compose up -d
Alternately, you could place those into a .env file locally:
cat > .env <<EOF
LATEST_SVC_TAG=1.1
LATEST_SVC2_TAG=2.3
EOF
docker-compose up -d

Resources