I am trying to launch Airflow with Swarm to orchestrate dockers. But as a result I have my running docker always on the same node ( in any circumstances). Could you give me suggestions please? I added Placement parameter, but still not successful. My code
t1 = DockerSwarmOperator(
api_version='auto',
docker_url='unix://var/run/docker.sock', # Set your docker URL
mounts=[
Mount(source="/mnt/gfd/airflow/data", target="/home/SCIP", type="bind"),
],
command=[
"/bin/bash",
"-c",
"/bin/pwd > pwd.file",
],
image='server:45000/scipminizinc:latest',
auto_remove=True,
task_id='dockerSwarmTaskSolver',
enable_logging=False,
pool="solvers",
pool_slots=2,
dag=dag,
)
Related
All my team members use the same server as docker remote context. I have set up a project using VSCode-Devcontainer with a devcontainer.json like this:
{
"name": "MyProject - DevContainer",
"dockerFile": "../Dockerfile",
"context": "..",
"workspaceMount": "source=vsc-myprojekt-${localEnv:USERNAME},target=/workspace,type=volume",
"workspaceFolder": "/workspace",
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance"
],
"postCreateCommand": "/opt/entrypoint.sh",
"mounts": [
"source=/media/Pool/,target=/Pool,type=bind",
"source=cache,target=/cache,type=volume"
]
}
This worked fine for me, but now as my colleges start their devcontainers, we have the problem, that a newly started devcontainer kill other already running devcontainers.
We found that the local folder of the projekt seems to by the way to identify already running devcontainers:
[3216 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=d:\develop\myproject
[3839 ms] Start: Run: docker inspect --type container 8ca7d3a44662
[4469 ms] Start: Removing Existing Container
As we all use the same path this identification based on the local folder is problematic. Is there a way to use other labels?
Seems to be a bug, because the issue I opened, was accepted as a bug report.
I am deploy the xxl-job application in Kubernetes(v1.15.2), now the application deploy success but registry client service failed.If deploy it in docker, it should look like this:
docker run -e PARAMS="--spring.datasource.url=jdbc:mysql://mysql-service.example.com/xxl-job?Unicode=true&characterEncoding=UTF-8 --spring.datasource.username=root --spring.datasource.password=<mysql-password>" -p 8180:8080 -v /tmp:/data/applogs --name xxl-job-admin -d xuxueli/xxl-job-admin:2.0.2
and when start application,the server side give me tips:
22:33:21.078 logback [http-nio-8080-exec-7] WARN o.s.web.servlet.PageNotFound - No mapping found for HTTP request with URI [/xxl-job-admin/api/registry] in DispatcherServlet with name 'dispatcherServlet'
I am searching from project issue and find the problem may be I could not pass the project name in docker to be part of it's url, so give me this tips.The client side give this error:
23:19:18.262 logback [xxl-job, executor ExecutorRegistryThread] INFO c.x.j.c.t.ExecutorRegistryThread - >>>>>>>>>>> xxl-job registry fail, registryParam:RegistryParam{registryGroup='EXECUTOR', registryKey='job-schedule-executor', registryValue='172.30.184.4:9997'}, registryResult:ReturnT [code=500, msg=xxl-rpc remoting fail, StatusCode(404) invalid. for url : http://xxl-job-service.dabai-fat.svc.cluster.local:8080/xxl-job-admin/api/registry, content=null]
so to solve the problem, I should execute command as possible as the same in kubernetes like execute with docker. The question is: How to pass the docker command --name to kubernetes environment? I have already tried this:
"env": [
{
"name": "name",
"value": "xxl-job-admin"
}
],
and also tried this:
"containers": [
{
"name": "xxl-job-admin",
"image": "xuxueli/xxl-job-admin:2.0.2",
}
]
Both did not work.
I would like to run Filebeat as Docker container in Azure IoT Edge. I would like Filebeat to get logs from others running containers.
I'm already able to run filebeat as Docker container, from the documentation (https://www.elastic.co/guide/en/beats/filebeat/6.8/running-on-docker.html#_volume_mounted_configuration)
docker run -d \
--name=filebeat \
--user=root \
--volume="$(pwd)/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro" \
--volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
--volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
docker.elastic.co/beats/filebeat:6.8.3 filebeat -e -strict.perms=false
With this command and with the correct filebeat.yml file I'm able to collect logs for every running containers on my device.
Now I would like to deploy this configuration as Azure IoT Edge Modules.
I created a docker image having the filebeat.yml file included with the following Dockerfile:
FROM docker.elastic.co/beats/filebeat:6.8.3
COPY filebeat.yml /usr/share/filebeat/filebeat.yml
USER root
RUN chmod go-w /usr/share/filebeat/filebeat.yml
RUN chown root:filebeat /usr/share/filebeat/filebeat.yml
USER filebeat
From documentation: https://www.elastic.co/guide/en/beats/filebeat/6.8/running-on-docker.html#_custom_image_configuration
I tested this Dockerfile by running locally
docker build -t filebeat .
and
docker run -d \
--name=filebeat \
--user=root \
--volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
--volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
filebeat:latest filebeat -e -strict.perms=false
This works fine, logs from other containers are collected as they should.
Now my question is :
In Azure IoT Edge, how can I mount volumes to access others Docker containers running on the devices, like it's done with
--volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
--volume="/var/run/docker.sock:/var/run/docker.sock:ro"
in order to collect logs?
From this other SO post (Mount path to Azure IoT Edge module) in the Azure IoT Edge portal I tried the following:
"HostConfig": {
"Mounts": [
{
"Target": "/var/lib/docker/containers",
"Source": "/var/lib/docker/containers",
"Type": "volume",
"ReadOnly: true
},
{
"Target": "/var/run/docker.sock",
"Source": "/var/run/docker.sock",
"Type": "volume",
"ReadOnly: true
}
]
}
}
But when I deploy this module I have the following error:
2019-11-25T10:09:41Z [WARN] - Could not create module FilebeatAgent
2019-11-25T10:09:41Z [WARN] - caused by: create /var/lib/docker/containers: "/var/lib/docker/containers" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path
I don't understand this error. How can I specify a path using only [a-zA-Z0-9][a-zA-Z0-9_.-] ?
Thanks for your help.
EDIT
In the Azure IoT Edge portal, createOptions json:
{
"HostConfig": {
"Binds": [
"/var/lib/docker/containers:/var/lib/docker/containers",
"/var/run/docker.sock:/var/run/docker.sock"
]
}
}
There is an article that describes how to mount storage from the host here: https://learn.microsoft.com/en-us/azure/iot-edge/how-to-access-host-storage-from-module
I couldn't find such a specific command around the internet so I kindly ask for your help with this one :)
Context
I have defined a podTemplate with a few containers, by using the containerTemplate methods:
ubuntu:trusty (14.04 LTS)
postgres:9.6
and finally, wurstmeister/kafka:latest
Doing some Groovy coding in Pipeline, I install several dependencies into my ubuntu:trusty container, such as latest Git, Golang 1.9, etc., and I also checkout my project from Github.
After all that dependencies are dealt with, I manage to compile, run migrations (which means Postgres is up and running and my app is connected to it), and spin up my app just fine until it complains that Kafka is not running because it couldn't connect to any broker.
Debugging sessions
After some debug sessions I have ps aux'ed each and every container to make sure all the services I needed were running in their respective containers, such as:
container(postgres) {
sh 'ps aux' # Show Postgres, as expected
}
container(linux) {
sh 'ps aux | grep post' # Does not show Postgres, as expected
sh 'ps aux | grep kafka' # Does not show Kafka, as expected
}
container(kafka) {
sh 'ps aux' # Does NOT show any Kafka running
}
I have also exported KAFKA_ADVERTISED_HOST_NAME var to 127.0.0.1 as explained in the image docs, without success, with the following code:
containerTemplate(
name: kafka,
image: 'wurstmeister/kafka:latest',
ttyEnabled: true,
command: 'cat',
envVars: [
envVar(key: 'KAFKA_ADVERTISED_HOST_NAME', value: '127.0.0.1'),
envVar(key: 'KAFKA_AUTO_CREATE_TOPICS_ENABLE', value: 'true'),
]
)
Questions
This image documentation details https://hub.docker.com/r/wurstmeister/kafka/ is explicit about starting a Kafka cluster with docker-compose up -d
1) How do I actually do that with this Kubernetes plugin + Docker + Groovy + Pipeline combo in Jenkins?
2) Do I actually need to do that? Postgres image docs (https://hub.docker.com/_/postgres/) also mentions about running the instance with docker run, but I didn't need to do that at all, which makes me think that containerTemplate is probably doing it automatically. So why is it not doing this for the Kafka container?
Thanks!
So... problem is with this image, and way how kubernetes works with them.
Kafka does not start because you override dockers CMD with command:'cat' which causes start-kafka.sh to never run.
Because of above I suggest using different image. Below template worked for me.
containerTemplate(
name: 'kafka',
image: 'quay.io/jamftest/now-kafka-all-in-one:1.1.0.B',
resourceRequestMemory: '500Mi',
ttyEnabled: true,
ports: [
portMapping(name: 'zookeeper', containerPort: 2181, hostPort: 2181),
portMapping(name: 'kafka', containerPort: 9092, hostPort: 9092)
],
command: 'supervisord -n',
envVars: [
containerEnvVar(key: 'ADVERTISED_HOST', value: 'localhost')
]
),
After executing this;
eval $(docker-machine env mymachine)
How do I check if the docker daemon on mymachine is a swarm manager?
To check general swarm membership, my preferred method is to use the formatted output from docker info. The possible values of this are currently inactive, pending, active, locked, and error:
case "$(docker info --format '{{.Swarm.LocalNodeState}}')" in
inactive)
echo "Node is not in a swarm cluster";;
pending)
echo "Node is not in a swarm cluster";;
active)
echo "Node is in a swarm cluster";;
locked)
echo "Node is in a locked swarm cluster";;
error)
echo "Node is in an error state";;
*)
echo "Unknown state $(docker info --format '{{.Swarm.LocalNodeState}}')";;
esac
To check for manager status, rather than just a node in a cluster, the field you want is .Swarm.ControlAvailable:
docker info --format '{{.Swarm.ControlAvailable}}'
That will output "true" for managers, and "false" for any node that is a worker or not in a swarm.
To identify worker nodes, you can join to two:
if [ "$(docker info --format '{{.Swarm.LocalNodeState}}')" = "active" \
-a "$(docker info --format '{{.Swarm.ControlAvailable}}')" = "false" ]; then
echo "node is a worker"
else
echo "node is not a worker"
fi
You could also use docker info to see the result of Swarm property (inactive or active).
For example:
function isSwarmNode(){
if [ "$(docker info | grep Swarm | sed 's/Swarm: //g')" == "inactive" ]; then
echo false;
else
echo true;
fi
}
I don't have a swarm node handy at the moment, but it looks as if you could simply run something like docker node ls. When targeting a docker daemon that is not in swarm node, that results in:
Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.
And it returns a nonzero exit code
$ echo $?
1
So the test would look something like:
if docker node ls > /dev/null 2>&1; then
echo this is a swarm node
else
echo this is a standalone node
fi
In addition to larsks answer, if you run docker node ls when pointing to a worker node, you'll get the following message:
Error response from daemon: This node is not a swarm manager. Worker nodes can't be used to view or modify cluster state. Please run this command on a manager node or promote the current node to a manager.
You can use this to differentiate between worker nodes and nodes not in a swarm at all.
Make sure your docker environment variables are set properly
$env | grep DOCKER_
Compare url and port values with the output from
$docker-machine ls
Select the swarm master machine name and you can reset the environment variables using
$eval $(docker-machine env your_master_machine_name)
Once environment variables are set properly, your command
$docker info | egrep '^Swarm: ' | cut -d ' ' -f2
should give the correct result
To get the IP address of a manager from any node (either worker or manager) using bash you can do:
read manager_ip _ <<<$(IFS=':'; echo $(docker info --format "{{ (index .Swarm.RemoteManagers 0).Addr }}"))
echo "${manager_ip}"
As mentioned above, the most direct way to identify if the current node is a manager is by using:
docker info --format '{{.Swarm.ControlAvailable}}'