I am trying to run my containers in mesos and outputting all logs to Cloudwatch. According to Docker documentation, I should pass the following to 'docker run':
--log-driver=awslogs --log-opt awslogs-region=us-east-1 --log-opt awslogs-group=myLogGroup
(from https://docs.docker.com/engine/admin/logging/awslogs/#awslogs-region)
to docker run. In order to achieve this using Marathon (or Chronos, I assume?) , I should add the following block to my configuration json file for my task
"parameters": [
{ "key": "hostname", "value": "a.corp.org" },
{ "key": "volumes-from", "value": "another-container" },
{ "key": "lxc-conf", "value": "..." }
]
https://mesosphere.github.io/marathon/docs/native-docker.html
It appears that this config appends the following to your docker run
--{key}={value}
Since I need to do something of the form "--log-opt awslogs-region=us-east-1", I am putting "key": "log-opt awslogs-region", "value": "us-east-1", which generates the docker run command correctly, as I can see from stderr in mesos. However, I then get an error saying
unknown flag: --log-opt awslogs-region
See 'docker run --help'.
Running the generated docker run command from stderr directly on the mesos slave works fine.
You can make use of the args parameter to pass custom docker run options,
"args": [
"--log-driver",
"awslogs"
]
Related
There's two distinct parameters in the /services/create endpoint, called Command and Args. The description says:
Command (array of string) – the command to be run in the image
Args (array of string) – arguments to the command
I was puzzled by the fact that Command is an array: if the command arguments can be passed to it (which seems to be the case as docker-py splits it as a shell command) what's the purpose of Args then?
My current guess: the purpose of Command and Args is the same as in Entrypoint and Cmd in regular Docker containers. Args are just appended to the Command:
POST /services/create
{
"Name": "test",
"TaskTemplate": {
"ContainerSpec": {
"Image": "ubuntu",
"Command": ["echo", "foo"],
"Args": ["bar"]
}
}
}
$ docker service logs test
test.1.npltmr9c69fz#hostname | foo bar
And if you omit the Command, it is taken from the image's entrypoint:
POST /services/create
{
"Name": "test2",
"TaskTemplate": {
"ContainerSpec": {
"Image": "ubuntu",
"Args": ["echo", "baz!"]
}
}
}
$ docker service logs test2
test2.1.c16waqqxtdgd#hostname | baz!
This is only a result of my empirical testing, so any solid documentation / source code proofs are appreciated!
I have been using docker to run images along with some options like:
docker run --net host --oom-kill-disable -d -p <port>:<port> image
How do I set values like --oom-kill-disable on marathon?
It is required for docker containers in their marathon spec to specify a boolean value for oom-kill-disable flag for executor to run properly.
So the spec would include:
"parameters": [
{ "key": "oom-kill-disable", "value": "true" }
]
I'm not quite sure if you can pass an empty value in this case, but you could go with something like this:
"container": {
"type": "DOCKER",
"docker": {
"network": "HOST",
"image": "your/image",
"parameters": [
{ "key": "oom-kill-disable", "value": "" }
]
}
}
You may read a little bit more here in "Privileged Mode and Arbitrary Docker Options" section.
Hi I have been successfull so far with simple jobs in marathon but it stuck when i have tried deploying a deocker job in mesos through marathon framework.
I am using a json file as below to deploy a docker job:
{
"id": "pga-docker",
"cpus": 0.2,
"mem": 1024.0,
"instances": 1,
"container": {
"type": "DOCKER",
"docker": {
"image": "pga",
"network": "BRIDGE",
"portMappings": [
{ "containerPort": 80, "hostPort": 6565, "servicePort": 0, "protocol": "tcp" }
]
}
}
}
My pga docker image have no problem when run as container, but through marathon its just not working. Its staying in the deploying state forever.
I am using the below command line:
curl -X POST http://10.141.141.10:8080/v2/apps -d #basic-3.json -H "Content-type: application/json"
But when I run the same image from marathon UI, its working. To run from marathon I used "docker run --publish 6060:80 --name test --rm pga" in the cmd field of the UI new job page.
Any one have idea why this is hanged in the command line approach?
This is what i have found during some trial and error with the json file.
I found that when we run docker image in local system, if we have mentioned an entry point or a cmd then that will execute while running the container. But this is not same for mesos/marathon. my observation is that if I explicitly mentioned cmd in the deployment json then its working fine.
"cmd":"sh pga-setup.sh"
I will love to know if anyone faced a similar issue an solved it by another way.
Made computation docker which runs fine locally. Uploaded it to Gcloud and could run it. But what I really need is to run hundreds of instances with different argument each.
docker run -t dxyz arg0
docker run -t dxyz arg1
docker run -t dxyz arg2
...
What is the best way to do it? I tried Kubctl pods but looks like they supposed to be identical
This is pretty clunky due to the nesting and because it requires you to specify the replication controller's name and image twice, but you can technically use
kubectl run dxyz0 --image=dxyz --overrides='{"apiVersion": "v1", "spec": {"template": {"spec": {"containers": [ {"name:" "dxyz0", "image": "dxyz", "args": [ "arg0" ] } ] } } } }'
kubectl run dxyz1 --image=dxyz --overrides='{"apiVersion": "v1", "spec": {"template": {"spec": {"containers": [ {"name:" "dxyz1", "image": "dxyz", "args": [ "arg1" ] } ] } } } }'
...
when I run a docker container as a marathon job, it creates a docker container in the active mesos slave system. when suspend or destroy the docker job what I expect that marathon should delete the docker container as its no longer required. But the container does not get deleted. I have to delete them manually every time marathon restart a docker container job.
is there any way to delete these unwanted containers automatically?
Edit:
Adding json file for initiating a marathon job
{
"id": "pga-docker",
"cmd":"sh pga-setup.sh",
"cpus": 0.5,
"mem": 1024.0,
"container": {
"type": "DOCKER",
"docker": {
"image": "pga:test",
"parameters": [
{ "key": "env", "value": "SERVER_HOST=value" },
{ "key": "env", "value": "SERVER_PORT=value" }
],
"network": "BRIDGE",
"portMappings": [
{ "containerPort": 80, "hostPort": 0}
]
}
}
}
Marathon will restart a docker container which failed so that you have the number of instances you requested. It could be that you see stopped/failed containers which were not cleaned up by Mesos. This could be related to the fact that Mesos delays container cleanup until GC.
see https://issues.apache.org/jira/browse/MESOS-1656
It is the behavior of Marathon, because it is meant for long running services, as soon the task is completed, Marathon assumes it has been terminated in that host and immediately it will assign a new instance for running the application. If you need one of task you can use Chronos, so it makes the task to run only one time. I have written a script to do this automatically for marathon.
start=$1
end=$2
for (( c=$start; c<=$end; c++ ))
do
echo "deleting:$c"
sleep 10
var=$(curl -X GET http://localhost:8080/v2/apps/docker-app-$c | grep "startedAt")
echo "$var"
if [[ $var == *"startedAt"* ]]
then
curl -X DELETE http://localhost:8080/v2/apps/docker-app-$c
echo "going to delete"
else
echo "application not started yet"
fi
sleep 1
done
echo "Completed!"