How to use volumes-from in marathon - docker

Question

I'm working with mesos + marathon + docker quite a while but I got stuck at some point. At the moment I try to deal with persistent container and I tried to play around with the "volumes-from" parameter but I can't make it work because I have no clue how I can figure out the name of the data box to put it as a key in the json. I tried it with the example from here
{
"id": "privileged-job",
"container": {
"docker": {
"image": "mesosphere/inky"
"privileged": true,
"parameters": [
{ "key": "hostname", "value": "a.corp.org" },
{ "key": "volumes-from", "value": "another-container" },
{ "key": "lxc-conf", "value": "..." }
]
},
"type": "DOCKER",
"volumes": []
},
"args": ["hello"],
"cpus": 0.2,
"mem": 32.0,
"instances": 1
}
I would really appreciate any kind of help :-)

Answer 1

From what I know :
docker --volume-from take the ID or the name of a container.
Since your datacontainer is launch with Marathon too, it get an ID (not sur how to get this ID from marathon) and a name of that form : mesos-0fb2e432-7330-4bfe-bbce-4f77cf382bb4 which is not related to task ID in Mesos nor docker ID.
The solution would be to write something like this for your web-ubuntu application :
"parameters": [
{ "key": "volumes-from", "value": "mesos-0fb2e432-7330-4bfe-bbce-4f77cf382bb4" }
]
Since this docker-ID is unknown from Marathon it is not practical to use datacontainer that are started with Marathon.
You can try to start a datacontainer directly with Docker (without using Marathon) and use it as you do before but since you don't know in advance where web-ubuntu will be scheduled (unless you add a constraint to force it) it is not practical.

Answer 2

{
"id": "data-container",
"container": {
"docker": {
"image": "mesosphere/inky"
},
"type": "DOCKER",
"volumes": [
{
"containerPath": "/data",
"hostPath": "/var/data/a",
"mode": "RW"
}
]
},
"args": ["data-only"],
"cpus": 0.2,
"mem": 32.0,
"instances": 1
}
{
"id": "privileged-job",
"container": {
"docker": {
"image": "mesosphere/inky"
"privileged": true,
"parameters": [
{ "key": "hostname", "value": "a.corp.org" },
{ "key": "volumes-from", "value": "data-container" },
{ "key": "lxc-conf", "value": "..." }
]
},
"type": "DOCKER",
"volumes": []
},
"args": ["hello"],
"cpus": 0.2,
"mem": 32.0,
"instances": 1
}
Something like that maybe?

Answer 3

Mesos support passing the parameter of volume plugin using "key" & "value". But the issue is how to pass the volume name which Mesos expects to be either an absolute path or if absolute path is not passed then it will merge the name provided with the slave container sandbox folder. They do that primarily to support checkpointing, in case slave goes down accidentally.
The only option, till the above get enhanced, is to use another key value pair parameter. For e.g. in above case
{ "key": "volumes-from", "value": "databox" },
{ "key": "volume", "value": "datebox_volume" }
I have tested above with a plugin and it works.

Answer 4

Another approach is to write a custom mesos framework capable of running the docker command you want. In order to know what offers to accept and where to place each task you can use marathon information from: /apps/v2/ (under tasks key).
A good starting point for writing a new mesos framework is: https://github.com/mesosphere/RENDLER