AWS ECS awsvpc no communication on localhost - docker

I am stuck on an issue with containers on ECS Fargate, more specifically 2 containers inside one task communicating with each other.
According the article linked (https://aws.amazon.com/blogs/compute/task-networking-in-aws-fargate/), once the two containers are configured inside the task, any other container in the task should be able to communicate with that container by just using localhost or 127.0.0.1.
However I can’t seem to get that behavior. I tried breaking it down to its simplest form - an nginx running on 1 container with port Mapping 80 and a simple curl command on another container, but no joy.
Am I missing something or some configuration that I need to do to get the 2 containers communicating within the ECS Task Definition over localhost or 127.0.0.1 ?
The JSON file of the Task Definition with 2 containers:
{
"requiresCompatibilities": [
"FARGATE"
],
"inferenceAccelerators": [],
"containerDefinitions": [
{
"name": "container-a",
"image": "******.dkr.ecr.eu-west-1.amazonaws.com/repository-a:latest",
"memoryReservation": "2048",
"cpu": "1024",
"resourceRequirements": null,
"essential": true,
"portMappings": [
{
"containerPort": "5000",
"protocol": "tcp"
}
],
"environment": null,
"environmentFiles": [],
"secrets": null,
"mountPoints": null,
"volumesFrom": null,
"hostname": null,
"user": null,
"workingDirectory": null,
"extraHosts": null,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/ecs/test-task-definition",
"awslogs-region": "eu-west-1",
"awslogs-stream-prefix": "ecs"
}
},
"ulimits": null,
"dockerLabels": null,
"dependsOn": null,
"repositoryCredentials": {
"credentialsParameter": ""
}
},
{
"name": "container-b",
"image": "******.dkr.ecr.eu-west-1.amazonaws.com/repository-b:latest",
"memoryReservation": "2048",
"cpu": "1024",
"resourceRequirements": null,
"essential": true,
"portMappings": [],
"environment": null,
"environmentFiles": [],
"secrets": null,
"mountPoints": null,
"volumesFrom": null,
"hostname": null,
"user": null,
"workingDirectory": null,
"extraHosts": null,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/ecs/test-task-definition",
"awslogs-region": "eu-west-1",
"awslogs-stream-prefix": "ecs"
}
},
"ulimits": null,
"dockerLabels": null,
"dependsOn": null,
"repositoryCredentials": {
"credentialsParameter": ""
}
}
],
"volumes": [],
"networkMode": "awsvpc",
"memory": "4096",
"cpu": "2048",
"executionRoleArn": "arn:aws:iam::******:role/ecr-execution-role",
"family": "test-task-definition",
"taskRoleArn": "arn:aws:iam::******:role/ecs-task-role",
"tags": []
}

Related

Connection the VerneMQ MQTT failing which is installed on local Windows 10 Docker engine

I got the Docker version - Docker Desktop 4.3.0 (71786) installed on my machine. I am trying to install the VerneMQ on Docker using the following command -
docker run -d -p 1883:1883 -e "DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on" -e "DOCKER_VERNEMQ_ACCEPT_EULA=yes" --name vernemq1 -d erlio/docker-vernemq
Docker logs for the contianer are as follows -
config is OK
-config /vernemq/data/generated.configs/app.2021.12.14.09.57.29.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args
Exec: /vernemq/bin/../erts-10.2.3/bin/erlexec -boot /vernemq/bin/../releases/1.10.0/vernemq -config /vernemq/data/generated.configs/app.2021.12.14.09.57.29.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args -pa /vernemq/bin/../lib/erlio-patches -- console -noshell -noinput
Root: /vernemq/bin/..
09:57:30.806 [info] writing state {[{[{actor,<<21,128,87,20,226,202,145,175,203,97,17,207,239,134,39,35,203,238,93,163>>}],1}],{dict,1,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[['VerneMQ#172.17.0.2',{[{actor,<<21,128,87,20,226,202,145,175,203,97,17,207,239,134,39,35,203,238,93,163>>}],1}]],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}} to disk <<75,2,131,80,0,0,0,248,120,1,203,96,206,97,96,96,96,204,96,130,82,41,12,172,137,201,37,249,69,185,64,81,17,209,134,112,145,71,167,38,174,63,157,40,120,254,125,155,186,242,233,119,177,139,179,18,25,179,50,56,83,24,88,82,50,147,75,18,25,19,5,128,144,35,49,32,209,32,67,32,11,13,100,48,130,196,192,6,131,8,166,20,6,161,176,212,162,188,84,223,64,7,67,115,35,61,67,115,61,3,61,35,210,108,70,181,1,225,14,6,194,238,64,213,10,0,36,44,78,211>>
09:57:30.827 [info] Datadir ./data/meta/meta/0 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,43756361}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:31.273 [info] Datadir ./data/meta/meta/1 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,54479016}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:31.585 [info] Datadir ./data/meta/meta/2 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,45466763}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:31.914 [info] Datadir ./data/meta/meta/3 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,33567655}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:32.245 [info] Datadir ./data/meta/meta/4 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,47595779}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:32.699 [info] Datadir ./data/meta/meta/5 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,39556020}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:33.062 [info] Datadir ./data/meta/meta/6 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,61685232}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:33.379 [info] Datadir ./data/meta/meta/7 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,55978245}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:33.688 [info] Datadir ./data/meta/meta/8 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,34857794}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:34.016 [info] Datadir ./data/meta/meta/9 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,59026771}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:34.474 [info] Datadir ./data/meta/meta/10 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,51575312}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:34.888 [info] Datadir ./data/meta/meta/11 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,49221877}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
09:57:35.579 [info] Try to start vmq_plumtree: ok
09:57:40.035 [info] Try to start vmq_generic_msg_store: ok
09:57:40.151 [info] loaded 0 subscriptions into vmq_reg_trie
09:57:40.153 [info] cluster event handler 'vmq_cluster' registered
09:57:44.398 [warning] session stopped abnormally due to '{cant_parse_connect_fixed_header,<<"GET / HTTP/1.1\r\nHost: localhost:1883\r\nConnection: Upgrade\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36\r\nUpgrade: websocket\r\nOrigin: chrome-extension://hemojaaeigabkbcookmlgmdigohjobjm\r\nSec-WebSocket-Version: 13\r\nAccept-Encoding: gzip, deflate, br\r\nAccept-Language: en-US,en;q=0.9,hi;q=0.8\r\nSec-WebSocket-Key: M4Jd7MuPOr8ENM1q4wsqpQ==\r\nSec-WebSocket-Extensions: permessage-deflate; client_max_window_bits\r\nSec-WebSocket-Protocol: mqttv3.1\r\n\r\n">>}'
Docker inspect result -
[
{
"Id": "28c7f34dfe5af201b7786a8b14d4a856914fac0d88d528a3f335c910",
"Created": "2021-12-14T09:57:24.5409484Z",
"Path": "start_vernemq",
"Args": [],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 19341,
"ExitCode": 0,
"Error": "",
"StartedAt": "2021-12-14T10:04:56.1632278Z",
"FinishedAt": "2021-12-14T10:03:05.3115524Z",
"Health": {
"Status": "healthy",
"FailingStreak": 0,
"Log": [
{
"Start": "2021-12-14T10:01:31.8393179Z",
"End": "2021-12-14T10:01:32.4156947Z",
"ExitCode": 0,
"Output": ""
},
{
"Start": "2021-12-14T10:02:02.5563049Z",
"End": "2021-12-14T10:02:03.0990371Z",
"ExitCode": 0,
"Output": ""
},
{
"Start": "2021-12-14T10:02:33.2237707Z",
"End": "2021-12-14T10:02:33.7814843Z",
"ExitCode": 0,
"Output": ""
},
{
"Start": "2021-12-14T10:03:03.9905476Z",
"End": "2021-12-14T10:03:04.5533888Z",
"ExitCode": 0,
"Output": ""
},
{
"Start": "2021-12-14T10:05:26.1645268Z",
"End": "2021-12-14T10:05:26.7281332Z",
"ExitCode": 0,
"Output": ""
}
]
}
},
"Image": "sha256:c313a97d55f84087e216fb18c06e0f410f5ded6ee089c5",
"ResolvConfPath": "/var/lib/docker/containers/28c7f34dfe5af201b6914fac0d88d528a3f335c910/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/28c7f34dfe5af8b14d4a856914fac0d88d528a3f335c910/hostname",
"HostsPath": "/var/lib/docker/containers/28c7f34dfe5af20d88d528a3f335c910/hosts",
"LogPath": "/var/lib/docker/containers/28c7f34dfe5af201ac0d88d528a3f335c910/28c7f34dfe5af201b77a941b00c86a8b14d4a856914fac0d88
d528a3f335c910-json.log",
"Name": "/vernemq1",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": null,
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "default",
"PortBindings": {
"1883/tcp": [
{
"HostIp": "",
"HostPort": "1883"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
24,
162
],
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": [
"/proc/asound",
"/proc/acpi",
"/proc/kcore",
"/proc/keys",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/proc/scsi",
"/sys/firmware"
],
"ReadonlyPaths": [
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
]
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/6738d5b2c9b935c3e90e370c21d03a5-init/diff:/var/lib/docker/overlay2/b48a41a3
92eb834cafd6f1579f005/diff:/var/lib/docker/overlay2/2237dca228f720bd3a37e53ba6b74fdfb/diff:/var/
lib/docker/overlay2/8f7ee085d6a04e/diff:/var/lib/docker/overlay2/18116350a96478ab3bb8250c294e58643d02a48bd89808e/diff:/var/lib/docker/overlay2/94f83d29c8304416d9709b034f/diff:/var/lib/docker/overlay2/1780d5340ed788d9c9f/diff",
"MergedDir": "/var/lib/docker/overlay2/6730c21d03a5/merged",
"UpperDir": "/var/lib/docker/overlay2/67383a5/diff",
"WorkDir": "/var/lib/docker/overlay2/6738d5b2370c21d03a5/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "77d02c57bafdc9275cc6bc",
"Source": "/var/lib/docker/volumes/77d02c57bafdc92750b6cc6bc/_data",
"Destination": "/vernemq/data",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
},
{
"Type": "volume",
"Name": "bb3402031f3fd09c821d41db",
"Source": "/var/lib/docker/volumes/bb3402031f3f9c821d41db/_data",
"Destination": "/vernemq/etc",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
},
{
"Type": "volume",
"Name": "3f79aa44ce3328f73c1093",
"Source": "/var/lib/docker/volumes/3f79aa44cead0ec3c1093/_data",
"Destination": "/vernemq/log",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "28c4dfe5a",
"Domainname": "",
"User": "vernemq",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"1883/tcp": {},
"4369/tcp": {},
"44053/tcp": {},
"8080/tcp": {},
"8883/tcp": {},
"8888/tcp": {},
"9100/tcp": {},
"9101/tcp": {},
"9102/tcp": {},
"9103/tcp": {},
"9104/tcp": {},
"9105/tcp": {},
"9106/tcp": {},
"9107/tcp": {},
"9108/tcp": {},
"9109/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on",
"DOCKER_VERNEMQ_ACCEPT_EULA=yes",
"PATH=/vernemq/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"DOCKER_VERNEMQ_KUBERNETES_LABEL_SELECTOR=app=vernemq",
"DOCKER_VERNEMQ_LOG__CONSOLE=console",
"VERNEMQ_VERSION=1.10.0"
],
"Cmd": [
"start_vernemq"
],
"Healthcheck": {
"Test": [
"CMD-SHELL",
"vernemq ping | grep -q pong"
]
},
"Image": "erlio/docker-vernemq",
"Volumes": {
"/vernemq/data": {},
"/vernemq/etc": {},
"/vernemq/log": {}
},
"WorkingDir": "/vernemq",
"Entrypoint": null,
"OnBuild": null,
"Labels": {}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "b3906e648f42f4ed49f98e521",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"1883/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "1883"
}
],
"4369/tcp": null,
"44053/tcp": null,
"8080/tcp": null,
"8883/tcp": null,
"8888/tcp": null,
"9100/tcp": null,
"9101/tcp": null,
"9102/tcp": null,
"9103/tcp": null,
"9104/tcp": null,
"9105/tcp": null,
"9106/tcp": null,
"9107/tcp": null,
"9108/tcp": null,
"9109/tcp": null
},
"SandboxKey": "/var/run/docker/netns/b3906e6f42",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "6e5e5d38da00cc6c6cc",
"Gateway": "172.17.0.1",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"MacAddress": "02:48:ac:11:11:02",
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "1a2a19e811b79bbb7e6dca53047fc1a",
"EndpointID": "6e5e5d38da00ca77724b7586c6cc",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:02",
"DriverOpts": null
}
}
}
}
]
And now when I try to connect to VerneMQ, I can't. I am trying to connect using TCP://172.17.0.2 over 1883 port.
Can somebody help me where I am making the mistake.
Note: I haven't changed the config file.
EDIT:
Using MQTTLens chrome extension to connect with MQTT.
EDIT2
I was trying to connect using various MQTT clients available but none of them seems to be working for me. but when accessing the broker using C# code (MQTTnet Nuget package) it seemed to be working. The only issue which I am facing right now is the MQTT Version.
When I set the Version as MqttProtocolVersion.V500 connection throws exception and the error which I will get is
10:48:57.303 [warning] invalid protocol version for {[],undefined} 5.
Now the question that remains is, how to configure the VerneMQ container to accept the latest Protocol V500?
var factory = new MqttFactory();
mqttClient = factory.CreateMqttClient();
var temp_options = new MqttClientOptions() { ClientId = clientId };
temp_options.ChannelOptions = new MqttClientTcpOptions()
{
Server = ipAddress,
Port = port
};
temp_options.CleanSession = true;
temp_options.KeepAlivePeriod = TimeSpan.FromHours(10);
temp_options.ProtocolVersion = MQTTnet.Formatter.MqttProtocolVersion.V311;
//temp_options.ProtocolVersion = MQTTnet.Formatter.MqttProtocolVersion.V500;
await mqttClient.ConnectAsync(temp_options, CancellationToken.None);

Traefik container has no mount on docker inspect

I have an error that only happens when I run docker compose up on one of my docker compose files and not the other, even though the services named "proxy" in which the error occurs are identical.
When I run docker compose -f docker-compose-v1.yml up -d the containers start without issue but on inspection the container named github_proxy_1 has no mounts.
If I run docker-compose -f docker-compose-v2.yml up -d the containers start without issue and on inspection the container named github_proxy_1 has mounts.
The proxy services are based on traefik windows images. I have read that many have had issues with binding volumes so that traefik can communicate with the underlying host, but the weird thing is that it works flawlessly in one setup but not the other.
Below are my docker version info, docker compose files, the results when inspecting the containers (showing the difference in mounts), and the error message from the container log shown in Docker Desktop.
Why does one proxy container manage to populate mounts while the other doesn't?
Thanks in advance!
docker version
Client:
Cloud integration: 1.0.14
Version: 20.10.6
API version: 1.41
Go version: go1.16.3
Git commit: 370c289
Built: Fri Apr 9 22:49:36 2021
OS/Arch: windows/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.6
API version: 1.41 (minimum version 1.24)
Go version: go1.13.15
Git commit: 8728dd2
Built: Fri Apr 9 22:45:40 2021
OS/Arch: windows/amd64
Experimental: false
docker-compose-v1.yml
version: "3.7"
services:
s4-web:
image: s4admin:484-2
build:
context: .
dockerfile: S4/Admin/Dockerfile
labels:
- "traefik.enable=true"
- "traefik.http.routers.web.rule=PathPrefix(`/`)"
networks:
- sg-net
proxy:
image: traefik:v2.3-windowsservercore-1809
command:
- "--providers.docker"
- "--providers.docker.endpoint=npipe:////./pipe/docker_engine"
- "--providers.docker.exposedbydefault=false"
- "--providers.docker.network=sg-net"
- "--entrypoints.web.address=:80"
- "--api.insecure=true"
ports:
- "8080:80"
- "8088:8080"
volumes:
- type: npipe
source: \\.\pipe\docker_engine
target: \\.\pipe\docker_engine
networks:
- sg-net
networks:
sg-net:
docker-compose-v2.yml
version: "3.7"
services:
signup-web:
image: signup-web:02-06
build:
context: ../..
dockerfile: docker/02-06-platform-integration/signup-web/v6/Dockerfile
environment:
- Dependencies:IReferenceDataLoader=SignUp.Web.ReferenceData.ApiReferenceDataLoader
- ReferenceDataApi:Url=http://reference-data-api/api
- Dependencies:IProspectSaveHandler=SignUp.Web.ProspectSave.AsynchronousProspectSaveHandler
- MessageQueue:Url=nats://${HOST_IP}:4222
labels:
- "traefik.enable=true"
- "traefik.http.routers.web.rule=PathPrefix(`/`)"
networks:
- signup-net
proxy:
image: traefik:v2.3-windowsservercore-1809
command:
- "--providers.docker"
- "--providers.docker.endpoint=npipe:////./pipe/docker_engine"
- "--providers.docker.exposedbydefault=false"
- "--providers.docker.network=signup-net"
- "--entrypoints.web.address=:80"
- "--api.insecure=true"
ports:
- "8080:80"
- "8088:8080"
volumes:
- type: npipe
source: \\.\pipe\docker_engine
target: \\.\pipe\docker_engine
networks:
- signup-net
networks:
signup-net:
Result from docker inspect github_proxy_1 showing no mounts:
[
{
"Id": "507bb53f9f7c772221ef858464877d310c4ab48a100f86e49b5e248de67c5921",
"Created": "2021-05-28T07:49:16.3839919Z",
"Path": "/traefik",
"Args": [
"--providers.docker",
"--providers.docker.endpoint=npipe:////./pipe/docker_engine",
"--providers.docker.exposedbydefault=false",
"--providers.docker.network=sg-net",
"--entrypoints.web.address=:80",
"--api.insecure=true"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 1312,
"ExitCode": 0,
"Error": "",
"StartedAt": "2021-05-28T07:49:22.9474168Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:0011bc7c1cd89741d499d69fe62ed8c50b251d90bb3dd01350d6f945dace6e89",
"ResolvConfPath": "",
"HostnamePath": "",
"HostsPath": "",
"LogPath": "C:\\ProgramData\\Docker\\containers\\507bb53f9f7c772221ef858464877d310c4ab48a100f86e49b5e248de67c5921\\507bb53f9f7c772221ef858464877d310c4ab48a100f86e49b5e248de67c5921-json.log",
"Name": "/github_proxy_1",
"RestartCount": 0,
"Driver": "windowsfilter",
"Platform": "windows",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "github_sg-net",
"PortBindings": {
"80/tcp": [
{
"HostIp": "",
"HostPort": "8080"
}
],
"8080/tcp": [
{
"HostIp": "",
"HostPort": "8088"
}
]
},
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": [],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "",
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 0,
"ConsoleSize": [
0,
0
],
"Isolation": "hyperv",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"dir": "C:\\ProgramData\\Docker\\windowsfilter\\507bb53f9f7c772221ef858464877d310c4ab48a100f86e49b5e248de67c5921"
},
"Name": "windowsfilter"
},
"Mounts": [],
"Config": {
"Hostname": "507bb53f9f7c",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": true,
"AttachStderr": true,
"ExposedPorts": {
"80/tcp": {},
"8080/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": [
"--providers.docker",
"--providers.docker.endpoint=npipe:////./pipe/docker_engine", "--providers.docker.exposedbydefault=false",
"--providers.docker.network=sg-net",
"--entrypoints.web.address=:80",
"--api.insecure=true"
],
"Image": "sha256:0011bc7c1cd89741d499d69fe62ed8c50b251d90bb3dd01350d6f945dace6e89",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/traefik"
],
"OnBuild": null,
"Labels": {
"com.docker.compose.config-hash": "ef67d49c829762590f72bf6caa8e34327f9b86915e9e985c9d2fa43177457894",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "github",
"com.docker.compose.project.config_files": "C:\\Dev\\Projects\\SurveyGenerator\\github\\docker-compose-s4.yml",
"com.docker.compose.project.working_dir": "C:\\Dev\\Projects\\SurveyGenerator\\github",
"com.docker.compose.service": "proxy",
"com.docker.compose.version": "1.0-alpha",
"org.opencontainers.image.description": "A modern reverse-proxy",
"org.opencontainers.image.documentation": "https://docs.traefik.io",
"org.opencontainers.image.title": "Traefik",
"org.opencontainers.image.url": "https://traefik.io",
"org.opencontainers.image.vendor": "Traefik Labs",
"org.opencontainers.image.version": "v2.3.7"
}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "507bb53f9f7c772221ef858464877d310c4ab48a100f86e49b5e248de67c5921",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"80/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8080"
}
],
"8080/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8088"
}
]
},
"SandboxKey": "507bb53f9f7c772221ef858464877d310c4ab48a100f86e49b5e248de67c5921",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"github_sg-net": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"proxy",
"github_proxy_1",
"507bb53f9f7c"
],
"NetworkID": "2e5931b8cba8fdd1b2b4e547089c21b85570450437599d13e8af3888263d22e3",
"EndpointID": "87d39c2b7cc69b3090026f8395312704e684d7fb925410895dca1a007473ee41",
"Gateway": "172.27.160.1",
"IPAddress": "172.27.163.11",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "00:15:5d:46:69:3c",
"DriverOpts": null
}
}
}
}
]
Result from docker inspect github_proxy_1 where mounts has values:
[
{
"Id": "a81aae03a323b5efd37f80656dccab0bd102aa903d34803554456f0661c22c99",
"Created": "2021-05-28T08:21:05.6675713Z",
"Path": "/traefik",
"Args": [
"--providers.docker",
"--providers.docker.endpoint=npipe:////./pipe/docker_engine",
"--providers.docker.exposedbydefault=false",
"--providers.docker.network=signup-net",
"--entrypoints.web.address=:80",
"--api.insecure=true"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 1328,
"ExitCode": 0,
"Error": "",
"StartedAt": "2021-05-28T08:21:10.4771044Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:0011bc7c1cd89741d499d69fe62ed8c50b251d90bb3dd01350d6f945dace6e89",
"ResolvConfPath": "",
"HostnamePath": "",
"HostsPath": "",
"LogPath": "C:\\ProgramData\\Docker\\containers\\a81aae03a323b5efd37f80656dccab0bd102aa903d34803554456f0661c22c99\\a81aae03a323b5efd37f80656dccab0bd102aa903d34803554456f0661c22c99-json.log",
"Name": "/03_proxy_1",
"RestartCount": 0,
"Driver": "windowsfilter",
"Platform": "windows",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "03_signup-net",
"PortBindings": {
"80/tcp": [
{
"HostIp": "",
"HostPort": "8080"
}
],
"8080/tcp": [
{
"HostIp": "",
"HostPort": "8088"
}
]
},
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": [],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "",
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 0,
"ConsoleSize": [
0,
0
],
"Isolation": "hyperv",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"Mounts": [
{
"Type": "npipe",
"Source": "\\\\.\\pipe\\docker_engine",
"Target": "\\\\.\\pipe\\docker_engine"
}
],
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"dir": "C:\\ProgramData\\Docker\\windowsfilter\\a81aae03a323b5efd37f80656dccab0bd102aa903d34803554456f0661c22c99"
},
"Name": "windowsfilter"
},
"Mounts": [
{
"Type": "npipe",
"Source": "\\\\.\\pipe\\docker_engine",
"Destination": "\\\\.\\pipe\\docker_engine",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "a81aae03a323",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"80/tcp": {},
"8080/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": [
"--providers.docker",
"--providers.docker.endpoint=npipe:////./pipe/docker_engine",
"--providers.docker.exposedbydefault=false",
"--providers.docker.network=signup-net",
"--entrypoints.web.address=:80",
"--api.insecure=true"
],
"Image": "traefik:v2.3-windowsservercore-1809",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/traefik"
],
"OnBuild": null,
"Labels": {
"com.docker.compose.config-hash": "53f350716aad96821fea12154c09bf70e538394230171e0139d28b0ea8b6a2c6",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "03",
"com.docker.compose.project.config_files": "app\\03\\v7-windows.yml",
"com.docker.compose.project.working_dir": "C:\\Dev\\Projects\\docker4dotnet\\app\\03",
"com.docker.compose.service": "proxy",
"com.docker.compose.version": "1.29.1",
"org.opencontainers.image.description": "A modern reverse-proxy",
"org.opencontainers.image.documentation": "https://docs.traefik.io",
"org.opencontainers.image.title": "Traefik",
"org.opencontainers.image.url": "https://traefik.io",
"org.opencontainers.image.vendor": "Traefik Labs",
"org.opencontainers.image.version": "v2.3.7"
}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "a81aae03a323b5efd37f80656dccab0bd102aa903d34803554456f0661c22c99",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"80/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8080"
}
],
"8080/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8088"
}
]
},
"SandboxKey": "a81aae03a323b5efd37f80656dccab0bd102aa903d34803554456f0661c22c99",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"03_signup-net": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"a81aae03a323",
"proxy"
],
"NetworkID": "1a464d92f0625acd600391c7b2204d33e54cb94b1bf4a098ca0a48e5f4067edc",
"EndpointID": "fde1dace014a898a4cee3fb624ce1a1ec74f87a565b3cb4696e895037774e57d",
"Gateway": "172.29.0.1",
"IPAddress": "172.29.12.143",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "00:15:5d:2f:e3:79",
"DriverOpts": null
}
}
}
}
]
Logged error message in docker desktop for the github_proxy_1 container without mounts:
time="2021-05-28T09:49:25+02:00" level=error msg="Failed to retrieve information of the docker client and server host: error during connect: This error may indicate that the docker daemon is not running.: Get \"http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.24/version\": open //./pipe/docker_engine: The system cannot find the file specified." providerName=docker
time="2021-05-28T09:49:25+02:00" level=error msg="Provider connection error error during connect: This error may indicate that the docker daemon is not running.: Get \"http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.24/version\": open //./pipe/docker_engine: The system cannot find the file specified., retrying in 552.330144ms" providerName=docker
So I found the problem. I was using docker-compose (using the docker-compose binary) for the compose that worked and docker compose for the one that didn't.
The question remains why one works and the other doesn't, what is the difference of running docker-compose vs docker compose? Does it perhaps somehow have to do with permissions?

Docker Container is disappeard after restart service

I stopped my docker container.
and i restart the service in linux (service docker restart)
after that my container is missing.
I typed docker ps -a but I can't find my splunk container.
I can find container directory and volume folder also.
also I have every folder in inspection result.
but I can't see when I type docker ps -a
how can I restore it ?
this is my docker inspect result, when container is exist.
[
{
"Id": "9e12bb7b2f8a926c0b7852aa9f1bf957363f327d6de53778059bf0fc05711039",
"Created": "2018-10-02T04:05:28.013507313Z",
"Path": "/sbin/entrypoint.sh",
"Args": [
"start-service"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 11513,
"ExitCode": 0,
"Error": "",
"StartedAt": "2020-09-22T05:22:09.939497539Z",
"FinishedAt": "2020-09-22T05:20:05.99542747Z"
},
"Image": "sha256:507021d7e77f9bdd337aeb47729a806162579f36e5d73b14f2a508e545adae72",
"ResolvConfPath": "/storage/docker/containers/9e12bb7b2f8a926c0b7852aa9f1bf957363f327d6de53778059bf0fc05711039/resolv.conf",
"HostnamePath": "/storage/docker/containers/9e12bb7b2f8a926c0b7852aa9f1bf957363f327d6de53778059bf0fc05711039/hostname",
"HostsPath": "/storage/docker/containers/9e12bb7b2f8a926c0b7852aa9f1bf957363f327d6de53778059bf0fc05711039/hosts",
"LogPath": "/storage/docker/containers/9e12bb7b2f8a926c0b7852aa9f1bf957363f327d6de53778059bf0fc05711039/9e12bb7b2f8a926c0b7852aa9f1bf957363f327d6de53778059bf0fc05711039-json.log",
"Name": "/splunk",
"RestartCount": 0,
"Driver": "overlay",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": null,
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "default",
"PortBindings": {
"8000/tcp": [
{
"HostIp": "",
"HostPort": "8000"
}
],
"8089/tcp": [
{
"HostIp": "",
"HostPort": "8089"
}
]
},
"RestartPolicy": {
"Name": "unless-stopped",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DiskQuota": 0,
"KernelMemory": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": -1,
"OomKillDisable": false,
"PidsLimit": 0,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0
},
"GraphDriver": {
"Data": {
"LowerDir": "/storage/docker/overlay/25444b7c23c5d0bf0a6d4d1563a587831b7c329e6623ee720e609a0a16787ef5/root",
"MergedDir": "/storage/docker/overlay/53d6023089444b76af280ef1220a2851c1b5045167c471f5f60f9faa0389ccd3/merged",
"UpperDir": "/storage/docker/overlay/53d6023089444b76af280ef1220a2851c1b5045167c471f5f60f9faa0389ccd3/upper",
"WorkDir": "/storage/docker/overlay/53d6023089444b76af280ef1220a2851c1b5045167c471f5f60f9faa0389ccd3/work"
},
"Name": "overlay"
},
"Mounts": [
{
"Type": "volume",
"Name": "cb9d0886db87a17304310bfa5a242972bf9c76464eb0c85c24902c7764443714",
"Source": "/storage/docker/volumes/cb9d0886db87a17304310bfa5a242972bf9c76464eb0c85c24902c7764443714/_data",
"Destination": "/opt/splunk/etc",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
},
{
"Type": "volume",
"Name": "5d637a70101b4d6a80261abfeac9534b8b77bde4669c6225dedf3a9bcfb4434a",
"Source": "/storage/docker/volumes/5d637a70101b4d6a80261abfeac9534b8b77bde4669c6225dedf3a9bcfb4434a/_data",
"Destination": "/opt/splunk/var",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "9e12bb7b2f8a",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"1514/tcp": {},
"8000/tcp": {},
"8088/tcp": {},
"8089/tcp": {},
"8191/tcp": {},
"9997/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"SPLUNK_START_ARGS=--accept-license",
"SPLUNK_USER=root",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"SPLUNK_PRODUCT=splunk",
"SPLUNK_VERSION=7.0.0",
"SPLUNK_BUILD=c8a78efdd40f",
"SPLUNK_FILENAME=splunk-7.0.0-c8a78efdd40f-Linux-x86_64.tgz",
"SPLUNK_HOME=/opt/splunk",
"SPLUNK_APPS=/opt/splunk/etc/apps",
"SPLUNK_GROUP=splunk",
"SPLUNK_BACKUP_DEFAULT_ETC=/var/opt/splunk",
"LANG=en_US.utf8",
"JAVA_HOME=/opt/jdk/jdk1.8.0_152/"
],
"Cmd": [
"start-service"
],
"ArgsEscaped": true,
"Image": "klustree/splunk:7.0.0",
"Volumes": {
"/opt/splunk/etc": {},
"/opt/splunk/var": {}
},
"WorkingDir": "/opt/splunk",
"Entrypoint": [
"/sbin/entrypoint.sh"
],
"OnBuild": null,
"Labels": {}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "57cf9a7d9ec5d31e685fd5ea9232a3126fab1a90406449cc4d48b43a2f7c4f57",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"1514/tcp": null,
"8000/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8000"
}
],
"8088/tcp": null,
"8089/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8089"
}
],
"8191/tcp": null,
"9997/tcp": null
},
"SandboxKey": "/var/run/docker/netns/57cf9a7d9ec5",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "b4d39d4b6615127cd6aaeda1e0e14e73d910fadf9b596d5b81698d3fc1c53945",
"Gateway": "172.17.1.1",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "172.17.1.4",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"MacAddress": "02:42:ac:11:01:04",
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "c563f5e5d0e2e4bd6d35f5802395f77874679e40a522dabaee59b93d3747619c",
"EndpointID": "b4d39d4b6615127cd6aaeda1e0e14e73d910fadf9b596d5b81698d3fc1c53945",
"Gateway": "172.17.1.1",
"IPAddress": "172.17.1.4",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:01:04",
"DriverOpts": null
}
}
}
}
]
please help...
This is the default behavior of the docker service, in the docs we learn:
By default, when the Docker daemon terminates, it shuts down running containers. Starting with Docker Engine 1.12, you can configure the daemon so that containers remain running if the daemon becomes unavailable. This functionality is called live restore. The live restore option helps reduce container downtime due to daemon crashes, planned outages, or upgrades.
Here you can learn how to keep your containers alive when occurring a service timeout.
And here you can configure for your containers still alive when the docker service is restarted.
Although live-restore and the restart policy might seem similar they have different purposes:
Restart policies are different from the --live-restore flag of the dockerd command. Using --live-restore allows you to keep your containers running during a Docker upgrade, though networking and user input are interrupted.

Inter docker communication (mqtt and python)

I am having two docker containers in IBM Bluemix. One is MQTT and other php-python.
The PHP application executes a python script that should connect to the MQTT docker container and subscribe to a topic.
However, the php-python docker fails to connect. It also fails to ping to the MQTT docker container.
The MQTT container is working perfect as I am able to connect to it from my local machine.
I have tried linking the two containers using the following command:
cf ic run -p 443 -p 80 --name connector --link moscacontainer:source
registry.ng.bluemix.net/fortspace/php-mqtt
I have exposed the ports 1883 and 80 for the MQTT container.
I have used the following link as reference:
https://github.com/dceejay/nrdock/blob/master/README.md#linking-containers
Can someone guide on this?
EDIT: The output for cf ic inspect
[
{
"BluemixApp": null,
"BluemixServices": null,
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [],
"Dns": "",
"Env": [
"logging_password=",
"space_id=f874124c-cea6-460e-9bf2-b633eeeefcdf",
"logstash_target=logmet.opvis.bluemix.net:9091",
"metrics_target=logmet.opvis.bluemix.net:9095"
],
"Hostname": "instance-0018c601",
"Image": "registry.ng.bluemix.net/fortspace/php-mqtt:latest",
"ImageArchitecture": "amd64",
"Labels": {},
"Memory": 256,
"MemorySwap": "",
"OpenStdin": true,
"PortSpecs": "",
"StdinOnce": false,
"Tty": true,
"User": "",
"VCPU": 1,
"VolumesFrom": "",
"WorkingDir": ""
},
"ContainerState": "Running",
"Created": "2017-01-11T06:21:38.000000000Z",
"Group": {},
"HostConfig": {
"Binds": [],
"CapAdd": [],
"CapDrop": [],
"ContainerIDFile": "",
"Links": [
"moscacontainer:source"
],
"LogConfig": {
"Config": {},
"Type": "json-file"
},
"LxcConf": [],
"PortBindings": {
"443/tcp": [
{
"HostPort": "443"
}
],
"80/tcp": [
{
"HostPort": "80"
}
]
},
"Privileged": false,
"PublishAllPorts": false
},
"HostId": "46f8e5e8739f205e67ec852485095e953f079ff5c8f95b9e7c16f331",
"Human_id": "connector",
"Id": "4920a824-66bf-49bd-be71-549d0adeeffc",
"Image": "sha256:080893a1e47927b2fdf3f24a8b0b803fab035c236551e32df9527654c8af698c",
"Mounts": [],
"Name": "connector",
"NetworkSettings": {
"Bridge": "",
"Gateway": "",
"IPAddress": "172.29.0.53",
"IPPrefixLen": 0,
"MacAddress": "fa:16:3e:3a:2e:04",
"Networks": {
"default": {
"Aliases": null,
"EndpointID": "4920a824-66bf-49bd-be71-549d0adeeffc",
"Gateway": "172.29.0.1",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAMConfig": null,
"IPAddress": "172.29.0.53",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"Links": null,
"MacAddress": "fa:16:3e:3a:2e:04",
"NetworkID": "bd84d27c-8909-4dcc-a9cf-7dca5453f55c"
}
},
"PortMapping": null,
"Ports": {
"443/tcp": [
{
"HostIp": "169.46.22.55",
"HostPort": "443"
}
],
"80/tcp": [
{
"HostIp": "169.46.22.55",
"HostPort": "80"
}
]
},
"PublicIpAddress": "169.46.22.55"
},
"Path": "date",
"ResolvConfPath": "/etc/resolv.conf",
"State": {
"Error": "",
"ExitCode": 0,
"FinishedAt": "0001-01-01T00:00:00Z",
"Ghost": "",
"Pid": 1,
"Running": true,
"StartedAt": "2017-01-11T06:22:54.000000000Z",
"Status": "Running"
},
"Volumes": {},
"VolumesRW": {}
}
]
You can review information in this page:
https://console.ng.bluemix.net/docs/containers/container_linking.html
I am afraid the the --link is incorrect and you will may have to ensure alias and name are correct.
This command output should help you:
cf ic inspect recipient_container_name

Docker local repository deletion does not free space

I am facing a problem. Please let me know if I am doing something wrong.
I have created a local docker repository (dev, file system) and tried to push some local images on to the same.
The local images were saved on the repository and the GET requests showed the JSON contents.
but then, I wanted to delete these images from the repository and for that I used the HTTP DELETE requests, deleting the "library" and the images from the repository.
The problem is that the space eaten up by the images that were pushed to the repo has not been freed up.
I have deleted the container running the repository and also the repository image itself, but the space is still not freed up.
Strange is that the same is not visible in any of the system directories even id I do a du -hs /* in /.
Contents of the repo query attached.
[{
"Args": [
"-c",
"exec docker-registry"
],
"Config": {
"AttachStderr": true,
"AttachStdin": false,
"AttachStdout": true,
"Cmd": [
"/bin/sh",
"-c",
"exec docker-registry"
],
"CpuShares": 0,
"Cpuset": "",
"Domainname": "",
"Entrypoint": null,
"Env": [
"HOME=/",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"DOCKER_REGISTRY_CONFIG=/docker-registry/config/config_sample.yml",
"SETTINGS_FLAVOR=dev"
],
"ExposedPorts": {
"5000/tcp": {}
},
"Hostname": "db6454154305",
"Image": "registry",
"Memory": 0,
"MemorySwap": 0,
"NetworkDisabled": false,
"OnBuild": null,
"OpenStdin": false,
"PortSpecs": null,
"StdinOnce": false,
"Tty": false,
"User": "",
"Volumes": {
"/docker/registry": {}
},
"WorkingDir": ""
},
"Created": "2014-08-25T14:12:03.711327013Z",
"Driver": "aufs",
"ExecDriver": "native-0.2",
"HostConfig": {
"Binds": null,
"CapAdd": null,
"CapDrop": null,
"ContainerIDFile": "",
"Devices": [],
"Dns": null,
"DnsSearch": null,
"Links": null,
"LxcConf": [],
"NetworkMode": "bridge",
"PortBindings": {
"5000/tcp": [
{
"HostIp": "",
"HostPort": "5000"
}
]
},
"Privileged": false,
"PublishAllPorts": false,
"RestartPolicy": {
"MaximumRetryCount": 0,
"Name": ""
},
"VolumesFrom": null
},
"HostnamePath": "/var/lib/docker/containers/db6454154305015c410c068e4d641d535fab1434edf91238ebd4bd5933fd159b/hostname",
"HostsPath": "/var/lib/docker/containers/db6454154305015c410c068e4d641d535fab1434edf91238ebd4bd5933fd159b/hosts",
"Id": "db6454154305015c410c068e4d641d535fab1434edf91238ebd4bd5933fd159b",
"Image": "cd3581c06bdcb2cfd877f79102b8e48a90e7fd723b4b4c6ee34d1f1dd6f64303",
"MountLabel": "",
"Name": "/dreamy_pike",
"NetworkSettings": {
"Bridge": "docker0",
"Gateway": "172.17.42.1",
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"PortMapping": null,
"Ports": {
"5000/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "5000"
}
]
}
},
"Path": "/bin/sh",
"ProcessLabel": "",
"ResolvConfPath": "/var/lib/docker/containers/db6454154305015c410c068e4d641d535fab1434edf91238ebd4bd5933fd159b/resolv.conf",
"State": {
"ExitCode": 0,
"FinishedAt": "2014-08-26T06:21:16.792306236Z",
"Paused": false,
"Pid": 5645,
"Restarting": false,
"Running": true,
"StartedAt": "2014-08-26T08:38:57.051361284Z"
},
"Volumes": {
"/docker/registry": "/var/lib/docker/vfs/dir/ae9ac9a2649acce729fde429586a7d1b4b7fe7ec834c12bea4b8b5519e2740a8"
},
"VolumesRW": {
"/docker/registry": true
}
}]
Posible answer:
Deleting images from a private docker registry
Cannot try if this works as I deleted my virtual machine.
But thanks anyway for the help Konrad Kleine
https://stackoverflow.com/users/835098/konrad-kleine

Resources