Nomad fails to start docker containers on macOs with colima - docker

Question

I am trying to get nomad to run on my mac with colima.
Colima seems to work just fine :
I can run and portforward to my host containers using docker with colima.
docker run --rm -p8080:80 nginx
I also configured nomad docker plugin to use the correct containerd socket :
plugin "docker" {
config {
// endpoint = "unix:///var/run/docker.sock"
// use colima
endpoint = "unix:///Users/antonin/.colima/default/docker.sock"
But nomad dev setup seems to be incorrect
I tried the sample application from nomad (a redis cache in a container), nomad agent fails to start the container. I guess there is too much network indirection ...
Here is my command to start nomad (according to nomad FAQ) :
sudo nomad agent -dev -config=docker.nomad -bind=0.0.0.0 -network-interface=en0
Here is the message logged by the nomad agent :
2023-02-15T22:57:24.796+0100 [DEBUG] client.driver_mgr.docker: failed to start container: driver=docker container_id=196030610c2795d6876386e799e03dd9f9afa3b391d6c093ab1d03616daf1949 attempt=5 error="API error (500): driver failed programming external connectivity on endpoint redis-bc2b6d1d-9241-ccb8-c5ad-3e649f018208 (df95904fed93d4be70fc0c3bf1682dbead84ba71f2a321443fba09ff0c43dd69): Error starting userland proxy: listen tcp4 192.168.1.120:28033: bind: cannot assign requested address"
192.168.1.120 is my host ip on my LAN.
additional information :
Here is the nomad job spec :
job "example" {
type = "service"
update {
max_parallel = 1
min_healthy_time = "10s"
healthy_deadline = "3m"
progress_deadline = "10m"
auto_revert = false
canary = 0
}
migrate {
max_parallel = 1
health_check = "checks"
min_healthy_time = "10s"
healthy_deadline = "5m"
}
group "cache" {
count = 1
network {
port "db" {
to = 6379
}
}
service {
name = "redis-cache"
tags = ["global", "cache"]
port = "db"
provider = "nomad"
check {
name = "alive"
type = "tcp"
interval = "10s"
timeout = "2s"
}
}
restart {
interval = "30m"
delay = "15s"
mode = "fail"
}
ephemeral_disk {
size = 300
}
task "redis" {
driver = "docker"
config {
image = "redis:7"
ports = ["db"]
auth_soft_fail = true
}
resources {
cpu = 200 # 500 MHz
memory = 100 # 256MB
}
}
}
}
Here is the output of docker info:
Client:
Context: colima
Debug Mode: false
Server:
Images: 2
Server Version: 20.10.20
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc version: 5fd4c4d144137e991c4acebb2146ab1483a97925
init version:
Security Options:
seccomp
Profile: default
Kernel Version: 5.15.82-0-virt
Operating System: Alpine Linux v3.16
OSType: linux
Architecture: aarch64
CPUs: 4
Total Memory: 3.828GiB
Name: colima
Docker Root Dir: /var/lib/docker
Debug Mode: false
Username: antonin42
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false