I have this terraform config:
locals {
service_set = toset(distinct(flatten([for _, v in flatten(fileset(path.module, "../services/**")) : basename(dirname(v))])))
}
resource "docker_image" "service" {
for_each = local.service_set
name = "${each.key}:latest"
keep_locally = true
}
resource "docker_container" "service" {
for_each = local.service_set
image = each.key
name = each.key
attach = true
logs = true
env = [
"Version=local",
]
depends_on = [
docker_container.mongo,
docker_container.minio,
docker_container.rabbitmq,
]
}
I'm trying to diagnose why my docker command isn't getting environment variables but for this I have to see the stdout and stderr of the container.
I set logs to true but I can't see where it gets saved too. What is the correct way to save the docker stdout and stderr to a file?
Related
I'm trying to upload a simple .yml file when creating an ECS task via Terraform, here is the code ./main.tf:
resource "aws_ecs_task_definition" "grafana" {
family = "grafana"
cpu = "256"
memory = "512"
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
container_definitions = jsonencode([
{
name = "grafana"
image = "grafana/grafana:latest"
portMappings = [
{
containerPort = 3000,
hostPort = 3000,
protocol = "tcp"
}
]
}
])
}
How do I go about adding ./datasource.yml (located on my host machine) to the container within the task definition so that when the task runs it can use it? I wasn't sure if volume { } could be used?
I think you have two alternatives here:
rebuild the docker image including your modified datasource.yaml.
COPY datasource.yaml /usr/share/grafana/conf/provisioning/datasource.yaml
or
mount a volume that you can easily mount and push files programmatically (EFS turns out to be a bit complicated to do this)
mount_points = [ {
sourceVolume = "grafana"
containerPath = "/var/lib/grafana/conf/provisioning"
readOnly = false
}
]
volumes = [
{
name = "grafana"
host_path = "/ecs/grafana-provisioning"}
]
I wasn't sure if volume { } could be used?
As a matter of fact you can, check the docs https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition#example-usage
volume {
name = "grafana-volume"
host_path = "./datasource.yml"
}
I have a create provisioner and a destroy provisioner. I've read that apparently, triggers might solve this problem, so they're integrated here, but while this succeeds to build the resources, it doesn't let them destroy this module.thingx.null_resource.script-stuff resource.
I'm not sure I'm using triggers correctly here, and it's more confusing that create time apply works fine, but the destroy time apply fails with the mentioned error.
Here is the module null resource that apparently the error is referring to; includes both the create and destroy time provisioners:
resource "null_resource" "script-stuff" {
### -- testing triggers
triggers = {
dns_zones = var.dns_zones[each.key]
dnat_ip = google_compute_instance.server[each.key].network_interface.0.access_config.0.nat_ip
pem = tls_private_key.node_ssh[each.key].private_key_pem
} ### -- end testing
depends_on = [google_compute_instance.server, google_project_iam_member.list-sa]
for_each = var.list_map
provisioner "remote-exec" {
when = create
inline = [
"cat ${var.dns_zones[each.key]} > /dev/null",
"sensitive-script.sh --create"
]
connection {
type = "ssh"
host = google_compute_instance.server[each.key].network_interface[0].access_config[0].nat_ip
user = "common-user"
private_key = tls_private_key.node_ssh[each.key].private_key_pem
}
}
provisioner "remote-exec" {
when = destroy
inline = [
# "echo ${var.dns_zones[each.key]} > /dev/null", #<-- this doesn't work when terraform is destroying
"echo ${self.triggers.dns_zones[each.key]} > /dev/null",
"sensitive-script.sh --destroy"
]
connection {
type = "ssh"
#host = google_compute_instance.server[each.key].network_interface[0].access_config[0].nat_ip #<-- this doesn't work when terraform is destroying
host = self.triggers.dnat_ip
user = "common-user"
#private_key = tls_private_key.node_ssh[each.key].private_key_pem #<-- this doesn't work when terraform is destroying
private_key = self.triggers.pem
}
}
}
destroy triggered provisioners do not support variables as explained in this GitHub issue:
Allow destroy-time provisioners to access variables
So you can't have any variable in "echo ${var.dns_zones[each.key]} > /dev/null".
with following job config. curl NOMAD_IP_http:NOMAD_PORT_http cannot access http-echo service.
there is no listenig port on localhost for incomming request.
why and how to access the http-echo service
job "job" {
datacenters = ["dc1"]
group "group" {
count = 2
network {
port "http" {}
}
service {
name = "http-echo"
port = "http"
tags = [
"http-echo",
]
check {
type = "http"
path = "/health"
interval = "30s"
timeout = "2s"
}
}
task "task" {
driver = "docker"
config {
image = "hashicorp/http-echo:latest"
args = [
"-listen", ":${NOMAD_PORT_http}",
"-text", "Hello and welcome to ${NOMAD_IP_http} running on port ${NOMAD_PORT_http}",
]
}
resources {}
}
}
}
UPDATE
after config driver network_mode, curl successfully.
network_mode = "host"
You forgot to add ports at job -> group -> task ->ports
Now it works on latest nomad(v1.1.3+).
job "job" {
datacenters = ["dc1"]
group "group" {
count = 2
network {
port "http" {}
# or maps to container's default port
# port "http" {
# to = 5678
# }
#
}
service {
name = "http-echo"
port = "http"
tags = [
"http-echo",
]
check {
type = "http"
path = "/health"
interval = "30s"
timeout = "2s"
}
}
task "task" {
driver = "docker"
config {
image = "hashicorp/http-echo:latest"
args = [
"-listen", ":${NOMAD_PORT_http}",
"-text", "Hello and welcome to ${NOMAD_IP_http} running on port ${NOMAD_PORT_http}",
]
ports = ["http"]
}
resources {}
}
}
}
Then run docker ps, you will get the mapped port, and curl works.
I'm currently running a docker container with Traefik as the load balancer using the following docker-compose file:
services:
loris:
image: bdlss/loris-grok-docker
labels:
- traefik.http.routers.loris.rule=Host(`loris.my_domain`)
- traefik.http.routers.loris.tls=true
- traefik.http.routers.loris.tls.certresolver=lets-encrypt
- traefik.port=80
networks:
- web
It is working fairly well. As part of one my first attempts using Nomad, I simply want to be able to start this container using a nomad job loris.nomad instead of using the docker-compose file.
The Docker container 'Labels' and the 'Network' identification are quite important for Traefik to do the dynamic routing.
My question is: where can I put this "label" information and "network" information in the loris.nomad file so that it starts the container in the same way that the docker-compose file currently does.
I've tried putting this information in the task.config stanza but this doesn't work and I'm having trouble following the documentation. I've seen examples where an additional "service" stanza has been added, but I"m still not sure.
Here's the basics of that nomad file I want to modify.
# loris.nomad
job "loris" {
datacenters = ["dc1"]
group "loris" {
network {
port "http" {
to = 5004
}
task "loris" {
driver = "docker"
config {
image = "bdlss/loris-openjpeg-docker"
ports = ["http"]
}
resources {
cpu = 500
memory = 512
}
}
}
}
Any advice is much appreciated.
Well, the most appropriate option for running traefik in nomad and load-balance between containers is using consul catalog (required for service discovery).
For this to run you have to confgure the consule connection when you start nomad. If you like to test things out locally you can do this by simply running sudo nomad agent -dev-connect. Consul can be started with consul agent -dev -client="0.0.0.0".
Now you can simply provide your traefik configuration using tags as it is shown here.
If you really need (which will cause issues in a clustered setup for sure) to run traefik in nomad with docker provider you can do the following:
First you need to enable host path mounting in the docker plugin. See this and this. You can place your configuration in an extra file like extra.hcl which looks like this:
plugin "docker" {
config {
volumes {
enabled = true
}
}
}
Now you can start nomad with this extra setting sudo nomad agent -dev-connect -config=extra.hcl. Now you can provide your traefik settings in the config/labels block, like (full):
job "traefik" {
region = "global"
datacenters = ["dc1"]
type = "service"
group "traefik" {
count = 1
task "traefik" {
driver = "docker"
config {
image = "traefik:v2.3"
//network_mode = "host"
volumes = [
"local/traefik.yaml:/etc/traefik/traefik.yaml",
"/var/run/docker.sock:/var/run/docker.sock"
]
labels {
traefik.enable = true
traefik.http.routers.from-docker.rule = "Host(`docker.loris.mydomain`)"
traefik.http.routers.from-docker.entrypoints = "web"
traefik.http.routers.from-docker.service = "api#internal"
}
}
template {
data = <<EOF
log:
level: DEBUG
entryPoints:
traefik:
address: ":8080"
web:
address: ":80"
api:
dashboard: true
insecure: true
accessLog: {}
providers:
docker:
exposedByDefault: false
consulCatalog:
prefix: "traefik"
exposedByDefault: false
endpoint:
address: "10.0.0.20:8500"
scheme: "http"
datacenter: "dc1"
EOF
destination = "local/traefik.yaml"
}
resources {
cpu = 100
memory = 128
network {
mbits = 10
port "http" {
static = 80
}
port "traefik" {
static = 8080
}
}
}
service {
name = "traefik"
tags = [
"traefik.enable=true",
"traefik.http.routers.from-consul.rule=Host(`consul.loris.mydomain`)",
"traefik.http.routers.from-consul.entrypoints=web",
"traefik.http.routers.from-consul.service=api#internal"
]
check {
name = "alive"
type = "tcp"
port = "http"
interval = "10s"
timeout = "2s"
}
}
}
}
}
(There might be a setting to bind to 0.0.0.0 I defined those domains in my /etc/hosts to point to my main interface IP).
You can test it with this modified webapp spec (I didn't figure out how to map ports correctly, like container:80 -> host:<random>, but I think it is enough to show how complicated it gets :)):
job "demo-webapp" {
datacenters = ["dc1"]
group "demo" {
count = 3
task "server" {
env {
// "${NOMAD_PORT_http}"
PORT = "80"
NODE_IP = "${NOMAD_IP_http}"
}
driver = "docker"
config {
image = "hashicorp/demo-webapp-lb-guide"
labels {
traefik.enable = true
traefik.http.routers.webapp-docker.rule = "Host(`docker.loris.mydomain`) && Path(`/myapp`)"
traefik.http.services.webapp-docker.loadbalancer.server.port = 80
}
}
resources {
network {
// Used for docker provider
mode ="bridge"
mbits = 10
port "http"{
// Used for docker provider
to = 80
}
}
}
service {
name = "demo-webapp"
port = "http"
tags = [
"traefik.enable=true",
"traefik.http.routers.webapp-consul.rule=Host(`consul.loris.mydomain`) && Path(`/myapp`)",
]
check {
type = "http"
path = "/"
interval = "2s"
timeout = "2s"
}
}
}
}
}
I hope this somehow answers your question.
I am on a Windows machine using Terraform 0.13.4 and trying to spin up some containers on a remote host using Terraform and the Docker provider:
provider "docker" {
host = "tcp://myvm:2376/"
registry_auth {
address = "myregistry:443"
username = "myusername"
password = "mypassword"
}
ca_material = file(pathexpand(".docker/ca.pem"))
cert_material = file(pathexpand(".docker/cert.pem"))
key_material = file(pathexpand(".docker/key.pem"))
}
data "docker_registry_image" "mycontainer" {
name = "myregistry:443/lvl1/lvl2/myimage:latest"
}
I am having a hard time with this as it cannot authenticate with my private registry. Always getting 401 Unauthorized.
If I don't do this to grab the sha256_digest and just use the docker_container resource, everything works but it forces replacements of the running containers.
Hello Angelos if you dont want to force replace the running container you should try this :
provider "docker" {
host = "tcp://myvm:2376/"
registry_auth {
address = "myregistry:443"
username = "myusername"
password = "mypassword"
}
ca_material = file(pathexpand(".docker/ca.pem"))
cert_material = file(pathexpand(".docker/cert.pem"))
key_material = file(pathexpand(".docker/key.pem"))
}
data "docker_registry_image" "mycontainer" {
name = "myregistry:443/lvl1/lvl2/myimage:latest"
}
resource "docker_image" "example" {
name = data.docker_registry_image.mycontainer.name
pull_triggers = [data.docker_registry_image.mycontainer.sha256_digest]
keep_locally = true
}
then in the container use :
resource "docker_container" "example" {
image = docker_image.example.latest
name = "container_name"
}
you shoukd use
docker_image.example.latest
Using the resource docker_image itself if it already exist he wont pull the image and doesn't restart the container but if you pass the name as a string he will replace the container everytime.
https://www.terraform.io/docs/providers/docker/r/container.html
Turns out that the code is correct and that the container service I am using (older version of ProGet) is not replying correctly for the auth calls. I tested the code using another registry and it all works as expected.