Im trying to use an docker image which is hosted on the docker hub registry with terraform. When i run terraform apply is says that it does not exist or docker login is required. And then it fails with an requested access to the resource is denied. What am i doing wrong?
main.tf:
provider "docker" {
host = "npipe:////.//pipe//docker_engine"
registry_auth {
address = "registry.hub.docker.com"
username = "username"
password = "passwword"
}
}
resource "docker_image" "ubuntu" {
name = "data.docker_registry_image.ubuntu.name"
pull_triggers = ["data.docker_registry_image.ubuntu.sha256_digest"]
}
data "docker_registry_image" "ubuntu" {
name = "ubuntu:latest"
}
versions.tf:
terraform {
required_providers {
docker = {
source = "terraform-providers/docker"
}
}
required_version = ">= 0.13"
}
error output:
Error: Unable to read Docker image into resource: Unable to pull image data.docker_registry_image.ubuntu.name: error pulling image data.docker_registry_image.ubuntu.name: Error response from daemon: pull access denied for data.docker_registry_image.ubuntu.name, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
on main.tf line 11, in resource "docker_image" "ubuntu":
11: resource "docker_image" "ubuntu" {
You need to remove the four quotation marks from within the resource "docker_image" "ubuntu" block.
Related
I am trying to setup auto deployment in Terraform to Docker of my golang server.
I have it working if I am deploying manually, but can't get the terraform config working.
Here is what I have.
resource "docker_image" "terraform_golang" {
name = "terraform_golang"
build {
path = "./../"
build_arg = {
tag : "golang-server"
}
label = {
author : "Bill"
}
}
}
resource "docker_container" "terraform_backend" {
image = docker_image.terraform_golang.latest
name = "Terraform_Backend"
env = [ "database=172.19.0.3:3306" ]
ports {
internal = 8080
external = 8080
}
network_mode = "bridge"
networks_advanced {
name = "golang-server"
}
}
This is the error I get:
│ Error: failed to solve with frontend dockerfile.v0: failed to read dockerfile: Error processing tar file(exit status 1): unexpected EOF
│
│
│
│ with module.terraform.docker_image.terraform_golang,
│ on terraform\backend.tf line 2, in resource "docker_image" "terraform_golang":
│ 2: resource "docker_image" "terraform_golang" {
Here is my manual deployment cmd, which is working:
docker build . -t golang-server
docker run --network=golang-server --name=golang-server --env database=172.19.0.3:3306 --expose=8080 -p=8080:8080/tcp golang-server
Here is my dockerfile:
FROM golang:1.18
WORKDIR $GOPATH/src
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
EXPOSE 8080
CMD ["golang-server"]
My folder structure is setup such that my terraform files are in a subfolder, in a module, I hope this doesn't mess with anything:
"path" only changes the context path, not the path where to find the docker file. Honestly feels like a bug since the docker command defaults to PATH/Dockerfile as a docker file path, but that's how the terraform plugin currently works.
What you need is to add a dockerfile parameter to point to the actual Dockerfile;
resource "docker_image" "terraform_golang" {
name = "terraform_golang"
build {
dockerfile = "../Dockerfile"
path = ".."
build_arg = {
tag : "golang-server"
}
label = {
author : "Bill"
}
}
}
I have a problem when I pull images from insecure registries(in my case, it is harbor). I receive a connection refused error when I use the command below.
sudo crictl pull 192.168.12.10/calculator/app2:new
Below is the full error message:
DEBU[0000] get image connection
DEBU[0000] connect using endpoint 'unix:///run/containerd/containerd.sock' with '2s' timeout
DEBU[0000] connected successfully using endpoint: unix:///run/containerd/containerd.sock
DEBU[0000] PullImageRequest: &PullImageRequest{Image:&ImageSpec{Image:192.168.12.10/calculator/app2:asdf,Annotations:map[string]string{},},Auth:nil,SandboxConfig:nil,}
DEBU[0000] PullImageResponse: nil
FATA[0000] pulling image: rpc error: code = Unknown desc = failed to pull and unpack image "192.168.12.10/calculator/app2:asdf": failed to resolve reference "192.168.12.10/calculator/app2:asdf": failed to do request: Head https://192.168.12.10/v2/calculator/app2/manifests/asdf: dial tcp 192.168.12.10:443: connect: connection refused
Added configuration to '/etc/containerd/config.toml' like this
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = ""
[plugins."io.containerd.grpc.v1.cri".registry.auths]
[plugins."io.containerd.grpc.v1.cri".registry.configs]
[plugins."io.containerd.grpc.v1.cri".registry.configs."192.168.12.10:80".tls]
insecure_skip_verify = true
[plugins."io.containerd.grpc.v1.cri".registry.configs."192.168.12.10:80".auth]
username = "admin12"
password = "Hada123213"
auth = "YWRtaW4xMjpIYWRhMTIzMjEzCg=="
[plugins."io.containerd.grpc.v1.cri".registry.headers]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.12.10:80"]
endpoint = ["http://192.168.12.10:80"]
To configure this file, I use the below like on stack overflow and github:
source one
source two
In my search on the web, more than 95 percent of websites said to add the insecure registry to /etc/docker/daemon.json to resolve this problem.
but its about docker not contaierd.
Per https://github.com/containerd/containerd/blob/main/docs/hosts.md
The old CRI config pattern for specifying registry.mirrors and registry.configs has been DEPRECATED. You should now point your registry config_path to the path where your hosts.toml files are located.
The right way to setup should be (in my example http://172.31.0.36 is the private insecure registry):
$ tree
.
├── certs.d
│ └── 172.31.0.36
│ └── hosts.toml
└── config.toml
where config.toml should include
version = 2
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"
and in hosts.toml:
server = "http://172.31.0.36"
[host."http://172.31.0.36"]
skip_verify = true
Restart the containerd service:
$ sudo systemctl restart containerd
To confirm it works:
$ sudo crictl pull 172.31.0.36/newpathfly/hello-world
Image is up to date for sha256:feb5d9fea6a5e9606aa995e879d862b825965ba48de054caab5ef356dc6b3412
in '/etc/containerd/config.toml' convert all of 192.168.12.10:80 to 192.168.12.10 so that will be working well.
because in your ctr command you refer to
sudo crictl pull 192.168.12.10/calculator/app2:new
and no refer to port number.
for more details look at this link.
We use gitlab-ci with docker images to test student code. This works fine with images published to docker-hub. We now want to test using images that are only available on the server.
The updated /etc/gitlab-runner/config.toml file with the settings shown below does not seem to be picked up, however, and we keep getting the error message shown below.
Any suggestions?
concurrent = 10
check_interval = 0
pull_policy = ["always", "if-not-present"]
ERROR: Job failed: Error response from daemon: pull access denied for vnijs/rsm-mgta453,
repository does not exist or may require 'docker login':
denied: requested access to the resource is denied (executor_docker.go:188:1s)
u should put pull_policy = ["always", "if-not-present"] under [[runners.docker]]
like this：
[[runners]]
name = "xxx"
url = "xxx"
token = "x"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
image = "pnpm-v6"
pull_policy = ["always", "if-not-present"]
I am trying to deploy apache docker image using Terraform on Kubernetes Cluster
I tried the following command and able to hit the URL localhost:8082 from browser successfully
docker run -it --rm -d -p 8082:80 webservice
I then created a kubernetes_deployment using Terraform but pod keeps crashing and there's nothing in logs
resource "kubernetes_deployment" "api" {
metadata {
name = "ex-api"
labels = {
app = "EX"
component = "api"
}
}
spec {
replicas = 1
selector {
match_labels = {
app = "EX"
}
}
template {
metadata {
name = "ex-api"
labels = {
app = "EX"
component = "api"
}
}
spec {
container {
image = "${var.web_service_image}:${var.web_service_image_tag}"
image_pull_policy = "IfNotPresent"
name = "api-image"
# All the other configuration options should be here too.
port {
container_port = 80
name = "web"
}
} # end of container block
} # end of spec block
} # end of template block
} # end of spec out-block
}
Pod's output
kubectl get pod
NAME READY STATUS RESTARTS AGE
ex-api-5458586bd8-ex6sp 0/1 CrashLoopBackOff 19 72m
I assume I should either add some command or daemonize (eg -itd when using docker) it so that it keeps running. I may be wrong here
Kindly let me know what should I do to overcome this
No logs or no events shown when you run the describe command generally suggests that that there is an issue with invoking your entrypoint in your Dockerfile. So, you may have to overwrite the command in your deployment.yaml
In your case - your deployment may need to use the command that you have or tried to use in your Dockerfile. Apparently, kubernetes pod is unable to use what you have defined in the Dockerfile.
i setup a dev-server in my homeoffice and installed gitlab via docker-compose. so far everything works fine, i can login, push commits and so on.
Now i wanted to setup a CI Pipeline to build composer packages when new tags are pushed. So i clicked the CI/CD Button and added the .gitlab-ci.yml file from the composer template. But the pipeline was only pending. So i figured i might need to register a runner first.
I installed gitlab-runner (via apt) on the same machine that runs the gitlab via docker and registered the runner with the domain and key given by gitlab (in the add runners page). I selected docker as executor, gave it a name and left everything else at its default value.
The runner is registered properly in gitlab and the ci pipeline is now working but it always fails.
The only output i have is:
Running with gitlab-runner 11.2.0 (11.2.0)
on **************
Using Docker executor with image curlimages/curl:latest ...
Pulling docker image gitlab-runner-helper:11.2.0 ...
The contents of the gitlab-ci file are:
# This file is a template, and might need editing before it works on your project.
# Publishes a tag/branch to Composer Packages of the current project
publish:
image: curlimages/curl:latest
stage: build
variables:
URL: "$CI_SERVER_PROTOCOL://$CI_SERVER_HOST:$CI_SERVER_PORT/api/v4/projects/$CI_PROJECT_ID/packages/composer?job_token=$CI_JOB_TOKEN"
script:
- version=$([[ -z "$CI_COMMIT_TAG" ]] && echo "branch=$CI_COMMIT_REF_NAME" || echo "tag=$CI_COMMIT_TAG")
- insecure=$([ "$CI_SERVER_PROTOCOL" = "http" ] && echo "--insecure" || echo "")
- response=$(curl -s -w "\n%{http_code}" $insecure --data $version $URL)
- code=$(echo "$response" | tail -n 1)
- body=$(echo "$response" | head -n 1)
# Output state information
- if [ $code -eq 201 ]; then
echo "Package created - Code $code - $body";
else
echo "Could not create package - Code $code - $body";
exit 1;
fi
Because i did not make any changes to the template file i suspect the gitlab-runner setup to need some configuration in order to work, maybe a group-assignment or something like that.
When running systemctl status gitlab-runner i can see:
Failed to create container volume for /builds/{group} Error response from daemon: pull access denied for gitlab-runner-helper, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (executor_docker.go:166:3s)" job=15 project=34 runner=******
So i went to the runners section in gitlab and enabled the runner fot the specific project. So i could avoid the error above but the pipeline still breaks.
The output in gitlab is still the same but the gitlab-runner log is different:
Not continuing with pull after error: errors:\ndenied: requested access to the resource is denied\nunauthorized: authentication required\n
Sadly - i am not getting any furhter from here
Everytime i press the retry button for the pipeline i get the following syslog entries:
Checking for jobs... received" job=19 repo_url="correct-url-for-repo" runner=******
This message appears twice
Not continuing with pull after error: errors:\ndenied: requested access to the resource is denied\nunauthorized: authentication required\n
Ignoring extra error returned from registry: unauthorized: authentication required
Failed to create container volume for /builds/{group} Error response from daemon: pull access denied for gitlab-runner-helper, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (executor_docker.go:166:3s)" job=19 project=34 runner=******
Job failed: Error response from daemon: pull access denied for gitlab-runner-helper, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (executor_docker.go:166:3s)" job=19 project=34 runner=******
Both messages appear twice
so either the gitlab-runner is not allowed to pull docker images or it is not allowed to access my gitlab project but i cant figure out the problem.
When running gitlab-runner restart as root i see the following "error"
ERRO[0000] Docker executor: prebuilt image helpers will be loaded from /var/lib/gitlab-runner.
Can someone please help me :) ?
Select the correct Docker image for the runner. Depending where are you executing it, and probably also depending on your GitLab version. Also, manually try it before executing the pipeline:
docker pull gitlab/gitlab-runner-helper:x86_64-latest
To use the selected image, modify the runner's config file:
[[runners]]
(...)
executor = "docker"
[runners.docker]
(...)
helper_image = "gitlab/gitlab-runner-helper:tag"
The images gitlab-runner-helper, gitlab/gitlab-runner-helper:11.2.0 do not exist. It seems the debian package installable in ubuntu is broken somehow... So i figured i might need to install the latest gitlab-runner version
Here is what i did: (I am on Ubuntu 20.04)
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
cat <<EOF | sudo tee /etc/apt/preferences.d/pin-gitlab-runner.pref
Explanation: Prefer GitLab provided packages over the Debian native ones
Package: gitlab-runner
Pin: origin packages.gitlab.com
Pin-Priority: 1001
EOF
Source
So i was able to update gitlab-runner to the latest version.
But still no success, now the service won't start without any error message, systemctl only tells mit that the process exited.
the syslog told me:
chdir /var/lib/gitlab-runner: no such file or directory
opening /etc/init.d/gitlab-runner showed me that path as --working-directory parameter for the service.
So i created that directory and changed its ownership to gitlab-runner
This time i could run the ci pipeline!
Still got an error
fatal: unable to access 'http://{mylocaldomain}/isat/typo3-gdpr.git/': Could not resolve host: {mylocaldomain}
Okay - dns resolution not possible because i use a local domain.
As stated here you can pass an extra_host to the docker executor.
To do so, simply adjust the /etc/gitlab-runner/config.toml file and add the extra_hosts option:
concurrent = 1
check_interval = 0
[[runners]]
name = "lab"
url = "http://{localDomain}/"
token = "******"
executor = "docker"
[runners.docker]
tls_verify = false
image = "ruby:2.1"
privileged = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
extra_hosts = ["{localDomain}:{ip}"]
[runners.cache]
Now i could sucessfully run the ci pipeline and my package is listed in the composer registry!