Can I change the port a GitLab runner uses to pull from my self-hosted GitLab instance? - docker

I have two virtual machines both running docker. On one, I am hosting a GitLab instance according to https://docs.gitlab.com/ee/install/docker.html. On the other, I have a GitLab runner running inside a container https://docs.gitlab.com/runner/install/docker.html.
Due to some port constraints, I am running the GitLab instance on non-standard ports (4443 instead of 443, for instance).
I am able to successfully register the runner, and GitLab can send a job that the runner will pick up. However, when that runner pulls the git repo it is apparently looking at the wrong port for that git pull:
My GitLab is on port 4443 not port 443. The GitLab runner config has the correct port in the url field and is, again, able to connect and receive jobs.
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "runner1"
url = "https://vm-pr01:4443/"
token = "egZUzK44hYVrhy6DTfey"
tls-ca-file = "/etc/gitlab-runner/certs/cert.crt"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "docker:20.10.16"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
Finally, I did try to change the GitLab ssh port according to this question:
Gitlab with non-standard SSH port (on VM with Iptable forwarding)
But that wasn't effective after restarting GitLab.
Are there other angles to try? My last thought was to host my own docker image with the correct ports changed in SSH config but I imagine there's a better way.

Use the clone_url configuration for the runner to change this.
[[runners]]
# ...
clone_url = "https://hostname:port"
Make sure the scheme matches your instance (http or https).
For whatever reason, the default clone url will not precisely respect the setting from url (scheme and port are assumed) so you must provide both url and clone_url in your configuration scenario.

Related

GitLab pipeline exiting with error code 137 when running Cypress

I'm creating a Docker image based on alpine:3.13, which is used for my test stage, running in a pipeline on GitLab.
There I install all the dependencies. The app consists of two components, which I will call front and back.
I run the following command to set up front and back and finally execute cypress in headless mode.
"e2e:run": "concurrently -n front, back \"yarn front\" \"yarn back\" \"yarn front:wait && yarn back:wait && yarn cypress:run\""
It builds front and back fine, but then the job log doesn't show any progress for a few minutes until I finally get this exit code:
ERROR: Job failed: command terminated with exit code 137
From my research so far, I concluded it seems to be related to a lack of memory.
Is there any other reasonable option?
What could I do to provide more memory/reduce memory consumption?
As #SamBob mentioned, this issue is likely due to low memory within the running docker container, and the shm_size parameter can increase it. However, since you're not directly running your image in the job (ie, doing docker run...) but rather the gitlab-runner process is, you'll have to set the shm_size parameter within the Runner's configuration for the Docker executor. To do this, you'll also have to run your own runners if you aren't already.
When running your own runners, each will have a config.toml file in /etc/gitlab-runner that looks like this by default:
listen_address = ":9252"
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "runner-1"
url = "https://gitlab.example.com"
token = "TOKEN"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.docker]
image = "alpine:latest"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
As you can see, by default the shm_size parameter is set to 0 bytes. You can edit this file to increase the shm_size, then restart the gitlab-runner service to reload the new config.
One other thing I do with my runners is to add a shm-increased tag on those runners I've increased since only a couple jobs in my pipelines need more shared memory.
To see more information on running your own Gitlab Runners, see here.
To see more on the shm_size parameter for Gitlab Runners, and other advanced runner configuration options, see here.
To see information on tagging runners and jobs, see here.

GitLab with Docker runner on localhost: how to expose host to container?

I'm learning to use GitLab CI.
Just now I'm using GitLab on localhost (external_url "http://localhost"). And I've registered a Docker runner with vanilla ubuntu:20.04 image and tried to run some test job on it.
Alas, it tries to clone my repo from localhost repository in the container, but cannot do it, because my localhost's port 80 is not visible from container.
Running with gitlab-runner 13.5.0 (ece86343)
on docker0 x8pHJPn7
Preparing the "docker" executor
Using Docker executor with image ubuntu:20.04 ...
Pulling docker image ubuntu:20.04 ...
Using docker image sha256:d70eaf7277eada08fca944de400e7e4dd97b1262c06ed2b1011500caa4decaf1 for ubuntu:20.04 with digest ubuntu#sha256:fff16eea1a8ae92867721d90c59a75652ea66d29c05294e6e2f898704bdb8cf1 ...
Preparing environment
Running on runner-x8phjpn7-project-6-concurrent-0 via gigant...
Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/root/ci_fuss/.git/
fatal: unable to access 'http://localhost:80/root/ci_fuss.git/': Failed to connect to localhost port 80: Connection refused
Uploading artifacts for failed job
Uploading artifacts...
WARNING: report.xml: no matching files
ERROR: No files to upload
Cleaning up file based variables
ERROR: Job failed: exit code 1
How can I can my Docker runner to expose host's localhost:80 as container's localhost:80?
Well, i have coped with this stuff.
I have added network_mode = "host"to my runner configuration in /etc/gitlab-runner/config.toml to make my docker use host network connections.
Also I've added --pull_policy="if-not-present" to first search for container image locally, then in remote repo.
[[runners]]
name = "docker0"
url = "http://localhost/"
token = "TTBRFis_W_yJJpN1LLzV"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "exposed_ctr:latest"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
network_mode = "host"
pull_policy = "if-not-present"

gitlab runner - network_mode = "host"

I want to setup CI/CD in GitLab.
So i installed docker and the gitlab-runner on linux, created a config for a runner and started everything. So far so good.
The runner works, and docker works.
But i am using the linux subsystem from windows, so i need to run the docker container with parameter "--network host" otherwise they not gonna work.
So right now i try to configure the gitlab-runner to use the host network via the "network_mode" parameter. But it does not work. I get the same error as if i would run a docker container directly and without the "--network host".
The error:
WARNING: Preparation failed: Error response from daemon: oci runtime error: container_linux.go:265: starting container process caused "process_linux.go:368: container init caused \"process_linux.go:351: running prestart hook 0 caused \\"error running hook: exit status 1, stdout: , stderr: time=\\\\"2019-04-12T18:42:33+02:00\\\\" level=fatal msg=\\\\"failed to add interface vethfc7c8d1 to sandbox: failed to get link by name \\\\\\\\"vethfc7c8d1\\\\\\\\": Link not found\\\\" \\n\\"\"" (executor_docker.go:423:16s) job=123project=123 runner=123
This is my config:
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "MyHostName"
url = "https://my.gitlab.url/"
token = "SoMeFaNcYcOdE-e"
executor = "docker"
[runners.docker]
tls_verify = false
image = "beevelop/ionic:latest"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
network_mode = "host"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
My question is how do i force the gitlab runner to create the containers to use the host network like with the docker parameter: "--network host"
I was unable to solve the problem directly, but i found an alternative way which is a lot better.
I configured the GitLab Container Registry
of the repository to upload and white list a custom docker image and then enabled the Shared Runners of my company. The custom image i uploaded was created via a Dockerfile using docker for windows, avoiding the struggle of the buggy docker in the linux subsystem of windows. Now i can execute my CI pipeline flawlessly and have full control over the used image and do not have to keep my local machine running.

Gitlab runner docker host setting

How should i enter the "host" value for the host params?
https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section
Thanks in advance!
I tried tcp://0.0.0.0:2375, 0.0.0.0:2375, 0.0.0.0, etc and all result in errors.
[runners.docker]
host = tcp://0.0.0.0:2375
tls_verify = false
image = "docker:latest"
The runner toml config file should be accepted without any error
You should leave it blank, if you run the command on your PC.
The format is 'ip-address:port-number'.
You can set the value of the host to 127.0.0.1:2375
[runners.docker]
host=127.0.0.1:2375

Gitlab-runner docker container is using the Gitlab container_id as the clone url

I am trying to configure a simple Gitlab-ci build pipeline and am running all of the components in docker containers. I followed the general guides on docs.gitlab.com and got a runner registered with gitlab. But when a build kicks off, the runner tries to clone the repository in question and seems to use the gitlab instance's container-id in place of the url, and I get an unreachable-host error:
Cloning repository...
Cloning into '/builds/root/ci-demo'...
fatal: unable to access 'http://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx#cdfd596f2bc4/root/ci-demo.git/': Could not resolve host: cdfd596f2bc4
ERROR: Job failed: exit code 1
Is there something obvious that I've overlooked? There are quite a few similar questions on SO and the internet in general, but none seem to have a problem with the target container-id being substituted for the url.
gitlab-runner's config.toml:
concurrent = 1
check_interval = 0
[[runners]]
name = "runner_name"
url = "http://[ipaddr]:[port]/"
token = "xxxxxxx"
executor = "docker"
[runners.docker]
tls_verify = false
image = "maven:latest"
privileged = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
[runners.cache]

Resources