Access GitLab running in a docker container - docker

I'm trying to set up GitLab as a docker container in an internal server. Let's assume the server's IP is 10.10.10.10. Below is my docker-compose file that I use to bring up the container. I'm unable to access the http url via localhost:4080 (from a browser within the server) OR via the IP 10.10.10.10:4080. I'd like to understand what I'm missing here.
version: '2'
services:
gitlab:
image: gitlab-ee-img:12.0.9-ee.0
container_name: gitlab
restart: always
hostname: 'localhost:4080'
# environment:
# GITLAB_OMNIBUS_CONFIG: |
# # Add any other gitlab.rb configuration here, each on its own line
# # external_url 'https://gitlab.example.com'
# external_url 'http://127.0.0.1:4080'
ports:
- '4080:80'
- '4443:443'
- '4022:22'
volumes:
- '/data/gitlab/config:/etc/gitlab'
- '/data/gitlab/logs:/var/log/gitlab'
- '/data/gitlab/data:/var/opt/gitlab'

Not entirely sure about if there's something else not working there but I'm fairly sure that the hostname: 'localhost:4080' block is not correct. It should be just the hostname without a port. Try to comment out that line and try without defining a hostname at all for testing.
src: https://docs.docker.com/compose/compose-file/#domainname-hostname-ipc-mac_address-privileged-read_only-shm_size-stdin_open-tty-user-working_dir

For anyone hitting this SO question:
The answer is to NOT map a custom port on to 80 in docker. Instead, this will work:
version: '2'
services:
gitlab:
image: gitlab-ee-img:12.0.9-ee.0
container_name: gitlab
restart: always
hostname: '10.10.10.10'
environment:
GITLAB_OMNIBUS_CONFIG: |
# Add any other gitlab.rb configuration here, each on its own line
# external_url 'https://gitlab.example.com'
external_url 'http://10.10.10.10:4080'
gitlab_rails['gitlab_shell_ssh_port'] = 4022
ports:
- '4080:4080'
- '4443:443'
- '4022:22'
volumes:
- '/data/gitlab/config:/etc/gitlab'
- '/data/gitlab/logs:/var/log/gitlab'
- '/data/gitlab/data:/var/opt/gitlab'
The reason is explained in this thread - Specifically, this answer
To summarize here (quoting the original answer),
The default port of gitlab is 80, but when you use the external_url
clause, gitlab changes the ngnix port to which it is going to listen
or resolve, it is not just an alias.
If you inside the container executes the command curl
http://localhost, after having placed external_url
http://10.10.10.10:4080, it will not answer for port 80, you can try
with the command curl http://10.10.10.10:4080

Related

Repo file links of gitlab-ce service in docker container redirect to 404 page

I try to run gitlab as a docker service on my local machine. The git and web functionality seems to work fine. Except when I try to open a file in a repo to view the content I'm redirected to a 404 page.
The URL which was generated to access the file looks as follows:
http://www/user/tesproject/-/blob/master/file.py
If I change the URL manually to:
http://192.168.122.168:8080/user/tesproject/-/blob/master/file.py
... everything works as expected. (192.168.122.168 is the host's ip.)
Does anybody know what changes I have to make so that it redirects to the correct URL?
My guess is, that domainname and hostname in my docker-compose file is somehow wrong.
My docker-compose.yml:
version: '3.7'
services:
# GitLab
gitlab-web:
image: gitlab/gitlab-ce:latest
restart: always
container_name: gitlabweb
domainname: gitlabweb.dev
hostname: www
volumes:
- ./gitlab-config:/etc/gitlab
- ./gitlab-logs:/var/log/gitlab
- ./gitlab-data:/var/opt/gitlab
ports:
- '2222:22'
- '8080:80'
- '443:443'
- '4567:4567'
environment:
GITLAB_OMNIBUS_CONFIG: |
gitlab_rails['gitlab_shell_ssh_port'] = 2222
registry_external_url 'http://localhost:4567'
registry['enable'] = true
unicorn['socket'] = '/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket'
networks:
- gitlab-network
networks:
gitlab-network:
name: gitlab-network
My /etc/hosts on the local host contains:
...
192.168.96.2 www.gitlabweb.dev
...
Pinging www.gitlabweb.dev works. (192.168.96.2 is the gitlab container's ip.)
I use:
Ubuntu Server 20.04,
Docker version 19.03.11
Thanks for your help!
Installing the docker image gitlab/gitlab-ce version 13.4.4 solved the problem. Before I had 13.2.4.

How to change target of the Spring Cloud Stream Kafka binder?

Using Spring cloud Stream 2.1.4 with Spring Boot 2.1.10, I'm trying to target a local instance of Kafka.
This is an extract of my projetc configuation so far:
spring.kafka.bootstrap-servers=PLAINTEXT://localhost:9092
spring.kafka.streams.bootstrap-servers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.binder.brokers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.binder.zkNodes=localhost:2181
spring.cloud.stream.kafka.streams.binder.brokers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.streams.binder.zkNodes=localhost:2181
But the binder keeps on calling a wrong target :
java.io.IOException: Can't resolve address: kafka.example.com:9092
How can can I specify the target if those properties won't do he trick?
More, I deploy the Kafka instance through a Docker Bitnami image and I'd prefer not to use SSL configuration (see PLAINTEXT protocol) but I'm don't find properties for basic credentials login. Does anyone know if this is hopeless?
This is my docker-compose.yml
version: '3'
services:
zookeeper:
image: bitnami/zookeeper:latest
container_name: zookeeper
environment:
- ZOO_ENABLE_AUTH=yes
- ZOO_SERVER_USERS=kafka
- ZOO_SERVER_PASSWORDS=kafka_password
networks:
- kafka-net
kafka:
image: bitnami/kafka:latest
container_name: kafka
hostname: kafka.example.com
depends_on:
- zookeeper
ports:
- 9092:9092
environment:
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://:9092
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_ZOOKEEPER_USER=kafka
- KAFKA_ZOOKEEPER_PASSWORD=kafka_password
networks:
- kafka-net
networks:
kafka-net:
driver: bridge
Thanks in advance
The hostname isn't the issue, rahter the advertised listeners protocol//:port mapping that causes the hostname to be advertised, by default. You should change that, rather than the hostname.
kafka:
image: bitnami/kafka:latest
container_name: kafka
hostname: kafka.example.com # <--- Here's what you are getting in the request
...
environment:
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://:9092 # <--- This returns the hostname to the clients
If you plan on running your code outside of another container, you should advertise localhost in addition to, or instead of the container hostname.
One year later, my comment still is not been merged into the bitnami README, where I was able to get it working with the following vars (changed to match your deployment)
KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_CFG_LISTENERS=PLAINTEXT://:29092,PLAINTEXT_HOST://:9092
KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka.example.com:29092,PLAINTEXT_HOST://localhost:9092
All right: got this to work by looking twice to the "dockerfile" (thx to cricket_007):
kafka:
...
hostname: localhost
For the record: I could get rid of all properties above, default being for Kafka localhost:9092

Expose port in docker-compose or configure second letsencrypt certificate

I'm running a selfhosted gitlab docker instance, but I'm facing some problems configuring the registry as I do get the error
Error response from daemon: Get https://example.com:4567/v2/: dial tcp <IP>:4567: connect: connection refused
for doing docker login example.com:4567.
So it seems that I have to expose the port 4567 somehow.
An (better) alternative would be to configure a second domain for the registry - like registry.example.com. As you can see below I'm using letsencrypt certificates for my gitlab instance. But how do I get a second certificate for the registry?
This is how my docker-compose looks like - I'm using jwilder/nginx-proxy for my reverse proxy.
docker-compose.yml
gitlab:
image: gitlab/gitlab-ce:11.9.0-ce.0
container_name: gitlab
networks:
- reverse-proxy
restart: unless-stopped
ports:
- '50022:22'
volumes:
- /opt/gitlab/config:/etc/gitlab
- /opt/gitlab/logs:/var/log/gitlab
- /opt/gitlab/data:/var/opt/gitlab
- /opt/nginx/conf.d:/etc/nginx/conf.d
- /opt/nginx/certs:/etc/nginx/certs:ro
environment:
VIRTUAL_HOST: example.com
VIRTUAL_PROTO: https
VIRTUAL_PORT: 443
LETSENCRYPT_HOST: example.com
LETSENCRYPT_EMAIL: certs#example.com
gitlab.rb
external_url 'https://example.com'
nginx['redirect_http_to_https'] = true
nginx['ssl_certificate'] = '/etc/nginx/certs/example.com/fullchain.pem'
nginx['ssl_certificate_key'] = '/etc/nginx/certs/example.com/key.pem'
gitlab_rails['backup_keep_time'] = 604800
gitlab_rails['backup_path'] = '/backups'
gitlab_rails['registry_enabled'] = true
registry_external_url 'https://example.com:4567'
registry_nginx['ssl_certificate'] = "/etc/nginx/certs/example.com/fullchain.pem"
registry_nginx['ssl_certificate_key'] = "/etc/nginx/certs/example.com/key.pem"
For the second alternative it would look like:
registry_external_url 'https://registry.example.com'
registry_nginx['ssl_certificate'] = "/etc/nginx/certs/registry.example.com/fullchain.pem"
registry_nginx['ssl_certificate_key'] = "/etc/nginx/certs/registry.example.com/key.pem"
But how do I set this up in my docker-compose?
Update
Im configuring nginx just via jwilder package, without changing anyhting. So this part of my docker-compose.yml file just looks like this:
services:
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx-proxy
networks:
- reverse-proxy
ports:
- "80:80"
- "443:443"
volumes:
- /opt/nginx-proxy/vhost.d:/etc/nginx/vhost.d:rw
- /opt/nginx/certs:/etc/nginx/certs:ro
- html:/usr/share/nginx/html
- /var/run/docker.sock:/tmp/docker.sock:ro
nginx-letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: nginx-letsencrypt
networks:
- reverse-proxy
depends_on:
- nginx-proxy
volumes:
- /opt/nginx-proxy/vhost.d:/etc/nginx/vhost.d:rw
- html:/usr/share/nginx/html
- /opt/nginx/certs:/etc/nginx/certs:rw
- /var/run/docker.sock:/var/run/docker.sock:rw
environment:
NGINX_PROXY_CONTAINER: "nginx-proxy"
TL; DR:
So it seems that I have to expose the port 4567 somehow.
Yes, however jwilder/nginx-proxy does not support more than one port per virtual host and port 443 is already exposed. There is a pull request for that feature but it has not been merged yet. You'll need to expose this port another way (see below)
You are using jwilder/nginx-proxy as reverse proxy to access a Gitlab instance in a container but with your current configuration onlyport 443 is exposed:
environment:
VIRTUAL_HOST: example.com
VIRTUAL_PROTO: https
VIRTUAL_PORT: 443
All other Gitlab services (including the registry on port 4567) are not proxied and therefore not reachable through example.com.
Unfortunately it is not possible yet to expose multiple port on a single hostname with jwilder/nginx-proxy. There is a pull request open for that use case but it had not been merged yet (you are not the only one with this kind of issue).
An (better) alternative would be to configure a second domain for the registry
This won't work if you keep using jwilder/nginx-proxy as even if you changed registry_external_url, you'll still be stuck with the port issue, and you cannot allocate the same port to two different services.
What you can do:
vote and comment for mentioned PR to be merged :)
try to build the Docker image from mentionned pull request's fork and configure your compose with something like VIRTUAL_HOST=example.com:443,example.com:4567
configure a reverse proxy manually fort port 4567 - you may wind-up a plain nginx container in addition with your current configuration which would specifically do this, or re-configure your entire proxying scheme without using jwilder images
update your configuration to expose example.com:4567 instead of example.com:443 but you'll lose HTTPS access. (though it's probably not what you are looking for)
I am aware this does not provide a finite solution but I hope it helps.

Running Gitlab in Docker

I want to host a private Gitlab server on my Debian VPS. I figured using Docker would be a good setup.
I tried running Gitlab with the following code:
version: '3'
services:
gitlab:
image: 'gitlab/gitlab-ce'
restart: always
hostname: 'gitlab.MYDOMAIN.com'
links:
- postgresql:postgresql
- redis:redis
environment:
GITLAB_OMNIBUS_CONFIG: |
postgresql['enable'] = false
gitlab_rails['db_username'] = "gitlab"
gitlab_rails['db_password'] = "gitlab"
gitlab_rails['db_host'] = "postgresql"
gitlab_rails['db_port'] = "5432"
gitlab_rails['db_database'] = "gitlabhq_production"
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'utf8'
redis['enable'] = false
gitlab_rails['redis_host'] = 'redis'
gitlab_rails['redis_port'] = '6379'
external_url 'http://gitlab.MYDOMAIN.com:30080'
gitlab_rails['gitlab_shell_ssh_port'] = 30022
ports:
# both ports must match the port from external_url above
- "30080:30080"
# the mapped port must match ssh_port specified above.
- "30022:22"
# the following are hints on what volumes to mount if you want to persist data
# volumes:
# - data/gitlab/config:/etc/gitlab:rw
# - data/gitlab/logs:/var/log/gitlab:rw
# - data/gitlab/data:/var/opt/gitlab:rw
postgresql:
restart: always
image: postgres:9.6.2-alpine
environment:
- POSTGRES_USER=gitlab
- POSTGRES_PASSWORD=gitlab
- POSTGRES_DB=gitlabhq_production
# the following are hints on what volumes to mount if you want to persist data
# volumes:
# - data/postgresql:/var/lib/postgresql:rw
redis:
restart: always
image: redis:3.0.7-alpine
Running this (docker-compose run -d) allows me to reach Gitlab on MYDOMAIN.com:30080, but not on gitlab.MYDOMAIN.com:30080.
Have I made an error in the configuration? Or do I need to use reverse proxies (NGINX or Traefik)?
I'm pretty sure the hostname: gitlab.MYDOMAIN.rocks needs to match the external_url 'http://gitlab.MYDOMAIN.com:30080' until the port exactly
So for example:
hostname: gitlab.MYDOMAIN.com
. . . more configuration . . .
external_url 'http://gitlab.MYDOMAIN.com:30080'
Did you check that the subdomain gitlab in dns is pointing to the right ip? Looks like an infrastructure problem more than a docker configuration one.
Regards
I managed to fix it myself!
I totally forgot to add an A-record, setting gitlab.mydomain.com to point to the same IP address as #.
I added the following block to the nginx configuration:
upstream gitlab.mydomain.com {
server 1.2.3.4:30080; # IP address of Docker container
}
server {
server_name gitlab.mydomain.com;
location / {
proxy_pass http://gitlab.mydomain.com;
}
}
I use upstream because otherwise the url set in new Gitlab projects is set to the IP address, as mentioned here.

docker and jwilder/nginx-proxy http/https issue

I'm using docker on osx via boot2docker.
I have 2 hosts: site1.loc.test.com and site2.loc.test.com pointed to ip address of docker host.
Both should be available via 80 and 443 ports.
So I'm using jwilder/nginx-proxy for reverse proxy purposes.
But in fact when I'm running all of them via docker-compose every time I try to open via 80 port I get redirect to 443 (301 Moved Permanently).
May be I've missed something in jwilder/nginx-proxy configuration?
docker-compose.yml
proxy:
image: jwilder/nginx-proxy
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- certs:/etc/nginx/certs
ports:
- "80:80"
- "443:443"
site1:
image: httpd:2.4
volumes:
- site1:/usr/local/apache2/htdocs
environment:
VIRTUAL_HOST: site1.loc.test.com
expose:
- "80"
site2:
image: httpd:2.4
volumes:
- site2:/usr/local/apache2/htdocs
environment:
VIRTUAL_HOST: site2.loc.test.com
expose:
- "80"
Just to keep this topic up to date, the jwilder/nginx-proxy meanwhile introduced a flag for that: HTTPS_METHOD=noredirect; To be set as environment variable.
Further reading on github
I think your configuration should be correct, but it seems that this is the intended behaviour of jwilder/nginx-proxy. See these lines in the file nginx.tmpl: https://github.com/jwilder/nginx-proxy/blob/master/nginx.tmpl#L89-L94
It seems that if a certificate is found, you will always be redirected to https.
EDIT: I found the confirmation in the documentation
The behavior for the proxy when port 80 and 443 are exposed is as
follows:
If a container has a usable cert, port 80 will redirect to 443 for that container so that HTTPS is always preferred when available.
You can still use a custom configuration. You could also try to override the file nginx.tmpl in a new Dockefile .
To serve traffic in both SSL and non-SSL modes without redirecting to SSL, you can include the environment variable HTTPS_METHOD=noredirect (the default is HTTPS_METHOD=redirect).
HTTPS_METHOD must be specified on each container for which you want to override the default behavior.
Here is an example Docker Compose file:
version: '3'
services:
nginx-proxy:
image: jwilder/nginx-proxy
ports:
- '80:80'
- '443:443'
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./config/certs:/etc/nginx/certs
environment:
DEFAULT_HOST: my.example.com
app:
build:
context: .
dockerfile: ./Dockerfile
environment:
HTTPS_METHOD: noredirect
VIRTUAL_HOST: my.example.com
Note: As in this example, environment variable HTTPS_METHOD must be set on the app container, not the nginx-proxy container.
Ref: How SSL Support Works section for the jwilder/nginx-proxy Docker image.

Resources