Hugo server in Docker container not reachable in Windows 10 - docker

A few days ago I started a little side project: Dockerizing my Hugo build on my Windows 10 machine. The Hugo container itself, which runs as a Linux container, was the easy part and seems to work (at least by looking at the console output
$ docker run --rm -it -p 1313:1313/tcp hugo:latest
Building sites …
Replace Autoprefixer browsers option to Browserslist config.
Use browserslist key in package.json or .browserslistrc file.
Using browsers option cause some error. Browserslist config
can be used for Babel, Autoprefixer, postcss-normalize and other tools.
If you really need to use option, rename it to overrideBrowserslist.
Learn more at:
https://github.com/browserslist/browserslist#readme
https://twitter.com/browserslist
WARN 2019/11/23 14:05:35 found no layout file for "HTML" for "section": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.
| DE | EN
+------------------+----+----+
Pages | 9 | 7
Paginator pages | 0 | 0
Non-page files | 0 | 0
Static files | 25 | 25
Processed images | 0 | 0
Aliases | 1 | 0
Sitemaps | 2 | 1
Cleaned | 0 | 0
Total in 680 ms
Watching for changes in /app/{assets,content,i18n,layouts,static}
Watching for config changes in /app/config.yaml
Environment: "development"
Serving pages from memory
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop
My Dockerfile the I run looks like this
FROM node:13-alpine
ENV VERSION 0.59.1
EXPOSE 1313
RUN apk add --no-cache git openssl py-pygments libc6-compat g++ curl
RUN curl -L https://github.com/gohugoio/hugo/releases/download/v${VERSION}/hugo_extended_${VERSION}_Linux-64bit.tar.gz | tar -xz \
&& cp hugo /usr/bin/hugo \
&& apk del curl \
&& hugo version
WORKDIR /app
COPY assets assets
COPY content content
COPY i18n i18n
COPY layouts layouts
COPY static static
COPY package.json package.json
COPY postcss.config.js postcss.config.js
COPY config.yaml config.yaml
RUN yarn
CMD [ "hugo", "server", "--buildDrafts","--watch" ]
The hard part for me now is to connect to the running Hugo server on my host's systems (Windows 10 Pro) browser.
I basically tried everything: localhost:1313 & http://172.17.0.2:1313/ (the container IP I get by running docker inspect <container ID>), with firewall enabled and disabled, but nothing seems to work.
To verify that it should work I ran hugo server --buildDrafts --watch directly on my host system and can access the server just fine. I also invested several hours in reading up on the issue, but none of the solutions seem to work in my case.
How can I solve this issue?

Here's your problem:
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Hugo is binding to the loopback address (127.0.0.1) inside the container. It does this by default because hugo serve is meant strictly as a development tool, not for actually serving pages in production. In order to avoid any security issues, it defaults to binding to the loopback interface so that you can only connect to it from the local machine.
Unfortunately, in the context of a container, localhost means "this container". So with Hugo bound to 127.0.0.1 inside a container you'll never be able to connect to it.
The solution is to provide a different bind address using the --bind option. You probably want to modify your Dockerfile so that it looks like:
CMD [ "hugo", "server", "--buildDrafts", "--watch", "--bind", "0.0.0.0" ]
This will cause hugo to bind to "all interfaces" inside the container, which should result in it working as you expect.

Related

Pass docker host ip as env var into devcontainer

I am trying to pass an environment variable into my devcontainer that is the output of a command run on my dev machine. I have tried the following in my devcontainer.json with no luck:
"initializeCommand": "export DOCKER_HOST_IP=\"$(ifconfig | grep -E '([0-9]{1,3}.){3}[0-9]{1,3}' | grep -v 127.0.0.1 | awk '{ print $2 }' | cut -f2 -d: | head -n1)\"",
"containerEnv": {
"DOCKER_HOST_IP1": "${localEnv:DOCKER_HOST_IP}",
"DOCKER_HOST_IP2": "${containerEnv:DOCKER_HOST_IP}"
},
and
"runArgs": [
"-e DOCKER_HOST_IP=\"$(ifconfig | grep -E '([0-9]{1,3}.){3}[0-9]{1,3}' | grep -v 127.0.0.1 | awk '{ print $2 }' | cut -f2 -d: | head -n1)\"
],
(the point of the ifconfig/grep piped command is to provide me with the IP of my docker host which is running via Docker for Desktop (Mac))
Some more context
Within my devcontainer I am running some kubectl deployments (to a cluster running on Docker for Desktop) where I would like to configure a hostAlias for a pod (docs) such that that pod will direct requests to https://api.cancourier.local to the ip of the docker host (which would then hit an ingress I have configured for that CNAME).
I could just pass in the output of the ifconfig command to my kubectl command when running from within the devcontainer. The problem is that I get two different results from this depending on whether I am running it on my host (10.0.0.89) or from within the devcontainer (10.1.0.1). 10.0.0.89 in this case is the "correct" IP as if I curl this from within my devcontainer, or my deployed pod, I get the response I'd expect from my ingress.
I'm also aware that I could just use the name of my k8s service (in this case api) to communicate between pods, but this isn't ideal. As for why, I'm running a Next.js application in a pod. The Next.js app on this pod has two "contexts":
my browser - the app serves up static HTML/JS to my browser where communicating with https://api.cancourier.local works fine
on the pod itself - running some things (ie. _middleware) on the pod itself, where the pod does not currently know what https://api.cancourier.local
What I was doing to temporarily get around this was to have a separate config on the pod, one for the "browser context" and the other for things running on the pod itself. This is less than ideal as when I go to deploy this Next.js app (to Vercel) it won't be an issue (as my API will be deployed on some publicly accessible CNAME). If I can accomplish what I was trying to do above, I'd be able to avoid this.
So I didn't end up finding a way to pass the output of a command run on the host machine as an env var into my devcontainer. However I did find a way to get the "correct" docker host IP and pass this along to my pod.
In my devcontainer.json I have this:
"runArgs": [
// https://stackoverflow.com/a/43541732/3902555
"--add-host=api.cancourier.local:host-gateway",
"--add-host=cancourier.local:host-gateway"
],
which augments the devcontainer's /etc/hosts with:
192.168.65.2 api.cancourier.local
192.168.65.2 cancourier.local
then in my Makefile where I store my kubectl commands I am simply running:
deploy-the-things:
DOCKER_HOST_IP = $(shell cat /etc/hosts | grep 'api.cancourier.local' | awk '{print $$1}')
helm upgrade $(helm_release_name) $(charts_location) \
--install \
--namespace=$(local_namespace) \
--create-namespace \
-f $(charts_location)/values.yaml \
-f $(charts_location)/local.yaml \
--set cwd=$(HOST_PROJECT_PATH) \
--set dockerHostIp=$(DOCKER_HOST_IP) \
--debug \
--wait
then within my helm chart I can use the following for the pod running my Next.js app:
hostAliases:
- ip: {{ .Values.dockerHostIp }}
hostnames:
- "api.cancourier.local"
Highly recommend following this tutorial: Container environment variables
In this tutorial, 2 methods are mentioned:
Adding individual variables
Using env file
Choose which is more comfortable for you, good luck))

How to run avahi in docker on a Linux host?

I am trying to setup avahi in a docker container running on a Linux host. The purpose is to let avahi announce a service of my own and form me find the host and IP of the docker host.
So far avahi seems to run nicely in the container but I can not find my services searching from outside of my host.
I have googled alot and there are suggestions what to do but they all seems to be contradictory and/or insecure.
This is what I got so far.
docker-compose.yml
version: "3.7"
services:
avahi:
container_name: avahi
build:
context: ./config/avahi
dockerfile: DockerFile
network: host
DockerFile:
FROM alpine:3.13
RUN apk add --no-cache avahi avahi-tools
ADD avahi-daemon.conf /etc/avahi/avahi-daemon.conf
ADD psmb.service /etc/avahi/services/mpsu.service
ENTRYPOINT avahi-daemon --no-drop-root --no-rlimits
avahi-daemon.conf:
[server]
enable-dbus=no
psmb.service: (my service)
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name replace-wildcards="yes">PSMB</name>
<service> <type>_mqtt._tcp</type> <port>1883</port>
<txt-record>info=MPS Service Host</txt-record>
</service>
</service-group>
This is from the terminal when starting avahi:
> docker-compose up
Starting avahi ... done
Attaching to avahi
avahi | avahi-daemon 0.8 starting up.
avahi | WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
avahi | Loading service file /etc/avahi/services/mpsu.service.
avahi | Loading service file /etc/avahi/services/sftp-ssh.service.
avahi | Loading service file /etc/avahi/services/ssh.service.
avahi | Joining mDNS multicast group on interface eth0.IPv4 with address 172.18.0.2.
avahi | New relevant interface eth0.IPv4 for mDNS.
avahi | Joining mDNS multicast group on interface lo.IPv4 with address 127.0.0.1.
avahi | New relevant interface lo.IPv4 for mDNS.
avahi | Network interface enumeration completed.
avahi | Registering new address record for 172.18.0.2 on eth0.IPv4.
avahi | Registering new address record for 127.0.0.1 on lo.IPv4.
avahi | Server startup complete. Host name is 8f220b5ac449.local. Local service cookie is 1841391818.
avahi | Service "8f220b5ac449" (/etc/avahi/services/ssh.service) successfully established.
avahi | Service "8f220b5ac449" (/etc/avahi/services/sftp-ssh.service) successfully established.
avahi | Service "PSMB" (/etc/avahi/services/mpsu.service) successfully established.
So,, how do I configure to be able to search for my service?
I would like to get the host information for the Host running docker.
So, I ran across this project https://gitlab.com/ydkn/docker-avahi
Dockerfile:
# base image
ARG ARCH=amd64
FROM $ARCH/alpine:3
# args
ARG VCS_REF
ARG BUILD_DATE
# labels
LABEL maintainer="Florian Schwab <me#ydkn.io>" \
org.label-schema.schema-version="1.0" \
org.label-schema.name="ydkn/avahi" \
org.label-schema.description="Simple Avahi docker image" \
org.label-schema.version="0.1" \
org.label-schema.url="https://hub.docker.com/r/ydkn/avahi" \
org.label-schema.vcs-url="https://gitlab.com/ydkn/docker-avahi" \
org.label-schema.vcs-ref=$VCS_REF \
org.label-schema.build-date=$BUILD_DATE
# install packages
RUN apk --no-cache --no-progress add avahi avahi-tools
# remove default services
RUN rm /etc/avahi/services/*
# disable d-bus
RUN sed -i 's/.*enable-dbus=.*/enable-dbus=no/' /etc/avahi/avahi-daemon.conf
# entrypoint
ADD docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
ENTRYPOINT [ "docker-entrypoint.sh" ]
# default command
CMD ["avahi-daemon"]
# volumes
VOLUME ["/etc/avahi/services"]
you'll have to change/uncomment ALLOW_INTERFACES and DENY_INTERFACES in /etc/avahi/avahi-daemon.conf to "allow" whatever eth port you are using. i.e.
avahi-daemon.conf:
...
use-ipv4=yes
use-ipv6=yes
allow-interfaces=eth1 # was commented out with eth0
deny-interfaces=eth0 # was commented out with eth1
...
and then you can simply run the container, with the ALLOW_INTERFACES=??? matching what was set in avahi-daemon.conf
run command:
docker run -d --restart always \
--net=host \
-e ALLOW_INTERFACES=eth1 \
-v $(pwd)/services:/etc/avahi/services \
ydkn/avahi:latest
Seems to work, I was able to ping computername.local from another computer on/connected to the router, where computername is whatever is on the terminal line, i.e. username#computername:~$
Looks like there is also a way to add a service file, mounted to /etc/avahi/services so I believe you can customize the service name to something else more useful. I need to figure out how to do that, will edit when I find out.

Time in Docker container out of sync with host machine

I'm trying to connect to CosmosDB through my SpringBoot app. I have all of this working if I run the app with Spring or via Intellij. But, when I run the app in Docker I get the following error message:
com.azure.data.cosmos.CosmosClientException: The authorization token is not valid at the current time.
Please create another token and retry
(token start time: Thu, 26 Mar 2020 04:32:10 GMT,
token expiry time: Thu, 26 Mar 2020 04:47:10 GMT, current server time: Tue, 31 Mar 2020 20:12:42 GMT).
Note that in the above error message the current server time is correct but the other times are 5 days behind.
What I find interesting is that I only ever receive this in the docker container.
FROM {copy of zulu-jdk11}
ARG JAR_FILE
#.crt file in the same folder as your Dockerfile
ARG CERT="cosmos.cer"
ARG ALIAS="cosmos2"
#import cert into java
COPY $CERT /
RUN chmod +x /$CERT
WORKDIR $JAVA_HOME/lib/security
RUN keytool -importcert -file /$CERT -alias $ALIAS -cacerts -storepass changeit -noprompt
WORKDIR /
COPY /target/${JAR_FILE} app.jar
COPY run-java.sh /
RUN chmod +x /run-java.sh
ENV JAVA_OPTIONS "-Duser.timezone=UTC"
ENV JAVA_APP_JAR "/app.jar"
# run as non-root to mitigate some security risks
RUN addgroup -S pcc && adduser -S nonroot -G nonroot
USER nonroot:nonroot
ENTRYPOINT ["/run-java.sh"]
One thing to note is ENV JAVA_OPTIONS "-Duser.timezone=UTC" but removing this didn't help me at all
I basically run the same step from IntelliJ and I have no issues with it but in docker the expiry date seems to be 5 days behind.
version: "3.7"
services:
orchestration-agent:
image: {image-name}
ports:
- "8080:8080"
network_mode: host
environment:
- COSMOSDB_URI=https://host.docker.internal:8081/
- COSMOSDB_KEY={key}
- COSMOSDB_DATABASE={database}
- COSMOSDB_POPULATEQUERYMETRICS=true
- COSMOSDB_ITEMLEVELTTL=60
I think it should also be mentioned that I changed the network_mode to host. And I also changed the CosmosDB URI from https://localhost:8081 to https://host.docker.internal:8081/
I would also like to mention that I built my dockerfile with the help of:
Importing self-signed cert into Docker's JRE cacert is not recognized by the service
How to add a SSL self-signed cert to Jenkins for LDAPS within Dockerfile?
Docker containers don't maintain a separate clock, it's identical to the Linux host since time is not a namespaced value. This is also why Docker removes the permission to change the time inside the container, since that would impact the host and other containers, breaking the isolation model.
However, on Docker Desktop, docker runs inside of a VM (allowing you to run Linux containers on non-Linux desktops), and that VM's time can get out of sync when the laptop is suspended. This is currently being tracked in an issue over on github which you can follow to see the progress: https://github.com/docker/for-win/issues/4526
Potential solutions include restarting your computer, restarting docker's VM, running NTP as a privileged container, or resetting the time sync in the windows VM with the following PowerShell:
Get-VMIntegrationService -VMName DockerDesktopVM -Name "Time Synchronization" | Disable-VMIntegrationService
Get-VMIntegrationService -VMName DockerDesktopVM -Name "Time Synchronization" | Enable-VMIntegrationService
With WSL 2, restarting the VM involves:
wsl --shutdown
wsl
There is recent known problem with WSL 2 time shift after sleep which has been fixed in 5.10.16.3 WSL 2 Linux kernel which is still not included in Windows 10 version 21H1 update but can be installed manually.
How to check WSL kernel version:
> wsl uname -r
Temporal workaround for the old kernel that helps until next sleep:
> wsl hwclock -s
Here's an alternative that worked for me on WSL2 with Docker Desktop on Windows:
Since it's not possible to set the date inside a Docker container, I just opened Ubuntu in WSL2 and ran the following command to synchronize the clock:
sudo date -s "$(wget -qSO- --max-redirect=0 google.com 2>&1 | grep Date: | cut -d' ' -f5-8)Z"
It worked well, so I added the following line in my root user's crontab:
# Edit root user's crontab
sudo crontab -e
# Add the following line to run it every minute of every day:
* * * * * sudo date -s "$(wget -qSO- --max-redirect=0 google.com 2>&1 | grep Date: | cut -d' ' -f5-8)Z"
After that, I just restarted my Docker containers, and the dates were correct since they seemed to use the WSL2 Ubuntu dates.
Date before (incorrect):
date
Thu Feb 4 21:50:35 UTC 2021
Date after (correct):
date
Fri Feb 5 19:01:05 UTC 2021

Setting up Gitlab using Docker on Windows host, issue with shared folders

TLDR;
Does anyone know how to solve the "Failed asserting that ownership of "/var/opt/gitlab/git-data" was git" error?
Background:
I want to set up the Gitlab Docker on WindowsServer2012R2 running Docker toolbox, version 17.04.0-ce, build 4845c56.
Issue/Question
I can't get the shared folder to work properly on the D drive of the server. I read that I needed to add the folder to the VirtualBox VM, which I did via the settings/shared folder menu in the VB GUI. I set a name "gitlab" to the path "D:\data\gitlab" then checked auto-mount, make permanent, and set it to full access.
I started the docker machine and ran "docker-machine ssh $machine-name". I noticed that there was no /media directory and so I added a folder at the home directory (/home/docker/gitlab) and then mounted the shared folder using the following command I found in several forums:
sudo mount -t vboxsf gitlab /home/docker/gitlab
At this point I can add files to the Windows host directory or the Docker VM and it seems to work fine and the test files show up.
Now when I spin up the Gitlab Docker image, I use the following command modified from their documentation:
docker run --detach --hostname gitlab.example.com --publish 80:80 --name gitlab --volume /home/docker/gitlab:/etc/gitlab:Z --volume /home/docker/gitlab/logs:/var/log/gitlab:Z --volume /home/docker/gitlab/data:/var/opt/gitlab:Z gitlab/gitlab-ce
Now I know that it appears to be writing to the shared drive, because all of these files are generated, but then it crashes after a few seconds and I receive the following error log.
Error Log:
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=9.3.6-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab vim /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Installing gitlab.rb config...
Generating ssh_host_rsa_key...
Generating public/private rsa key pair.
Your identification has been saved in /etc/gitlab/ssh_host_rsa_key.
Your public key has been saved in /etc/gitlab/ssh_host_rsa_key.pub.
The key fingerprint is:
SHA256:GyFlf9tl7ZuEbuE+dwZUYiyahdsRzpC1T7kwyUvoD+o root#gitlab.example.com
The key's randomart image is:
+---[RSA 2048]----+
| o .+oo |
| o .o*+o+.o|
| . . o*#+oo+|
| . o+o.Oo= |
| S o o++..|
| + oo + o|
| o .+ + |
| . o. .o|
| E .o..|
+----[SHA256]-----+
Generating ssh_host_ecdsa_key...
Generating public/private ecdsa key pair.
Your identification has been saved in /etc/gitlab/ssh_host_ecdsa_key.
Your public key has been saved in /etc/gitlab/ssh_host_ecdsa_key.pub.
The key fingerprint is:
SHA256:Kb99jG8EtMuTSdIuqBT3GLeD1D0wwTEcQhKgVJUlBjs root#gitlab.example.com
The key's randomart image is:
+---[ECDSA 256]---+
| .o+=*=+=+ |
|.. oo..=.. |
|. E . * . |
| o + +.B |
| +.BS* * |
| . +o= B . |
| . . .o = |
| . o. + |
| . .+. |
+----[SHA256]-----+
Generating ssh_host_ed25519_key...
Generating public/private ed25519 key pair.
Your identification has been saved in /etc/gitlab/ssh_host_ed25519_key.
Your public key has been saved in /etc/gitlab/ssh_host_ed25519_key.pub.
The key fingerprint is:
SHA256:lVxpu0UoyNPWVY6D9c+m/bUTyvKP6vuR4cTOYwQ0j+U root#gitlab.example.com
The key's randomart image is:
+--[ED25519 256]--+
| . o +.=o..|
| +.=o#o.+ |
| o+=.Eo o|
| . + .o.|
| S B +|
| B o= |
| .Oo +|
| ..o+.+|
| .+*+.oo|
+----[SHA256]-----+
Preparing services...
Starting services...
Configuring GitLab package...
/opt/gitlab/embedded/bin/runsvdir-start: line 24: ulimit: pending signals: cannot modify limit: Operation not permitted
/opt/gitlab/embedded/bin/runsvdir-start: line 34: ulimit: max user processes: cannot modify limit: Operation not permitted
/opt/gitlab/embedded/bin/runsvdir-start: line 37: /proc/sys/fs/file-max: Read-only file system
Configuring GitLab...
================================================================================
Error executing action `run` on resource 'ruby_block[directory resource: /var/opt/gitlab/git-data]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Failed asserting that ownership of "/var/opt/gitlab/git-data" was git
---- Begin output of set -x && [ "$(stat --printf='%U' $(readlink -f /var/opt/gitlab/git-data))" = 'git' ] ----
STDOUT:
STDERR: + readlink -f /var/opt/gitlab/git-data
+ stat --printf=%U /var/opt/gitlab/git-data
+ [ UNKNOWN = git ]
---- End output of set -x && [ "$(stat --printf='%U' $(readlink -f /var/opt/gitlab/git-data))" = 'git' ] ----
Ran set -x && [ "$(stat --printf='%U' $(readlink -f /var/opt/gitlab/git-data))" = 'git' ] returned 1
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/storage_directory_helper.rb:124:in `validate_command'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/storage_directory_helper.rb:112:in `block in validate'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/storage_directory_helper.rb:111:in `each_index'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/storage_directory_helper.rb:111:in `validate'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/storage_directory_helper.rb:87:in `validate!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/definitions/storage_directory.rb:35:in `block (3 levels) in from_file'
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/definitions/storage_directory.rb
26: ruby_block "directory resource: #{params[:path]}" do
27: block do
28: # Ensure the directory exists
29: storage_helper.ensure_directory_exists(params[:path])
30:
31: # Ensure the permissions are set
32: storage_helper.ensure_permissions_set(params[:path])
33:
34: # Error out if we have not achieved the target permissions
35: storage_helper.validate!(params[:path])
36: end
37: not_if { storage_helper.validate(params[:path]) }
38: end
39: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/definitions/storage_directory.rb:26:in `block in from_file'
ruby_block("directory resource: /var/opt/gitlab/git-data") do
params {:path=>"/var/opt/gitlab/git-data", :owner=>"git", :group=>nil, :mode=>"0700", :name=>"/var/opt/gitlab/git-data"}
action [:run]
retries 0
retry_delay 2
default_guard_interpreter :default
block_name "directory resource: /var/opt/gitlab/git-data"
declared_type :ruby_block
cookbook_name "gitlab"
recipe_name "gitlab-shell"
block #<Proc:0x000000054a99a8#/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/definitions/storage_directory.rb:27>
not_if { #code block }
end
Platform:
---------
x86_64-linux
Does anyone know how to solve the "Failed asserting that ownership of "/var/opt/gitlab/git-data" was git" error? I'm still somewhat new to Docker/setting up Gitlab, so it's very possible I could have overlooked something simple. I've spent several hours Googling this, and it seems that others also have a lot of issues getting shared folders to work from Windows using the Docker Toolbox, so hopefully this will help others as well.
Background
One solution (maybe not the best) for those of us stuck in a world without native docker, is to use vdi drives and shared folders. The vdi drive can live on an drive we want (which is important if you don't want to use the C drive) and is used to allow the Gitlab docker the ability to chown anything it wants, so this is where we'll store the persistent volumes. The downside is that a vdi is not as transparent as a simple shared folder, thus for backups, a shared folder makes things a little bit easier/transparent.
Disclaimer
I'm not an expert on any of this, so please use caution and take what I say with a grain of salt.
Steps to perform
Create a new vdi drive and shared folder on any drive you'd like
Turn off your docker machine you want to use for gitlab
In virtualbox go into the settings on your docker-machine, then Storage, and click Add Hard Disk icon, then Create new disk
Select VDI (VirtualBox Disk Image) and click Next
Select Dynamically allocated and click Next
Select the name and location you want to store the vdi by clicking the folder with green carrot symbol, then select the max size the vdi can grow to, and click Create
Now in the settings menu, switch to Shared Folders and click Adds new shared folder icon
Create a gitlabbackups folder to where ever you want and select Auto-mount and Make Permanent
Now partition and format the drive
Start/enter the docker machine (either use VBox window or docker-machine ssh <your docker machine name> from cmd prompt)
Run fdisk -l to list the available drives, and if you've only mounted the one extra vdi drive, you should see something like /dev/sdb
The next steps are irreversible, so perform it at your own discretion: enter command fdisk /dev/sdb then n for new partition, p for primary, and 1
Now format the new partition (you might need sudo as well): mkfs.ext4 /dev/sdb1
Run docker with persistent volumes on second vdi and backups in shared folder
Sample Dockerfile:
FROM gitlab/gitlab-ce:latest
RUN apt-get update
RUN apt-get install -y cron
# Add a cron job to backup everyday
RUN echo "0 5 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create STRATEGY=copy CRON=1" | crontab -
# For an unknown reason, the cron job won't actually run unless cron is restarted
CMD service cron restart && \
/assets/wrapper
Sample docker-compose.yml:
version: "3.0"
services:
gitlab:
build: .
restart: always
ports:
- "80:80"
volumes:
# These volumes are on the vdi we created above
- "/mnt/sdb1/etc/gitlab/:/etc/gitlab"
- "/mnt/sdb1/var/log/gitlab:/var/log/gitlab"
- "/mnt/sdb1/var/opt/gitlab:/var/opt/gitlab"
# This volume sits in the shared folder defined above
- "/gitlabbackups:/var/opt/gitlab/backups"
cap_add:
# These seem to be necessary for the mounted drive to work properly
# https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
- SYS_ADMIN
- DAC_READ_SEARCH
Because there seems to be an issue with auto mounting the vdi, use a startup script, for example (assuming you used a D drive, just replace anything inside <...> as needed), sample run.bat:
#cd /d D:\<path to docker-compose.yml, assuming it's on the D drive>
#docker-machine start <docker machine name>
#FOR /f "tokens=*" %%i IN ('docker-machine env <docker machine name>') DO #%%i
#docker-machine ssh <docker machine name> sudo mount /dev/sdb1 /mnt/sdb1
#docker-compose build
#docker-compose up -d
#REM If the docker machine was completely off, running only 'docker-compose up -d' will
#REM not mount the volumes properly. Stopping and restarting the container results in
#REM the volumes mounting properly.
#docker stop <gitlab container name>
#docker start <gitlab container name>
#pause
Note: the gitlab container name can be found by running docker-compose up once and then docker ps -a to check it, but it's usually follows the convention <directory compose file is in>_<name in the compose file, e.g. gitlab here>_1
Assuming all went well and you change the stuff in the <...>'s above for your situation, you should be able to run the batch file and have gitlab up and running in such a way that it stores everything on the alternate drive, persistent working files in the vdi (to get around VBox POSIX limitations), and backups transparently stored in the shared folder.
Hope this helps other poor souls that don't have access to native docker yet.

ActiveMQ within Wildfly on a Docker container gives: Invalid "host" value "0.0.0.0" detected

I have Wildfly running in a Docker container.
Within Wildfly the messaging-activemq subsystem is active.
The subsystem and extension defaults are taken from the standalone-full.xml file.
After starting wildfly, following output is displayed
[org.apache.activemq.artemis.jms.server] (ServerService Thread Pool -- 64)
AMQ121005: Invalid "host" value "0.0.0.0" detected for "http-connector" connector.
Switching to "eeb79399d447".
If this new address is incorrect please manually configure the connector to use the proper one.
The eeb79399d447 is the docker container id.
It's also impossible to connect to jms from my java client. While connecting it gives the following error.
AMQ214016: Failed to create netty connection: java.net.UnknownHostException: eeb79399d447
When I start wildfly on my local workstation (outside docker) the problem does not occur and I can connect to jms and send my messages.
Here are a few options. Option 1 & 2 may be what you asked for, but in the end didn't work for me. Option 3 however, I think will better address your intent.
Option 1) You can do this by adding some scripting to your docker image ( and not touching your standalone-full.xml. The basic idea ( credit goes to git-hub user kwart ) is to make a docker entry point that can determine the IPv4 address of the docker container before calling standalone.sh.
see : https://github.com/kwart/dockerfiles/tree/master/wildfly-ext and check out the usage of WILDFLY_BIND_ADDR. I forked it.
Notes:
GetIp.java will print out the IPv4 address ( and is copied into the container )
dockerentry-point.sh calls GetIp.java as needed
WILDFLY_BIND_ADDR=${WILDFLY_BIND_ADDR:-0.0.0.0}
if [ "${WILDFLY_BIND_ADDR}" = "auto" ]; then
WILDFLY_BIND_ADDR=`java -cp /opt/jboss GetIp`
fi
Option 2) Alternatively, using some script-fu, you may be able to do everything you need in a Dockerfile:
#CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-c", "standalone-full.xml", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]
CMD ["sh", "-c", "DOCKER_IPADDR=$(hostname --ip-address) && echo IP Address was $DOCKER_IPADDR && /opt/jboss/wildfly/bin/standalone.sh -c standalone-full.xml -b=$DOCKER_IPADDR -bmanagement=$DOCKER_IPADDR"]
Your mileage may very.
I was working with the helloworld-jms quickstart from the WildFly docs, and had to jump through some extra hoops to get the JMS queue created. Even at that point, the sample java code wasn't able to connect with either option 1 or option 2.
Option 3) ( This worked for me btw ) Start your container with binding to 0.0.0.0, expose your 8080 port for your JMS client running on the host, and add an entry in your hosts' /etc/hosts file:
Dockerfile:
FROM jboss/wildfly
# CP foo.war /opt/jboss/wildfly/standalone/deployments/
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin --silent
RUN /opt/jboss/wildfly/bin/add-user.sh -a quickstartUser quickstartPwd1! --silent
RUN echo "quickstartUser=guest" >> /opt/jboss/wildfly/standalone/configuration/application-roles.properties
# use standalone-full.xml to enable the JMS feature
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-c", "standalone-full.xml", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]
Build & run ( expose 8080 if your client is on your host machine )
docker build -t mywildfly .
docker run -it --rm --name jboss -p127.0.0.1:8080:8080 -p127.0.0.1:9990:9990 my_wildfly
Then on the host machine ( I'm running OSX; my jboss container's id was 46d04508b92b ) add an entry in your /etc/hosts for the docker-host-name that points to 127.0.0.1:
127.0.0.1 46d04508b92b # <-- replace with your container's id
Once the wildfly container is running, you create/configure the testQueue via scripts or in the management console. My config came from https://github.com/wildfly/quickstart.git under the helloworld-jms folder:
docker cp configure-jms.cli jboss:/tmp/
docker exec jboss /opt/jboss/wildfly/bin/jboss-cli.sh --connect --file=/tmp/configure-jms.cli
and SUCCESS from mvn clean compile exec:java the host machine (from w/in the helloworld-jms folder):
Mar 28, 2018 9:03:15 PM org.jboss.as.quickstarts.jms.HelloWorldJMSClient main
INFO: Found destination "jms/queue/test" in JNDI
Mar 28, 2018 9:03:16 PM org.jboss.as.quickstarts.jms.HelloWorldJMSClient main
INFO: Sending 1 messages with content: Hello, World!
Mar 28, 2018 9:03:16 PM org.jboss.as.quickstarts.jms.HelloWorldJMSClient main
INFO: Received message with content Hello, World!
You need to edit the standalone-full.xml to cope with jms behind NAT and when you run the docker container pass though the ip and port that your jms client can use to connect, which is the ip of the machine running docker in Dockers' default config

Resources