GitLab Pipeline / Runner fails after update: failed adding cache volume ... AppArmor enabled ... docker-default profile could not be loaded - docker

After updating GitLab and GitLab Runner to v15.8.1 on a Ubuntu 20.04.05 LTS suddenly none of our pipelines succeeded anymore.
Every job showed the following error message:
ERROR: Preparation failed: adding cache volume: set volume permissions: running permission container "XXXXXXX" for volume "runner-zx-dagtx-project-41-concurrent-0-cache-YYYYYYY": starting permission container: Error response from daemon: AppArmor enabled on system but the docker-default profile could not be loaded: running apparmor_parser apparmor_parser --version failed with output:
error: exec: "apparmor_parser": executable file not found in $PATH (linux_set.go:105:0s)
Google miss leaded me to re-installations of Docker and the GitLab Runner, to deleting all docker volumes, images and containers and so on. All without success.
How to get GitLab Runner back to work?

What fixed the problem was the installation of AppArmour and the AppArmour Utilities.
sudo apt-get install apparmor apparmor-utils
Another solution would have been to disable AppArmour during the boot process by adding apparmor=0 as boot parameter to the grub file (what I didn't try, so no guarantee for that).

Related

Docker Installation Problems

I'm new to using Docker (never used it before) and I'm running into these errors:
I installed the Docker for Windows and following the steps on this
tutorial, but Docker Desktop doesn't load up for me at all.
I tried to run the docker pull hello-world command but I'm getting the error: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.
How do I fix this? Thanks

Docker container cannot start due to apparmor, why?

I have a fresh install of ubuntu 18.04. The only program installed is docker.
But if I try to run the hello-world container, to test the installation, the run failed:
docker: Error response from daemon: AppArmor enabled on system but the docker-default profile could not be loaded: running `/sbin/apparmor_parser apparmor_parser -Kr /var/lib/docker/tmp/docker-default847992402` failed with output: apparmor_parser: Unable to replace "docker-default". Permission denied; attempted to load a profile while confined?
error: exit status 243.
ERRO[0000] error waiting for container: context canceled
There are no apparmor profiles installed on the system:
apparmor module is loaded.
0 profiles are loaded.
0 profiles are in enforce mode.
0 profiles are in complain mode.
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
How can I install the profile? And fix the error?
I am trying to run:
sudo docker run hello-world
I installed docker via:
sudo apt install docker.io
The system is Ubuntu 18.04 on a LXC vserver
Jonas
To run docker properly on an LXC vServer, there are some settings to change outside the vServer, in the LXC configurations. In my case I had to enable docker support on my hosting account´s dashboard.

Set up Docker Build Step in Bamboo

I am currently despairing at the attempt of setting up a docker build step in Atlassian Bamboo.
For starters, I just want to create a build configuration that runs the hello-world image as a proof of confluence. So far, I have failed.
I have tried following the steps on https://confluence.atlassian.com/bamboo0609/using-bamboo/jobs-and-tasks/configuring-tasks/configuring-the-docker-task-in-bamboo , but to no avail.
My setup is this:
We have Bamboo installed on an Ubuntu server. I also installed Docker on that server and added the bamboo user to the docker usergroup and restarted the server to make sure the permission change takes effect. At this point, docker run hello-world works when I run it directly on the server. I can also confirm that this is the server that Bamboo runs on since Bamboo went offline whenever I restarted the server that I installed Docker on.
Then, I have added the docker capability to the server (the agent is the default agent, so it inherits this capability from the server). As the docker path, I have tried various things, none of which worked (aka, the following errors remained the same for each of these):
/snap/docker (the first folder that I found on a manual search)
/usr/bin/docker (the recommended path, though on inspecting the Ubuntu server I quickly found out that no docker folder exists under /usr/bin on the Ubuntu derver)
/var/snap/docker/common/var-lib-docker (the path that Docker returns as its Root Directory when I run docker info on the Ubuntu server)
/var/snap/docker (for good measure)
Now, for the runner, I have tried two different approaches.
First, I tried using a Docker runner with the following settings:
Command: Run a Docker container
Docker image: hello-world
This returns the following error message:
┊
Error occurred while running Task 'Hello World Docker Test(5)' of type com.atlassian.bamboo.plugins.bamboo-docker-plugin:task.docker.cli.com.atlassian.bamboo.task.TaskException: Failed to execute task
┊
Caused by: com.atlassian.bamboo.docker.DockerException: Error running Docker run command
┊
Caused by: com.atlassian.utils.process.ProcessException: Error executing /snap/docker run --volume /var/atlassian/application-data/bamboo/xml-data/build-dir/CAM-DOC-JOB1:/data --workdir /data --rm hello-world
┊
The second was just to run a shell runner for the command docker run hello-world, which returned the following error:
docker: not found
At this point, I feel like I'm out of ideas. Everything points towards Bamboo for some reason not finding Docker on the server, even though I can clearly confirm that it is there. I have tried various different approaches of telling Bamboo where to find Docker, but none of them have worked.
It's obvious that I'm doing something wrong, but I can't figure out what. Or maybe the problem lies in an entirely different direction altogether? Anyway, I would be grateful for any insight shared on this matter.
Okay, I found out what caused this strange behaviour.
The problem was that I installed Docker using sudo snap install docker, and apparently installing docker via snap causes problems with Bamboo.
So I got it to work using these simple steps:
[Server] Uninstalled Snap Docker using sudo snap remove docker
[Server] Reinstalled Docker using sudo apt install docker.io
[Bamboo] Changed the path to Docker in the Server Capabilities to /usr/bin/docker
After that, the hello-world image build succeeded and printed the expected output to the log.

Google Cloud Logging Driver cannot find credentials after reboot

I've followed the directions here, and everything works well until I restart my computer. After restarting, it seems like the docker daemon loses track of the Google credentials.
$ docker run --log-driver=gcplogs ...
fails with:
docker: Error response from daemon: failed to initialize logging driver: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
ERRO[0000] error waiting for container: context canceled
This is strange to me, because running $ systemctl show --property=Environment docker returns the value in my systemd configuration:
Environment=GOOGLE_APPLICATION_CREDENTIALS=/etc/path/to/application_default_credentials.json
If I $ sudo systemctl restart docker, then docker runs sucessfully and logs are sent to stackdriver. But I want this docker image to run automatically on startup, and restarting docker with sudo gets in the way.
Is there a way to initialize the docker daemon with the necessary environment variables, so gcplogs is ready on boot without restarting docker?
I had two versions of docker installed -- one through adding docker's repo to apt, and one through snap. Running
sudo systemctl list-unit-files| grep docker | grep enabled
showed two installations of docker:
docker.service enabled
snap.docker.dockerd.service enabled
Having two docker installations was causing problems for startup. I removed the snap installation, rebooted, and everything now works.
I think you may try to edit the systemd: Unit dependencies and order, let docker.service start after google-accounts-daemon.service.
You can see all the service in google vm by
sudo systemctl list-unit-files| grep google | grep enabled
And you will see
google-accounts-daemon.service enabled
google-clock-skew-daemon.service enabled
google-instance-setup.service enabled
google-network-daemon.service enabled
google-shutdown-scripts.service enabled
google-startup-scripts.service enabled

Docker Login: Error when manually logging into private Registry

I can't manually log into my private GitLab Docker Registry from CLI:
# docker login -u "${DOCKER_USER}" -p "${DOCKER_PASS}" "${DOCKER_URL}"
error getting credentials - err: exit status 1, out: `Cannot autolaunch D-Bus without X11 $DISPLAY`
System info:
Ubuntu 18.04
docker-ce 18.03.1~ce~3-0~ubuntu (from official repo, without install script)
There is no ~/.docker/config.json for any users and I'm executing the docker login as root.
On Google, I just find recommendations to export DISPLAY... Can docker only login to remote registries in a GUI environment?
Exporting DISPLAY=0:0 yields:
error getting credentials - err: exit status 1, out: `Failed to execute child process “dbus-launch” (No such file or directory)`
Am I missing some dependency? Docker runs fine otherwise, but login doesn't work. I know there are backends to store credentials, but I don't want to store credentials. I'm just trying to authenticate against my registry to pull an image, doesn't that work in Docker ootb?
The docker-compose package unnecessarily depend on the broken golang-github-docker-docker-credential-helpers package. Removing the executable fixes this.
rm /usr/bin/docker-credential-secretservice
Note: This is a workaround and will need to be repeated each time the package is updated.
This affects the Ubuntu 18.04 (and possibly other non-LTS releases) and some Debian releases.

Resources