I'm trying out teamcity to build GCC binaries with docker agents on centos. I setup a docker agent to connect to builder2 TC server.
$ docker pull jetbrains/teamcity-agent
$ mkdir -p /mnt/builders/teamcity/agent1/conf
$ mkdir -p /mnt/builders/teamcity/agent/work
$ mkdir -p /mnt/builders/teamcity/agent/system
docker run -it --name agent1 \
-e SERVER_URL="http://builder2:8111" \
-e AGENT_NAME="builder2_agent1" \
--hostname builder2_agent \
--dns="xx.xxx.xx.xx" \
-v /mnt/builders/teamcity/agent1/conf:/data/teamcity_agent/conf \
-v /mnt/builders/teamcity/agent/work:/opt/buildagent/work \
-v /mnt/builders/teamcity/agent/system:/opt/buildagent/system \
--network='BuilderNetwork' \
jetbrains/teamcity-agent
All that works good but in order to make a build you must import the devtoolset like this
scl enable devtoolset-10 "/bin/bash"
$ which make
/opt/rh/devtoolset-10/root/usr/bin/make
So how is this done with docker agent? Are these tools to be built in the image or do you expose the /opt/rh dir to the container? Also if you were to expose the volume then how do you install /usr/bin/scl (i.e rh package scl-utils-20130529-19.el7.x86_64) into the docker container? Does it even make sense to run an agent in docker for this?
Related
I am trying to run this docker file: A-LOAM-Docker.
My setup is docker desktop over WSL2 with ubuntu 20.04LTS and XLaunch. When I run roscore and then rviz in different terminal, it opens and shows ~30 FPS. Then, I try running the docker file with script:
#!/bin/bash
docker run -it --rm --net=host -e DISPLAY=$DISPLAY -v $1:/root/catkin_ws/src/A-LOAM/ a-loam:latest bash -i -c \
"source ~/.bashrc; \
cd /root/catkin_ws/; \
catkin config \
--cmake-args \
-DCMAKE_BUILD_TYPE=Release; \
catkin build; \
source devel/setup.bash; \
roslaunch aloam_velodyne aloam_velodyne_VLP_16.launch rviz:=true; \
source /opt/ros/kinetic/setup.bash; \
"
It instantly runs rviz, but this time from the very beginning it shows 1-3 FPS. I wonder what can cause such a difference and how to fix that.
I am experimenting with running X11 GUI programs from a docker container on localhost:
FROM ubuntu:20.04
RUN apt-get update && apt-get install -y firefox
ARG home=/root
WORKDIR $home
COPY entrypoint.sh .
ENTRYPOINT ["./entrypoint.sh"]
where the entrypont.sh file is:
#! /bin/bash
firefox &
exec bash
Building the image with:
docker build -t firefox-ubuntu-2004 .
And running the container (localhost: Ubuntu 20.04):
XSOCK=/tmp/.X11-unix
XAUTH=/tmp/.docker.xauth
DISPLAY="localhost:0"
xauth nlist $DISPLAY | sed -e 's/^..../ffff/' | xauth -f $XAUTH nmerge -
docker run -it --rm \
-e DISPLAY=$DISPLAY \
-v $XAUTH:$XAUTH \
-v $XSOCK:$XSOCK \
-e XAUTHORITY=$XAUTH \
firefox-ubuntu-2004
The output error message from running the last command is:
Unable to init server: Broadway display type not supported: localhost:0
Error: cannot open display: localhost:0
The --net=host in the docker command should do the job
docker run --name myContainer -it --net=host fromMyimage:latest
along with using host.docker.internal instead of localhost for connecting to the docker's host on OSX.
In the log file I have this error:
./worker: error while loading shared libraries: libcares.so.2: cannot open shared object file: No such file or directory
I tried everything with the library it exists and its linked to the path.
My Dockerfile :
FROM ubuntu:20.04
RUN apt update -y && apt install libssl-dev -y
WORKDIR /worker
COPY build/worker ./
COPY build/lib /usr/lib
EXPOSE 50051
CMD ./worker
My makefile:
all: clean build
build:
mkdir -p build/lib && \
cd build && cmake .. && make
clean:
rm -rf build
clean-containers :
docker container stop `docker container ls -aq`
docker container rm `docker container ls -a -q`
create-workers :
docker run --name worker1 -p 2001:50051 -d workerimage
docker run --name worker2 -p 2002:50051 -d workerimage
docker run --name worker3 -p 2003:50051 -d workerimage
docker run --name worker4 -p 2004:50051 -d workerimage
docker run --name worker5 -p 2005:50051 -d workerimage
docker run --name worker6 -p 2006:50051 -d workerimage
docker run --name worker7 -p 2007:50051 -d workerimage
docker run --name worker8 -p 2008:50051 -d workerimage
docker run --name worker9 -p 2009:50051 -d workerimage
docker run --name worker10 -p 2010:50051 -d workerimage
make sure libcares.so.2 and other shared libraries are present inside /usr/lib of the container.
am running the progrium/consul container with the gliderlabs/registrator container. I am trying to create health checks to monitor if my docker containers are up or down. However I noticed some very strange activity with with health check I was able to make. Here is the command I used to create the health check:
curl -v -X PUT http://$CONSUL_IP_ADDR:8500/v1/agent/check/register -d #/home/myUserName/health.json
Here is my health.json file:
{
"id": "docker_stuff",
"name": "echo test",
"docker_container_id": "4fc5b1296c99",
"shell": "/bin/bash",
"script": "echo hello",
"interval": "2s"
}
First I noticed that this check would automatically delete the service whenever the container was stopped properly, but would do nothing when the container was stopped improperly (i.e. durring a node failure).
Second I noticed that the docker_container_id did not matter at all, this health check would attach itself to every container running on the consul node it was attached to.
I would like to just have a working tcp or http health test run for every docker container running on a consul node (yes I know my above json file runs a script, I just created that one following the documentation example). I just want consul to be able to tell if a container is stopped or running. I don't want my services to delete themselves when a health check fails. How would I do this.
Note: I find the consul documentation on Agent Health Checks very lacking, vague and inaccurate. So please don't just link to it and tell me to go read it. I am looking for a full explanation on exactly how to set up docker health checks the right way.
Update: Here is how to start consul servers with the most current version of the official consul container (right now its the dev versions, soon ill update it with the production versions):
#bootstrap server
docker run -d \
-p 8300:8300 \
-p 8301:8301 \
-p 8301:8301/udp \
-p 8302:8302 \
-p 8302:8302/udp \
-p 8400:8400 \
-p 8500:8500 \
-p 53:53/udp \
--name=dev-consul0 consul agent -dev -ui -client 0.0.0.0
#its IP address will then be the IP of the host machine
#lets say its 172.17.0.2
#start the other two consul servers, without web ui
docker run -d --name --name=dev-consul1 \
-p 8300:8300 \
-p 8301:8301 \
-p 8301:8301/udp \
-p 8302:8302 \
-p 8302:8302/udp \
-p 8400:8400 \
-p 8500:8500 \
-p 53:53/udp \
consul agent -dev -join=172.17.0.2
docker run -d --name --name=dev-consul2 \
-p 8300:8300 \
-p 8301:8301 \
-p 8301:8301/udp \
-p 8302:8302 \
-p 8302:8302/udp \
-p 8400:8400 \
-p 8500:8500 \
-p 53:53/udp \
consul agent -dev -join=172.17.0.2
# then heres your clients
docker run -d --net=host --name=client0 \
-e 'CONSUL_LOCAL_CONFIG={"leave_on_terminate": true}' \
consul agent -bind=$(hostname -i) -retry-join=172.17.0.2
https://hub.docker.com/r/library/consul/
progrium/consul image has old version of consul (https://hub.docker.com/r/progrium/consul/tags/) and currently seems to be not maintained.
Please try to use official image with current version for consul https://hub.docker.com/r/library/consul/tags/
You can also use registrator to register checks in consul connected with your service. eg.
SERVICE_[port_]CHECK_SCRIPT=nc $SERVICE_IP $SERVICE_PORT | grep OK
More examples: http://gliderlabs.com/registrator/latest/user/backends/#consul
So a solution that works around using any version of the consul containers is to just directly install consul on the host machine. This can be done by following these steps from https://sonnguyen.ws/install-consul-and-consul-template-in-ubuntu-14-04/:
sudo apt-get update -y
sudo apt-get install -y unzip curl
sudo wget https://releases.hashicorp.com/consul/0.6.4/consul_0.6.4_linux_amd64.zip
sudo unzip consul_0.6.4_linux_amd64.zip
sudo rm consul_0.6.4_linux_amd64.zip
sudo chmod +x consul
sudo mv consul /usr/bin/consul
sudo mkdir -p /opt/consul
cd /opt/consul
sudo wget https://releases.hashicorp.com/consul/0.6.4/consul_0.6.4_web_ui.zip
sudo unzip consul_0.6.4_web_ui.zip
sudo rm consul_0.6.4_web_ui.zip
sudo mkdir -p /etc/consul.d/
sudo wget https://releases.hashicorp.com/consul-template/0.14.0/consul-template_0.14.0_linux_amd64.zip
sudo unzip consul-template_0.14.0_linux_amd64.zip
sudo rm consul-template_0.14.0_linux_amd64.zip
sudo chmod a+x consul-template
sudo mv consul-template /usr/bin/consul-template
sudo nohup consul agent -server -bootstrap-expect 1 \
-data-dir /tmp/consul -node=agent-one \
-bind=$(hostname -i) \
-client=0.0.0.0 \
-config-dir /etc/consul.d \
-ui-dir /opt/consul/ &
echo 'Done with consul install!!!'
Then after you do this create your consul health check json files, info on how to do that can be found here. After you create your json files just put them in the /etc/consul.d directory and restart consul with consul reload. If after the reload consul does not add your new health checks then there is something wrong with the syntax of your json files. Go back edit them and try again.
I have a dockerfile:
FROM jenkins:1.651.1
COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt
USER root
RUN groupadd docker
RUN usermod -a -G docker jenkins
USER jenkins
I add my user jenkins to the group docker.
When I access my container:
jenkins#bc145b8cfc1d:/$ docker ps
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
jenkins#bc145b8cfc1d:/$ whoami
jenkins
This is the content of my /etc/groupon my container
jenkins:x:1000:
docker:x:1001:jenkins
my jenkins user is in the docker group
jenkins#bc145b8cfc1d:/$ groups jenkins
jenkins : jenkins docker
What am I doing wrong? I want to use docker-commands with my jenkins user. I'm on Amazon EC2 Container Service.
This is how I start a container from my image:
docker run -d -v /var/run/docker.sock:/var/run/docker.sock -v
/usr/bin/docker:/usr/bin/docker:ro -v
/lib64/libdevmapper.so.1.02:/usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02
-v /lib64/libudev.so.0:/usr/lib/x86_64-linux-gnu/libudev.so.0
-p 8080:8080 --name jenkins -u jenkins --privileged=true -t -i
my-jenkins:1.0
This was my 'solution' but it only worked on Ubuntu (not on my centos).
Dockerfile
FROM jenkins:1.651.1
USER root
RUN apt-get update \
&& apt-get install -y apt-transport-https ca-certificates \
&& echo "deb https://apt.dockerproject.org/repo debian-jessie main" > /etc/apt/sources.list.d/docker.list \
&& apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D \
&& apt-get update -y \
&& apt-get install -y docker-engine
RUN gpasswd -a jenkins docker
USER jenkins
Run command:
docker run -d -it -v /var/run/docker.sock:/var/run/docker.sock test-jenkins
On Ubuntu:
jenkins#c73c683b02d7:/$ whoami
jenkins
jenkins#c73c683b02d7:/$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c73c683b02d7 test-jenkins "/bin/tini -- /usr/lo" 2 minutes ago Up 2 minutes 8080/tcp, 50000/tcp
condescending_wing
It has something to do with gid I think:
cat /etc/group in container (on ubuntu and centos).
jenkins:x:1000:
docker:x:999:jenkins
cat /etc/group on Ubuntu (also 999)
docker:x:999:ubuntu
cat /etc/group on Centos (different gid)
docker:x:983:centos
There is probably a solution for this. But I only needed Ubuntu so did not go further in this.
Once your container is running, you can "patch" into the running container using different users using
docker exec -ti -u 0 jenkins bash // root
docker exec -ti -u 1 jenkins bash // probably jenkins
Using the root user, you can su jenkins if you need to switch to the jenkins user from the root user.
If you want to run docker containers inside your existing container (it seems like that is what you're trying), remember to start your docker container with the --privileged flag, eg docker run --privileged ...