I am trying to get jenkins running inside docker using marathon and mesos to lunch a jenkins docker image.
I used the create application button which produces the following json
{
"type": "DOCKER",
"volumes": [
{
"containerPath": "/var/jenkins_home",
"hostPath": "jenkins_home",
"mode": "RW"
},
{
"containerPath": "jenkins_home",
"mode": "RW",
"persistent": {
"size": 200
}
}
],
"docker": {
"image": "jenkins",
"network": "HOST",
"privileged": false,
"parameters": [],
"forcePullImage": false
}
}
stdout shows
--container="mesos-c8bd5b26-6e71-4e18-b490-821dbf7edd9d-S0.ac0b4dbb-10e4-4684-a4df-9539258d77ee" --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" --initialize_driver_logging="true" --launcher_dir="/home/ajazam/mesos-0.28.0/build/src" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/var/lib/mesos/data/slaves/c8bd5b26-6e71-4e18-b490-821dbf7edd9d-S0/frameworks/6079a596-90a8-4fa5-9c92-9215558737d1-0000/executors/jenkins-t7.9be44260-f99c-11e5-b0ac-e4115bb26fcc/runs/ac0b4dbb-10e4-4684-a4df-9539258d77ee" --stop_timeout="0ns"
Registered docker executor on slave4
Starting task jenkins-t7.9be44260-f99c-11e5-b0ac-e4115bb26fcc
Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?
stderr shows
I0403 14:04:51.026866 6569 exec.cpp:143] Version: 0.28.0
I0403 14:04:51.032097 6585 exec.cpp:217] Executor registered on slave c8bd5b26-6e71-4e18-b490-821dbf7edd9d-S0
WARNING: Your kernel does not support swap limit capabilities, memory limited without swap.
touch: cannot touch ‘/var/jenkins_home/copy_reference_file.log’: Permission denied
I am using
marathon 1.0.0 RC1
mesos 0.28.0
docker 1.10.3
OS is ubuntu 14.04.4 LTS
Does anybody have any pointers to where I'm going wrong? My feeling is that the problem is todo with the persistent volume and the mapping of it into the jenkins container.
I got it working.
git clone https://github.com/jenkinsci/docker.git on to your agent nodes. I've done it on all mine
insert # in front of lines 16 and 17 in Dockerfile e.g
# RUN groupadd -g ${gid} ${group} \
# && useradd -d "$JENKINS_HOME" -u ${uid} -g ${gid} -m -s /bin/bash ${user}
run sudo docker build .
use sudo docker tag xyz jenkins to rename the repo to jenkins and then create an application using docker, jenkins and persistent volumes.
Related
I have a basic container that opens up a ssh tunnel to a machine.
Recently I noticed the container has exited with error code 255 with an error message saying the task already exists:
"Id": "7eb92418992a1a1c3e44d6b47257dc503d4fa4d0f26050956533d617ac369479",
"Created": "2022-08-29T18:19:41.286843867Z",
"Path": "sh",
"Args": [
"-c",
"apk update && apk add openssh-client &&\n chmod 400 ~/.ssh/abc.pem\n while true; do \n exec ssh -o StrictHostKeyChecking=no -i ~/.ssh/abc.pem -nNT -L *:33333:localhost:5001 abc#192.168.1.1; \n done"
],
"State": {
"Status": "exited",
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 255,
"Error": "task 7eb92418992a1a1c3e44d6b47257dc503d4fa4d0f26050956533d617ac369479: already exists",
"StartedAt": "2022-08-30T19:43:58.575463029Z",
"FinishedAt": "2022-08-30T19:51:23.511624168Z"
},
More importantly even though the restart policy is always, the docker engine did not start the container after the container exit.
abc:
container_name: abc
image: alpine:latest
restart: always
command: >
sh -c "apk update && apk add openssh-client &&
chmod 400 ~/.ssh/${PEM_FILENAME}
while true; do
exec ssh -o StrictHostKeyChecking=no -i ~/.ssh/${PEM_FILENAME} -nNT -L *:33333:localhost:5001 abc#${IP};
done"
volumes:
- ./ssh:/root/.ssh:rw
expose:
- 33333
Does anyone know under what situation error task already exists can happen?
Also any idea why docker engine did not start the container after exit?
Update 1:
Also any idea why docker engine did not start the container after exit? [Answered by #Mihai]
According to Restart policy details:
A restart policy only takes effect after a container starts
successfully. In this case, starting successfully means that the
container is up for at least 10 seconds and Docker has started
monitoring it. This prevents a container which does not start at all from going into a restart loop.
Sine we have:
"StartedAt": "2022-08-30T19:43:58.575463029Z",
"FinishedAt": "2022-08-30T19:51:23.511624168Z"
then FinishedAt - StartedAt ~ 8 seconds < 10 seconds that's why docker engine is not restarting the container. Which I think it is not a good logic. docker engine should have a retry mechanism to retry for instance at least 3 times before giving up.
I would suggest this solution:
create Dockerfile in an empty folder as:
FROM alpine:latest
RUN apk update && apk add openssh-client
build the image:
docker build -t alpinessh .
Run it with docker run:
docker run -d \
--restart "always" \
--name alpine_ssh \
-u $(id -u):$(id -g) \
-v $HOME/.ssh:/user/.ssh \
-p 33333:33333 \
alpinessh \
ssh -o StrictHostKeyChecking=no -i /user/.ssh/${PEM_FILENAME} -nNT -L :33333:localhost:5001 abc#${IP}
(make sure to set the env variables that you need)
Running with docker-compose follows the same logic.
** NOTE **
Mapping ~/.ssh inside the container is not the best of ideas. It would be better to copy the key to a different location and use it from there. Reason is: inside the container you are root and any files created in your ~/.ssh by the container would be created/accessed by root (uid=0). For example known_hosts - if you don't already have one, you will get a fresh new one owned by root.
For this reason I am running the container as the current UID:GID on the host.
I have been having really hard time figuring out the problem with my docker image not loading on localhost. I am using Docker for Windows 7. Following is some information regarding my docker image, network, etc, if it can be any helpful in proposing solutions.
I have EXPOSE 80 in my docker file. and listen 8080 in my shinyserver.conf file. I run my image by typing docker run -p 80:8080 imagename. It goes back to command line. Then I check http://localhost or http://localhost:8080, or http://172.17.0.1 or http://172.17.0.1:8080 or http://192.168.99.100 and no image shows up.
When I run docker ps there is no running container (makes sense, since image is not running anyway on localhost). All exited containers show with docker ps -a. There some containers that I haven't even created by show (i guess it comes with creating images).
I run docker inspect to find the IP address and here is what I get (there is no IP on fitfarmz which is my imagename)
$ docker inspect -f '{{.Name}} - {{.NetworkSettings.IPAddress }}' $(docker ps -aq)
/mystifying_johnson -
/wonderful_newton -
/flamboyant_brown -
/gallant_austin -
/fitfarmz-2 -
/fitfarmz -
/amazing_easley -
/nervous_lewin -
/silly_wiles -
/lucid_hopper -
/quizzical_kirch -
/boring_gates -
/clever_booth -
/determined_mestorf -
/pedantic_wozniak -
/goofy_goldstine -
/sharp_ardinghelli -
/xenodochial_lamport -
/keen_panini -
/blissful_lamarr -
/suspicious_boyd -
/confident_hodgkin -
/vigorous_lewin - 172.17.0.2
/quirky_khorana -
/agitated_knuth -
I also got info on docker network:
$ docker network inspect bridge
[
{
"Name": "bridge",
"Id": "9105bd8d679ad2e7d814781b4fa99f375cff3a99a047e70ef63e463c35c5ae28"
,
"Created": "2018-09-08T22:22:51.075519268Z",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.17.0.0/16",
"Gateway": "172.17.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {},
"Options": {
"com.docker.network.bridge.default_bridge": "true",
"com.docker.network.bridge.enable_icc": "true",
"com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
"com.docker.network.bridge.name": "docker0",
"com.docker.network.driver.mtu": "1500"
},
"Labels": {}
}
]
Here is the dockerfile:
FROM r-base:3.5.0
# Install Ubuntu packages
RUN apt-get update && apt-get install -y \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev/unstable \
libxt-dev \
libssl-dev
# Add shiny user
RUN groupadd shiny \
&& useradd --gid shiny --shell /bin/bash --create-home shiny
# Download and install ShinyServer
RUN wget --no-verbose https://download3.rstudio.org/ubuntu-14.04/x86_64/shiny-server-1.5.7.907-amd64.deb && \
gdebi shiny-server-1.5.7.907-amd64.deb
# Install R packages that are required
RUN R -e "install.packages(c('Benchmarking', 'plotly', 'DT'), repos='http://cran.rstudio.com/')"
RUN R -e "install.packages('shiny', repos='https://cloud.r-project.org/')"
# Copy configuration files into the Docker image
COPY shiny-server.conf /etc/shiny-server/shiny-server.conf
COPY /app /srv/shiny-server/
# Make the ShinyApp available at port 80
EXPOSE 80
# Copy further configuration files into the Docker image
COPY shiny-server.sh /usr/bin/shiny-server.sh
CMD ["/usr/bin/shiny-server.sh"]
And shinyserver.conf:
# Define the user we should use when spawning R Shiny processes
run_as shiny;
# Define a top-level server which will listen on a port
server {
# Instruct this server to listen on port 80. The app at dokku-alt need expose PORT 80, or 500 e etc. See the docs
listen 8080;
# Define the location available at the base URL
location / {
# Run this location in 'site_dir' mode, which hosts the entire directory
# tree at '/srv/shiny-server'
site_dir /srv/shiny-server;
# Define where we should put the log files for this location
log_dir /var/log/shiny-server;
# Should we list the contents of a (non-Shiny-App) directory when the user
# visits the corresponding URL?
directory_index on;
}
}
Here is the shiny-server.sh file:
mkdir -p /var/log/shiny-server
chown shiny.shiny /var/log/shiny-server
exec shiny-server >> /var/log/shiny-server.log 2>&1
Any ideas what is going wrong? I use the command: docker run -p 80:8080 fitfarmz to run the image.
I am using Ansible local inside a Packer script to configure a Docker image. I have a role test that has a main.yml file that's supposed to output some information and create a directory to see that the script actually ran. However, the main.yml doesn't seem to get run.
Here is my playbook.yml:
---
- name: apply configuration
hosts: all
remote_user: root
roles:
- test
test/tasks/main.yml:
---
- name: Test output
shell: echo 'testing output from test'
- name: Make test directory
file: path=/test state=directory owner=root
When running this via packer build packer.json I get the following output from the portion related to Ansible:
docker: Executing Ansible: cd /tmp/packer-provisioner-ansible-local/59a33ccb-bd9f-3b49-65b0-4cc20783f193 && ANSIBLE_FORCE_COLOR=1 PYTHONUNBUFFERED=1 ansible-playbook /tmp/packer-provisioner-ansible-local/59a33ccb-bd9f-3b49-65b0-4cc20783f193/playbook.yml --extra-vars "packer_build_name=docker packer_builder_type=docker packer_http_addr=" -c local -i /tmp/packer-provisioner-ansible-local/59a33ccb-bd9f-3b49-65b0-4cc20783f193/packer-provisioner-ansible-local037775056
docker:
docker: PLAY [apply configuration] *****************************************************
docker:
docker: TASK [setup] *******************************************************************
docker: ok: [127.0.0.1]
docker:
docker: PLAY RECAP *********************************************************************
docker: 127.0.0.1 : ok=1 changed=0 unreachable=0 failed=0
I used to run a different more useful role this way and it worked fine. I hadn't run this for a few months and now it stopped working. Any ideas what I am doing wrong? Thank you!
EDIT:
here is my packer.json:
{
"builders": [
{
"type": "docker",
"image": "ubuntu:latest",
"commit": true,
"run_command": [ "-d", "-i", "-t", "--name", "{{user `ansible_host`}}", "{{.Image}}", "/bin/bash" ]
}
],
"provisioners": [
{
"type": "shell",
"inline": [
"apt-get -y update",
"apt-get -y install ansible"
]
},
{
"type": "ansible-local",
"playbook_file": "ansible/playbook.yml",
"playbook_dir": "ansible",
"role_paths": [
"ansible/roles/test"
]
}
]
}
This seems to be due to a bug in Packer. Everything works as expected with any Packer version other than 1.0.4. I recommend either downgrading to 1.0.3 or installing the yet to be released 1.1.0 version.
My best guess is that this is being caused by a known and fixed issue about how directories get copied by the docker builder when using Ansible local provisioner.
I have the following Dockerfile for my container:
FROM centos:centos7
# Install software
RUN yum -y update && yum clean all
RUN yum install -y tar gzip wget && yum clean all
# Install io.js
RUN mkdir /root/iojs
RUN wget https://iojs.org/dist/v1.1.0/iojs-v1.1.0-linux-x64.tar.gz
RUN tar -zxvf iojs-v1.1.0-linux-x64.tar.gz -C /root/iojs
RUN rm -f iojs-v1.1.0-linux-x64.tar.gz
# add io.js to path
RUN echo "PATH=$PATH:/root/iojs/iojs-v1.1.0-linux-x64/bin" >> /root/.bashrc
# go to /src
WORKDIR /src
CMD /bin/bash
I build this container and start the image with docker run -i -t -p 8080:8080 -v /srv/source:/usr/src/app -w /usr/src/app --rm iojs-dev bash. Docker binds the port 8080 to the host port 8080, so that I can access the iojs-application from my client. Everything works fine.
Now I want to start my container with docker-compose, using the following docker-compose.yml
webfrontend:
image: iojs-dev
links:
- db
command: bash -c "iojs test.js"
ports:
- "127.0.0.1:8080:8080"
volumes:
- /srv/source:/usr/src/app
- /logs:/logs
db:
image: mariadb
environment:
MYSQL_ROOT_PASSWORD: 12345
When I now run docker-compose run webfrontend bash I can not access the port 8080 on my host. No port was binded. The result of docker ports is empty and also the result of docker inspect is empty at the port settings:
"NetworkSettings": {
"Bridge": "docker0",
"Gateway": "172.17.42.1",
"IPAddress": "172.17.0.51",
"IPPrefixLen": 16,
"MacAddress": "02:42:ac:11:00:33",
"PortMapping": null,
"Ports": {
"8080/tcp": null
}
},
"HostConfig": {
"Binds": [
"/srv/source:/usr/src/app:rw",
"/logs:/logs:rw"
],
"CapAdd": null,
"CapDrop": null,
"ContainerIDFile": "",
"Devices": null,
"Dns": null,
"DnsSearch": null,
"ExtraHosts": null,
"Links": [
"/docker_db_1:/docker_webfrontend_run_34/db",
"/docker_db_1:/docker_webfrontend_run_34/db_1",
"/docker_db_1:/docker_webfrontend_run_34/docker_db_1"
],
"LxcConf": null,
"NetworkMode": "bridge",
"PortBindings": null,
"Privileged": false,
"PublishAllPorts": false,
"RestartPolicy": {
"MaximumRetryCount": 0,
"Name": ""
},
"SecurityOpt": null,
"VolumesFrom": []
},
This is intentional behavior for docker-compose run, as per documentation:
When using run, there are two differences from bringing up a container normally:
...
by default no ports will be created in case they collide with already opened ports.
One way to overcome this is to use up instead of run, which:
Builds, (re)creates, starts, and attaches to containers for a service.
Another way, if you're using version 1.1.0 or newer, is to pass the --service-ports option:
Run command with the service's ports enabled and mapped to the host.
P.S. Tried editing the original answer, got rejected, twice. Stay classy, SO.
This is intentional behavior for fig run.
Run a one-off command on a service.
One-off commands are started in new containers with the same config as a normal container for that service, so volumes, links, etc will all be created as expected. The only thing different to a normal container is the command will be overridden with the one specified and no ports will be created in case they collide.
source.
fig up is probably the command you're looking for, it will (re)create all containers based on your fig.yml and start them.
I'm trying to build and provision a docker container, but when I run this it hangs when running the provisioning script.
I'm running on a OSX using:
Boot2Docker-cli version: v1.3.1
Packer v0.7.2
**docker version**
Client version: 1.3.1
Client API version: 1.15
Server version: 1.3.1
Server API version: 1.15
Running this:
packer build ./packer-build-templates/docker/testsite/testsite.json
packer-build-templates/docker/testsite/testsite.json
{
"builders": [
{
"type": "docker",
"image": "centos:centos6",
"commit": "true"
}
],
"provisioners": [
{
"type": "shell",
"script": "script.sh"
}
],
"post-processors": [
[
{
"type": "docker-tag",
"repository": "test/stuff",
"tag": "latest"
}
]
]
}
script.sh
#!/bin/sh -x
echo foo
Output:
==> docker: Creating a temporary directory for sharing data...
==> docker: Pulling Docker image: centos:centos6
docker: centos:centos6: The image you are pulling has been verified
docker: Status: Image is up to date for centos:centos6
==> docker: Starting docker container...
docker: Run command: docker run -v /var/folders/z2/nm_4_yyx2ss9z8wn4h0bfd1jw_pj8j/T/packer-docker208208953:/packer-files -d -i -t centos:centos6 /bin/bash
docker: Container ID: 3ab21c7c21bc4af84e0f0c7bdbac91ee600d1ea0a469bfa51a959faba73fa7e4
==> docker: Provisioning with shell script: script.sh
This is as far as it gets. Then it just sits there. Any idea what's going on here?