I added a new worker/slave via a Docker plugin agent template that uses an Alpine image. The agent is instantiated, invoked, and authentication succeeds, but the launch of the agent on the slave fails with the following error (which I get from Jenkins > Nodes > my_node_id > See log for more details):
[03/27/19 18:11:39] [SSH] Authentication successful.
SSH connection reports a garbage before a command execution.
Check your .bashrc, .profile, and so on to make sure it is quiet.
The received junk text is as follows:
nologin: this account is not available
null
[03/27/19 18:11:39] Launch failed - cleaning up connection
[03/27/19 18:11:39] [SSH] Connection closed.
There's not enough info for me to debug this - just the error output from some unknown command. Looks like the agent is issueing some command with a nologin parameter that is being misinterpreted as a userid. My guess would be it's some slave image configuration or package install problem.
Is there any way I can find out exactly what command(s) the agent issued? Any way to get more complete agent launch logs?
FWIW here's my slave Dockerfile:
FROM docker:stable-dind
RUN apk update; \
apk upgrade; \
apk add git; \
apk add python3; \
pip3 install pyyaml pexpect requests ruamel.yaml; \
apk add openjdk8; \
apk add curl; \
apk add sudo; \
apk add bash; \
apk add openssh-server; \
rm -rf /etc/ssh/ssh_host_rsa_key /etc/ssh/ssh_host_dsa_key; \
/usr/bin/ssh-keygen -A
EXPOSE 22
# Docker and Jenkins users
RUN addgroup -g 1001 -S docker; \
adduser -D -u 1001 -S appuser -G docker; \
addgroup -S jenkins; adduser -S jenkins -G jenkins; \
echo "jenkins:xxxxxxxxx" | chpasswd; \
chown -R jenkins:jenkins /home/jenkins
CMD ["/usr/sbin/sshd", "-D"]
The nologin: this account is not available problem was because Jenkins agent requires bash, not sh. So changing the Dockerfile adduser fixed that problem:
adduser -S jenkins -G jenkins --shell /bin/bash
Now on to the next error :-)
Note I did not find out how to get more detailed info about the agent commands or logs. If anyone knows that I would be interested to know how.
Related
Idea here is simple - dbt provides a way to generate static files and serve them by using commands dbt docs generate and dbt docs serve and I want to share in a way that everyone in my organization can see them (bypassing security concerns as of now). For this task I thought Cloud Run would be ideal solution as I already have Dockerfile and bash scrips which do some background work (cron job to clone git repo every x hours, etc.). Running this container locally works fine. But deploying this image in Cloud Run wasn't successful - it fails on the last step (which is dbt docs server --port 8080) with default error message Cloud Run error: The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable. Logs for this revision might contain more information. No additional information in logs before that wasn't printed.
Dockerfile:
FROM --platform=$build_for python:3.9.9-slim-bullseye
WORKDIR /usr/src/dbtdocs
RUN apt-get update && apt-get install -y --no-install-recommends git apt-transport-https ca-certificates gnupg curl cron \
&& apt-get clean
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install tzdata
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - && apt-get update -y && apt-get install google-cloud-sdk -y
RUN python -m pip install --upgrade pip setuptools wheel --no-cache-dir
RUN pip install dbt-bigquery
RUN ln -s /usr/local/bin/dbt /usr/bin/
RUN rm -rf /var/lib/apt/lists/*
COPY ./api-entrypoint.sh /usr/src/dbtdocs/
COPY ./cron_dbt_docs.sh /usr/src/dbtdocs/
COPY ./cron_script.sh /usr/src/dbtdocs/
ENV PORT=8080
RUN chmod 755 api-entrypoint.sh
RUN chmod 755 cron_dbt_docs.sh
RUN chmod 755 cron_script.sh
ENTRYPOINT ["/bin/bash", "-c", "/usr/src/dbtdocs/api-entrypoint.sh" ] ```
api-entrypoint.sh
#!/bin/bash
#set -e
#catch() {
# echo 'catching!'
# if [ "$1" != "0" ]; then
# echo "Error $1 occurred on $2"
# fi
#}
#trap 'catch $? $LINENO' EXIT
exec 2>&1
echo 'Starting DBT Workload'
echo 'Checking dependencies'
dbt --version
git --version
mkdir -p /data/dbt/ && cd /data/dbt/
echo 'Cloning dbt Repo'
git clone ${GITLINK} /data/dbt/
echo 'Working on dbt directory'
export DBT_PROFILES_DIR=/data/dbt/profile/
echo "Authentificate at GCP"
echo "Decrypting and saving sa.json file"
mkdir -p /usr/src/secret/
echo "${SA_SECRET}" | base64 --decode > /usr/src/secret/sa.json
gcloud auth activate-service-account ${SA_EMAIL} --key-file /usr/src/secret/sa.json
echo 'The Project set'
if test "${PROJECT_ID}"; then
gcloud config set project ${PROJECT_ID}
gcloud config set disable_prompts true
else
echo "Project Name not in environment variables ${PROJECT_ID}"
fi
echo 'Use Google Cloud Secret Manager Secret'
if test "${PROFILE_SECRET_NAME}"; then
#mkdir -p /root/.dbt/
mkdir -p /root/secret/
gcloud secrets versions access latest --secret="${PROFILE_SECRET_NAME}" > /root/secret/creds.json
export GOOGLE_APPLICATION_CREDENTIALS=/root/secret/creds.json
else
echo 'No Secret Name described - GCP Secret Manager'
fi
echo 'Apply cron Scheduler'
sh -c "/usr/src/dbtdocs/cron_script.sh install"
/etc/init.d/cron restart
touch /data/dbt_docs_job.log
sh -c "/usr/src/dbtdocs/cron_dbt_docs.sh"
touch /data/cron_up.log
tail -f /data/dbt_docs_job.log &
tail -f /data/cron_up.log &
dbt docs serve --port 8080
Container port is set to 8080 when creating Cloud Run service, so I don't think it's a problem here.
Have someone actually encountered similar problems using Cloud Run?
Logs in Cloud Logging
Your container is not listening/responding on port 8080 and has been terminated before the server process starts listening.
Review the last line in the logs. The previous line is building catalog.
Your container is taking too long to startup. Containers should start within 10 seconds because Cloud Run will only keep pending requests for 10 seconds.
All of the work I see in the logs should be performed before the container is deployed and not during container start.
The solution is to redesign how you are building and deploying this container so that the application begins responding to requests as soon as the container starts.
I am running an Elastic and Kibana service within a container using an Azure Web app container service. I was keen on checking the SSH connectivity for this container using Azures Web SSH console feature. Followed the microsoft documentation for SSH into custom containers https://learn.microsoft.com/en-us/azure/app-service/configure-custom-container?pivots=container-linux#enable-ssh which shows the example of running the container as default root user.
My issue is Elasticsearch process does not run as a root user so I had to make the sshd process run as an elastic user. I was able to get the sshd process running which accepts the SSH connection from my host however the credentials I am setting in the docker file (elasticsearch:Docker!) are throwing Access Denied error.Any idea where i am going wrong here?
Dockerfile
FROM openjdk:jre-alpine
ARG ek_version=6.5.4
RUN apk add --quiet --no-progress --no-cache nodejs wget \
&& adduser -D elasticsearch \
&& apk add openssh \
&& echo "elasticsearch:Docker!" | chpasswd
# Copy the sshd_config file to the /etc/ssh/ directory
COPY startup.sh /home/elasticsearch/
RUN chmod +x /home/elasticsearch/startup.sh && \
chown elasticsearch /home/elasticsearch/startup.sh
COPY sshd_config /home/elasticsearch/
USER elasticsearch
WORKDIR /home/elasticsearch
ENV ES_TMPDIR=/home/elasticsearch/elasticsearch.tmp ES_DATADIR=/home/elasticsearch/elasticsearch/data
RUN wget -q -O - https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-${ek_version}.tar.gz \
| tar -zx \
&& mv elasticsearch-${ek_version} elasticsearch \
&& mkdir -p ${ES_TMPDIR} ${ES_DATADIR} \
&& wget -q -O - https://artifacts.elastic.co/downloads/kibana/kibana-oss-${ek_version}-linux-x86_64.tar.gz \
| tar -zx \
&& mv kibana-${ek_version}-linux-x86_64 kibana \
&& rm -f kibana/node/bin/node kibana/node/bin/npm \
&& ln -s $(which node) kibana/node/bin/node \
&& ln -s $(which npm) kibana/node/bin/npm
EXPOSE 9200 5601 2222
ENTRYPOINT ["/home/elasticsearch/startup.sh"]
startup.sh script
#!/bin/sh
# Generating hostkey
ssh-keygen -f /home/elasticsearch/ssh_host_rsa_key -N '' -t rsa
# starting sshd process
echo "Starting SSHD"
/usr/sbin/sshd -f sshd_config
# Staring the ES stack
echo "Starting ES"
sh elasticsearch/bin/elasticsearch -E http.host=0.0.0.0 & kibana/bin/kibana --host 0.0.0.0
sshd_config file
Port 2222
HostKey /home/elasticsearch/ssh_host_rsa_key
PidFile /home/elasticsearch/sshd.pid
ListenAddress 0.0.0.0
LoginGraceTime 180
X11Forwarding yes
Ciphers aes128-cbc,3des-cbc,aes256-cbc,aes128-ctr,aes192-ctr,aes256-ctr
MACs hmac-sha1,hmac-sha1-96
StrictModes yes
SyslogFacility DAEMON
PasswordAuthentication yes
PermitEmptyPasswords no
PermitRootLogin yes
Subsystem sftp internal-sftp
Error i am getting
Please check and verify that your docker image supports SSH. It would appear that you have done everything correctly so one of the final troubleshooting steps left as this point is to verify that your image supports SSH to begin with.
I have defined a node in the jenkins cloud config with the following docker file, everything is fine, except when I run a job it runs as root user.
FROM jenkins/inbound-agent:alpine as jnlp
FROM maven:3.6.3-jdk-11
ARG DOCKER_VERSION=18.03.0-ce
ARG DOCKER_COMPOSE_VERSION=1.21.0
ARG USER=jenkins
USER root
COPY --from=jnlp /usr/local/bin/jenkins-agent /usr/local/bin/jenkins-agent
COPY --from=jnlp /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
RUN apt-get install ca-certificates wget -y \
&& rm -r /var/lib/apt/lists /var/cache/apt/archives \
&& wget https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl -q -O /usr/local/bin/kubectl \
&& chmod a+x /usr/local/bin/kubectl
RUN curl -fsSL https://download.docker.com/linux/static/stable/`uname -m`/docker-$DOCKER_VERSION.tgz | tar --strip-components=1 -xz -C /usr/local/bin docker/docker
RUN curl -fsSL https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose
RUN touch /debug-flag
RUN useradd jenkins
USER jenkins
ENTRYPOINT ["/usr/local/bin/jenkins-agent"]
When my job start, the user is root. And I need to run with a non root user to some jobs, my shared library on jenkins work well.
The only problem that I have is when I run some test on some projects with a database embedded, due to it needs to run in a non root user.
Check your Pod Template and Container Template to see if you set the "runAsUser" and "runAsGroup". You can set to 1000 as default "jenkins" user uid and gid.
I'm trying to create a Jenkins Docker agent that has Go.
The following is my Dockerfile.
After I build it, if I try: docker run myimage:0.0.1 go version returns the Go version, however if I try this, it doesn't find Go at all.
docker run --privileged --dns 9.0.128.50 --dns 9.0.130.50 -d -P --name slave myimage:0.0.1
docker ps ## grab the port number
ssh -p PORT_NUMBER jenkins#localhost
What am I missing in order to make Go available under the Jenkins user?
FROM golang:1.11.5-alpine
RUN apk add --no-cache \
bash \
curl \
wget \
git \
openssh \
tar
COPY ssh/*key /etc/ssh/
COPY skel/ /home/jenkins
COPY id_rsa /home/jenkins/.ssh/id_rsa
COPY id_rsa.pub /home/jenkins/.ssh/id_rsa.pub
RUN addgroup docker \
&& adduser -s /bin/bash -h /home/jenkins -G docker -D jenkins \
&& echo "jenkins ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers \
&& echo "jenkins:jenkinspass" | chpasswd \
&& chmod u+s /bin/ping \
&& chown -R jenkins:docker /home/jenkins \
&& mv /etc/profile.d/color_prompt /etc/profile.d/color_prompt.sh \
&& mv /bin/sh /bin/sh.bak \
&& ln -s /bin/bash /bin/sh
# Standard SSH port
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
If you run:
docker run myimage:0.0.1 which go
You will see that go executable in path /usr/local/go/bin/go
If you connect as jenkins user via ssh and run /usr/local/go/bin/go version all work as well.
Conclusion:
Go installation provided as root user
jenkins user added after go installed and haven't /usr/local/go/bin/go in his $PATH environment variable.
Solution:
Add /usr/local/go/bin/go to $PATH for user jenkins
Use go executable with full path.
I deployed my first scala project on docker but i have a problem, the problem is the docker says that the server has been started, but surprisingly it doesn't listen to any request, even i exposed the port to the host, when i tried to request a get, it says that the connection is refused, also i tried to telnet to the port and it seems that there are no listener on port 9000 neither 3200 an 3000, please find bellow what i have wrote in dockerFile
FROM jelastic/sbt
# Env variables
ENV SCALA_VERSION 2.12.4
ENV SBT_VERSION 1.1.0
# Scala expects this file
RUN touch /usr/lib/jvm/java-8-openjdk-amd64/release
# Install Scala
## Piping curl directly in tar
RUN \
curl -fsL https://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz | tar xfz - -C /root/ && \
echo >> /root/.bashrc && \
echo "export PATH=~/scala-$SCALA_VERSION/bin:$PATH" >> /root/.bashrc
# Install sbt
RUN \
curl -L -o sbt-$SBT_VERSION.deb https://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb && \
apt-get update && \
apt-get install sbt && \
sbt sbtVersion
WORKDIR /
ADD play /
RUN tree /
EXPOSE 9000
CMD sbt run
and my run command was
docker run -p 9000:9000 -t bee while bee is my image name
as you see the server is started properly.
please find bellow the attached picture to be more clearly
here is the docker ps
If you see your screenshot, it clear states the docker machine is located at 192.168.99.100. So that is the address you need to use.
Open http://192.168.99.100:9000 and it should work