I am facing a memory leak issue in job-server which is present in docker container. To analyze what is causing the issue I need to attach the jprofiler or yourkit to docker container process. I am not sure how to do that. can someone put some light on it?
You can try and follow "Configure JProfiler 9.2 to profiling applications running in Docker containers" from Andrew Liu:
It would involve completing your existing Dockerfile with:
RUN wget http://download-keycdn.ej-technologies.com/jprofiler/jprofiler_linux_9_2.tar.gz -P /tmp/ &&\
tar -xzf /tmp/jprofiler_linux_9_2.tar.gz -C /usr/local &&\
rm /tmp/jprofiler_linux_9_2.tar.gz
ENV JPAGENT_PATH="-agentpath:/usr/local/jprofiler9/bin/linux-x64/libjprofilerti.so=nowait"
EXPOSE 8849
That would enable you to exec a bash to the running container:
docker exec -it [container-name] bash
cd /usr/local/jrofiler9/
bin/jpenable
Alternatively, if you want to enable JProfiler agent at your web server start up and wait for JProfiler GUI connecting from host, instead of putting "ENV JPAGENT_PATH="-agentpath:/usr/local/jprofiler9/bin/linux-x64/libjprofilerti.so=nowait"" in the Dockerfile. Add following line to the JAVA_OPTS. For tomcat, it will be CATALINA_OPTS.
Note: the config.xml will be the place to put your JProfiler license key.
JAVA_OPTS="$JAVA_OPTS -agentpath:/usr/local/jprofiler9/bin/linux-x64/libjprofilerti.so=port=8849,wait,config=/usr/local/jprofiler9/config.xml"
Now you are done at the docker container side. The container is ready to be attached to your JProfiler GUI. The steps below are to be done on the host machine.
Download JProfiler 9.2 from https://www.ej-technologies.com/download/jprofiler/files and install it.
Open JProfiler and open a new session by press Ctrl + N or Click 'New Session' in Session menu.
Select 'Attach to profiled JVM (local or remote)' in Session Type section. Enter the IP address and 8849 as profiling port in Profiled JVM Settings section. Leave the other settings as default. Then click OK.
You can attach your jProfiler to the application inside your docker container like this:
EXPOSE 8849
Exposing the profiling port is important (8849 is default)
RUN wget http://download-keycdn.ej-technologies.com/jprofiler/jprofiler_linux_9_2_1.tar.gz --no-verbose -P /tmp/ && \
tar -xzf /tmp/jprofiler_linux_9_2_1.tar.gz -C /usr/local && \
rm /tmp/jprofiler_linux_9_2_1.tar.gz
This downloads and extracts the jProfiler inside your docker container, when the container is build.
ENTRYPOINT exec java -jar /app.jar & \
echo $! >>/tmp/process.pid && \
sleep 60s && \
/usr/local/jprofiler9/bin/jpenable --pid=$(cat /tmp/process.pid) --gui --port=8849 && \
while true; do sleep 2147483647; done
This is how I dealt with the fact that you can't run two applications inside one docker container. First we execute the jar and save the processId to a file. Then we simply wait 60 seconds and after that we start the jProfiler (jpenable) and attach it to our process (via processId). The while loop is necessary to keep the container running afterwards.
Related
I'm currently facing the problem of Emacs not being able to connect to MELPA. When researching this problem, I found out that this problem is discussed in many topics. However, every suggested solution I tried didn't work for me, beside that, the way I've set up my Emacs instance makes me think the root of the problem might be somewhere else (not emacs related itself).
So let me explain my setup:
I have a Windows 10 host, and I'm trying to get GUI Emacs for Linux running on that host. I've tried various methods (VMs, Emacs for Windows, ...). Currently I'm trying to run Emacs inside a Docker container with X11 forwarding with XMing on my host.
Docker version on host: Docker version 19.03.2, build 6a30dfc
I'm using this docker container as a base. However I modified the Dockerfile a bit to fit my needs (I should mention that I've never worked with docker before):
FROM alpine:edge
MAINTAINER Daniel Guerra <daniel.guerra69#gmail.com>
ARG authorizedKeys=authorized_keys
RUN apk add --update openssh util-linux dbus ttf-freefont xauth xf86-input-keyboard emacs-x11 bash git sudo\
&& rm -rf /tmp/* /var/cache/apk/*
RUN addgroup alpine \
&& adduser -G alpine -s /bin/bash -D alpine \
&& echo "alpine:alpine" | /usr/sbin/chpasswd \
&& echo "alpine ALL=(ALL) ALL" >> /etc/sudoers
RUN cp -r /etc/ssh /ssh_orig
RUN rm -rf /etc/ssh/*
ADD etc /etc
ADD docker-entrypoint.sh /usr/local/bin
VOLUME ["/etc/ssh"]
RUN mkdir -p /home/alpine/.ssh
ADD $authorizedKeys /home/alpine/.ssh/authorized_keys
RUN mkdir -p /home/alpine/.config
RUN git clone https://github.com/minikN/dotemacs.git /home/alpine/.config/emacs/
RUN mkdir -p /home/alpine/.emacs.d
RUN ln -s /home/alpine/.config/emacs/init.el /home/alpine/.emacs.d/init.el
RUN ln -s /home/alpine/.config/emacs/config.el /home/alpine/.emacs.d/config.el
RUN ln -s /home/alpine/.config/emacs/config.org /home/alpine/.emacs.d/config.org
RUN chown -R alpine:alpine /home/alpine/.emacs.d
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["/usr/sbin/sshd","-D"]
I basically copy my hosts authorized_keys to the container. I do this because I'm using Putty and it's ssh-agent to connect to the machine via SSH.
After that I just copy my emacs config from my GitHub to the container.
You can find the emacs config here.
If I now start the container connect to it via SSH and start emacs, it opens in XMing. However it just hangs (in fact the whole container hangs up), I just see a white screen. Upon restarting the container and running emacs --daemon I can see that it hangs at
alpine-sshdx:~$ emacs --daemon
Contacting host: melpa.org:443
BTW I've checked several mirrors. melpa.org:443 is just the latest try. Also tried both HTTP/HTTPS.
However doing ping 8.8.8.8 works just fine.
I have absolutely no idea what the cause of this is. But I believe it's something in the container.
Would appreciate any help.
Dockerfile
FROM drupal
RUN apt-get update
RUN apt-get install openssh-server -y
RUN apt-get install -y supervisor
#SS Related Fix : https://github.com/Microsoft/WSL/issues/3621
RUN mkdir -p /run/sshd
# SS Access Configuration
RUN echo "root:Docker!" | chpasswd
#Project Uplaod
RUN rm -rf /var/www/html/*
COPY ./html/ /var/www/html/
# Startup Configuration
COPY servername.conf /etc/apache2/conf-enabled/servername.conf
ADD supervisord.conf /etc/supervisor/conf.d/supervisord.conf
CMD ["/usr/bin/supervisord"]
Start Command : docker -D run -p 80:80 -p 2222:22 -it /bin/bash
[supervisord]
nodaemon=true
[program:SSH]
command=/usr/sbin/sshd start
[program:Apache]
command=/etc/init.d/apache2 start
when i jump into Shell and run that command it works but when i start container its not starting up the web server.
As standing in documentation
To start supervisord, run $BINDIR/supervisord. The resulting process
will daemonize itself and detach from the terminal. It keeps an
operations log at $CWD/supervisor.log by default.
You may start the supervisord executable in the foreground by passing
the -n flag on its command line. This is useful to debug startup
problems.
So systemd detach from main process what means for docker that process ended - exit container. To solve your problem you need to change CMD section to
CMD ["/usr/bin/supervisord", "-n"]
When you run
docker -D run -p 80:80 -p 2222:22 -it /bin/bash
The last part of the command, /bin/bash, replaces the CMD in the Dockerfile, so you only get the GNU bash shell. You should remove that part of the line and the standard command from your image will run.
You might consider how much you actually need an interactive shell in your Docker environment. Most application images are set up to run totally on their own without manual setup steps; compare the stock mysql or nginx images, for instance, which don't include any kind of remote login system. Also consider that anyone who can run docker history can now trivially find out your root password, and you have no way to manage the sshd host keys. I'd suggest removing this entire supervisord/sshd system and just packaging your application.
Given a dyno in which a container is running, what's the Heroku equivalent of docker exec -it blarg /bin/bash? That is, how can one open a shell into the already-running container?
Example Dockerfile:
FROM heroku/heroku:16
CMD while true; do sleep 1; done
Example run:
$ heroku container:push my_app
<wait a minute>
$ heroku ps
=== my_app (Free): /bin/sh -c while\ true\;\ do\ sleep\ 1\;\ done (1)
my_app.1: up 2017/10/09 12:13:07 -0600 (~ 4m ago)
So far so good.
But now...
$ heroku ps:exec --dyno=my_app.1
Establishing credentials... error
▸ Could not connect to dyno!
▸ Check if the dyno is running with `heroku ps'
For good measure I check heroku ps at this point and it shows that the dyno is still running.
Yes, I've done all the things Heroku suggests to enable Docker support. Per the documentation, I have tried using a base image of my choice while ensuring that bash, curl, openssh, and python are present. I have also tried using the Heroku-16 base image, as shown in the above example.
(The linked documentation also references steps required for Private Spaces. Since I'm not using Private Spaces I have not applied those steps.)
EDIT CIRCA 2022 This was the accepted answer in 2017. I'm not so sure anymore, so I'm unaccepting my answer here to avoid misleading anyone. I'm not fiddling with Heroku + Docker these days, so I'm not in a good position to accept an answer.
TL;DR Ensure that bash is installed in the image and add this to your Dockerfile:
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
Explanation
Contrary to what the documentation would lead one to believe, Heroku does not, out of the box, support heroku ps:exec into a Docker container already running in a dyno.
Quoting from responses I have received from the Heroku team:
Our ps:exec feature ... works ... by injecting a bash file into dynos,
opening an additional port in the background, and allowing you to
connect to it.
[T]he default
shell used by Docker is /bin/sh, which is not compatible with the
Heroku Exec script (it's requires /bin/bash).
There is a workaround you can use though. Put the following in your
Dockerfile:
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
This is definitely a gap in
our product, and we'll work to make this better.
If bash is installed, run heroku run bash. This will get you into the shell from your command line.
You can also use the GUI and go to "More" -> "Run Console" on your heroku app, and input "bash" to bring it up there.
Edited:
In order to run heroku ps:exec on apps with Docker and deployed via the Container Registry you have to enable runtime-heroku-exec.
You can do heroku features:enable runtime-heroku-exec to enable it
Here you can see the documentation of exec with the instructions to enable docker support
in my situation, to get this working with Ubuntu 20.04 (focal), i had to additionally install the python-is-python3 package into the docker image, to get heroku-exec working.
here is a working example (october 2020) of an ubuntu-based dockerfile, that works with heroku-exec:
FROM ubuntu:focal
# install required packages
RUN export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get install -y python3 curl python-is-python3 openssh-server iproute2 nginx && apt-get clean
# simplfy nginx config to enable ENV variable substitution
RUN echo 'server { listen PORT_NUMBER; }' > /etc/nginx/sites-enabled/default
# add config required for HEROKU_EXEC
# ENV HEROKU_EXEC_DEBUG=1
RUN rm /bin/sh \
&& ln -s /bin/bash /bin/sh \
&& mkdir -p /app/.profile.d/ \
&& printf '#!/usr/bin/env bash\n\nset +o posix\n\n[ -z "$SSH_CLIENT" ] && source <(curl --fail --retry 7 -sSL "$HEROKU_EXEC_URL")\n' > /app/.profile.d/heroku-exec.sh \
&& chmod +x /app/.profile.d/heroku-exec.sh
# configure NGINX to listen on dynamic $PORT env variable supplied by Heroku.
CMD sed -i 's/PORT_NUMBER/'"$PORT"'/g' /etc/nginx/sites-enabled/default; nginx -g 'daemon off;'
then connect with this command:
heroku ps:exec -a name-of-app-12345
Both heroku run /bin/bash and heroku ps:exec won't work in my situation. The former opens a new container which is different from the real one running! The latter just doesn't work in my container of alpine3, though heroku features:enable runtime-heroku-exec can succeed. My solution is to bring up a shell server and a traffic forwarder in the container. Then on the client connect to the shell server with tunnel created by traffic forwarder.
traffic flow:
localhost:2023 -> chisel client -> ...tunnel... -> chisel server -> localhost:8182
In conainer, start up a shell server with socat and a tunnel server with chisel:
nohup socat tcp-l:8182,reuseaddr,fork exec:/bin/bash,pty,setsid,setpgid,stderr,ctty > /tmp/socat.log 2>&1 &
nohup ./chisel server --port $PORT --proxy http://httpbin.org > /tmp/chisel.log 2>&1 &
On client side, start a chisel client to forward traffic from localhost:8182 to the socat on the server
chisel client http://yourapp.herokuapp.com/ 0.0.0.0:2023:localhost:8182
on client side, open another terminal window:
socat -,raw,echo=0 tcp:127.0.0.1:2023
How to get the chisel on server? Download it or just compile from source in Dockerfile
download chisel
One probably root cause is the docker not running in detached mode on Heroku.
https://docs.docker.com/language/nodejs/run-containers/#run-in-detached-mode
Have someone do know how to activate -d option when the container is being executed by Heroku?
Im having a recurring issue while trying to set up a Docker container so that it stays running.
Here is a sample of the Dockerfile that I am wanting to use:
RUN wget -O /usr/local/nexus-2.11.3-01-bundle.tar.gz http://www.sonatype.org/downloads/nexus-2.11.3-01-bundle.tar.gz
WORKDIR /usr/local
RUN tar xvzf /usr/local/nexus-2.11.3-01-bundle.tar.gz
RUN ln -s nexus-2.11.3-01 nexus
ENV NEXUS_HOME /usr/local/nexus
ENV RUN_AS_USER root
CMD ["/usr/local/nexus/bin/nexus", "start"]
EXPOSE 8081
Basically when I build this, and then run it, the container just dies, and doing a docker ps command returns that there are no running containers.
As far as I know, (please correct me if I'm wrong...) the docker container should stay running so long as theres a process with a pid of 1. Would the usage of the previous commands use PID 1, and if so, how can I force the nexus start command to use it? Or to just keep the container alive...
The contents of a docker logs nexus gives:
****************************************
WARNING - NOT RECOMMENDED TO RUN AS ROOT
****************************************
Starting Nexus OSS...
Started Nexus OSS.
It seems to suggest that Nexus has started, but then again when I do a docker ps, I don't see it running.
If the process running with PID 1 exits, then the container is automatically stopped. You can check on the sonatype/nexus repository here, using the concept of Launcher.
Here is how they are avoiding the container to exit:
...
RUN mkdir -p /opt/sonatype/nexus \
&& curl --fail --silent --location --retry 3 \
https://download.sonatype.com/nexus/professional-bundle/nexus-professional-${NEXUS_VERSION}-bundle.tar.gz \
| gunzip \
| tar x -C /tmp nexus-professional-${NEXUS_VERSION} \
&& mv /tmp/nexus-professional-${NEXUS_VERSION}/* /opt/sonatype/nexus/ \
&& rm -rf /tmp/nexus-professional-${NEXUS_VERSION}
RUN useradd -r -u 200 -m -c "nexus role account" -d ${SONATYPE_WORK} -s /bin/false nexus
...
EXPOSE 8081
WORKDIR /opt/sonatype/nexus
USER nexus
ENV CONTEXT_PATH /
ENV MAX_HEAP 768m
ENV MIN_HEAP 256m
ENV JAVA_OPTS -server -XX:MaxPermSize=192m -Djava.net.preferIPv4Stack=true
ENV LAUNCHER_CONF ./conf/jetty.xml ./conf/jetty-requestlog.xml
CMD java \
-Dnexus-work=${SONATYPE_WORK} -Dnexus-webapp-context-path=${CONTEXT_PATH} \
-Xms${MIN_HEAP} -Xmx${MAX_HEAP} \
-cp 'conf/:lib/*' \
${JAVA_OPTS} \
org.sonatype.nexus.bootstrap.Launcher ${LAUNCHER_CONF}
Since it is an open repository, you can directly refer to their repo, if you like.
A quick guess from the logs is that running /usr/local/nexus/bin/nexus start would start it as a daemon.
That would cause another process to spawn and the one that started the daemon would exit, terminating the container.
One solution is to start the process not as a daemon, but I couldn't find a option to do this in your nexus case.
Another is to use something like supervisord as the CMD to docker. Then make it start your process.
We have docker running on one machine
Workstation running on other machine
I want to do bootstrap from workstation on docker container then our image should be ssh enabled
How to make docker image ssh enabled.
Before you add ssh you should see if docker exec will be sufficient for what you need. (doc link)
If you do need SSH, the following Dockerfile should help (copied from Docker docs):
# sshd
#
# VERSION 0.0.2
FROM ubuntu:14.04
MAINTAINER Sven Dowideit <SvenDowideit#docker.com>
RUN apt-get update && apt-get install -y openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:screencast' | chpasswd
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
# SSH login fix. Otherwise user is kicked off after login
RUN sed 's#session\s*required\s*pam_loginuid.so#session optional pam_loginuid.so#g' -i /etc/pam.d/sshd
ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
Using the CMD command in your Dockerfile will indeed enable ssh
CMD ["/usr/sbin/sshd", "-D"]
But there is a huge downside. If you already have a CMD command (that starts MySQL for example), then you are facing a problem not easily resolved in Docker. You can use only one CMD in Dockerfile. But there is a workaround for that, using supervisor. What you do is tell Dockerfile to install Supervisor:
RUN apt-get install -y openssh-server supervisor
Using supervisor, you can start as many processes as you want on container startup. These processes are defined in supervisor.conf file (naming is arbitrary) located in the directory with your Dockerfile. In your Dockerfile you tell Docker to copy this file during building:
ADD supervisor-base.conf /etc/supervisor.conf
Then you tell Docker to start supervisor when container starts (when supervisor starts, supervisor will also start all processes listed in the supervisor.conf file mentioned above).
CMD ["supervisord", "-c", "/etc/supervisor.conf"]
Your supervisor.conf file may look like this:
[supervisord]
nodaemon=true
[program:sshd]
directory=/usr/local/
command=/usr/sbin/sshd -D
autostart=true
autorestart=true
redirect_stderr=true
There is one issue to be careful about. Supervisor needs to start as a root, otherwise it will throw errors. So if your Dockerfile defines an user to start container with (e.g USER jboss), then you should put USER root at the end of your Dockerfile, so that supervisor starts with root. In your supervisor.conf file you simply define a user for each process:
[program:wildfly]
user=jboss
command=/opt/jboss/wildfly/bin/standalone.sh -b 0.0.0.0 -bmanagement 0.0.0.0
[program:chef]
user=chef
command=/bin/bash -c chef-2.1/bin/start.sh
Of course, these users need to be pre-defined in your dockerfile. E.g.
RUN groupadd -r -f jboss -g 2000 && useradd -u 2000 -r -g jboss -m -d /opt/jboss -s /sbin/nologin -c "JBoss user" jboss
You can learn more about Supervisor+Docker+SSH in more details in this article.
Notice: this answer promotes a tool I've written.
Some answers here suggest to place an SSH server inside your container. Conceptually running multiple processes in one container is not the right approach (https://docs.docker.com/articles/dockerfile_best-practices/). A more favorable solution is one that involves multiple containers each running their own process/service. Linking them together would result in a coherent application.
I've created a containerized SSH server that you can 'stick' to any running container. This way you can create compositions with every container, without that container even knowing about ssh. The only requirement is that the container has bash.
The following example would start an SSH server attached to a container with name 'sshd-web-server1'.
docker run -ti --name sshd-web-server1 -e CONTAINER=web-server1 -p 2222:22 \
-v /var/run/docker.sock:/var/run/docker.sock -v $(which docker):/usr/bin/docker \
jeroenpeeters/docker-ssh
You connect to the SSH server with your ssh client of choice, just as you normally would.
Be adviced: Docker-SSH is currently still under development, but it does work! Please let me know what you think
For more pointers and documentation see: https://github.com/jeroenpeeters/docker-ssh
You can find prebuilt images with SSH installed, for instance CentOS tutum/centos and Debian tutum/debian
And the Dockerfiles used to build them
https://github.com/tutumcloud/tutum-centos/blob/master/Dockerfile
https://github.com/tutumcloud/tutum-debian/blob/master/Dockerfile