Why dockered centos doesn't recognize pip? - docker

I want to create a container with python and few packages over centos. I've tried to run several commands inside raw centos container. Everything worked fine I've installed everything I want. Then I created Dockerfile with the same commands executed via RUN and I'm getting /bin/sh: pip: command not found What could be wrong? I mean the situation at all. Why everything could be executed in the command line but not be executed with RUN? I've tried both variants:
RUN command
RUN command
RUN pip install ...
and
RUN command\
&& command\
&& pip install ...
Commands that I execute:
from centos
run yum install -y centos-release-scl\
&& yum install -y rh-python36\
&& scl enable rh-python36 bash\
&& pip install django
UPD: Full path to the pip helped. What's wrong?

You need to install pip first using
yum install python-pip
or if you need python3 (from epel)
yum install python36-pip
When not sure, ask yum:
yum whatprovides /usr/bin/pip
python2-pip-18.1-1.fc29.noarch : A tool for installing and managing Python 2 packages
Repo : #System
Matched from:
Filename : /usr/bin/pip
python2-pip-18.1-1.fc29.noarch : A tool for installing and managing Python 2 packages
Repo : updates
Matched from:
Filename : /usr/bin/pip
python2-pip-18.0-4.fc29.noarch : A tool for installing and managing Python 2 packages
Repo : fedora
Matched from:
Filename : /usr/bin/pip
This output is from Fedora29, but you should get similar result in Centos/RHEL
UPDATE
From comment
But when I execute same commands from docker run -ti centos everything
is fine. What's the problem?
Maybe your PATH is broken somehow? Can you try full path to pip?

As it has already been mentioned by #rkosegi, it must be a PATH issue. The following seems to work:
FROM centos
ENV PATH /opt/rh/rh-python36/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
RUN yum install -y centos-release-scl
RUN yum install -y rh-python36
RUN scl enable rh-python36 bash
RUN pip install django
I "found" the above PATH by starting a centos container and typing the commands one-by-one (since you've mentioned that it is working).
There is a nice explanation on this, in the slides of BMitch which can be found here: sudo-bmitch.github.io/presentations/dc2018/faq-stackoverflow.html#24
Q: Why doesn't RUN work?
Why am I getting ./build.sh is not found?
RUN cd /app/srcRUN ./build.sh
The only part saved from a RUN is the filesystem (as a new layer).
Environment variables, launched daemons, and the shell state are all discarded with the temporary container when pid 1 exits.
Solution: merge multiple lines with &&:
RUN cd /app/src && ./build.sh

I know this was asked a while ago, but I just had this issue when building a Docker image, and wasn't able to find a good answer quickly, so I'll leave it here for posterity.
Adding the scl enable command wouldn't work for me in my Dockerfile, so I found that you can enable scl packages without the scl command by running:
source /opt/rh/<package-name>/enable.
If I remember correctly, you won't be able to do:
RUN source /opt/rh/<package-name>/enable
RUN pip install <package>
Because each RUN command creates a different layer, and shell sessions aren't preserved, so I just ran the commands together like this:
RUN source /opt/rh/rh-python36/enable && pip install <package>
I think the scl command has issues running in Dockerfiles because scl enable <package> bash will open a new shell inside your current one, rather than adding the package to the path in your current shell.
Edit:
Found that you can add packages to your current shell by running:
source scl_source enable <package>

Related

Run protoc command into docker container

I'm trying to run protoc command into a docker container.
I've tried using the gRPC image but protoc command is not found:
/bin/sh: 1: protoc: not found
So I assume I have to install manually using RUN instructions, but is there a better solution? An official precompiled image with protoc installed?
Also, I've tried to install via Dockerfile but I'm getting again protoc: not found.
This is my Dockerfile
#I'm not using "FROM grpc/node" because that image can't unzip
FROM node:12
...
# Download proto zip
ENV PROTOC_ZIP=protoc-3.14.0-linux-x86_32.zip
RUN curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.14.0/${PROTOC_ZIP}
RUN unzip -o ${PROTOC_ZIP} -d ./proto
RUN chmod 755 -R ./proto/bin
ENV BASE=/usr/local
# Copy into path
RUN cp ./proto/bin/protoc ${BASE}/bin
RUN cp -R ./proto/include/* ${BASE}/include
RUN protoc -I=...
I've done RUN echo $PATH to ensure the folder is in path and is ok:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Also RUN ls -la /usr/local/bin to check protoc file is into the folder and it shows:
-rwxr-xr-x 1 root root 4849692 Jan 2 11:16 protoc
So the file is in /bin folder and the folder is in the path.
Have I missed something?
Also, is there a simple way to get the image with protoc installed? or the best option is generate my own image and pull from my repository?
Thanks in advance.
Edit: Solved downloading linux-x86_64 zip file instead of x86_32. I downloaded the lower architecture requirements thinking a x86_64 machine can run a x86_32 file but not in the other way. I don't know if I'm missing something about architecture requirements (It's probably) or is a bug.
Anyway in case it helps someone I found the solution and I've added an answer with the neccessary Dockerfile to run protoc and protoc-gen-grpc-web.
The easiest way to get non-default tools like this is to install them through the underlying Linux distribution's package manager.
First, look at the Docker Hub page for the node image. (For "library" images like node, construct the URL https://hub.docker.com/_/node.) You'll notice there that there are several variations named "alpine", "buster", or "stretch"; plain node:12 is the same as node:12-stretch and node:12.20.0-stretch. The "alpine" images are based on Alpine Linux; the "buster" and "stretch" ones are different versions of Debian GNU/Linux.
For Debian-based packages, you can then look up the package on https://packages.debian.org/ (type protoc into the "Search the contents of packages" form at the bottom of the page). That leads you to the protobuf-compiler package. Knowing that contains the protoc binary, you can install it in your Dockerfile with:
FROM node:12 # Debian-based
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --no-install-recommends --assume-yes \
protobuf-compiler
# The rest of your Dockerfile as above
COPY ...
RUN protoc ...
You generally must run apt-get update and apt-get install in the same RUN command, lest a subsequent rebuild get an old version of the package cache from the Docker build cache. I generally have only a single apt-get install command if I can manage it, with the packages list alphabetically one to a line for maintainability.
If the image is Alpine-based, you can do a similar search on https://pkgs.alpinelinux.org/contents to find protoc, and similarly install it:
FROM node:12-alpine
RUN apk add --no-cache protoc
# The rest of your Dockerfile as above
Finally I solved my own issue.
The problem was the arch version: I was using linux-x86_32.zip but works using linux-x86_64.zip
Even #David Maze answer is incredible and so complete, it didn't solve my problem because using apt-get install version 3.0.0 and I wanted 3.14.0.
So, the Dockerfile I have used to run protoc into a docker container is like this:
FROM node:12
...
# Download proto zip
ENV PROTOC_ZIP=protoc-3.14.0-linux-x86_64.zip
RUN curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.14.0/${PROTOC_ZIP}
RUN unzip -o ${PROTOC_ZIP} -d ./proto
RUN chmod 755 -R ./proto/bin
ENV BASE=/usr
# Copy into path
RUN cp ./proto/bin/protoc ${BASE}/bin/
RUN cp -R ./proto/include/* ${BASE}/include/
# Download protoc-gen-grpc-web
ENV GRPC_WEB=protoc-gen-grpc-web-1.2.1-linux-x86_64
ENV GRPC_WEB_PATH=/usr/bin/protoc-gen-grpc-web
RUN curl -OL https://github.com/grpc/grpc-web/releases/download/1.2.1/${GRPC_WEB}
# Copy into path
RUN mv ${GRPC_WEB} ${GRPC_WEB_PATH}
RUN chmod +x ${GRPC_WEB_PATH}
RUN protoc -I=...
Because this is currently the highest ranked result on Google and the above instructions above won't work, if you want to use docker/dind for e.g. gitlab, this is the way how you can get the glibc-dependency working for protoc there:
#!/bin/bash
# install gcompat, because protoc needs a real glibc or compatible layer
apk add gcompat
# install a recent protoc (use a version that fits your needs)
export PB_REL="https://github.com/protocolbuffers/protobuf/releases"
curl -LO $PB_REL/download/v3.20.0/protoc-3.20.0-linux-x86_64.zip
unzip protoc-3.20.0-linux-x86_64.zip -d $HOME/.local
export PATH="$PATH:$HOME/.local/bin"

Unable to create superuser in cvat

I am able to build and run cvat tool. But when I trying to create a superuser then it is giving me below error.
ImportError: No module named 'gitdb.utils.compat'
I am running below command for creating a superuser.
docker exec -it cvat bash -ic 'python3 ~/manage.py createsuperuser'
Does anyone have any idea or suggestion for the above problem?
It seems the newer version of gitdb does not work with cvat (default version is 4.0.2), you can follow Furkan Kirac answer but with gitdb version is 0.6.4:
# pip uninstall gitdb
# pip install gitdb==0.6.4
This problem is most probably due to a newer gitdb2 python package.
If cvat is already built as a docker container, for testing, you must log into the container as root, uninstall it and install an older gitdb.
docker exec -it -u root cvat bash
pip3 uninstall gitdb2
pip3 install gitdb
Then, running python script must work. If that is the case, then a persistent solution is to rebuild the containers.
You need to edit Dockerfile as below:
# Install git application dependencies
...
fi
RUN pip3 uninstall -y gitdb2
RUN pip3 install --no-cache-dir gitdb
Run "docker-compose build".
Hope this helps.

How can I edit tomcat configuration files inside the docker container?

I am trying to nano/vim inside a docker container to edit the tomcat config files but i am getting an error that nano/vim is unknown command. I tried to yum install, still yum is unknown comand. How do I go about it
The most common editor is vi. To install some packages into your container you have to know it's base image. Most of distros create a special file in /etc/ with all necessary information: something-release, you can find it out with this command:
cat /etc/*release
And then use the package manager of current distro.
for Alpine it will be apk update && apk add vim.
for Ubuntu/Debian - apt update && apt install vim.
for Centos/RedHat/Fedora - yum install vim
etc

Invoke Ansible playbook in Jenkins

I have jenkins build and I am trying to invoke a ansible playbook file for an s3 upload. When I execute a post-build-script for invoking an ansible playbook file, I am ending with below error.
Cannot run program "ansible-playbook" (in directory "/var/jenkins_home/workspace/mybuild"): error=2, No such file or directory
Below screenshot is ansible post build script configuration.
FYI: There is a file(ansibledemo.yml) in my build folder. I tried giving absolute path(/var/jenkins_home/workspace/mybuild/ansibledemo.yml). Still no go.
When I try running ansible-playbook myplaybook.yml directly in jenkins image(terminal) I am ending up with bash: ansible-playbook: command not found
When I tried installing ansible in my jenkins server, I couldn't execute any installation commands. Please see the below screenshot.
Ansible is not install on your Jenkins machine, first you need to install the ansible on the jenkins machine:
On Ubuntu/Debian:
sudo apt-add-repository ppa:ansible/ansible
sudo apt-get update
sudo apt-get install ansible
On CentOS/RedHat:
sudo yum install epel-release
sudo yum install ansible
After that you will be able to run the ansible-playbook.
You can try to install using pip version as an alternative and try, Please see the below steps,
$ virtualenv venv
$ source venv/bin/activate
$ pip install ansible-container[docker,openshift]
You can see more options to install in docs: https://docs.ansible.com/ansible-container/installation.html
But always it is a good option to keep a separate vm / docker like "ansible-controller" and use that as a slave to jenkins, So that you don't need ansible plugins in ansible. And jenkins will be always stable without much load
Download package information from the configured sources.
# apt update
Install ansible
# apt install ansible
That's it.
If you run official jenkins container (based on debian) than repo with ansible build in already and you don't need "apt-add-repository". But you could install apt-add-repository by installing software-properties-common for further using.
dpkg -S apt-add-repository tells that this packet belongs to software-properties-common.
Error appears because the author of container always tries to make it as light as possible and remove package information.
You don't need sudo, because you become root in container by default. You become another user only if you mention it in intentionally.
Please, add information that you work in container to your question.

Docker, Supervisord and supervisor-stdout

I'm trying to centralize output from supervisord and its processes using supervisor-stdout. But with this supervisord configuration:
#supervisord.conf
[supervisord]
nodaemon = true
[program:nginx]
command = /usr/sbin/nginx
stdout_events_enabled = true
stderr_events_enabled = true
[eventlistener:stdout]
command = supervisor_stdout
buffer_size = 100
events = PROCESS_LOG
result_handler = supervisor_stdout:event_handler
(Note that the config section of supervisor-stoud is exactly the same as the example on the supervisor-stoud site).
...and this Dockerfile:
#Dockerfile
FROM python:3-onbuild
RUN apt-get update && apt-get install -y nginx supervisor
# Setup supervisord
RUN pip install supervisor-stdout
RUN mkdir -p /var/log/supervisor
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
COPY nginx.conf /etc/nginx/nginx.conf
# restart nginx to load the config
RUN service nginx stop
# Start processes
CMD supervisord -c /etc/supervisor/conf.d/supervisord.conf -n
I can build the image just fine, but running a container from it gives me:
Error: supervisor_stdout:event_handler cannot be resolved within [eventlistener:stdout]
EDIT
The output from running:
supervisord -c /etc/supervisor/conf.d/supervisord.conf -n
is:
Error: supervisor_stdout:event_handler cannot be resolved within [eventlistener:stdout]
For help, use /usr/bin/supervisord -h
I had the same problem, in short, you need to install the Python package that provides that supervisor_stdout:event_handler handler. You should be able to by issuing the following commands:
apt-get install -y python-pip
pip install supervisor-stdout
If you have pip installed on that container, a simple:
pip install supervisor-stdout should suffice, more info about that specific package can be found here:
https://pypi.python.org/pypi/supervisor-stdout
AFAIK, there is no debian package that provides the supervisor-stdout, so the easiest method to install it is through pip.
Hope it helps whoever comes here as I did.
[Edit]
As Vin-G suggested, if you still have a problem even after going through these steps, supervisord might be stuck in an old version. Try updating it.
Cheers!
I had the exact same problem and solved it by using Ubuntu 14.04 as a base image instead of Debian Jessie (I was using python:2.7 image which is based on Jessie).
You can refer to this complete working implementation: https://github.com/rehabstudio/docker-gunicorn-nginx.
EDIT: as pointed out by #Vin-G in his comment, it might be because the supervisor version shipped with Debian Jessie is too old. You could try to remove it from apt and install it with pip instead.
Very similar to the above, but I don't think that there is a complete answer.
I had to remove from apt
apt-get remove supervisor
Then reinstall with pip, but with pip2 as the current version of supervisor doesn't support python 3
apt-get install -y python-pip
pip2 install supervisor
pip2 install supervisor-stdout
This all then worked.
Although the supervisord path is now
/usr/local/bin/supervisord
Hope that helps.
I used this hacky way to get it to work. It works in Debian Jessie as well.
I simply pasted the guy's file to one of my own in my project directory. Like /app/supervisord_stdout.py
I then added it to the conf like this. /app is the project directory for my project files in the container.
[eventlistener:stdout]
command = python supervisord_stdout.py
buffer_size = 100
events = PROCESS_LOG
directory=/app
result_handler=supervisord_stdout:event_handler
environment=PYTHONPATH=/app

Resources