There seems to be some issue with pipenv lock. I used to update the requirements file by running pipenv lock -r | sed 's|https://.*#nexus|https://${PYPI_USER}:'"'"'${PYPI_PASSWORD}'"'"'#nexus|g' | tee requirements.txt. I however observed that this doesn't work anymore. Is there an alternative that I could use. pipenv lock --help does not list this option -r anymore.
I'm usiong pipenmv version 2022.10.10 and pip version 21.3.1
As explained here, it's replaced by pipenv requirements > requirements.txt
https://github.com/pypa/pipenv/issues/5253#issuecomment-1214344569
Related
I have a dockerfile where I want to RUN a bash command. This command can be different based on conditions. Therefore I had decided to pass this command as an argument.
Is it possible for me to inject an argument in RUN?
It would look like this:
RUN ${MYCOMMAND}
I also need to know if I can run multiple commands in the same layer of the dockerfile.
Edit:-
I need to install a bunch of apps similar to pip install <app>, in the dockerfile.
If I had 4 apps the command would look similar to this.
RUN pip install app_1 && \
pip install app_2 && \
pip install app_3 && \
pip install app_4
However, the list of apps provided are dynamic.
Which is why I thought I could inject the entire set of command string as an argument.
I ended up using a for loop. after passing the argument APP_LIST = "app1 app2 app3"
for app in ${APP_LIST}; do pip install $app; done
I'm trying to run protoc command into a docker container.
I've tried using the gRPC image but protoc command is not found:
/bin/sh: 1: protoc: not found
So I assume I have to install manually using RUN instructions, but is there a better solution? An official precompiled image with protoc installed?
Also, I've tried to install via Dockerfile but I'm getting again protoc: not found.
This is my Dockerfile
#I'm not using "FROM grpc/node" because that image can't unzip
FROM node:12
...
# Download proto zip
ENV PROTOC_ZIP=protoc-3.14.0-linux-x86_32.zip
RUN curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.14.0/${PROTOC_ZIP}
RUN unzip -o ${PROTOC_ZIP} -d ./proto
RUN chmod 755 -R ./proto/bin
ENV BASE=/usr/local
# Copy into path
RUN cp ./proto/bin/protoc ${BASE}/bin
RUN cp -R ./proto/include/* ${BASE}/include
RUN protoc -I=...
I've done RUN echo $PATH to ensure the folder is in path and is ok:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Also RUN ls -la /usr/local/bin to check protoc file is into the folder and it shows:
-rwxr-xr-x 1 root root 4849692 Jan 2 11:16 protoc
So the file is in /bin folder and the folder is in the path.
Have I missed something?
Also, is there a simple way to get the image with protoc installed? or the best option is generate my own image and pull from my repository?
Thanks in advance.
Edit: Solved downloading linux-x86_64 zip file instead of x86_32. I downloaded the lower architecture requirements thinking a x86_64 machine can run a x86_32 file but not in the other way. I don't know if I'm missing something about architecture requirements (It's probably) or is a bug.
Anyway in case it helps someone I found the solution and I've added an answer with the neccessary Dockerfile to run protoc and protoc-gen-grpc-web.
The easiest way to get non-default tools like this is to install them through the underlying Linux distribution's package manager.
First, look at the Docker Hub page for the node image. (For "library" images like node, construct the URL https://hub.docker.com/_/node.) You'll notice there that there are several variations named "alpine", "buster", or "stretch"; plain node:12 is the same as node:12-stretch and node:12.20.0-stretch. The "alpine" images are based on Alpine Linux; the "buster" and "stretch" ones are different versions of Debian GNU/Linux.
For Debian-based packages, you can then look up the package on https://packages.debian.org/ (type protoc into the "Search the contents of packages" form at the bottom of the page). That leads you to the protobuf-compiler package. Knowing that contains the protoc binary, you can install it in your Dockerfile with:
FROM node:12 # Debian-based
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --no-install-recommends --assume-yes \
protobuf-compiler
# The rest of your Dockerfile as above
COPY ...
RUN protoc ...
You generally must run apt-get update and apt-get install in the same RUN command, lest a subsequent rebuild get an old version of the package cache from the Docker build cache. I generally have only a single apt-get install command if I can manage it, with the packages list alphabetically one to a line for maintainability.
If the image is Alpine-based, you can do a similar search on https://pkgs.alpinelinux.org/contents to find protoc, and similarly install it:
FROM node:12-alpine
RUN apk add --no-cache protoc
# The rest of your Dockerfile as above
Finally I solved my own issue.
The problem was the arch version: I was using linux-x86_32.zip but works using linux-x86_64.zip
Even #David Maze answer is incredible and so complete, it didn't solve my problem because using apt-get install version 3.0.0 and I wanted 3.14.0.
So, the Dockerfile I have used to run protoc into a docker container is like this:
FROM node:12
...
# Download proto zip
ENV PROTOC_ZIP=protoc-3.14.0-linux-x86_64.zip
RUN curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.14.0/${PROTOC_ZIP}
RUN unzip -o ${PROTOC_ZIP} -d ./proto
RUN chmod 755 -R ./proto/bin
ENV BASE=/usr
# Copy into path
RUN cp ./proto/bin/protoc ${BASE}/bin/
RUN cp -R ./proto/include/* ${BASE}/include/
# Download protoc-gen-grpc-web
ENV GRPC_WEB=protoc-gen-grpc-web-1.2.1-linux-x86_64
ENV GRPC_WEB_PATH=/usr/bin/protoc-gen-grpc-web
RUN curl -OL https://github.com/grpc/grpc-web/releases/download/1.2.1/${GRPC_WEB}
# Copy into path
RUN mv ${GRPC_WEB} ${GRPC_WEB_PATH}
RUN chmod +x ${GRPC_WEB_PATH}
RUN protoc -I=...
Because this is currently the highest ranked result on Google and the above instructions above won't work, if you want to use docker/dind for e.g. gitlab, this is the way how you can get the glibc-dependency working for protoc there:
#!/bin/bash
# install gcompat, because protoc needs a real glibc or compatible layer
apk add gcompat
# install a recent protoc (use a version that fits your needs)
export PB_REL="https://github.com/protocolbuffers/protobuf/releases"
curl -LO $PB_REL/download/v3.20.0/protoc-3.20.0-linux-x86_64.zip
unzip protoc-3.20.0-linux-x86_64.zip -d $HOME/.local
export PATH="$PATH:$HOME/.local/bin"
My Pipenv seems to always think it is in a virtual environment, even when it clearly is not:
$ cd /tmp/
/tmp $ mkdir hello
/tmp $ cd hello/
/t/hello $ pipenv install --python 3.8 --dev pylint
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
How can I fix this?
In running the Docker build (using Jenkins CI), it fails on upgrading pip (last line of the docker file). I need it to upgrade version 8.1.1, as it suggest in the log, as my deploy fails on PIP versions mismatch.
Dockerfile
FROM ubuntu:14.04
FROM python:3.4
# Expose a port for gunicorn to listen on
EXPOSE 8002
# Make a workdir and virtualenv
WORKDIR /opt/documents_api
# Install everything else
ADD . /opt/documents_api
# Set some environment varialbes for PIP installation and db management
ENV CQLENG_ALLOW_SCHEMA_MANAGEMENT="True"
RUN apt-get update
RUN apt-get install -y python3-pip
RUN pip3 install --upgrade pip
Here's the error:
Step 15 : RUN pip3 install --upgrade pip
19:46:00 ---> Running in 84e2bcc850c0
19:46:04 Collecting pip
19:46:04 Downloading pip-8.1.1-py2.py3-none-any.whl (1.2MB)
19:46:04 Installing collected packages: pip
19:46:04 Found existing installation: pip 7.1.2
19:46:04 Uninstalling pip-7.1.2:
19:46:05 Successfully uninstalled pip-7.1.2
19:46:10 Exception:
19:46:10 Traceback (most recent call last):
19:46:10 File "/usr/local/lib/python3.4/shutil.py", line 424, in _rmtree_safe_fd
19:46:10 os.unlink(name, dir_fd=topfd)
19:46:10 FileNotFoundError: [Errno 2] No such file or directory: 'pip'
19:46:10 You are using pip version 7.1.2, however version 8.1.1 is available.
When you use two FROM directives, docker creates two output images, that's why it's messed up.
First, remove FROM ubuntu:14.04 and don't apt-get update in a Dockerfile, it's a bad practice (your image will be different every time you build, defeating the whole purpose of containers/Docker).
Second, you can check official python images Dockerfile to know which version of pip is installed, for example, python:3.4 (it's already v8.1.1).
Third, there is a special image for you case (external application): python:3.4-onbuild. Your Dockerfile can be reduced to:
FROM python:3.4-onbuild
ENV CQLENG_ALLOW_SCHEMA_MANAGEMENT="True"
EXPOSE 8002
CMD python myapp.py
One last thing, try to use alpine based images, they're much smaller (for python, it's almost 10 time smaller than the ubuntu based).
turns out the host I was running had no outside (internet) access. So the upgrade was failing. We solved it by adding another package to the DTR that had the necessary version in it.
use /usr/bin/ for run pip. Example :
/usr/bin/pip install --upgrade pip
running this command solved the same problem for me (python 3.9):
RUN /usr/local/bin/python -m pip install --upgrade pip
I'm trying to centralize output from supervisord and its processes using supervisor-stdout. But with this supervisord configuration:
#supervisord.conf
[supervisord]
nodaemon = true
[program:nginx]
command = /usr/sbin/nginx
stdout_events_enabled = true
stderr_events_enabled = true
[eventlistener:stdout]
command = supervisor_stdout
buffer_size = 100
events = PROCESS_LOG
result_handler = supervisor_stdout:event_handler
(Note that the config section of supervisor-stoud is exactly the same as the example on the supervisor-stoud site).
...and this Dockerfile:
#Dockerfile
FROM python:3-onbuild
RUN apt-get update && apt-get install -y nginx supervisor
# Setup supervisord
RUN pip install supervisor-stdout
RUN mkdir -p /var/log/supervisor
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
COPY nginx.conf /etc/nginx/nginx.conf
# restart nginx to load the config
RUN service nginx stop
# Start processes
CMD supervisord -c /etc/supervisor/conf.d/supervisord.conf -n
I can build the image just fine, but running a container from it gives me:
Error: supervisor_stdout:event_handler cannot be resolved within [eventlistener:stdout]
EDIT
The output from running:
supervisord -c /etc/supervisor/conf.d/supervisord.conf -n
is:
Error: supervisor_stdout:event_handler cannot be resolved within [eventlistener:stdout]
For help, use /usr/bin/supervisord -h
I had the same problem, in short, you need to install the Python package that provides that supervisor_stdout:event_handler handler. You should be able to by issuing the following commands:
apt-get install -y python-pip
pip install supervisor-stdout
If you have pip installed on that container, a simple:
pip install supervisor-stdout should suffice, more info about that specific package can be found here:
https://pypi.python.org/pypi/supervisor-stdout
AFAIK, there is no debian package that provides the supervisor-stdout, so the easiest method to install it is through pip.
Hope it helps whoever comes here as I did.
[Edit]
As Vin-G suggested, if you still have a problem even after going through these steps, supervisord might be stuck in an old version. Try updating it.
Cheers!
I had the exact same problem and solved it by using Ubuntu 14.04 as a base image instead of Debian Jessie (I was using python:2.7 image which is based on Jessie).
You can refer to this complete working implementation: https://github.com/rehabstudio/docker-gunicorn-nginx.
EDIT: as pointed out by #Vin-G in his comment, it might be because the supervisor version shipped with Debian Jessie is too old. You could try to remove it from apt and install it with pip instead.
Very similar to the above, but I don't think that there is a complete answer.
I had to remove from apt
apt-get remove supervisor
Then reinstall with pip, but with pip2 as the current version of supervisor doesn't support python 3
apt-get install -y python-pip
pip2 install supervisor
pip2 install supervisor-stdout
This all then worked.
Although the supervisord path is now
/usr/local/bin/supervisord
Hope that helps.
I used this hacky way to get it to work. It works in Debian Jessie as well.
I simply pasted the guy's file to one of my own in my project directory. Like /app/supervisord_stdout.py
I then added it to the conf like this. /app is the project directory for my project files in the container.
[eventlistener:stdout]
command = python supervisord_stdout.py
buffer_size = 100
events = PROCESS_LOG
directory=/app
result_handler=supervisord_stdout:event_handler
environment=PYTHONPATH=/app