I'm currently working on a simple Python script and using pipenv to manage dependencies.
Here's the problem:
When running pipenv shell and then python script.py, I'm getting module not found errors for things that are definitely in Pipfile (e.g. requests).
Here's what's interesting:
I don't get these problems when I use pipenv run python script.py and I don't get these problems when I use bash instead of zsh, i.e. when I do:
bash
pipenv shell
python script.py
the script runs happily and resolves the imports. I tried re-installing pipenv using Homebrew, but zsh still doesn't correctly activate my pipenv environment. Any ideas on how to get pipenv to work with zsh or why this would even be problematic?
Using zsh:
Using bash:
Related
The background is, I'm responsible for maintaining a fancy Docker image that is used by our team for analytics. It uses a Jupyter notebook image as the base, and then adds various customisations, extra packages, etc.
One of the team members recently wanted to run Tensorflow. No problem, I'll just run mamba install and add it to the image. However, this created an issue: Tensorflow 2.4.3 (the latest version) is somehow incompatible with R 4.1.1 (also the latest version) or something else in the ecosystem, causing R to to be downgraded to 3.6.3. So I created a new environment and installed TF into that:
FROM hongooi/jupytermodelrisk:1.0.0
RUN mamba create -n tensorflow --clone base
# Make RUN commands use the new environment
RUN echo "conda activate tensorflow" >> ~/.bashrc
SHELL ["/bin/bash", "--login", "-c"]
RUN mamba install -y 'tensorflow=2.4.3'
But when I rebuilt the image, I found that while the tensorflow env had been created, the Tensorflow package had been installed into the base env, not the tensorflow env. Has anyone else encountered this? I can verify, if I login to the container, that the tensorflow env has been created: it just doesn't contain the Tensorflow package.
I don't get this problem if run the create, activate and install commands from inside the container. It's only when I try to do it in the Dockerfile.
I use mamba instead of conda because the latter takes forever to run, given the number of packages installed. In fact, trying to run conda install tensorflow crashes after ~5 hours.
Not an expert on dockerfiles, but in general you could just use the -n flag to the install command to specify the target environment for the installation like so:
mamba install -n tensorflow -y tensorflow=2.4.3
I ran pipenv install to create a Pipfile in the current directory that doesn't have a Pipfile. It gave the following output but did not create a Pipfile. Why not?
Installing dependencies from Pipfile.lock (639627)…
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
It looks like it found a Pipfile.lock somewhere and used it? (similar to git behavior)
Use the PIPENV_NO_INHERIT environment variable to ignore inheriting from directories above the current directory, e.g.,
PIPENV_NO_INHERIT=True pipenv install
In your case, pipenv searched directories above the current directory and found a Pipfile there that it used (the location of which can be seen with pipenv --where).
(Incidently, I looked at the pipenv documentation but was unable to find where it discussed this behavior, so please add a link here to that documentation if you find it.)
I ran pipenv shell first (creat Pipfile), then pipenv install (creat Pipfile.lock)
'pip install pipenv' command returns "Requirement already satisfied ..."
Is it possible to tell pipenv where the venv is located? Perhaps there's something you can put in the pipfile, or something for the .env file?
I fairly frequently have to recreate my venv because pipenv seemingly loses track of where it is.
For example, I started a project using Pycharm to configure the file system and create my pipenv interpreter. It created the venv in ~/.local/share/virtualenvs/my-project-ZbEWMGNA and it was able to keep track of where that interpreter was located.
Switching to a terminal window & running pipenv commands then resulted in;
Warning: No virtualenv has been created for this project yet! Consider running pipenv install first to automatically generate one for you or seepipenv install --help for further instructions.
At which point I ran pipenv install from the terminal & pointed pycharm at that venv, so the path would become ~/apps/my-project-ZbEWMGNA (which sits alongside the project files ~/apps/my-project)
Now I've got venvs in both paths and pipenv still can't find them.
mwalker#Mac my-project % pipenv --where
/Users/mwalker/apps/my-project
mwalker#Mac my-project % pipenv --venv
No virtualenv has been created for this project yet!
Aborted!
mwalker#Mac my-project % ls ~/apps
my-project
my-project-ZbEWMGNA
mwalker#Mac my-project % ls ~/.local/share/virtualenvs
my-project-ZbEWMGNA
Yes, it is possible by setting environment variables. You can set a path for virtual environments via the WORKON_HOME. Or have the virtual environment created in the project with PIPENV_VENV_IN_PROJECT.
Pipenv automatically honors the WORKON_HOME environment variable, if you have it set — so you can tell pipenv to store your virtual environments wherever you want
-- https://pipenv-fork.readthedocs.io/en/latest/advanced.html#custom-virtual-environment-location
or
PIPENV_VENV_IN_PROJECT
If set, creates .venv in your project directory.
-- https://pipenv-fork.readthedocs.io/en/latest/advanced.html#pipenv.environments.PIPENV_VENV_IN_PROJECT
In my experience, PyCharm will uses the existing venv created by Pipenv. Otherwise it will create it in the directory that PyCharm is configured to create it.
I am new to pipenv so there might be something I'm not understanding here. However it seems like the virtual environment which is created depends on the current directory, which seems bad to me.
Here is what I did:
Checked out code from Github which already had Pipfile and Pipfile.lock
Did some unrelated stuff... at this point I was in a directory called /home/user/me/miniconda3/bin/
Ran /home/user/me/miniconda3/bin/pipenv run python /home/user/me/my-script-dir/my-script.py
This caused Pipenv to create a virtual environment. Output:
Creating a virtualenv for this project...
Using /home/user/me/miniconda3/bin/python (3.6.4) to create virtualenv…
Already using interpreter /home/user/me/miniconda3/bin/python
Using base prefix '/home/user/me/miniconda3'
New python executable in /home/user/me/.local/share/virtualenvs/bin-YnM8YhRk/bin/python
Installing setuptools, pip, wheel...done.
Virtualenv location: /home/user/me/.local/share/virtualenvs/bin-YnM8YhRk
Creating a Pipfile for this project…
Then I realized that I needed to run pipenv install so this time I cd'd to the directory where the script is actually stored, /home/user/me/my-script-dir/, and ran /home/user/me/miniconda3/bin/pipenv install. Then I got this output:
Creating a virtualenv for this project…
Using /home/user/me/miniconda3/bin/python (3.6.4) to create virtualenv…
Already using interpreter /home/user/me/miniconda3/bin/python
Using base prefix '/home/user/me/miniconda3'
New python executable in /home/user/me/.local/share/virtualenvs/my-script-dir-Ex37BY7g/bin/python
Installing setuptools, pip, wheel...done.
Virtualenv location: /home/user/me/.local/share/virtualenvs/my-script-dir-Ex37BY7g
Installing dependencies from Pipfile.lock (6c24e4)…
So as you can see I actually was running the same script each time, but somehow it created two different virtual environments. And the virtual environments are named after what happened to be my current directory at the time, not the directory of the script. This seems like it would be very unwieldy unless I am missing something.
You are correct, the virtualenv Pipenv uses does depend on the current directory.
I'm just recently started to using the Windows Subsystem for Linux. I was trying to install Angular and ran into an error. I found a potential solution, but I don't understand part of the solution. In the script bellow, what do the keywords USER, ENV, and RUN mean, and what are they called? I tried running "USER node" and i got an error
USER node
RUN mkdir /home/node/.npm-global
ENV PATH=/home/node/.npm-global/bin:$PATH
ENV NPM_CONFIG_PREFIX=/home/node/.npm-global
RUN npm install -g #angular/cli
Here is the entire answer in case you need more context https://github.com/angular/angular-cli/issues/7389
USER sets a username to use when executing commands that follow later in Dockerfile. See Dockerfile docs
That is not a script. Those directives have no meaning in Ubuntu.
That is a dockerfile. It is used by Docker to build images.