Activate conda environment in docker and using image in WDL - docker

Question

I am developing dockerfile where i need to activate conda environment(working fine)
Dockerfile:
FROM continuumio/miniconda3
WORKDIR /app
# Create the environment:
COPY environment.yml .
RUN conda env create -f environment.yml
# Make RUN commands use the new environment:
SHELL ["conda", "run", "-n", "myenv", "/bin/bash", "-c"]
# The code to run when container is started:
COPY test.py .
ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "myenv"]
environment.yml
name: myenv
channels:
- conda-forge
dependencies:
- python=3.8
- flask
- numpy
test.py
import numpy as np
# Creating a rank 1 Array
arr = np.array([1, 2, 3])
print("Array with Rank 1: \n",arr)
Building and pushing docker image:
docker build -t docker.io/myaccount/condaset:latest .
docker push docker.io/myaccount/condaset:latest
I tested the docker image which seems to be working fine.
docker run docker.io/myaccount/condaset python test.py
Array with Rank 1:
[1 2 3]
but when i use the same docker image in WDL it gives error.
test.wdl
version 1.0
task my_run
{
input
{
String somevar
}
parameter_meta {
}
command {
set -exo pipefail
python /app/test.py
}
output {
File out = ''
}
runtime {
docker: "docker.io/myaccount/condaset:latest"
}
}
workflow my_wokflow
{
input
{
String somevar
}
call my_run
{
input: somevar=somevar
}
}
input.json
{
"my_wokflow.somevar": "hello_world"
}
Error while running WDL:
java -jar ~/bin/cromwell-58.jar run test.wdl -i input.json
[First 3000 bytes]:+ python /app/test.py
Traceback (most recent call last):
File "/app/test.py", line 1, in <module>
import numpy as np
ModuleNotFoundError: No module named 'numpy'
It seems conda environment become deactivated before calling dockerimage in WDL. How can i fix such issue?

Answer 1

When cromwell runs a container, it overrides the ENTRYPOINT script, so your conda environment does not get activated when the WDL is run.
There are a a couple of different ways you can get the behavior you want.
Add your conda environment to the your PATH in the Dockerfile:
ENV PATH="/opt/conda/envs/myenv/bin:$PATH"
In your WDL, supply an absolute path to python:
/opt/conda/envs/myenv/bin/python /app/test.py
Additionally, in your WDL, the my_run output stanza will result in an error. You need to give a non-empty file name for out:
command {
set -exo pipefail
/opt/conda/envs/myenv/bin/python /app/test.py > out.txt
}
output {
File out = 'out.txt'
}