Why is my singularity pull failing in my nextflow script? - docker

Question

I have looked hard for an answer, but haven't managed to find one, so I'm hoping someone here can help me understand this error and what is happening during the singularity pull command.
Here is the error:
Error executing process > 'QC_TRIM_READS (1)'
Caused by:
Failed to pull singularity image
command: singularity pull --name quay.io-biocontainers-sickle-trim-1.33--2.img.pulling.1632264509884 docker://quay.io/biocontainers/sickle-trim:1.33--2 > /dev/null
status : 127
message:
WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull. Use Singularity Registry
WARNING: (shub://) to pull exactly equivalent images.
/usr/bin/env: ‘python’: No such file or directory
ERROR: pulling container failed!
Here is the script (excuse the mess, I am just getting used to nextflow)
#!/usr/bin/env Nextflow
nextflow.enable.dsl=2
params.ref_genome = "./data/GmaxFiskeby_678_v1.0.fa"
params.ref_annotation = "./data/GmaxFiskeby_678_v1.1.gene_exons.gff3"
params.intermediate_dir = "$workDir/intermediate/"
workflow {
ref_genome_ch = Channel.fromPath("$params.ref_genome")
ref_annotation_ch = Channel.fromPath("$params.ref_annotation")
input_fastq_ch = Channel.fromPath("./data/*.fastq")
ref_genome_ch.view()
QC_TRIM_READS(input_fastq_ch)
STAR_INDEX_GENOME(ref_genome_ch, ref_annotation_ch)
}
process GZIP_VERSION {
echo true
script:
"""
gzip --version
"""
}
process UNZIP {
publishDir "intermediate/"
input:
path file
output:
path "${file.baseName}"
script:
"""
gzip -dfk ${file}
"""
}
process QC_TRIM_READS {
publishDir "intermediate/"
container 'quay.io/biocontainers/sickle-trim:1.33--2'
input:
path fastqFile
output:
path "${fastqFile.baseName}_trimmed.${fastqFile.getExtension()}"
script:
"""
sickle se \\
-f $fastqFile \\
-t sanger \\
-o ${fastqFile.baseName}_trimmed.${fastqFile.getExtension()} \\
-q 35 \\
-l 45
"""
}
process STAR_INDEX_GENOME {
publishDir "intermediate/indexedGenome/"
/*if (worflow.containerEngine == 'singularity'){
container "https://depot.galaxyproject.org/singularity/mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2:59cdd445419f14abac76b31dd0d71217994cbcc9-0"
} else {*/
container "quay.io/biocontainers/mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2:59cdd445419f14abac76b31dd0d71217994cbcc9-0" //'quay.io/biocontainers/star:2.6.1d--0'
//}
input:
path genome
path gtf
output:
path "star" , emit: index
script:
"""
STAR \\
--runMode genomeGenerate \\
--genomeDir star/ \\
--genomeFastaFiles ${genome}\\
--sjdbGTFfile ${gtf} \\
--sjdbGTFtagExonParentTranscript Parent \\
--sjdbOverhang 100 \\
--runThreadN 2
"""
}
Here is my configuration file:
//docker.enabled = false
singularity.enabled = true
singularity.autoMounts = true
I built my environment as a conda environment, here is the yml file:
name: nf-core
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- _openmp_mutex=4.5=1_gnu
- attrs=21.2.0=pyhd3eb1b0_0
- brotlipy=0.7.0=py38h27cfd23_1003
- bzip2=1.0.8=h7b6447c_0
- c-ares=1.17.1=h27cfd23_0
- ca-certificates=2021.7.5=h06a4308_1
- cairo=1.14.12=h7636065_2
- cattrs=1.7.1=pyhd3eb1b0_0
- certifi=2021.5.30=py38h06a4308_0
- cffi=1.14.6=py38h400218f_0
- charset-normalizer=2.0.4=pyhd3eb1b0_0
- click=8.0.1=pyhd3eb1b0_0
- colorama=0.4.4=pyhd3eb1b0_0
- commonmark=0.9.1=pyhd3eb1b0_0
- coreutils=8.32=h7b6447c_0
- cryptography=3.4.7=py38hd23ed53_0
- curl=7.78.0=h1ccaba5_0
- expat=2.4.1=h2531618_2
- fontconfig=2.12.6=h49f89f6_0
- freetype=2.8=hab7d2ae_1
- fribidi=1.0.10=h7b6447c_0
- future=0.18.2=py38_1
- gettext=0.21.0=hf68c758_0
- git=2.32.0=pl5262hc120c5b_1
- gitdb=4.0.7=pyhd3eb1b0_0
- gitpython=3.1.18=pyhd3eb1b0_1
- glib=2.69.1=h5202010_0
- graphite2=1.3.14=h23475e2_0
- graphviz=2.40.1=h25d223c_0
- harfbuzz=1.7.6=h5f0a787_1
- hdf5=1.10.6=hb1b8bf9_0
- icu=58.2=he6710b0_3
- idna=3.2=pyhd3eb1b0_0
- importlib-metadata=4.8.1=py38h06a4308_0
- importlib_metadata=4.8.1=hd3eb1b0_0
- itsdangerous=2.0.1=pyhd3eb1b0_0
- jinja2=3.0.1=pyhd3eb1b0_0
- jpeg=9d=h7f8727e_0
- jsonschema=3.2.0=pyhd3eb1b0_2
- krb5=1.19.2=hac12032_0
- ld_impl_linux-64=2.35.1=h7274673_9
- libcurl=7.78.0=h0b77cf5_0
- libedit=3.1.20210714=h7f8727e_0
- libev=4.33=h7b6447c_0
- libffi=3.3=he6710b0_2
- libgcc-ng=9.3.0=h5101ec6_17
- libgfortran-ng=7.5.0=ha8ba4b0_17
- libgfortran4=7.5.0=ha8ba4b0_17
- libgomp=9.3.0=h5101ec6_17
- libiconv=1.15=h63c8f33_5
- libnghttp2=1.41.0=hf8bcb03_2
- libpng=1.6.37=hbc83047_0
- libssh2=1.9.0=h1ba5d50_1
- libstdcxx-ng=9.3.0=hd4cf53a_17
- libtiff=4.2.0=h85742a9_0
- libtool=2.4.6=h7b6447c_1005
- libwebp-base=1.2.0=h27cfd23_0
- libxcb=1.14=h7b6447c_0
- libxml2=2.9.12=h03d6c58_0
- lz4-c=1.9.3=h295c915_1
- markupsafe=2.0.1=py38h27cfd23_0
- ncbi-ngs-sdk=2.10.4=hdf6179e_0
- ncurses=6.2=he6710b0_1
- nextflow=21.04.0=h4a94de4_0
- nf-core=2.1=pyh5e36f6f_0
- openjdk=8.0.152=h7b6447c_3
- openssl=1.1.1l=h7f8727e_0
- ossuuid=1.6.2=hf484d3e_1000
- packaging=21.0=pyhd3eb1b0_0
- pango=1.42.0=h377f3fa_0
- pcre=8.45=h295c915_0
- pcre2=10.35=h14c3975_1
- perl=5.26.2=h14c3975_0
- perl-app-cpanminus=1.7044=pl526_1
- perl-business-isbn=3.004=pl526_0
- perl-business-isbn-data=20140910.003=pl526_0
- perl-carp=1.38=pl526_3
- perl-constant=1.33=pl526_1
- perl-data-dumper=2.173=pl526_0
- perl-encode=2.88=pl526_1
- perl-exporter=5.72=pl526_1
- perl-extutils-makemaker=7.36=pl526_1
- perl-file-path=2.16=pl526_0
- perl-file-temp=0.2304=pl526_2
- perl-mime-base64=3.15=pl526_1
- perl-parent=0.236=pl526_1
- perl-uri=1.76=pl526_0
- perl-xml-libxml=2.0132=pl526h7ec2d77_1
- perl-xml-namespacesupport=1.12=pl526_0
- perl-xml-sax=1.02=pl526_0
- perl-xml-sax-base=1.09=pl526_0
- perl-xsloader=0.24=pl526_0
- pip=21.2.2=py38h06a4308_0
- pixman=0.40.0=h7b6447c_0
- prompt-toolkit=3.0.17=pyhca03da5_0
- prompt_toolkit=3.0.17=hd3eb1b0_0
- pycparser=2.20=py_2
- pygments=2.10.0=pyhd3eb1b0_0
- pyopenssl=20.0.1=pyhd3eb1b0_1
- pyparsing=2.4.7=pyhd3eb1b0_0
- pyrsistent=0.17.3=py38h7b6447c_0
- pysocks=1.7.1=py38h06a4308_0
- python=3.8.11=h12debd9_0_cpython
- python_abi=3.8=2_cp38
- pyyaml=5.4.1=py38h27cfd23_1
- questionary=1.10.0=pyhd8ed1ab_0
- readline=8.1=h27cfd23_0
- requests=2.26.0=pyhd3eb1b0_0
- requests-cache=0.7.4=pyhd8ed1ab_0
- rich=10.10.0=py38h578d9bd_0
- setuptools=58.0.4=py38h06a4308_0
- singularity=2.4.2=0
- six=1.16.0=pyhd3eb1b0_0
- smmap=4.0.0=pyhd3eb1b0_0
- sqlite=3.36.0=hc218d9a_0
- sra-tools=2.11.0=pl5262h314213e_0
- tabulate=0.8.9=py38h06a4308_0
- tk=8.6.10=hbc83047_0
- typing-extensions=3.10.0.2=hd3eb1b0_0
- typing_extensions=3.10.0.2=pyh06a4308_0
- url-normalize=1.4.3=pyhd8ed1ab_0
- urllib3=1.26.6=pyhd3eb1b0_1
- wcwidth=0.2.5=pyhd3eb1b0_0
- wheel=0.37.0=pyhd3eb1b0_1
- xz=5.2.5=h7b6447c_0
- yaml=0.2.5=h7b6447c_0
- zipp=3.5.0=pyhd3eb1b0_0
- zlib=1.2.11=h7b6447c_3
- zstd=1.4.9=haebb681_0
prefix: /home/mkozubov/miniconda3/envs/nf-core
Here is the log file:
Sep-21 16:00:49.076 [main] DEBUG nextflow.cli.Launcher - $> nextflow run rnaseq.nf
Sep-21 16:00:49.318 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 21.04.0
Sep-21 16:00:49.367 [main] INFO nextflow.cli.CmdRun - Launching `rnaseq.nf` [reverent_jepsen] - revision: 0fc00d31fc
Sep-21 16:00:49.414 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /mnt/c/Users/mkozubov/Desktop/nextflow_tutorial/rnaseq/nextflow.config
Sep-21 16:00:49.418 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /mnt/c/Users/mkozubov/Desktop/nextflow_tutorial/rnaseq/nextflow.config
Sep-21 16:00:49.506 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Sep-21 16:00:50.238 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/mkozubov/.nextflow/plugins
Sep-21 16:00:50.240 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Sep-21 16:00:50.242 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins local root: .nextflow/plr/empty
Sep-21 16:00:50.258 [main] INFO org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Sep-21 16:00:50.262 [main] INFO org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Sep-21 16:00:50.266 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Sep-21 16:00:50.289 [main] INFO org.pf4j.AbstractPluginManager - No plugins
Sep-21 16:00:50.366 [main] DEBUG nextflow.Session - Session uuid: 22a13149-e9f8-47cc-8f09-98a6b000a83a
Sep-21 16:00:50.367 [main] DEBUG nextflow.Session - Run name: reverent_jepsen
Sep-21 16:00:50.372 [main] DEBUG nextflow.Session - Executor pool size: 5
Sep-21 16:00:50.418 [main] DEBUG nextflow.cli.CmdRun -
Version: 21.04.0 build 5552
Created: 02-05-2021 16:22 UTC (09:22 PDT)
System: Linux 5.10.16.3-microsoft-standard-WSL2
Runtime: Groovy 3.0.7 on OpenJDK 64-Bit Server VM 1.8.0_152-release-1056-b12
Encoding: UTF-8 (UTF-8)
Process: 10590#DESKTOP-UJ90D1J [127.0.1.1]
CPUs: 5 - Mem: 1.9 GB (311.8 MB) - Swap: 1 GB (783.4 MB)
Sep-21 16:00:50.539 [main] DEBUG nextflow.file.FileHelper - Can't check if specified path is NFS (1): /mnt/c/Users/mkozubov/Desktop/nextflow_tutorial/rnaseq/work
v9fs
Sep-21 16:00:50.541 [main] DEBUG nextflow.Session - Work-dir: /mnt/c/Users/mkozubov/Desktop/nextflow_tutorial/rnaseq/work [null]
Sep-21 16:00:50.545 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /mnt/c/Users/mkozubov/Desktop/nextflow_tutorial/rnaseq/bin
Sep-21 16:00:50.585 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Sep-21 16:00:50.616 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Sep-21 16:00:50.999 [main] DEBUG nextflow.Session - Session start invoked
Sep-21 16:00:51.461 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Sep-21 16:00:51.511 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: QC_TRIM_READS, UNZIP, STAR_INDEX_GENOME, GZIP_VERSION
Sep-21 16:00:51.643 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Sep-21 16:00:51.643 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Sep-21 16:00:51.651 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Sep-21 16:00:51.656 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=5; memory=1.9 GB; capacity=5; pollInterval=100ms; dumpInterval=5m
Sep-21 16:00:51.868 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Sep-21 16:00:51.869 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Sep-21 16:00:51.904 [main] DEBUG nextflow.Session - Ignite dataflow network (5)
Sep-21 16:00:51.963 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > QC_TRIM_READS
Sep-21 16:00:51.965 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > STAR_INDEX_GENOME
Sep-21 16:00:51.966 [main] DEBUG nextflow.script.ScriptRunner - > Await termination
Sep-21 16:00:51.968 [main] DEBUG nextflow.Session - Session await
Sep-21 16:00:51.969 [PathVisitor-3] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: ./data/; pattern: *.fastq; options: [:]
Sep-21 16:00:52.300 [Actor Thread 8] WARN nextflow.container.SingularityCache - Singularity cache directory has not been defined -- Remote image will be stored in the path: /mnt/c/Users/mkozubov/Desktop/nextflow_tutorial/rnaseq/work/singularity -- Use env variable NXF_SINGULARITY_CACHEDIR to specify a different location
Sep-21 16:00:52.300 [Actor Thread 8] INFO nextflow.container.SingularityCache - Pulling Singularity image docker://quay.io/biocontainers/sickle-trim:1.33--2 [cache /mnt/c/Users/mkozubov/Desktop/nextflow_tutorial/rnaseq/work/singularity/quay.io-biocontainers-sickle-trim-1.33--2.img]
Sep-21 16:00:52.300 [Actor Thread 7] INFO nextflow.container.SingularityCache - Pulling Singularity image docker://quay.io/biocontainers/mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2:59cdd445419f14abac76b31dd0d71217994cbcc9-0 [cache /mnt/c/Users/mkozubov/Desktop/nextflow_tutorial/rnaseq/work/singularity/quay.io-biocontainers-mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2-59cdd445419f14abac76b31dd0d71217994cbcc9-0.img]
Sep-21 16:00:52.433 [Actor Thread 5] ERROR nextflow.processor.TaskProcessor - Error executing process > 'QC_TRIM_READS (1)'
Caused by:
Failed to pull singularity image
command: singularity pull --name quay.io-biocontainers-sickle-trim-1.33--2.img.pulling.1632265252300 docker://quay.io/biocontainers/sickle-trim:1.33--2 > /dev/null
status : 127
message:
WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull. Use Singularity Registry
WARNING: (shub://) to pull exactly equivalent images.
/usr/bin/env: ‘python’: No such file or directory
ERROR: pulling container failed!
java.lang.IllegalStateException: java.lang.IllegalStateException: Failed to pull singularity image
command: singularity pull --name quay.io-biocontainers-sickle-trim-1.33--2.img.pulling.1632265252300 docker://quay.io/biocontainers/sickle-trim:1.33--2 > /dev/null
status : 127
message:
WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull. Use Singularity Registry
WARNING: (shub://) to pull exactly equivalent images.
/usr/bin/env: ‘python’: No such file or directory
ERROR: pulling container failed!
at nextflow.container.SingularityCache.getCachePathFor(SingularityCache.groovy:304)
at nextflow.container.SingularityCache$getCachePathFor.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
at nextflow.container.ContainerHandler.createSingularityCache(ContainerHandler.groovy:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:193)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:61)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:194)
at nextflow.container.ContainerHandler.normalizeImageName(ContainerHandler.groovy:68)
at nextflow.container.ContainerHandler$normalizeImageName.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
at nextflow.processor.TaskRun.getContainer(TaskRun.groovy:587)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
at org.codehaus.groovy.runtime.metaclass.MethodMetaProperty$GetBeanMethodMetaProperty.getProperty(MethodMetaProperty.java:76)
at org.codehaus.groovy.runtime.callsite.GetEffectivePogoPropertySite.getProperty(GetEffectivePogoPropertySite.java:85)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:341)
at nextflow.processor.TaskProcessor.createTaskHashKey(TaskProcessor.groovy:1939)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:193)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:61)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:171)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
at nextflow.processor.TaskProcessor.invokeTask(TaskProcessor.groovy:591)
at nextflow.processor.InvokeTaskAdapter.call(InvokeTaskAdapter.groovy:59)
at groovyx.gpars.dataflow.operator.DataflowOperatorActor.startTask(DataflowOperatorActor.java:120)
at groovyx.gpars.dataflow.operator.ForkingDataflowOperatorActor.access$001(ForkingDataflowOperatorActor.java:35)
at groovyx.gpars.dataflow.operator.ForkingDataflowOperatorActor$1.run(ForkingDataflowOperatorActor.java:58)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Failed to pull singularity image
command: singularity pull --name quay.io-biocontainers-sickle-trim-1.33--2.img.pulling.1632265252300 docker://quay.io/biocontainers/sickle-trim:1.33--2 > /dev/null
status : 127
message:
WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull. Use Singularity Registry
WARNING: (shub://) to pull exactly equivalent images.
/usr/bin/env: ‘python’: No such file or directory
ERROR: pulling container failed!
at nextflow.container.SingularityCache.runCommand(SingularityCache.groovy:256)
at nextflow.container.SingularityCache.downloadSingularityImage0(SingularityCache.groovy:223)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1268)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1029)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1012)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:101)
at nextflow.container.SingularityCache$_downloadSingularityImage_closure1.doCall(SingularityCache.groovy:191)
at nextflow.container.SingularityCache$_downloadSingularityImage_closure1.call(SingularityCache.groovy)
at nextflow.file.FileMutex.lock(FileMutex.groovy:107)
at nextflow.container.SingularityCache.downloadSingularityImage(SingularityCache.groovy:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1268)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1029)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1012)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:101)
at nextflow.container.SingularityCache$_getLazyImagePath_closure2.doCall(SingularityCache.groovy:281)
at nextflow.container.SingularityCache$_getLazyImagePath_closure2.call(SingularityCache.groovy)
at groovyx.gpars.dataflow.LazyDataflowVariable$1.run(LazyDataflowVariable.java:70)
... 3 common frames omitted
Sep-21 16:00:52.443 [Actor Thread 5] DEBUG nextflow.Session - Session aborted -- Cause: java.lang.IllegalStateException: Failed to pull singularity image
command: singularity pull --name quay.io-biocontainers-sickle-trim-1.33--2.img.pulling.1632265252300 docker://quay.io/biocontainers/sickle-trim:1.33--2 > /dev/null
status : 127
message:
WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull. Use Singularity Registry
WARNING: (shub://) to pull exactly equivalent images.
/usr/bin/env: ‘python’: No such file or directory
ERROR: pulling container failed!
Sep-21 16:00:52.494 [Actor Thread 5] DEBUG nextflow.Session - The following nodes are still active:
[process] QC_TRIM_READS
status=ACTIVE
port 0: (queue) closed; channel: fastqFile
port 1: (cntrl) - ; channel: $
Sep-21 16:00:52.507 [main] DEBUG nextflow.Session - Session await > all process finished
Sep-21 16:00:52.510 [main] DEBUG nextflow.Session - Session await > all barriers passed
Sep-21 16:00:52.521 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=0; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=0ms; failedDuration=0ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=0; peakCpus=0; peakMemory=0; ]
Sep-21 16:00:52.685 [main] DEBUG nextflow.CacheDB - Closing CacheDB done
Sep-21 16:00:52.752 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
I have been using nf-core's rnaseq pipeline to guide me a bit: https://github.com/nf-core/rnaseq
If it helps, here is the pipeline I am trying to automate: https://bioinformatics.uconn.edu/resources-and-events/tutorials-2/rna-seq-tutorial-with-reference-genome/#
My computer has a Windows 10 operating system, and I have enabled WSL2 and got Ubuntu.
I am fairly new to Docker, Singularity, and Nextflow so I am hoping someone can explain the error. I don't even understand why python is being mentioned. Is the issue that singularity cannot pull from Quay.io? I am a bit lost and would appreciate a nudge in the right direction.
Also the reason I am trying to get singularity to work is STAR immediately gives me a Segmentation fault error on my local machine (i'm assuming I run out of memory), and I would like to test this pipeline on our HPC (but I don't have root privileges).

Answer 1

You can ignore the Singularity warnings, but not the errors. The problem looks to be that you're missing python in your environment:
/usr/bin/env: ‘python’: No such file or directory
ERROR: pulling container failed!
You need to make sure you have Python 3 installed. If you have, you should be able to see it here with:
/usr/bin/python --version
You didn't mention the version of Ubuntu you are using, but if you have Ubuntu 20.04 then you should already have Python 3 pre-installed. If this is the case, and you have Python 3 already installed (i.e. you find that /usr/bin/python3 --version works, note the '3') but the above doesn't, try:
sudo apt-get install python-is-python3
This will install a symlink to point the /usr/bin/python interpreter at the current default python3.