I am using Singularity 3.0 and trying to pull a container image from DockerHub and run it on a university cluster. I followed this recipe to get started.
singularity pull --name rstudio-3.5.2.sif docker://rocker/rstudio:3.5.2
singularity exec --bind example-project-1/:/home/rstudio/ rstudio-3.5.2.sif rserver --www-port 8787
The container process starts but when I try to connect via a browser I get the following error log.
01 Jan 2019 12:07:22 [rsession-pughdr] ERROR system error 30 (Read-only file system) [path=/home/pughdr/.rstudio, target-dir=]; OCCURRED AT: rstudio::core::Error rstudio::core::FilePath::createDirectory(const string&) const /home/ubuntu/rstudio/src/cpp/core/FilePath.cpp:846; LOGGED FROM: rstudio::core::FilePath rstudio::core::system::userSettingsPath(const rstudio::core::FilePath&, const string&) /home/ubuntu/rstudio/src/cpp/core/system/PosixSystem.cpp:486
01 Jan 2019 12:07:22 [rsession-pughdr] ERROR system error 30 (Read-only file system) [path=/home/pughdr/.rstudio, target-dir=]; OCCURRED AT: rstudio::core::Error rstudio::core::FilePath::createDirectory(const string&) const /home/ubuntu/rstudio/src/cpp/core/FilePath.cpp:846; LOGGED FROM: int main(int, char* const*) /home/ubuntu/rstudio/src/cpp/session/SessionMain.cpp:1689
Seems that the file system in the container is only read-only. How do I build (or run) the container such that the container's file system is writable?
Update:
I was able to get the RStudio Server running on the university cluster as follows.
singularity exec --home my-project-directory rstudio-3.5.2.sif rserver --www-port 8787
This seems to work because Singularity automatically mounts the user's home directory on the host into the container and I redefined my home directory to be my-project-directory in the above.
However I still cannot install R packages into the container as the file system is not writable.
> install.packages(c("plyr", "dply", "tidyr", "ggplot2"))
Installing packages into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
Warning in install.packages :
'lib = "/usr/local/lib/R/site-library"' is not writable
Would you like to use a personal library instead? (yes/No/cancel) cancel
Error in install.packages : unable to install packages
Original question still stands: how can I create a Singularity 3.* container that has a writable filesystem? If this is not possible, explanation as to why would be appreciated.
Reading in this article, apparently there is a solution to the problem concerning package installation:
Create an .Renviron file in your home directory and set R_LIBS_USER as follows:
# User-installed R packages go into their home directory
echo 'R_LIBS_USER=~/R/%p-library/%v' >> ${HOME}/.Renviron
This solved the package-installation-issue for me.
try --writable flag in your container:
singularity build --writable rstudio-3.5.2.sif docker://rocker/rstudio:3.5.2
singularity exec --writable --bind example-project-1/:/home/rstudio/ rstudio-3.5.2.sif rserver --www-port 8787
You can create a sandbox folder with --sandbox option .
Also you probably can create a Virtual Machine / vagrant for singularity 2.5
Related
I had Docker Toolbox installed on my Windows 7 PC and I wanted to upgrade my Docker installation to the most recent version. To do that, I decided to delete Docker Toolbox from my system and reinstall it. I uninstalled Docker Toolbox, uninstalled VirtualBox, and removed all instances of both in my files (such as files in AppData). After reinstalling Docker Toolbox and launching the Quickstart Terminal, I ran into the following error:
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this vi
rtual machine, run: C:\Program Files\Docker Toolbox\docker-machine.exe env defau
lt
Looks like something went wrong in step 'Setting env'... Press any key to contin
ue...
So it seems like it failed when "setting env". I'm not sure what that means in this context and I wish there was a way to check some extended logs to get more detail. I tried following the Docker documentation pointing the location of daemon logs in AppData, however, I could not find anything relevant. Something I did find was a file called "no-error-report", though it was empty.
I tried uninstalling everything again and reinstalling with the attribute NDIS5 network type checked, I've ran the Quickstart Terminal as admin, and I still ran into the same exact error.
Any suggestions on how I may approach this issue?
I got this same issue.
I fixed this by doing the below procedures.
I changed the below lines in start.sh
STEP="Setting env"
eval "$("${DOCKER_MACHINE}" env --shell=bash --no-proxy "${VM}" | sed -e "s/export/SETX/g" | sed -e "s/=/ /g")" &> /dev/null #for persistent Environment Variables, available in next sessions
eval "$("${DOCKER_MACHINE}" env --shell=bash --no-proxy "${VM}")" #for transient Environment Variables, available in current session
Changed --no-proxy to --http_proxy since I am using http proxy
Using this docker image:
docker build -t batocera-docker https://github.com/batocera-linux/batocera.docker.git
I launch a container this way, so that the sources are available in F:\docker Windows folder for browsing.
docker run -it -v F:\docker:/build batocera-docker
The following commands start the build process:
git clone git://git.buildroot.net/buildroot
cd buildroot/
make pc_x86_64_bios_defconfig
make
Which fails when processing the "host-gmp" component:
>>> host-gmp 6.1.2 Building
The build fails with the following error (but from experiment, it seems it does not always fail on the same files).
m4: cannot open `invert_limb_table.asm': No such file or directory
This is a "strange" because, the following command shows the file exists where it should (and issuing the "cat" command shows a valid file content!).
root#fe9bc1b08539:/build/buildroot# ls -la
/build/buildroot/output/build/host-gmp-6.1.2/mpn/invert_limb_table.asm lrwxrwxrwx 1 root root 35 Feb 12 22:01 /build/buildroot/output/build/host-gmp-6.1.2/mpn/invert_limb_table.asm -> ../mpn/x86_64/invert_limb_table.asm
Sometimes, the error states "File handle stale".
However, such errors always occur on symbolic links files (symlinks or hardlinks?)
I'm confused because creating a symbolic link in the mounted folder seems to work (it works using the ln command), but then it fails at some point, as if the overlay file system of the container was not synchronizing its content with the mounted folder "fast enough"?
Would there be any work around?
(I could build in a container folder, but that is trivial and not much useful to me as the sources are not available from outside).
According to the documentation at bazelbuild/rules_docker, it should be possible to work with these container images on OSX, and it also claims that it's possible to do so without docker.
These rules do not require / use Docker for pulling, building, or pushing images. This means:
They can be used to develop Docker containers on Windows / OSX without boot2docker or docker-machine installed.
They do not require root access on your workstation.
How do I do that? Here's a simple rule:
go_image(
name = "helloworld_image",
importpath = "github.com/nictuku/helloworld",
library = ":go_default_library",
visibility = ["//visibility:public"],
)
I can build the image with bazel build :helloworld_image. It produces a tar ball in blaze-bin, but it won't run it:
INFO: Running command line: bazel-bin/helloworld_image
Loaded image ID: sha256:08d312b529d30431c68741fd3a31468a02533f27a8c2c29eedc969dae5a39852
Tagging 08d312b529d30431c68741fd3a31468a02533f27a8c2c29eedc969dae5a39852 as bazel:helloworld_image
standard_init_linux.go:185: exec user process caused "exec format error"
ERROR: Non-zero return code '1' from command: Process exited with status 1.
It's trying to run the linux this is OSX, which is silly.
I also tried doing a "docker load" on the .tar content but it doesn't seem to like that format.
$ docker load -i bazel-bin/helloworld_image-layer.tar
open /var/lib/docker/tmp/docker-import-330829602/app/json: no such file or directory
Help? Thanks!
You are building for your host platform by default so you need to build for the container platform if you want to do that.
Since you are using a go binary, you can do cross compilation by specifying --cpu=k8 on the command line. Ideally we would be able to just say that the docker image needs a linux binary (so no need to specify the --cpu command-line flag) but this is still a work in progress in Bazel.
I am currently trying to learn Hyperledger Fabric. I have followed all the instructions till here. I have gotten all the necessary binaries and downloaded all the required samples. When I go into the Fabcar folder and use the "ls" command, I can see all the files expected according to the document EXCEPT chaincode.
$ ls
creds/ invoke.js package.json query.js startFabric.sh*
When I run "./startFabric.sh" command, I get the following errors:
$ ./startFabric.sh
cp: cannot stat '/c/Users/Vaibhav': No such file or directory
cp: cannot stat 'Shah/fabric-samples/fabcar/creds/*': No such file or directory
I am attaching a screenshot of the entire Docker interface below.
Thank you very much.
Screenshot of error in Docker
Check this example in https://github.com/hyperledger/fabric/tree/release/examples/e2e_cli
This is the end-to-end test of the fabric using the command line interface. It helps you understand installation, setting up channels and executing if I recall chaincode_example02. Obviously, this runs on Ubuntu 14.04.
If you are a Windows user, you may want to spin up an Ubuntu 14.04 VM and try out, after installing all the pre-requisites.
I have setup an automated build on Docker hub here (the sources are here).
The build goes well locally. I have also tried to rebuild it with --no-cache option:
docker build --no-cache .
And the process completes successfully
Successfully built 68b34a5f493a
However, the automated build fails on Docker hub with this error log:
...
Cloning into 'nerdtree'...
[91mVim: Warning: Output is not to a terminal
[0m
[91mVim: Warning: Input is not from a terminal
[0m
[m[m[0m[H[2J[24;1HError detected while processing command line:
E492: Not an editor command: PluginInstall
E492: Not an editor command: GoInstallBinaries
[91mmv: cannot stat `/go/bin/*': No such file or directory
[0m
This build apparently fails on the following vim command:
vim +PluginInstall +GoInstallBinaries +qall
Note that the warnings Output is not to a terminal and Input is not to a terminal appears also in the local build.
I cannot understand how this can happen. I am using a standard Ubuntu 14.04 system.
I finally figured it out. The issue was related to this one.
I am using Docker 1.0 in my host machine, however a later version is in production in Docker Hub. Without the explicit ENV HOME=... line in the Dockerfile, version 1.0 uses / as home directory, while /root is used by the later version. The result is that vim was not able to find its .vimrc file, since it was copied in / instead of /root. The solution I used is to explicitly define ENV HOME=/root in my Dockerfile, so there are no differences between the two versions.