I've created a MLOPS project on BlueData 4.0 and mounted the Project Repo (NFS) folder. I created the NFS service on Centos 7x as below:
sudo yum -y install nfs-utils
sudo mkdir /nfsroot
echo '/nfsroot *(rw,no_root_squash,no_subtree_check)' | sudo tee /etc/exports
sudo exportfs -r
sudo systemctl enable nfs-server.service
sudo systemctl start nfs-server.service
I'm now trying to access a data set stored in the NFS Project Repo, but I'm receiving the following error:
PermissionError: [Errno 13] Permission denied: '/bd-fs-mnt/path/data.csv'
Any idea how I can fix this?
It appears the project repo is created with root as owner and no write permissions on group level.
To fix, you need to:
create a notebook cluster
open a Jupyter Terminal
sudo chmod -R 777 /bd-fs-mnt/nfsrepo (this only works if you create that cluster as tenant admin, as user you don't have sudo permission)
Related
I have a docker container built from the following image : FROM debian:9.11-slim
I try to install rust using the following line in my Dockerfile and it works fine until the last line. I get a permission denied error whenever I try to run /rust/cargo. However, if I connect to the container and run it from there via the command line it works. However, I need to be able to run rust/cargo commands from the docker file. Any help?
ENV RUSTUP_HOME=/rust/rustup
ENV CARGO_HOME=/rust/cargo
RUN set -eux; \
url="https://raw.githubusercontent.com/rust-lang/rustup/1.22.1/rustup-init.sh"; \
wget -O rustup-init.sh "$url"; \
echo "b273275cf4d83cb6b991c1090baeca54 rustup-init.sh" | md5sum -c -; \
echo "8928261388c8fae83bfd79b08d9030dfe21d17a8b59e9dcabda779213f6a3d14 rustup- init.sh" | sha256sum -c -; \
bash ./rustup-init.sh --profile=minimal -y -t thumbv7em-none-eabihf; \
rm rustup-init.sh; \
chmod -R go+rwX /rust; \
/rust/cargo --version
The problem is chmod -R go+rwX
How to reproduce:
We have file:
#!/bin/bash
echo good
~ $ ls -l file
total 0
-rw-r--r-- 1 user staff 0 Jun 30 11:49 file
~ $ ./file
-bash: ./file: Permission denied
~ $ chmod go+rwX file
~ $ ls -l file
-rw-rw-rw- 1 user staff 23 Jun 30 11:50 file
~ $ ./file
-bash: ./file: Permission denied
As you can see -rw-rw-rw- permissions don't allow to execute file
Solution is to use something of below:
chmod -R ug+rwx /rust (add all permissions to user and group)
chmod -R ugo+rwx /rust (add all permissions to all users)
chmod -R 777 /rust (add all permissions to all users (same as ugo+rwx))
chmod -R 755 /rust (add execution permissions to all users)
chmod 755 /rust/cargo (add execution permissions to all users only for execution file)
[if permissions already correct] don't set permissions at all (remove chmod -R go+rwX /rust) ← Best way
I faced a similar issue but in a slightly different situation. I was using docker-compose pipeline in GitHub actions on EC2 Self-Hosted Runner, based on the native GitHub pipeline. I didn't remove the Rust toolchain installation, which caused reinstallation of cargo in every build on EC2 instance changing permissions and sourcing binaries from cargo source ~/.cargo/env, hence the permission error on the default system user.
In my case, the solution was simply removing the installation of Rust from the workflow.yml and sourcing the system rust source ~/.bashrc.
I am trying to run a Docker container to analyze data in a Google Cloud Bucket.
I have been able to successfully mount the Bucket using gcsfuse, and I tested that I could do things like create and delete files within the Bucket.
In order to be able to install other programs (and mount the bucket), I installed Docker (and didn't use the Docker-optimized instance option). If I run Docker in interactive mode (without mounting a drive), it looks like it is working OK.
However, if I try to run Docker in interactive mode with the mounted drive (which is the gcsfuse-mounted Bucket), I get an error message:
user#instance:~/bucket-name/subfolder$ docker run -it -v /home/user/bucket-name:/mnt/bucket-name gcr.io/deepvariant-docker/deepvariant
docker: Error response from daemon: error while creating mount source path '/home/user/bucket-name': mkdir /home/user/bucket-name: file exists.
I hope that I am close to having this working: does anybody have any ideas about a relatively simple fix for this error message?
BTW, I realize that there are other ways to run DeepVariant on Google Cloud, but I am trying to makes things as similar as possible to what I am doing on AWS (plus, I may need to do some extra troubleshooting for analysis of one of my files).
Thank you very much for your help!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FYI, this is how I mounted the Bucket:
#mount directory: https://github.com/GoogleCloudPlatform/gcsfuse/blob/master/docs/installing.md
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt-get update
sudo apt-get -y install gcsfuse
#restart and mount directory: https://cloud.google.com/storage/docs/gcs-fuse
#NOTE: please make sure you are in your home directory (I encounter issues if I try to mount from /mnt)
mkdir [bucket-name]
gcsfuse -o allow_other --file-mode 777 --dir-mode 777 [bucket-name] ./[bucket-name]
and this is how I installed Docker:
#install Docker for Debian: https://docs.docker.com/install/linux/docker-ce/debian/
sudo apt-get update
sudo apt-get -y install \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable"
sudo apt-get update
sudo apt-get -y --allow-unauthenticated install docker-ce docker-ce-cli containerd.io
#fix Docker sock issue: https://stackoverflow.com/questions/47854463/got-permission-denied-while-trying-to-connect-to-the-docker-daemon-socket-at-uni
sudo usermod -a -G docker [user]
#have to restart after this
For anyone experiencing a similar error / issue - here is what worked for me. Steps I took:
First unmount the disk if it's already mounted: sudo umount /mounted_folder
Remount the disk using the below command, listing the credentials file to be used explicitly
sudo GOOGLE_APPLICATION_CREDENTIALS=/home/user/credentials/example-asdf21b0af7.json gcsfuse -o allow_other bucket_name /mounted_folder
Should now be connected successfully without further errors :)
NOTE: This command needs to be run everytime after restarting the computer / VM. Formatting this into fstab could probably be done so one does not need to manually execute these steps upon each restart.
EXPLANATION: What I did here was explicitly specifying the credentials via a credentials JSON for the user / service account with appropriate access (Not explained here on how to get this but should be googl-able) and referring to that json in the GOOGLE_APPLICATION_CREDENTIALS environment variable option, as suggested by this answer: https://stackoverflow.com/a/39047673/10002593. The need for this environment variable option is likely due to gcsfuse not registering the same level of access as the activated acount in gcloud config for some reason.
I think I figured out at least a partial solution to my problem:
As mentioned in this tutorial, you also need to run gcloud auth configure-docker.
I found you also needed to exit and restart your instance, but this strictly solved the original error message for this post.
I think got a strange message, but perhaps that is more about the specific container. So, I ran another test:
docker run -it -v /home/user/bucket-name:/mnt/bucket-name cwarden45/dnaseq-dependencies
This time, I got an error message about storage space on the instance (to be able to download and run the Docker container). So, I went back and created a new instance with a larger local hard drive:
1) From the Google Cloud Console, I selected "Compute Instance" and "VM instances"
2) I clicked "create instance" (similar to before)
3) I select "change" under "boot disk"
4) I set size to 300 GB instead of 10 GB (currently, towards bottom-right, under "Size (GB)")
Similar to before, I choose 8 vCPUs for the "Machine type", I selected "Allow full access to all Cloud APIs" under "Identity and API access", and I checked boxes for both "Allow HTTP traffic" and "Allow HTTPS traffic" (under "Firewall").
I am not selecting "Deploy a container image to this VM instance," which I believe is how I got Docker installed with "sudo" to be able to install gcsfuse.
I also have to call this a "parital" solution because this allows me to run the Docker container successfully in interactive mode, but the mounted bucket appears empty within Docker.
For another project, I noticed that executables could work if I installed them on the local hard drive under /opt, but not if I tried to install them on my bucket (in order to save the installation time for those programs each time). On AWS, I believe I needed to use EFS storage instead of S3 storage to do something similar, but I will keep learning more about using the Google Cloud Bucket for mounted storage / analysis.
Also, it is a different issue, but I noticed that I could fix an issue with running exectuable files from the bucket from changing the command from gcsfuse [bucket-name] ./[bucket-name] to gcsfuse --file-mode 777 --dir-mode 777 [bucket-name] ./[bucket-name] (and I changed the example code accordingly)
I noticed more recently that the set of commands above is no longer sufficient to be able to have a functional directory (I can't add or edit files, for example).
Based upon this discussion, I thought that I needed to add the -o allow_other parameter.
However, if that is all I do, I get the following error message
fusermount: option allow_other only allowed if 'user_allow_other' is set in /etc/fuse.conf
I can resolve that error message if I uncomment the corresponding line in that file. However, that still doesn't resolve having the right file permissions in the mounted directory.
So, I then tried editing my /etc/fstab file, by adding the following entry
[bucket-name] /home/[username]/[bucket-name] gcsfuse rw,allow_other,file_mode=777,dir_mode=777
I am also accordingly editing the content at the top (for whatever seems like it might help).
Also, please note that this was not a Docker-specific issue. This was necessary to essentially do anything within the bucket. Plus, I haven't actually solved this new problem.
For example, I still can't create files as root, after changing to the superuser via sudo su - (as described here)
I installed ubuntu 18.04 LTS and checked a setting for docker (17.06.2-ce) to install at the same time.
I tested by starting the hello-world (sudo docker run hello-world) :
[...]
Hello from Docker!
This message shows that your installation appears to be working correctly.
[...]
I mounted a software raid on the folder named /raid, and make a folder /docker-data in it.
I try to change the root dir of my docker to put it in /raid/docker-data/ by following the few tutorials on the network... in vain.
these solutions don't work either :
/etc/default/docker : I can't find this
As in the 2nd solution : docker can't find his folder.
Docker Root Dir: /var/snap/docker/common/var-lib-docker
Has anyone managed to do this feat in recent months?
(this is my 3rd installation of ubuntu and I just broke it...)
Apparently on Ubuntu 18.04 LTS, docker 17.06.2-it needs to work with snap, I'm going to dig this way. I will try to return answer later...
The common solution is to move the data and create a symlink:
systemctl stop docker
mv /var/lib/docker /raid/docker-data
ln -s /raid/docker-data /var/lib/docker
systemctl start docker
You can also tell docker about the new location with a setting in /etc/docker/daemon.json. If you don't have this file, you could create one with the contents:
{
"data-root": "/raid/docker-data"
}
I would recommend the first solution since you will find many 3rd party tools expect docker to be located in /var/lib/docker.
Sorry for this late response.
to come to my problem, after having looked at it more closely:
I am on Ubuntu 18.04, I can add or remove the docker service only via snap install (or remove) docker.
a docker party installs in /var/snap/
so I transpose your solution like this:
mv /var/snap/ /raid/snap
ln -s /raid/snap /var/snap
and finally I install docker via snap install docker
I have a test with sudo docker info, and I have this error message that appears :
cannot perform operation: mount --rbind /var/snap /tmp/snap.rootfs_RRAjdq//var/snap: Permission denied
(snap.rootfs_* because the end does not stop changing at each command launch)
and yet the installation went well some docker is apparently of course on /raid/snap.
I come back to you to give you the solution that allowed me to solve this problem.
cannot perform operation: mount --rbind /var/snap /tmp/snap.rootfs_RRAjdq//var/snap: Permission denied
I know why : https://bugs.launchpad.net/snapcraft/+bug/1620771 :
When /home is a symlink snaps don't work.
When /home is a real directory snaps work, see output below
In my case :
When /raid/snap is a symlink snaps don't work.
When /var/snap is a real directory snaps work.
I deleted docker. I had to reinstall snapcraf (snapd) because I was left on file modifications of it (wrong way)
from there, I stopped the snapd service:
sudo mv /var/snap/ /raid/snap
sudo mount --rbind /raid/snap /var/snap
I started the snapd service.
sudo snap install docker
sudo docker info <= to test
sudo docker run hello-world <= to test
I fixed my mount on fstab:
/raid/snap /var/snap none bind
I restarted my OS : it worked, at least for my case. (I checked all along this file consistency handling to see if the docker files was going well on the raid...)
Change Docker root storage (data path):
run this command to find docker data path:
$ sudo docker info | grep -i root
default path:
root#user-testing-HP-ProBook-4540s:/etc/docker# docker info | grep -i root
Root Dir: /var/lib/docker/aufs
WARNING: No swap limit support
Docker Root Dir: /var/lib/docker
first, stop the docker:
sudo service docker stop
copy the corrent data path to new path:
sudo rsync -aqxP /var/lib/docker /data/docker-data/
add the following on (/etc/docker/daemon.json) file:
(if the file is not there create the file with vim or your fav editor(sudo vim /etc/docker/deamon.json) )
{
"data-root": "/data/docker-data/docker"
}
conform with cat command:
cat /etc/docker/deamon.json
output will be like this:
root#user-testing-HP-ProBook-4540s:/home/user/Downloads# cat /etc/docker/daemon.json
{
"data-root": "/data/docker-data/docker"
}
root#user-testing-HP-ProBook-4540s:/home/user/Downloads#
start docker:
sudo service docker start
check the root (data path) path now:
$ sudo docker info | grep -i root
out put will be like this:
root#user-testing-HP-ProBook-4540s:/home/user/Downloads# sudo docker info | grep -i root
Root Dir: /data/docker-data/docker/aufs
WARNING: No swap limit support
Docker Root Dir: /data/docker-data/docker
root#user-testing-HP-ProBook-4540s:/home/user/Downloads#
I finished the docker installation with non-root access, namely
1.define a docker user group
2.add my current user to the docker group
pass the test
docker run --rm hello-world
but when I start to provision my docker containers, somewhere in the procedure I got the error:
Error loading config file:stat /home/user/.docker/config.json:Permission Denied
seems to be docker is trying to access some resources but got denied
What is happening here? How could I fix this ?
thx
It may be that the .docker folder and the .docker/json.config file is not owned by the current user.
Try this:
sudo chown $USER:docker ~/.docker
sudo chown $USER:docker ~/.docker/config.json
sudo chmod g+rw ~/.docker/config.json
I've got the same error after upgrate from 1.4 to latest.
Fix: manually create .docker/config.json ( put empty json object there {} ) and set ownership like this :
-rw-r--r-- 1 myuser docker /home/myuser/.docker/config.json
Simply grant your current user to access the /.docker folder and your warning will get vanished in thin air.
Run this on your terminal:
sudo chown $USER:docker ~/.docker
I am setting up my VPS to deploy my rails app, but permission denied error keeps happening on my server side. I had to reinstall perl and curl, and now I'm getting the following:
deployer#max:~$ rbenv bootstrap-ubuntu-12-04
bash: /usr/bin/python: permission denied
I tried reinstalling python but came up with no success. How should I solve it?
Update:
root#max:/home/deployer# cd /usr
root#max:/usr# cd bin
root#max:/usr/bin# cd python
bash: cd: python: not a directory
Does this mean python is not installed correctly?
Update:
deployer#max:~$ ls -l 'which python'
total 0
deployer#max:~$ python
bash: /usr/bin/python: permission denied
I thought this meant python was not installed correctly, so I computed the following
deployer#d:~$ su root
root#d:/home/deployer# apt-get install python
python package is already the latest version
So I failed to find the solution yet.
As it turned out it was a permissions issue, here are some general tips on how to debug a permission denied error when trying to execute a binary, python in this case:
ls -l `which python` - tries to retrieve full path and permissions, like -rwxr-xr-x 1 nobody nobody 9644 2012-12-02 13:15 /tmp/env/bin/python
Has to be run as a privileged user if permissions are 0000, for example.
sudo chmod 0755 /usr/bin/python usually helps.