jenkins doesn't have rights to copy files - or does it? - jenkins

I have a strange problem in jenkins, I cannot copy files in a job, however, with the user jenkins on the command line, I can do that without problem.
I am using jenkins on debian running under the user "jenkins".
I added the user "jenkins" to the group "www-data", so that I can copy files to the www-folder of apache.
The folder rights of the target folder look like this:
drwxrwxr-x 9 www-data www-data 4096 Jun 23 16:19 .
drwxrwxr-x 4 www-data www-data 4096 Jun 23 12:45 ..
-rw-rw-r-- 1 volker www-data 368 Jun 23 17:10 about.php
-rw-rw-r-- 1 volker www-data 366 Jun 23 17:10 bio.php
-rw-rw-r-- 1 volker www-data 370 Jun 23 17:10 contact.php
drwxrwxr-x 3 volker www-data 4096 Jun 23 16:19 content
drwxrwxr-x 3 volker www-data 4096 Jun 23 16:19 css
drwxrwxr-x 8 volker www-data 4096 Jun 23 16:19 default
drwxrwxr-x 3 volker www-data 4096 Jun 23 16:19 fonts
drwxrwxr-x 2 volker www-data 4096 Jun 23 13:40 image
drwxrwxr-x 3 volker www-data 4096 Jun 23 16:19 images
-rw-rw-r-- 1 volker www-data 372 Jun 23 17:10 impressum.php
-rw-rw-r-- 1 volker www-data 367 Jun 23 17:10 index.php
-rw-rw-r-- 1 volker www-data 296 Jun 23 13:52 kontakt.php
drwxrwxr-x 3 volker www-data 4096 Jun 23 16:19 layout
-rw-rw-r-- 1 volker www-data 367 Jun 23 17:10 news.php
-rw-rw-r-- 1 volker www-data 370 Jun 23 17:10 termine.php
-rw-rw-r-- 1 volker www-data 369 Jun 23 17:10 videos.php
So everything is writable for group www-data.
If I am using the jenkins user to copy the files in the shell, I get no error:
jenkins#rootserver:~/jobs/deploy_notundellende/workspace$ whoami
jenkins
jenkins#rootserver:~/jobs/deploy_notundellende/workspace$ cp -R * /var/www/nue
jenkins#rootserver:~/jobs/deploy_notundellende/workspace$
But if I use the same command in jenkins itself, it fails with permission error:
pwd
/var/lib/jenkins/jobs/deploy_notundellende/workspace
whoami
jenkins
cp -R about.php bio.php contact.php content css fonts images impressum.php index.php layout news.php termine.php videos.php /var/www/nue
cp: cannot create regular file `/var/www/nue/about.php': Permission denied
cp: cannot create regular file `/var/www/nue/bio.php': Permission denied
cp: cannot create regular file `/var/www/nue/contact.php': Permission denied
cp: cannot create regular file `/var/www/nue/content/videos.php': Permission denied
How is that possible? Does anyone have an idea?

OK, I got it to work, I restarted the jenkins server and it worked. I assume it did not work before, because the jenkins server was already running when I changed its permissions. Makes sense to me now, come to think of it :) Anyway, thanks for anybody reading and thinking about this!

Solution 1: Restart Jenkins
(jenkins_url)/safeRestart - Allows all running jobs to complete. New jobs will remain in the queue to run after the restart is complete.
(jenkins_url)/restart - Forces a restart without waiting for builds to complete.
Solution 2: Check the user and permission for the same user
Check user: whoami
change permission: sudo chmod -R 777 /var/www/html/* or sudo chmod a+rwx /var/szDirectoryName
Solution 3:
If you get error/warning like Linux: 'Username' is not in the sudoers file. This incident will be reported ref the link

Related

How to mount volumes inside a container that is started from a bash script in Google Cloud Build?

I have a cloudbuild.yaml file that looks like this:
steps:
- name: 'gcr.io/cloud-builders/gsutil'
args: [ "-m", "rsync", "-r", "gs://${_BUCKET}/maven-repository", "/cache/.m2" ]
volumes:
- path: '/cache/.m2'
name: 'm2_cache'
- name: docker/compose:debian-1.29.2
entrypoint: bash
args:
- -c
- |
./test.sh
volumes:
- path: '/cache/.m2'
name: 'm2_cache'
timeout: 2700s
substitutions:
_BUCKET: 'my-bucket'
In the first step we download our maven settings.xml file from GCS. This file is crucial for subsequent build steps since it contain the username/password to our Artifact Registry Maven repository (I've simplified this example as we don't actually store the credential in the settings.xml as plain text). Without these credentials, our Maven build won't run. Normally the script that we call in the second step starts several docker containers and then run our maven tests. But I've replaced it with test.sh to easier show what the problem is. The test.sh file is shown below:
#!/bin/bash
echo "### [Host] Contents in /cache/.m2"
ls -la /cache/.m2
mkdir ~/test
echo "Johan" > ~/test/ikk.txt
echo "### [Host] Contents in ~/test"
ls -la ~/test
docker run --rm -v /cache/.m2:/cache/.m2 -v ~/test:/root/test -w /usr/src/somewhere ubuntu bash -c 'echo "### [Docker] Contents in /cache/.m2" && ls -la /cache/.m2 && echo "### [Docker] Contents in /root/test" && ls -la /root/test'
I.e. we try to mount two volumes to the ubuntu container that we start in the test.sh file. I list the contents in two directors both outside (### [Host]) and inside (### [Docker]) the ubuntu container. Here's the relevant output of running this in cloud build:
### [Host] Contents in /cache/.m2
total 16
drwxr-xr-x 2 root root 4096 Sep 15 08:55 .
drwxr-xr-x 3 root root 4096 Sep 15 08:55 ..
-rw-r--r-- 1 root root 8063 Sep 13 11:03 settings.xml
### [Host] Contents in ~/test
total 12
drwxr-xr-x 2 root root 4096 Sep 15 08:55 .
drwxr-xr-x 6 root root 4096 Sep 15 08:55 ..
-rw-r--r-- 1 root root 6 Sep 15 08:55 ikk.txt
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
Digest: sha256:20fa2d7bb4de7723f542be5923b06c4d704370f0390e4ae9e1c833c8785644c1
Status: Downloaded newer image for ubuntu:latest
### [Docker] Contents in /cache/.m2
total 8
drwxr-xr-x 2 root root 4096 Sep 15 08:55 .
drwxr-xr-x 3 root root 4096 Sep 15 08:55 ..
### [Docker] Contents in /root/test
total 8
drwxr-xr-x 2 root root 4096 Sep 15 08:55 .
drwx------ 1 root root 4096 Sep 15 08:55 ..
As you can see, the volume mounts doesn't seem to work when I run the ubuntu container from the test.sh file in cloud build (since the contents of /root/test and /cache/.m2 are empty).
Running the test.sh locally on my machine yields the expected outcome:
### [Host] Contents in /cache/.m2
total 40
drwxr-xr-x 7 johan staff 224 Mar 15 2022 .
drwxr-x---+ 87 johan staff 2784 Sep 15 10:58 ..
-rw-r--r-- 1 johan staff 2344 Sep 14 11:37 copy_reference_file.log
drwxr-xr-x 221 johan staff 7072 Sep 14 10:52 repository
-rw-r--r-- 1 johan staff 327 Nov 24 2021 settings-docker.xml
-rw-r--r--# 1 johan staff 9842 Mar 15 2022 settings.xml
drwxr-xr-x 3 johan staff 96 Nov 19 2021 wrapper
### [Host] Contents in ~/test
total 8
drwxr-xr-x# 3 johan staff 96 Sep 15 10:53 .
drwxr-xr-x# 135 johan staff 4320 Sep 15 10:49 ..
-rw-r--r-- 1 johan staff 6 Sep 15 10:58 ikk.txt
### [Docker] Contents in /cache/.m2
total 24
drwxr-xr-x 7 root root 224 Mar 15 2022 .
drwxr-xr-x 3 root root 4096 Sep 15 08:58 ..
-rw-r--r-- 1 root root 2344 Sep 14 09:37 copy_reference_file.log
drwxr-xr-x 221 root root 7072 Sep 14 08:52 repository
-rw-r--r-- 1 root root 327 Nov 24 2021 settings-docker.xml
-rw-r--r-- 1 root root 9842 Mar 15 2022 settings.xml
drwxr-xr-x 3 root root 96 Nov 19 2021 wrapper
### [Docker] Contents in /root/test
total 8
drwxr-xr-x 3 root root 96 Sep 15 08:53 .
drwx------ 1 root root 4096 Sep 15 08:58 ..
-rw-r--r-- 1 root root 6 Sep 15 08:58 ikk.txt
Here you can see that the volumes are mounted correctly and I can access the files inside the ubuntu container.
How can I mount volumes inside a container in cloud build?

Why is my symbolic link not linking, but creating a file inside the diretory I'm trying to link

Im trying to do a proyect on Drupal using Docker and Composer. This is my docker-compose.yml
app:
build: ./app/.
volumes:
- /home/username/practicas/docker_drupal/drupal_ensi:/var/www
links:
- db
ports:
- 86:80
db:
image: mariadb:10.5.3
volumes:
- /home/username/practicas/docker_drupal/db:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_USER=admin
- MYSQL_PASSWORD=root
- MYSQL_DATABASE=drupalDb
This is where I create the Drupal project with composer.
~/practicas/docker_drupal$ docker exec -it docker_drupal_app_1 bash
root#11da03a5baab:/var/www/html# php -d memory_limit=-1 /usr/local/bin/composer create-project drupal-composer/drupal-project:8.x-dev some-dir --no-interaction
Then I move all the content of the some-dir directory to the /var/www/ level
root#11da03a5baab:/var/www/html# ls -l
total 4
drwxr-xr-x 6 root root 4096 May 22 09:57 some-dir
root#11da03a5baab:/var/www/html# cd ..
root#11da03a5baab:/var/www# ls -l
total 4
drwxr-xr-x 3 root root 4096 May 22 09:56 html
root#11da03a5baab:/var/www# mv html/some-dir/* ./
root#11da03a5baab:/var/www# ls -l
total 396
-rw-r--r-- 1 root root 18046 May 22 09:56 LICENSE
-rw-r--r-- 1 root root 6451 May 22 09:56 README.md
-rw-r--r-- 1 root root 2370 May 22 09:56 composer.json
-rw-r--r-- 1 root root 343241 May 22 09:57 composer.lock
drwxr-xr-x 4 root root 4096 May 22 09:56 drush
drwxr-xr-x 3 root root 4096 May 22 09:56 html
-rw-r--r-- 1 root root 321 May 22 09:56 load.environment.php
-rw-r--r-- 1 root root 481 May 22 09:56 phpunit.xml.dist
drwxr-xr-x 3 root root 4096 May 22 09:56 scripts
drwxr-xr-x 54 root root 4096 May 22 09:57 vendor
drwxr-xr-x 7 root root 4096 May 22 09:57 web
And then try to do the symbolic link between web and html. When the server tries to go to the html directory by default, it should redirect to the web directory, because there's where the Drupal project is. (I dont know if I should do this inside the container or outside, but I would assume it really doesnt matter since its mounted in the docker-composer.yml, right? I dont know anymore at this point).
Whenever I try do the ln -s web/ html, it just creates a file inside the html folder, it doesnt create the link :c
~/practicas/docker_drupal/drupal_ensi$ sudo ln -s web/ html
[sudo] password for yindazai:
yindazai#yindazai-Predator-PH317-52:~/practicas/docker_drupal/drupal_ensi$ ls -l
total 396
-rw-r--r-- 1 root root 2370 may 22 11:56 composer.json
-rw-r--r-- 1 root root 343241 may 22 11:57 composer.lock
drwxr-xr-x 4 root root 4096 may 22 11:56 drush
drwxr-xr-x 3 root root 4096 may 22 12:07 html
-rw-r--r-- 1 root root 18046 may 22 11:56 LICENSE
-rw-r--r-- 1 root root 321 may 22 11:56 load.environment.php
-rw-r--r-- 1 root root 481 may 22 11:56 phpunit.xml.dist
-rw-r--r-- 1 root root 6451 may 22 11:56 README.md
drwxr-xr-x 3 root root 4096 may 22 11:56 scripts
drwxr-xr-x 54 root root 4096 may 22 11:57 vendor
drwxr-xr-x 7 root root 4096 may 22 11:57 web
yindazai#yindazai-Predator-PH317-52:~/practicas/docker_drupal/drupal_ensi/html$ ls -l
total 4
drwxr-xr-x 2 root root 4096 may 22 11:58 some-dir
lrwxrwxrwx 1 root root 4 may 22 12:07 web -> web/
This last part was done outside the container, but if I do it inside, it doesnt create the link but a file inside the html folder.
I think I'am doing all the steps right, but obviously its not work, so any help would be indeed much appreciated. Thx.
May be you can have a little help at github: https://github.com/wayoflegend/drupalindocker . Because in fact you have a little miss understand of the composer commands and may be you didn't have the right vhost configuration.

Graphileon InterActor Settings Page not rendering properly on first startup

I am trying to get InterActor Community Edition running on my ubuntu 16.04 machine by following the instructions on this page
http://docs.graphileon.com/interactor/Getting_started/Setup_InterActor/Installation.html
I am fine up to the point I do the docker run...-command.
Also, I have neo4j CE 3.2.5 already running at that time....
When I open the Startpage http://localhost:8000 it shows following
login page
Contrary to the description no settings page is shown.
I also opened it directly by typing localhost:8000/settings in the browser and I got the settings page obviously with missing form-inputs and no styles rendered (compared to picture on installation instruction page)
I thought files were missing, therefore did a docker exec -it interactor /bin/bash to check sources (especially js an css-files) within the running container under the directory /var/www/html/interactor/, seemed to be ok, but i do not know how the permissions have to be set, so here they are in the runing container:
root#ffb64b944023:/var/www/html/interactor# ll
total 168
drwxr-xr-x 21 www-data www-data 4096 Oct 2 18:32 ./
drwxr-xr-x 5 root root 4096 Oct 2 18:32 ../
-rw-r--r-- 1 www-data www-data 446 Sep 6 14:26 .htaccess
-rw-r----- 1 www-data www-data 90804 Sep 6 14:26 INTERACTOR_END_USER_LICENSE_AGREEMENT.pdf
drwxr-x--- 4 www-data www-data 4096 Sep 6 14:27 css/
drwxr-x--- 4 www-data www-data 4096 Sep 6 14:27 dashboard/
-rw-r----- 1 www-data www-data 5430 Jun 21 15:19 favicon.ico
-rw-r--r-- 1 www-data www-data 5738 Sep 6 14:26 favicon.png
drwxr-x--- 2 www-data www-data 4096 Sep 6 14:27 images/
-rw-r--r-- 1 www-data www-data 6929 Sep 6 14:26 index.php
drwxr-x--- 4 www-data www-data 4096 Sep 6 14:27 js/
drwxr-x--- 3 www-data www-data 4096 Oct 1 19:31 persistent/
drwxr-x--- 14 www-data www-data 4096 Sep 6 14:27 php/
drwxr-x--- 2 www-data www-data 4096 Sep 6 14:27 scripts/
drwxr-x--- 11 www-data www-data 4096 Oct 2 18:32 settings/
drwxr-x--- 2 www-data www-data 4096 Sep 6 14:27 templates/
-rw-r----- 1 www-data www-data 98 Sep 6 14:26 version.json
thus I tried to open the linked sources in the pagesource within the browser and got some of the files opened and some not, due to 403-Error.
I wonder if the provided image is somehow misconfigured regarding the permissions or if I am doing something wrong.
The strange thing is that I already had it running after I clicked all of the linked sources in the pagesource and reloaded the settings page.
When I finished playing around in interactor I used a docker stop interactor. neither a docker start interactor nor a docker restart interactor gave me a working interactor-instance back and I cannot get it working anymore...what I am doing wrong?
I can reproduce this issue but so far i cannot find a permanent solution.
The only thing to make it work is to connect to the container:
sudo docker exec -i -t interactor /bin/bash
And the run:
chown www-data:www-data /var/www/html/interactor -R
even if all the files are already set like that.
You have to do this everytime you start the container.
We are working on a fix for this.
Disclaimer: i am a developer for Graphileon.

docker cp - "Error response from daemon: not a directory"

I am trying to copy file from docker to host using the below command,
docker cp <container_name>:<file FQN> ./
But getting the below error,
Error response from daemon: not a directory
As verified, the file name and container name are valid.
Note: Using Docker in Mac
Thanks for all the answers. After a bit of struggle found out that the error message was not actually directly related to the docker cp command.
The scenario was, I ran the docker with the link to a local file. When the docker was running I deleted it. Then the file got created as a folder somehow (Probably, when I restarted the docker).
And whenever I am executing some command, the docker was giving me that error. Then once I created the file the error disappeared.
It seems your command is correct. You please try like the below from your local machine not from inside the container. sometimes unfortunately if we run this command with in the container we will get this kind of errors.
docker cp [container_name]:[docker dir abs path] [host dir path]
Hope it will help you.
Here is a full example on how to copy a file:
$ docker run -it ubuntu /bin/bash
root#9fc8a1af7f23:/#
root#9fc8a1af7f23:/# ll
total 72
drwxr-xr-x 34 root root 4096 Jul 13 21:51 ./
drwxr-xr-x 34 root root 4096 Jul 13 21:51 ../
-rwxr-xr-x 1 root root 0 Jul 13 21:51 .dockerenv*
drwxr-xr-x 2 root root 4096 Feb 14 23:29 bin/
drwxr-xr-x 2 root root 4096 Apr 12 2016 boot/
drwxr-xr-x 5 root root 360 Jul 13 21:51 dev/
drwxr-xr-x 45 root root 4096 Jul 13 21:51 etc/
drwxr-xr-x 2 root root 4096 Apr 12 2016 home/
drwxr-xr-x 8 root root 4096 Sep 13 2015 lib/
drwxr-xr-x 2 root root 4096 Feb 14 23:29 lib64/
drwxr-xr-x 2 root root 4096 Feb 14 23:28 media/
drwxr-xr-x 2 root root 4096 Feb 14 23:28 mnt/
drwxr-xr-x 2 root root 4096 Feb 14 23:28 opt/
dr-xr-xr-x 288 root root 0 Jul 13 21:51 proc/
drwx------ 2 root root 4096 Feb 14 23:29 root/
drwxr-xr-x 6 root root 4096 Feb 27 19:41 run/
drwxr-xr-x 2 root root 4096 Feb 27 19:41 sbin/
drwxr-xr-x 2 root root 4096 Feb 14 23:28 srv/
dr-xr-xr-x 13 root root 0 Jul 13 21:51 sys/
drwxrwxrwt 2 root root 4096 Feb 14 23:29 tmp/
drwxr-xr-x 11 root root 4096 Feb 27 19:41 usr/
drwxr-xr-x 13 root root 4096 Feb 27 19:41 var/
root#9fc8a1af7f23:/# cd tmp/
root#9fc8a1af7f23:/tmp# ls
root#9fc8a1af7f23:/tmp# echo "hello docker" > docker_test.txt
root#9fc8a1af7f23:/tmp# cat docker_test.txt
hello docker
root#9fc8a1af7f23:/tmp#
Then, in another terminal
dali#dali-X550JK:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9fc8a1af7f23 ubuntu "/bin/bash" 2 minutes ago Up 2 minutes fervent_hodgkin
dali#dali-X550JK:~$ docker cp fervent_hodgkin:/tmp/docker_test.txt /tmp/
dali#dali-X550JK:~$ cat /tmp/docker_test.txt
hello docker
dali#dali-X550JK:~$
Please follow these instruction, make sure your don't have typo in the file paths, otherwise share a reproducible error.
This error also appears when trying to copy a file that is actually a volume in the container, but the file has been deleted on the host.
This is simply an error in the path you want to copy.
You may not believe it, but that it is.

sh in docker image does not see executable on Windows 10

I have a docker image https://github.com/carnellj/spmia-chapter1 which does not find its CMD ./run.sh executable although it is there in the file system.
I was able to run /bin/sh in the container, and I can ls -l:
D:\Dokumente\ws\spring-microservices\spmia-chapter1 (master)
λ docker run -i -t johncarnell/tmx-simple-service:chapter1 /bin/sh
/ # ls -l
total 56
drwxr-xr-x 2 root root 4096 Mar 3 11:20 bin
drwxr-xr-x 5 root root 360 Apr 22 07:10 dev
drwxr-xr-x 1 root root 4096 Apr 22 07:10 etc
drwxr-xr-x 2 root root 4096 Mar 3 11:20 home
drwxr-xr-x 1 root root 4096 Apr 22 06:01 lib
drwxr-xr-x 5 root root 4096 Mar 3 11:20 media
drwxr-xr-x 2 root root 4096 Mar 3 11:20 mnt
dr-xr-xr-x 123 root root 0 Apr 22 07:10 proc
drwx------ 1 root root 4096 Apr 22 07:10 root
drwxr-xr-x 2 root root 4096 Mar 3 11:20 run
-rwxr-xr-x 1 root root 245 Apr 22 06:50 run.sh
drwxr-xr-x 2 root root 4096 Mar 3 11:20 sbin
drwxr-xr-x 2 root root 4096 Mar 3 11:20 srv
dr-xr-xr-x 13 root root 0 Apr 22 07:10 sys
drwxrwxrwt 2 root root 4096 Mar 3 11:20 tmp
drwxr-xr-x 1 root root 4096 Mar 7 01:04 usr
drwxr-xr-x 1 root root 4096 Mar 7 01:04 var
/ # ./run.sh
/bin/sh: ./run.sh: not found
/ # ls run.sh
run.sh
/bin/sh does not find ./run.sh although it is there in the file system, as proven by ls run.sh. Also, cat shows the content of run.sh:
/ # cat run.sh
#!/bin/sh
echo "********************************************************"
echo "Starting simple-service "
echo "********************************************************"
java -jar /usr/local/simple-service/simple-service-0.0.1-SNAPSHOT.jar
When I run vi from sh and copy the content of run.sh into a new file myrun.sh and make myrun.sh executable, I can execute ./myrun.sh and the spring service starts.
What is going on here? Why would sh not see an executable which is there in the filesystem? Executables from PATH or executables which I add manually run fine.
I am running Docker on Windows 10.
OK the reason is, run.sh is created with Windows line endings in the docker image if you check out with automatic lf->crlf conversion. One possible solution is to tell git not to convert line endings.

Resources