Rocker/Shiny not seeing apps after reboot - docker

What can be the reason for rocker/shiny to not recognize shiny apps in host OS?
I'm using rocker/shiny to experiment with shiny apps on windows.
This is how I start the image:
docker run -d -p 80:3838 -v D:/Projects/DockedShiny/apps/:/srv/shiny-server/ -v D:/Projects/DockedShiny/logs:/var/log/shiny-server/ rocker/shiny
I can see shiny was started:
C:\Users\Honza>docker logs gallant_thompson
*** warning - no files are being watched ***
[2019-02-08T06:49:04.860] [INFO] shiny-server - Shiny Server v1.5.9.1 (Node.js v8.11.3)
[2019-02-08T06:49:04.887] [INFO] shiny-server - Using config file "/etc/shiny-server/shiny-server.conf"
[2019-02-08T06:49:04.927] [WARN] shiny-server - Running as root unnecessarily is a security risk! You could be running more securely as non-root.
[2019-02-08T06:49:04.931] [INFO] shiny-server - Starting listener on http://[::]:3838
Edit: Anybody has any hints what *** warning - no files are being watched *** exactly means? I have suspition this might be a clue.
By inspecting the configuration I can see it's configured to files and folders that I specified during image startup:
root#778e307632ab:/etc/shiny-server# more shiny-server.conf
# Instruct Shiny Server to run applications as the user "shiny"
run_as shiny;
# Define a server that listens on port 3838
server {
listen 3838;
# Define a location at the base URL
location / {
# Host the directory of Shiny Apps stored in this directory
site_dir /srv/shiny-server;
# Log all Shiny output to files in this directory
log_dir /var/log/shiny-server;
# When a user visits the base URL rather than a particular application,
# an index of the applications available in this directory will be shown.
directory_index on;
}
}
Despite this the shiny web directory listing is empty and when specifying app by name (e.g. http://localhost/myapp) I'm getting 'page not found'. There is no new logs in Shiny log location.
The exact same scenario worked just fine for months. I recently needed to reboot my host machine and now I cannot make shiny to recognize my apps.

Server is complaining about running as root. I'd start by adding --user shiny. Also you're not sticking to a particular version but latest, and I can see the rocker/shiny image was updated a few days ago, so it could be broken or work different than previous version. I'd stick to 3.5.1 or whatever version you prefer, like rocker/shiny:3.5.1.

LSS: The issue was in recently changed username on affected machine. Docker is caching old one and that needed to be explicitly reset in docker settings -> Shared Drives menu
More details: https://github.com/rocker-org/shiny/issues/59

Related

fail2ban won't start using nextcloud.log with jail

I have nextcloud installed and working fine in a docker but want to have fail2ban monitor the log files for brute force attempts. I know nextcloud has it's own baked in but it just throttles the log in attempts and I would like to all out ban them (I also have this problem with other containers as well). The docker-compose is set to create the nextcloud.log file to /mnt/nextcloud/log/nextcloud.log. I followed this guide to create the jail
https://www.c-rieger.de/nextcloud-installation-guide-ubuntu/#c06
Fail2ban is running on the host machine however, fail2ban fails to start with:
[447]: ERROR Failed during configuration: Have not found any log file for nextcloud jail
[447]: ERROR Async configuration of server failed
Thinking it was simply a permission issue, I chowned everything to root and tried to start again but still the service won't start. What am I doing wrong?
Thanks for the help!
The docker-compose is set to create the nextcloud.log file to /mnt/nextcloud/log/nextcloud.log
Be sure this file really exists and your jail.local has correct entry logpath:
[nextcloud]
...
logpath = /mnt/nextcloud/log/nextcloud.log
You can also check resulting config using dump:
fail2ban-client -d | grep 'nextcloud.*logpath'
But I'm still not sure the error message you provide was throwed by fail2ban, because its error messages look different, see https://github.com/fail2ban/fail2ban/commit/27947407bc7910f0f50972113218ebc73c4a22c7
It should be something like:
-have not found a log file for nextcloud log
+Have not found any log file for nextcloud jail

is /usr/bin/ authorized as a default location for volume sharing in docker prefrences?

I am sharing a docker-compose file with a team member to easily build our app. We're both on OSX and it works fine from my machine, but my colleague is getting the following error:
ERROR: for backend Cannot start service backend: b'Mounts denied: \r\nThe path /usr/bin/docker\r\nis not shared from OS X and is not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\nSee https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.\r\n.'
I assume it is due to the following statement in the docker-compose.yaml.
volumes:
- "/usr/bin/docker:/usr/bin/docker"
I didn't have to alter my docker-> preferences-> file sharing to make this work. I only have the default dirs shared: /Users, /Volumes, /tmp, /private.
How come it isn't working on his machine? Does he have to add the /usr dir as a shared dir? If so, how come I don't have to?
UPDATE
The problem was that the docker executable was located in /usr/local/bin/ rather than /usr/bin. I have no idea why docker installed the executable differently despite both machines being OSX.
That's a misleading error, should have asked you to check if the path /usr/bin/docker exists on the host machine. Docker (at least on Mac) will attempt to create a directory if it doesn't exist. Apparently, your team mate isn't logged in with sufficient privileges to create /usr/bin/docker.

Unable to start any container when Volumes are enabled Docker Toolbox

I am running Docker Toolbox v. 1.13.1a on Windows 7 Pro Service pack 1 x64OS.
with Virtual Box Version 5.1.14 r112924
when I try to run any docker image e.g. official postgres image from Docker Hub with volumes disabled, it works fine!
But when I enable the volumes it fails.
I tried all official documentations
The VM has shared folder as required and has full access to it also
shared folder screenshot
In case of my example of postgresql it crashes with following log
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... LOG: could not link file "pg_xlog/xlogtemp.27" to "pg_xlog/000000010000000000000001": Operation not permitted
FATAL: could not open file "pg_xlog/000000010000000000000001": No such file or directory
child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/postgresql/data"
I know its the problem with folder permissions. But kinda stuck!
A ton of thanks in advance
I've been busy with this problem all day and my conclusion that it's currently simply not possible to run postgresql inside a docker container while keeping your data persistent in a separate volume.
I even tried running the container without linking to a volume and copying the data that was originally in /var/lib/postgresql into a folder of my host OS (Windows 10 Home), then copy that into the folder that got then linked to the container itself.
Alas, I got the next error:
FATAL: data directory "/var/lib/postgresql/data/pgadmin" has wrong ownership
HINT: The server must be started by the user that owns the data directory.
In conclusion: There's something going wrong with the ownership and the correct user owning it and to be able to fix it, you'll need a unix commandline on Windows that is able to run docker (something currently not possible with Bash on Ubuntu on Windows that is running using Ubuntu 16.04 binaries).
Maybe, in the future, you'll be able to run the needed commands (found here, under Arbitrary --user Notes), but these are *nix commands and powershell (started by Kitematic) can't run those. Bash for Ubuntu for Windows could run those, but that shell has no connection to the docker daemon/service on windows...
TL;DR: Lost a day of work: It is currently impossible on Windows.
I have been trying to fix this issue also ..
At first I thought it was a symlink problem (because the first error fails on " could not link .. operation not permitted)
To be sure symlink is permitted you have to :
share a folder in virtualbox
run virtualbox as administrator (if you account is in administrator group) Right click virtualbox.exe and select run as Administrator
if your account is not administrator, add the symlink privilege with secpol.msc > "Local Policies-User Rights Assignments" add your user to "Create symbolic links"
enable symlink for your shared folder in virtualbox :
VBoxManage setextradata VM_NAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARED_FOLDER_NAME 1
Alternatively you can also use the c:\User\username folder which is shared and symlink enabled by default dockertools installation
Now I can create symlinks in the shared folder from the docker container .. but I still have the same error "could not link ... operation not permitted"
So the reason must be somewhere else ... in the file permissions as you said but I do not see why ?

OrientDB " Error on moving existent database"

I'm trying to setup OrientDB distributed configuration with docker. But I'm getting error when starting second node -
2015-10-09 17:14:14:066 WARNI
[node1444321499719]->[[node1444321392311]] requesting deploy of
database 'testDB' on local server... [OHazelcastPlugin] 2015-10-09
17:14:14:117 INFO [node1444321499719]<-[node1444321392311] received
updated status node1444321499719.testDB=SYNCHRONIZING
[OHazelcastPlugin] 2015-10-09 17:14:14:119 INFO
[node1444321499719]<-[node1444321392311] received updated status
node1444321392311.testDB=SYNCHRONIZING [OHazelcastPlugin] 2015-10-09
17:14:15:935 WARNI [node1444321499719] moving existent database
'testDB' located in '/orientdb/databases/testDB' to
'/orientdb/databases/../backup/databases/testDB' and get a fresh copy
from a remote node... [OHazelcastPlugin] 2015-10-09 17:14:15:936 SEVER
[node1444321499719] error on moving existent database 'testDB' located
in '/orientdb/databases/testDB' to
'/orientdb/databases/../backup/databases/testDB'. Try to move the
database directory manually and retry
[OHazelcastPlugin][node1444321499719] Error on starting distributed
plugin
com.orientechnologies.orient.server.distributed.ODistributedException:
Error on moving existent database 'testDB' located in
'/orientdb/databases/testDB' to
'/orientdb/databases/../backup/databases/testDB'. Try to move the
database directory manually and retry
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.backupCurrentDatabase(OHazelcastPlugin.java:1007)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.requestDatabase(OHazelcastPlugin.java:954)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.installDatabase(OHazelcastPlugin.java:893)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.installNewDatabases(OHazelcastPlugin.java:1426)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.startup(OHazelcastPlugin.java:184)
at com.orientechnologies.orient.server.OServer.registerPlugins(OServer.java:979)
at com.orientechnologies.orient.server.OServer.activate(OServer.java:346)
at com.orientechnologies.orient.server.OServerMain.main(OServerMain.java:41)
I don't have this error if I'm starting orientdb cluster without docker.
Also I can move it in container
[root#64f6cc1eba61 orientdb]# mv -v /orientdb/databases/testDB
/orientdb/databases/../backup/databases/testDB
'/orientdb/databases/testDB' ->
'/orientdb/databases/../backup/databases/testDB'
'/orientdb/databases/testDB/distributed-config.json' ->
'/orientdb/databases/../backup/databases/testDB/distributed-config.json'
removed '/orientdb/databases/testDB/distributed-config.json' removed
directory: '/orientdb/databases/testDB' [root#64f6cc1eba61 orientdb]#
ls -l /orientdb/databases/../backup/databases/testDB total 4
-rw-r--r--. 1 root root 455 Oct 9 11:32 distributed-config.json [root#64f6cc1eba61 orientdb]#
I'm using OrientDB version 2.1.3
This was reported and fixed:
https://github.com/orientechnologies/orientdb/issues/4891
Set the 'distributed.backupDirectory' variable to a specific directory and the issue should be gone.
By the way, running orient distributed in docker is our experience currently a no go:
- Docker does not support multicast yet, you can work around it, but it's painful. But the main problem:
- Docker doesn't reuse ip addresses on restart, so a container restart will give it a new ip address, this messes up your cluster big time.
We abandoned using orient distributed with docker until docker is fixed on both issues (I believe it is both on the roadmap).
If you experience otherwise, I'm happy to hear your thoughts.

how to kill uWSGI process

So I have finally gotten nginx + uWSGI running successfully for my Django install
however the problem I am having now is when I make changes to the code I need to restart the uWSGI process to view my changes
I feel like I am running the correct command here (i am very new to linux as well btw):
uwsgi --stop /var/run/uwsgi.pid
uwsgi --reload /var/run/uwsgi.pid
I get no error when I run these commands however my old code is still what loads
I also know its not a coding issue because I ran my django app in its development server and everything ran fine
The recommended way to signal reloading of application data is to use the --touch-reload option. Sample syntax on a .ini fine is:
touch-reload /var/run/uwsgi/app/myapp/reload
Where myappis your application name. /var/run/uwsgi/app is the recommended place for such files (could be anywhere). The reload file is an empty file whose timestamp is watched by uwsgi, whenever it changes (by, for example, using touch) uWSGI detects that change and restarts the corresponding uWSGI application instance.
So, whenever you update your code you should touch the file in order to update the in-memory version of the application. For example, on bash:
sudo touch /var/run/uwsgi/app/myapp/reload
Note --reload is an undocumented option on current uWSGI version.

Resources