Oauth2 prometheus config - oauth-2.0

Has anyone set up and used oauth2 on prometheus config in order to integrate SSO with keycloak? I tried the configuration below and I couldn't make it work.
prometheus.yml
oauth2:
client_id: "dsdsf-sdfssd-vfdsd-23sdfs"
client_secret: "sl;k96,;di0l;skdf;'lsgjs;';fks"
scopes : "https://keycloak/oauth2/authorize"
auth_url : "https://keycloak/oauth2/authorize"
token_url : "https://keycloak/oauth2/token"
api_url : "https://keycloak/oauth2/userinfo"
The prometheus service is not starting after adding the above oauth2 lines.
[root#root]# systemctl status prometheus
● prometheus.service - Prometheus Server
Loaded: loaded (/usr/lib/systemd/system/prometheus.service; enabled; vendor preset: disabled)
Active: deactivating (stop-sigterm) (Result: exit-code) since Thu 2022-02-03 15:45:39 CET; 5s ago
Process: 21596 ExecStart=/opt/prometheus-2.30.3.linux-amd64/prometheus --config.file /opt/prometheus/prometheus.yml --storage.tsdb.path=/data/prometheus --web.console.templates=/opt/prometheus/consoles --web.console.libraries=/opt/prometheus/console_libraries --storage.tsdb.min-block-duration=1h --storage.tsdb.max-block-duration=12h --storage.tsdb.retention.time=62d --log.level=debug --web.route-prefix=/prometheus --web.external-url=https://example.com/prometheus (code=exited, status=2)
Main PID: 21596 (code=exited, status=2)
CGroup: /system.slice/prometheus.service
└─21182 /opt/prometheus-2.30.3.linux-amd64/prometheus --config.file /opt/prometheus/prometheus.yml

Related

Although Docker is running, daemon cannot be connected

System: Ubuntu 20.02:
Whe I run systemctl status docker, I got:
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-10-20 14:52:17 JST; 26s ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 811329 (dockerd)
Tasks: 40
Memory: 26.7M
CGroup: /system.slice/docker.service
└─811329 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
but when I run docker container list, I got:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
What is the matter?

Docker won't startup

If I try to start docker server like so:
systemctl start docker
I get:
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
Running:
systemctl status docker.service
Shows:
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor p>
Active: failed (Result: exit-code) since Thu 2020-06-04 03:25:10 EDT; 36s >
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Process: 84866 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=1/F>
Main PID: 84866 (code=exited, status=1/FAILURE)
lines 1-7/7 (END)...skipping...
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2020-06-04 03:25:10 EDT; 36s ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Process: 84866 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=1/FAILURE)
Main PID: 84866 (code=exited, status=1/FAILURE)
I'm on Solus linux and installed docker normally like sudo eopkg install docker. How should I go about trying to get it to work ?
This seems to have been a Solus linux specific issue due to an update. I fixed it by running sudo rm -rf /var/lib/docker/overlay as suggested here (the issue is tracked here).

Cannot start memcached

I cannot get memcached to run on my server.
This is what I tried so far:
% sudo systemctl start memcached # no output
% sudo systemctl status memcached.service
● memcached.service - memcached daemon
Loaded: loaded (/lib/systemd/system/memcached.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2020-02-16 17:45:09 CET; 4s ago
Process: 22725 ExecStart=/usr/share/memcached/scripts/systemd-memcached-wrapper /etc/memcached.conf (code=exited, status=71)
Main PID: 22725 (code=exited, status=71)
systemd[1]: Started memcached daemon.
systemd-memcached-wrapper[22725]: bind(): Cannot assign requested address
systemd-memcached-wrapper[22725]: failed to listen on TCP port 11211: Cannot assign requested address
systemd[1]: memcached.service: Main process exited, code=exited, status=71/n/a
systemd[1]: memcached.service: Unit entered failed state.
systemd[1]: memcached.service: Failed with result 'exit-code'.
I am running Ubuntu 16.04.6 LTS
How can I start my memcached service?
Have a look into /etc/memcached.conf there might be written sth. like
-l xxx.xx.xx.xx
If you are trying to connect via localhost: just comment the line.
If you are trying to connect from somewhere else check the IP for correctness.

Does dockerd support WatchdogSec sd_notify health checks?

We've been having issues where the Docker daemon will occasionally stop responding on one of our Kubernetes systems, but Systemd still thinks the service is running:
systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2019-04-15 20:40:57 UTC; 3 months 22 days ago
Docs: https://docs.docker.com
Main PID: 1281 (dockerd)
Tasks: 1409
Memory: 31.0G
CPU: 5d 17h 3min 4.758s
CGroup: /system.slice/docker.service
├─ 1281 /usr/bin/dockerd -H fd://
...
There isn't anything in the journalctl -u docker or syslog files to indicate what the issue is, but the Docker daemon no longer responds to requests (docker ps just hangs). We are currently using the 17.03.2~ce-0~ubuntu-xenial package for Ubuntu 16.04, which has the following service unit:
cat /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket firewalld.service
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I noticed that even though it is a Type=notify service, there isn't a WatchdogSec= defined in the service unit.
Does the Docker daemon support setting a watchdog timeout for sd_notify based health checks?
No, currently the components/engine/cmd/dockerd/daemon_linux.go file only implements systemdDaemon.SdNotifyReady to notify Systemd when the process has started. For watchdog support it would have to use something like SdWatchdogEnabled to continually send SdNotifyWatchdog = "WATCHDOG=1" notifications.
If you try and set WatchdogSec=60s on the docker.service file it will kill and restart the service because the daemon doesn't send the required notifications.
systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2019-08-08 02:09:52 UTC; 50s ago
systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: deactivating (stop-sigabrt) (Result: watchdog) since Thu 2019-08-08 02:10:02 UTC; 45ms ago
systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: activating (start) since Thu 2019-08-08 02:10:04 UTC; 777ms ago
# Log entries:
Aug 08 02:09:14 kam1 systemd[1]: Starting Docker Application Container Engine...
Aug 08 02:09:15 kam1 systemd[1]: Started Docker Application Container Engine.
Aug 08 02:10:15 kam1 systemd[1]: docker.service: Watchdog timeout (limit 60s)!
Aug 08 02:10:15 kam1 systemd[1]: docker.service: Killing process 12383 (dockerd) with signal SIGABRT.
Aug 08 02:10:16 kam1 systemd[1]: docker.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 08 02:10:16 kam1 systemd[1]: docker.service: Failed with result 'watchdog'.
Aug 08 02:10:18 kam1 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Aug 08 02:10:18 kam1 systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Aug 08 02:10:18 kam1 systemd[1]: Stopped Docker Application Container Engine.
Aug 08 02:10:18 kam1 systemd[1]: Starting Docker Application Container Engine...

connecting to neo4j via py2neo on digitalocean error

I have a droplet on digital ocean and have neo4j installed and running, the results of service neo4j status is:
Loaded: loaded (/etc/init.d/neo4j; bad; vendor preset: enabled)
Active: active (exited) since Wed 2017-04-19 10:35:43 UTC; 21h ago
Docs: man:systemd-sysv-generator(8)
Process: 725 ExecStop=/etc/init.d/neo4j stop (code=exited, status=0/SUCCESS)
Process: 806 ExecStart=/etc/init.d/neo4j start (code=exited, status=0/SUCCESS)
Tasks: 0
Memory: 0B
CPU: 0
But when I run the pyhthon code I get:
py2neo.packages.httpstream.http.SocketError: Connection refused
I have authentication disabled.
In the graph all I explicitly referred to the db:
G=Graph('http://localhost:7474/db/data')
which results in the following error:
File "/usr/local/lib/python3.5/dist-packages/py2neo/packages/neo4j/v1/bolt.py", line 156, in _recv
raise ProtocolError("Server closed connection")
py2neo.packages.neo4j.v1.exceptions.ProtocolError: Server closed connection

Resources