After restoring from server backup, Jenkins stopped working - devops

in the logs the following information
The command journalctl -xe issues failrul
-- Support: http://www.ubuntu.com/support
--
-- Automatic restarting of the unit jenkins.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
Dec 05 04:11:12 vwkdenicdn systemd[1]: Stopped Jenkins Continuous Integration Server.
-- Subject: A stop job for unit jenkins.service has finished
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A stop job for unit jenkins.service has finished.
--
-- The job identifier is 10799 and the job result is done.
Dec 05 04:11:12 vwkdenicdn systemd[1]: jenkins.service: Start request repeated too quickly.
Dec 05 04:11:12 vwkdenicdn systemd[1]: jenkins.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- The unit jenkins.service has entered the 'failed' state with result 'exit-code'.
Dec 05 04:11:12 vwkdenicdn systemd[1]: Failed to start Jenkins Continuous Integration Server.
-- Subject: A start job for unit jenkins.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit jenkins.service has finished with a failure.
--
-- The job identifier is 10799 and the job result is failed.
and
systemctl status jenkins.service
● jenkins.service - Jenkins Continuous Integration Server
Loaded: loaded (/lib/systemd/system/jenkins.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2022-12-05 04:11:12 UTC; 1min 41s ago
Process: 137379 ExecStart=/usr/bin/jenkins (code=exited, status=1/FAILURE)
Main PID: 137379 (code=exited, status=1/FAILURE)
Dec 05 04:11:12 vwkdenicdn systemd[1]: jenkins.service: Scheduled restart job, restart counter is at 5.
Dec 05 04:11:12 vwkdenicdn systemd[1]: Stopped Jenkins Continuous Integration Server.
Dec 05 04:11:12 vwkdenicdn systemd[1]: jenkins.service: Start request repeated too quickly.
Dec 05 04:11:12 vwkdenicdn systemd[1]: jenkins.service: Failed with result 'exit-code'.
Dec 05 04:11:12 vwkdenicdn systemd[1]: Failed to start Jenkins Continuous Integration Server.
Tried to update Jenkins as after the same error the first time it worked
and I also tried changing the Jenkins config and setting the version:
openjdk version "1.8.0_352"
OpenJDK Runtime Environment (build 1.8.0_352-8u352-ga-1~20.04-b08)
OpenJDK 64-Bit Server VM (build 25.352-b08, mixed mode)

the problem was that the new version of django does not support version 8 of java
this command caused a problem when starting jenkins:
journalctl -f --unit=jenkins
next, I'm just install java11:
sudo apt install openjdk-11-jdk
and change default java,
sudo update-alternatives --config java
after this command choice java version by number
There are 2 choices for the alternative java (providing /usr/bin/java).
Selection Path Priority Status
0 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1111 auto mode
1 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1111 manual mode
2 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java 1081 manual mode
Press to keep the current choice[*], or type selection number: 0
next, start jenkins
systemctl start jenkins.service

Related

Jupyterhub (TLJH): Failure setting up Development Environment: "Failed to start jupyterhub.service."

since last week, I am unable to set up a TLJH Development Environment as described here:
https://tljh.jupyter.org/en/latest/contributing/dev-setup.html.
Up until January 23 2023 the process went smoothly, but after rebuilding the image last Friday (27 January), after running bootstrap.py in the container, the server doesn't spawn. Browsing to my localhost:12000 I just see the message "404 page not found".
I am running TLJH in a container in an Ubuntu 22.04 host, running Docker version 20.10.21.
On the same host, when using the image I build before January 23 with the same method I can spawn the server ok. There were no new commits on the TLJH repo since, so the issue should not be there.
Here is the output of journalctl -u jupyterhub -f ran inside the container:
-- Logs begin at Mon 2023-01-30 12:44:34 UTC. --
Jan 30 12:44:41 87f857896c91 python3[64]: raise exc.ArgumentError(msg, code=code) from err
Jan 30 12:44:41 87f857896c91 python3[64]: sqlalchemy.exc.ArgumentError: Column expression, FROM clause, or other columns clause element expected, got [1]. Did you mean to say select(1)?
Jan 30 12:44:41 87f857896c91 python3[64]:
Jan 30 12:44:41 87f857896c91 systemd[1]: jupyterhub.service: Main process exited, code=exited, status=1/FAILURE
Jan 30 12:44:41 87f857896c91 systemd[1]: jupyterhub.service: Failed with result 'exit-code'.
Jan 30 12:44:41 87f857896c91 systemd[1]: jupyterhub.service: Scheduled restart job, restart counter is at 5.
Jan 30 12:44:41 87f857896c91 systemd[1]: Stopped jupyterhub.service.
Jan 30 12:44:41 87f857896c91 systemd[1]: jupyterhub.service: Start request repeated too quickly.
Jan 30 12:44:41 87f857896c91 systemd[1]: jupyterhub.service: Failed with result 'exit-code'.
Jan 30 12:44:41 87f857896c91 systemd[1]: Failed to start jupyterhub.service.
Since the logs show an error in sqlalchemy, I tried dowongrading sqlalchemy from 2.0.0 to 1.4.46, and then reloaded the hub, but that didn't solve the problem
Is anyone else experiancing the same issue? Any ideas for further troubleshooting?
Thanks!
EDIT: I applied the solution described here and it worked.
Commands to run:
pip3 install --upgrade 'SQLAlchemy<2.0.0'
tljh-config reload hub
I applied the solution described here and it worked.
The reason seems to be an issue introduced in SQLAlchemy 2.0.0, is resolved by downgrading SQLAlchemy.
Commands to run:
pip3 install --upgrade 'SQLAlchemy<2.0.0'
tljh-config reload hub

Docker install on armv7

I am having trouble with my docker installation.
I rent a arm storage server for my setup.
The kernel is
4.5.2-armada375 #1 SMP Tue Oct 25 11:52:56 CEST 2016 armv7l
But when i try to insall docker-ce and the other packages ive got the following error:
Jul 09 20:54:50 storage-fr systemd[1]: Failed to start Docker Application Container Engine.
Jul 09 20:54:50 storage-fr systemd[1]: docker.service: Unit entered failed state.
Jul 09 20:54:50 storage-fr systemd[1]: docker.service: Failed with result 'exit-code'.
dpkg: error processing package docker-ce (--configure):
subprocess installed post-installation script returned error exit status 1
Setting up liberror-perl (0.17024-1) ...
Setting up rename (0.20-4) ...
update-alternatives: using /usr/bin/file-rename to provide /usr/bin/rename (rename) in auto mode
Setting up aufs-dkms (4.9+20161219-1) ...
Loading new aufs-4.9+20161219 DKMS files...
It is likely that 4.5.2-armada375 belongs to a chroot's host
I think the kernel is not compatible.
IS it possible to install docker on it?
I tried this guide https://docs.docker.com/install/linux/docker-ce/debian/
But it doesnt work.
Any idea ?

Getting "neo4j.service: Failed with result 'start-limit-hit'." on installation of Neo4j Server 3.5.7

Trying to upgrade local Neo4j instances and getting errors on start.
Ubuntu 16.04: Trying to upgrade local instances of Neo4j databases which are currently at 3.3.1.
Installed Neo4j Desktop thinking I could do it with that and found the lowest version it upgrades from is 3.4. I now can't remove that. Ubuntu software fails (I just click remove, restart and Neo4j Desktop is still there). dpkg --list doesn't list neo4j desktop, so I can't use "apt-get remove" to remove it.
Uninstalled 3.3.1 and installed 3.3.9 (latest version of 3.3.x). Started Neo4j and ran fine, updating the database stores to 3.3.9.
Uninstalled 3.3.9 and installed 3.5.7.
Expecting Neo4j to start normally with "sudo service neo4j start", but now getting the following:
neo4j.service - Neo4j Graph Database
Loaded: loaded (/lib/systemd/system/neo4j.service; disabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Tue 2019-07-09 14:00:22 BST; 58s ago
Process: 1417 ExecStart=/usr/share/neo4j/bin/neo4j console (code=exited, status=1/FAILURE)
Main PID: 1417 (code=exited, status=1/FAILURE)
Jul 09 14:00:22 doug-ubuntu systemd[1]: neo4j.service: Main process exited, code=exited, status=1/FAILURE
Jul 09 14:00:22 doug-ubuntu systemd[1]: neo4j.service: Unit entered failed state.
Jul 09 14:00:22 doug-ubuntu systemd[1]: neo4j.service: Failed with result 'exit-code'.
Jul 09 14:00:22 doug-ubuntu systemd[1]: neo4j.service: Service hold-off time over, scheduling restart.
Jul 09 14:00:22 doug-ubuntu systemd[1]: Stopped Neo4j Graph Database.
Jul 09 14:00:22 doug-ubuntu systemd[1]: neo4j.service: Start request repeated too quickly.
Jul 09 14:00:22 doug-ubuntu systemd[1]: Failed to start Neo4j Graph Database.
Jul 09 14:00:22 doug-ubuntu systemd[1]: neo4j.service: Unit entered failed state.
Jul 09 14:00:22 doug-ubuntu systemd[1]: neo4j.service: Failed with result 'start-limit-hit'.
Checked that I am running JVE 1.8:
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (build 1.8.0_212-8u212-b03-0ubuntu1.16.04.1-b03)
OpenJDK 64-Bit Server VM (build 25.212-b03, mixed mode)
Unsure how to proceed. Any ideas welcome please. Thank you!
Found the answer by running "journalctl -e -u neo4j" to view the full error log. Turns out that it was an APOC jar that was installed for 3.3.x but not compatible with 3.5.x. Removing the jar file solved it.

Run timer exactly every second with systemd

How to run some service exactly every second? Overlapping allowed.
Here is my timer unit for systemd:
[Unit]
Description=Send jobs every second
[Timer]
OnBootSec=1min
OnUnitActiveSec=1s
Unit=app.jobs-send.service
[Install]
WantedBy=multi-user.target
But app.jobs-send.service service doesn't executed every second:
May 07 17:57:38 app.deployer.org systemd[1]: Started Send jobs to queue.
May 07 17:57:48 app.deployer.org systemd[1]: Starting Send jobs to queue...
May 07 17:57:48 app.deployer.org systemd[1]: Started Send jobs to queue.
May 07 17:57:58 app.deployer.org systemd[1]: Starting Send jobs to queue...
May 07 17:57:58 app.deployer.org systemd[1]: Started Send jobs to queue.
May 07 17:58:08 app.deployer.org systemd[1]: Starting Send jobs to queue...
May 07 17:58:08 app.deployer.org systemd[1]: Started Send jobs to queue.
There is a gap about 17:57:58 - 17:57:48 = 10 seconds.
How to exec task every second?
As documented in man systemd.time, the accuracy of OnUnitActiveSec= is subject to the AccuracySec= setting, also documented there. Set to AccuracySec=1us for best resolution.

neo4j 3.0 (community edition) - unable to start server on debian 8

I tried neo4j 3.0 on two machines.
Had a working neo4j 2.x and upgraded to neo4j 3.0 through apt-get upgrade
Fresh installation of neo4j 3.0
In both cases, neo4j does not start and I see the following error message
[console]
sudo service neo4j start
Job for neo4j.service failed. See 'systemctl status neo4j.service' and 'journalctl -xn' for details.
$ sudo systemctl status neo4j.service -l
● neo4j.service - LSB: Neo4j Graph Database server
Loaded: loaded (/etc/init.d/neo4j)
Active: failed (Result: exit-code) since Thu 2016-05-05 18:24:14 UTC; 8min ago
Process: 1377 ExecStart=/etc/init.d/neo4j start (code=exited, status=1/FAILURE)
May 05 18:24:14 abc-com neo4j[1377]: Starting Neo4j.
May 05 18:24:14 abc-com neo4j[1377]: WARNING: Max 1024 open files allowed, minimum of 40000 recommended. See the Neo4j manual.
May 05 18:24:14 abc-com neo4j[1377]: /usr/share/neo4j/bin/neo4j: line 149: /var/run/neo4j/neo4j.pid: No such file or directory
May 05 18:24:14 abc-com systemd[1]: neo4j.service: control process exited, code=exited status=1
May 05 18:24:14 abc-com systemd[1]: Failed to start LSB: Neo4j Graph Database server.
May 05 18:24:14 abc-com systemd[1]: Unit neo4j.service entered failed state.
cat /var/log/neo4j/neo4j.log
Caused by: org.neo4j.kernel.StoreLockException: Store and its lock file has been locked by another process: /var/lib/neo4j/data/databases/graph.db/store_lock. Please ensure no other process is using this database, and that the directory is writable (required even for read-only access)
at org.neo4j.kernel.internal.StoreLocker.storeLockException(StoreLocker.java:90)
at org.neo4j.kernel.internal.StoreLocker.checkLock(StoreLocker.java:76)
at org.neo4j.kernel.internal.StoreLockerLifecycleAdapter.start(StoreLockerLifecycleAdapter.java:40)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
... 12 more

Resources