How to add accounts to Jenkins without the web interface? - jenkins

I want to automate the entire installation of Jenkins, given a list of user names I want to be able to create user accounts for each. The only method I've read to set up user accounts is here:
https://wiki.jenkins-ci.org/display/JENKINS/Standard+Security+Setup
Tried seeing if there was an option to configure with command line at:
https://localhost:8080/cli/
But does not seem to the be case.
Is it possible to add user accounts without using the web interface? More specifically a method that is scriptable.
My last resort is to do raw post requests but hoping there is a nicer way.

Yes of course, it is possible to script provisioning for jenkins. But not with the cli tool alone.
I guess you want to use "Jenkins own user database" with Project Matrix Authorization Strategy.
Steps to prepare provisioning:
Configure your Jenkins manually (enable security, add rolls and at least one user)
Shutdown your jenkins (to let him write all in-memory changes to disk)
Copy $JENKINS_HOME/config.xml to your provisioning script (as as seed data)
Copy $JENKINS_HOME/users/ (as seed data)
Get the cli tool: cd /tmp; wget -nv http://localhost:8080/jnlpJars/jenkins-cli.jar
If you do not want to have static seed data (one config.xml for each user) you can generate a (users/username/)config.xml using a bash script or a more advanced tool. But for simplicity sake you can take users/username1/config.xml as a template. Replace relevant data with a placeholder e.g. "PLACEHOLDER_FULLNAME" for full user name.
e.g.:
change
"<fullName>sample full username</fullName>"
to
"<fullName>PLACEHOLDER_FULLNAME</fullName>"
In your provisioning script, iterate over all users. For each user, replace each placeholder with the correct value.
e.g.
cp $SEED_DATA/templates/user/config.xml /tmp/config.xml
sed -e "s/\${PLACEHOLDER_USERNAME}/1/" -e "s/\${ChuckNorris}/dog/" /tmp/config.xml
sed -e "s/\${PLACEHOLDER_EMAIL}/1/" -e "s/\${he#findsyou.com}/dog/" /tmp/config.xml
...
mkdir -p $SEED_DATA/users/$USERNAME/
cp /tmp/config.xml $SEED_DATA/users/$USERNAME/config.xml
When you want to use generated users config.xml please generate for each user some permission settings in $JENKINS_HOME/config.xml:
<authorizationStrategy class="hudson.security.ProjectMatrixAuthorizationStrategy">
...
<permission>hudson.model.View.Create:username1</permission>
<permission>hudson.model.View.Delete:username1</permission>
<permission>hudson.model.View.Read:username1</permission>
...
<permission>hudson.model.View.Create:username2</permission>
<permission>hudson.model.View.Delete:username2</permission>
<permission>hudson.model.View.Read:username2</permission>
...
</authorizationStrategy>
Provisioning steps:
Install jenkins as you did before & maybe dynamic config generator (see above)
cp $SEED_DATA/config.xml $JENKINS_HOME/
cp -R $SEED_DATA/users/ $JENKINS_HOME/
chown -R "jenkins:jenkins" $JENKINS_HOME/users/ (maybe optional)
cd /tmp; java -jar jenkins-cli.jar -s http://localhost:8080/ reload-configuration

Related

How can I use a linux system variable in my Jenkinsfile?

I want to use a curl with a username and password that I set in the bashrc. ie:
curl -u $jenkuser:$jenkpass foobar.org
but this isn't working for me. So what is a good way to set secret credentials that I don't want in my repo/Jenkinsfile
Create a Jenkins project with Execute shell build step. In that shell you can run curl command and to set credentials, there is one option named This build is parameterized, where you can create Password Parameter. These parameters can be used in shell with curl command. Here is screenshot of my test project.
This way is secure because password is stored in encrypted format.
.

Jenkins Build rpm sign error

I am trying to copy my entire Jenkins configuration from RHEL 6.7 to RHEL 6.9 , On doing this everything looks good, but only one jenkins build is failing with below error
Enter pass phrase:
can't connect to `/usr/share/tomcat6/.gnupg/S.gpg-agent': No such file or directory
gpg: skipped "Credit": Bad passphrase
gpg: signing failed: Bad passphrase
Pass phrase check failed
The gpg private key 1.4.5 exists in jenkins configuration. Strange thing is , all other builds are able to sign rpm but only one build is failing
Anyone know how to fix it ?
RPM reads the passphrase uses getpass(3) and sends to gnupg through an additional file descriptor.
This creates two problems that need to be handled by automating signing mechanisms:
1) Some versions of rpm use getpass(3) which will use a tty (to disable echoing) and will require setting up a pseudo tty so that the automated password can be passed to RPM. Make sure you have the pty file system mounted, and expect(1) is one way to setup the pty from which the password can be read. There's another approach using /proc file descriptors that can be attempted on linux. The password is then sent to gnupg using --passphrase-fd.
2) gnupg2 can also handle persistent passwords in a separate agent process which is sometimes tricky to setup and keep running "automatically" because the detection depends on both the user/process id's. Your report seems to have an agent (which means gnupg2 or special gpg1 configuration) even though you mention 1.4.5 (which would seem to use gnupg1).
I see two separate issues in your log that need to be addressed.
can't connect to `/usr/share/tomcat6/.gnupg/S.gpg-agent': No such file
or directory
gpg-agent needs to be running as a daemon on the build host, where it will connect to a socket to listen for requests. Perhaps it is already running, but Jenkins is looking for its socket in the wrong directory because GNUPGHOME is set to some unusual value. Or perhaps gpg-agent isn't running and a new instance needs to be started.
Something like this script can be used to safely attach to an existing gpg-agent or spin up a new instance.
#!/bin/bash
# Decide whether to start gpg-agent daemon.
# Create necessary symbolic link in $GNUPGHOME/.gnupg/S.gpg-agent
SOCKET=S.gpg-agent
PIDOF=`pgrep gpg-agent`
RETVAL=$?
if [ "$RETVAL" -eq 1 ]; then
echo "Starting gpg-agent daemon."
eval `gpg-agent --daemon `
else
echo "Daemon gpg-agent already running."
fi
# Nasty way to find gpg-agent's socket file...
GPG_SOCKET_FILE=`find /tmp/gpg-* -name $SOCKET`
echo "Updating socket file link."
cp -fs $GPG_SOCKET_FILE $GNUPGHOME/.gnupg/S.gpg-agent
You may want to substitute pgrep for pidof, depending on your shell.
If you do end up starting a new agent, you can check to see that your keys have been loaded into it by running gpg --list-keys. If you don't see it listed, you'll need to add it using gpg --import. Follow the Jenkins docs for Using Credentials.
Resolving the gpg-agent issue may resolve your other issue, so check to see if your job is working before doing anything else.
References:
www.linuxquestions.org
gpg: skipped "Credit": Bad passphrase
The GPG key is protected by a passphrase. rpm is asking for this passphrase and expects it to be manually entered. Of course, Jenkins is running things non-interactively, so that's not going to be possible. We need some way to supply the passphrase to rpm so it can forward it along to gpg, or else we need to supply the passphrase to gpg directly via some sort of caching mechanism.
The Expect Method
By wrapping our rpm --addsign call in an expect script, we can use expect to enter the passphrase headlessly. This practice is fairly common. Assuming the following script named rpm_sign.exp:
#!/usr/bin/expect -f
set password [lindex $argv 0]
set files [lrange $argv 1 1]
spawn rpm --define --addsign $files
expect "Enter pass phrase:"
send -- "$password\r"
expect eof
This script can be used in a Jenkins shell step or pipeline as follows:
echo "Signing rpms ..."
sh "./rpm_sign.exp '${GPG_PASSPHRASE}' <list-of-files>"
Please note that, with some modifications, it is possible to specify which GPG identity you want to sign your RPMs with. This is done by passing --define {_gpg_name $YOUR_KEY_ID_HERE} as an argument to rpm inside the wrapper script. Note the TCL syntax. Since we're doing this on Jenkins, which may offer multiple sets of credentials, I assume this is relevant info.
References:
aaronhawley.livejournal.com
lists.fedoraproject.org
Other Methods
There are other solutions out there that may be more appropriate to your configuration. One such solution is to use RpmSignPlugin, which uses expect under the hood. Other solutions can be found in this posting on unix.stackexchange.com.

How to export credentials from one jenkins instance to another?

I am using the credentials plugin in Jenkins to manage credentials for git and database access for my team's builds. I would like to copy the credentials from one jenkins instance to another, independent jenkins instance. How would I go about doing this?
UPDATE: TL;DR Follow the link provided below in a comment by Filip Stachowiak it is the easiest way to do it. In case it doesn't work for you go on reading.
Copying the $HUDSON_HOME/credentials.xml is not the solution because Jenkins encrypts paswords and these can't be decrypted by another instance unless both share a common key.
So, either you use the same encription keys in both Jenkins instances (Where's the encryption key stored in Jenkins? ) or what you can do is:
Create the same user/password, you need to share, in the 2nd Jenkins instance so that a valid password is generated
What is really important is that user ids in both credentials.xml are the same. For that (see the credentials.xml example below) for user: Jenkins the identifier <id>c4855f57-5107-4b69-97fd-298e56a9977d</id> must be the same in both credentials.xml
<com.cloudbees.plugins.credentials.SystemCredentialsProvider plugin="credentials#1.22">
<domainCredentialsMap class="hudson.util.CopyOnWriteMap$Hash">
<entry>
<com.cloudbees.plugins.credentials.domains.Domain>
<specifications/>
</com.cloudbees.plugins.credentials.domains.Domain>
<java.util.concurrent.CopyOnWriteArrayList>
<com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl>
<scope>GLOBAL</scope>
<id>c4855f57-5107-4b69-97fd-298e56a9977d</id>
<description>Para SVN</description>
<username>jenkins</username>
<password>J1ztA2vSXHbm60k5PjLl5jg70ZooSFKF+kRAo08UVts=
</password>
</com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl>
</java.util.concurrent.CopyOnWriteArrayList>
</entry>
</domainCredentialsMap>
</com.cloudbees.plugins.credentials.SystemCredentialsProvider>
I was also facing the same problem. What worked for me is I copied the credentials.xml, config.xml and the secrets folder from existing jenkins to the new instance. After the restart of jenkins things worked fine.
This is what worked for me.
Create a job in Jenkins that takes the credentials and writes them to output. If Jenkins replaces the password in the output with ****, just obfuscate it first (add a space between each character, reverse the characters, base64 encode it, etc.)
I used a Powershell job to base64 encode it:
[convert]::ToBase64String([text.encoding]::Default.GetBytes($mysecret))
And then used Powershell to convert the base64 string back to a regular string:
[text.encoding]::Default.GetString([convert]::FromBase64String("bXlzZWNyZXQ="))
After trying quite a few things for several days this is the best solution I found for migrating my secrets from a Jenkins 2.176 to a new clean Jenkins 2.249.1 jenkins-cli was the best approach for me.
The process is quite simple just dump the credentials from the old instance to a local machine, or Docker pod with java installed, as a XML file (unencrypted) and then uploaded to the new instance.
Before starting you should verify the following:
Access to the credentials section on both Jenkins instances
Download the jenkins-ccli.jar from one of the instances (https://www.your-jenkins-url.com/cli/)
Have User and Password/Token at hand.
Notice: In case your jenkins uses an oAuth service you will need to
create a token for your user. Once logged into jenkins at the top
right if you click your profile you can verify both username and
generate password.
Now for the special sauce, you have to execute both parts from the same machine/pod:
Notice: If your instances are using valid Certificates and you want to
secure your connection you must remove the -noCertificateCheck
flag from both commands.
# OLD JENKINS DUMP # 
export USER=madox#example.com
export TOKEN=f561banana6ead83b587a4a8799c12c307
export SERVER=https://old-jenkins-url.com/
java -jar jenkins-cli.jar -noCertificateCheck -s $SERVER -auth $USER:$TOKEN list-credentials-as-xml "system::system::jenkins" > /tmp/jenkins_credentials.xml
# NEW JENKINS IMPORT # 
export USER=admin
export TOKEN=admin
export SERVER=https://new-jenkins-url.com/
java -jar jenkins-cli.jar -noCertificateCheck -s $SERVER -auth $USER:$TOKEN import-credentials-as-xml "system::system::jenkins" < /tmp/jenkins_credentials.xml
If you have the credentials.xml available and the old Jenkins instance still running, there is a way to decrypt individual credentials so you can enter them in the new Jenkins instance via the UI.
The approach is described over at the DevOps stackexchange by kenorb.
This does not convert all the credentials for an easy, automated migration, but helps when you have only few credentials to migrate (manually).
To summarize, you visit the /script page over at the old Jenkins instance, and use the encrypted credential from the credentials.xml file in the following line:
println(hudson.util.Secret.decrypt("{EncryptedCredentialFromCredentialsXml=}"))
To migrate all credentials to a new server, from Jenkins: Migrating credentials:
Stop Jenkins on new server.
new-server # /etc/init.d/jenkins stop
Remove the identity.key.enc file on new server:
new-server # rm identity.key.enc
Copy secret* and credentials.xml to new server.
current-server # cd /var/lib/jenkins
current-server # tar czvf /tmp/credentials.tgz secret* credentials.xml
current-server # scp credentials.tgz $user#$new-server:/tmp/
new-server # cd /var/lib/jenkins
new-server # tar xzvf /tmp/credentials.tgz -C ./
Start Jenkins.
new-server # /etc/init.d/jenkins start
Migrating users from a Jenkins instance to another Jenkins on a new server -
I tried following https://stackoverflow.com/a/35603191 which lead to https://itsecureadmin.com/2018/03/26/jenkins-migrating-credentials/. However I did not succeed in following these steps.
Further, I experimented exporting /var/lib/jenkins/users (or {JENKINS_HOME}/users) directory to the new instance on new server. After restarting the Jenkins on new server - it looks like all the user credentials are available on new server.
Additionally, I cross-checked if the users can log in to the new Jenkins instance. It works for now.
PS: This code is for redhat servers
Old server:
cd /var/lib/jeknins
or cd into wherever your Jenkins home is
tar cvzf users.tgz ./users
New server:
cd /var/lib/jeknins
scp <user>#<oldserver>:/var/lib/jenkins/user.tgz ~/var/lib/jenkins/.
sudo tar xvzf users.tgz
systemctl restart jenkins
Did you try to copy the $JENKINS_HOME/users folder and the $JENKINS_HOME/credentials.xml file to the other Jenkins instance?

How to have 2 Neo4j DB's on same computer: dev and test

I'm using Neo4j with Node.js to build a REST API.
I'd like to write some tests for this API. How do I use a "test database" during those tests?
With MySQL or MongoDB, I'd fiddle the resource URL to use a different database like "app-test" vs "app".
What's the smart way to do that in neo4j?
Thanks!
SOLUTION: This is what I did:
Made a directory db/test. In that dir, put:
two bash scripts below
a test.zip with backup of the database you want
install.sh
#!/bin/bash
VERSION=neo4j-community-2.1.5
# Download a copy of the server
wget http://dist.neo4j.org/$VERSION-unix.tar.gz
# Unpack it here
tar -xvzf $VERSION-unix.tar.gz
# Change the default port to http->7475 https->7476
sed -i.bak s/7474/7475/g $VERSION/conf/neo4j-server.properties
sed -i.bak s/7473/7476/g $VERSION/conf/neo4j-server.properties
restart.sh
#!/bin/bash
VERSION=neo4j-community-2.1.5
echo === stop the server
$VERSION/bin/neo4j stop
echo --- replace the database
rm -rf $VERSION/data/graph.db
unzip -q test.zip -d $VERSION/data/graph.db
echo --- start the server
$VERSION/bin/neo4j start
Then added it to gruntfile using grunt-run:
run: {
restartTestDb: {
exec: 'cd db/test && ./restart.sh',
}
},
Works.
You can do the same thing with neo4j. Just install another copy of neo4j in a separate location, and configure it to use a different port.
After each test, you can delete the graph.db file to clear all the data.
I'm currently busy with a project that can manage and switch databases from the command line.
I'm busy with fixing the neo4j download feature, this is still in WIP but the more users, the more quickly stable it can be.
I should push on github within an hour or two.
EDIT: Repository here https://github.com/neoxygen/neo4j-toolkit
Live video here http://recordit.co/YRVhOJKXdj

RoR: Shell/System commands with write property not working in production mode?

System("ls")
System("pwd")
Both these commands just work fine in both production & development mode on the same server.
However System("mkdir test") or any other command that involves creating a new file/dir does not go through in production mode, but works just fine in development mode. Any ideas here?
My guess is it has something to do with permissions but not sure where.
On your server you should have a user different from root for security reasons. Than this user should be added to sudoers list:
https://askubuntu.com/questions/7477/how-can-i-add-a-new-user-as-sudoer-using-the-command-line
now, depending where you want to create this folder, if it's in your app folder where your user has permissions to read/write, (search chmod 755 and chown to set the owner of the folder, better use chown -R to apply this to all subfolders), after this you'll be able to create that folder with:
System("mkdir test")
but only in folders where your user has access to read/write.
If you want to create the test folder in some other path where you need to use sudo you'll have to run:
System("sudo mkdir test")
normally this is running in a background and you won't be there to write the password, so you'll have to add your command to not require password while running sudo, with NOPASSWD directive you can do that:
https://askubuntu.com/questions/159007/how-do-i-run-specific-sudo-commands-without-a-password
sudo visudo -f /etc/sudoers #!important visudo, read in the upper link more about it before trying this.
after doing all this you'll be able to create a folder in your path using:
System("sudo mkdir test")
without requiring a password.

Resources