How to export credentials from one jenkins instance to another? - jenkins

I am using the credentials plugin in Jenkins to manage credentials for git and database access for my team's builds. I would like to copy the credentials from one jenkins instance to another, independent jenkins instance. How would I go about doing this?

UPDATE: TL;DR Follow the link provided below in a comment by Filip Stachowiak it is the easiest way to do it. In case it doesn't work for you go on reading.
Copying the $HUDSON_HOME/credentials.xml is not the solution because Jenkins encrypts paswords and these can't be decrypted by another instance unless both share a common key.
So, either you use the same encription keys in both Jenkins instances (Where's the encryption key stored in Jenkins? ) or what you can do is:
Create the same user/password, you need to share, in the 2nd Jenkins instance so that a valid password is generated
What is really important is that user ids in both credentials.xml are the same. For that (see the credentials.xml example below) for user: Jenkins the identifier <id>c4855f57-5107-4b69-97fd-298e56a9977d</id> must be the same in both credentials.xml
<com.cloudbees.plugins.credentials.SystemCredentialsProvider plugin="credentials#1.22">
<domainCredentialsMap class="hudson.util.CopyOnWriteMap$Hash">
<entry>
<com.cloudbees.plugins.credentials.domains.Domain>
<specifications/>
</com.cloudbees.plugins.credentials.domains.Domain>
<java.util.concurrent.CopyOnWriteArrayList>
<com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl>
<scope>GLOBAL</scope>
<id>c4855f57-5107-4b69-97fd-298e56a9977d</id>
<description>Para SVN</description>
<username>jenkins</username>
<password>J1ztA2vSXHbm60k5PjLl5jg70ZooSFKF+kRAo08UVts=
</password>
</com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl>
</java.util.concurrent.CopyOnWriteArrayList>
</entry>
</domainCredentialsMap>
</com.cloudbees.plugins.credentials.SystemCredentialsProvider>

I was also facing the same problem. What worked for me is I copied the credentials.xml, config.xml and the secrets folder from existing jenkins to the new instance. After the restart of jenkins things worked fine.

This is what worked for me.
Create a job in Jenkins that takes the credentials and writes them to output. If Jenkins replaces the password in the output with ****, just obfuscate it first (add a space between each character, reverse the characters, base64 encode it, etc.)
I used a Powershell job to base64 encode it:
[convert]::ToBase64String([text.encoding]::Default.GetBytes($mysecret))
And then used Powershell to convert the base64 string back to a regular string:
[text.encoding]::Default.GetString([convert]::FromBase64String("bXlzZWNyZXQ="))

After trying quite a few things for several days this is the best solution I found for migrating my secrets from a Jenkins 2.176 to a new clean Jenkins 2.249.1 jenkins-cli was the best approach for me.
The process is quite simple just dump the credentials from the old instance to a local machine, or Docker pod with java installed, as a XML file (unencrypted) and then uploaded to the new instance.
Before starting you should verify the following:
Access to the credentials section on both Jenkins instances
Download the jenkins-ccli.jar from one of the instances (https://www.your-jenkins-url.com/cli/)
Have User and Password/Token at hand.
Notice: In case your jenkins uses an oAuth service you will need to
create a token for your user. Once logged into jenkins at the top
right if you click your profile you can verify both username and
generate password.
Now for the special sauce, you have to execute both parts from the same machine/pod:
Notice: If your instances are using valid Certificates and you want to
secure your connection you must remove the -noCertificateCheck
flag from both commands.
# OLD JENKINS DUMP # 
export USER=madox#example.com
export TOKEN=f561banana6ead83b587a4a8799c12c307
export SERVER=https://old-jenkins-url.com/
java -jar jenkins-cli.jar -noCertificateCheck -s $SERVER -auth $USER:$TOKEN list-credentials-as-xml "system::system::jenkins" > /tmp/jenkins_credentials.xml
# NEW JENKINS IMPORT # 
export USER=admin
export TOKEN=admin
export SERVER=https://new-jenkins-url.com/
java -jar jenkins-cli.jar -noCertificateCheck -s $SERVER -auth $USER:$TOKEN import-credentials-as-xml "system::system::jenkins" < /tmp/jenkins_credentials.xml

If you have the credentials.xml available and the old Jenkins instance still running, there is a way to decrypt individual credentials so you can enter them in the new Jenkins instance via the UI.
The approach is described over at the DevOps stackexchange by kenorb.
This does not convert all the credentials for an easy, automated migration, but helps when you have only few credentials to migrate (manually).
To summarize, you visit the /script page over at the old Jenkins instance, and use the encrypted credential from the credentials.xml file in the following line:
println(hudson.util.Secret.decrypt("{EncryptedCredentialFromCredentialsXml=}"))

To migrate all credentials to a new server, from Jenkins: Migrating credentials:
Stop Jenkins on new server.
new-server # /etc/init.d/jenkins stop
Remove the identity.key.enc file on new server:
new-server # rm identity.key.enc
Copy secret* and credentials.xml to new server.
current-server # cd /var/lib/jenkins
current-server # tar czvf /tmp/credentials.tgz secret* credentials.xml
current-server # scp credentials.tgz $user#$new-server:/tmp/
new-server # cd /var/lib/jenkins
new-server # tar xzvf /tmp/credentials.tgz -C ./
Start Jenkins.
new-server # /etc/init.d/jenkins start

Migrating users from a Jenkins instance to another Jenkins on a new server -
I tried following https://stackoverflow.com/a/35603191 which lead to https://itsecureadmin.com/2018/03/26/jenkins-migrating-credentials/. However I did not succeed in following these steps.
Further, I experimented exporting /var/lib/jenkins/users (or {JENKINS_HOME}/users) directory to the new instance on new server. After restarting the Jenkins on new server - it looks like all the user credentials are available on new server.
Additionally, I cross-checked if the users can log in to the new Jenkins instance. It works for now.
PS: This code is for redhat servers
Old server:
cd /var/lib/jeknins
or cd into wherever your Jenkins home is
tar cvzf users.tgz ./users
New server:
cd /var/lib/jeknins
scp <user>#<oldserver>:/var/lib/jenkins/user.tgz ~/var/lib/jenkins/.
sudo tar xvzf users.tgz
systemctl restart jenkins

Did you try to copy the $JENKINS_HOME/users folder and the $JENKINS_HOME/credentials.xml file to the other Jenkins instance?

Related

How to properly build a private Go project? [duplicate]

I'm searching for the way to get $ go get work with private repository, after many google try.
The first try:
$ go get -v gitlab.com/secmask/awserver-go
Fetching https://gitlab.com/secmask/awserver-go?go-get=1
https fetch failed.
Fetching http://gitlab.com/secmask/awserver-go?go-get=1
Parsing meta tags from http://gitlab.com/secmask/awserver-go?go-get=1 (status code 200)
import "gitlab.com/secmask/awserver-go": parse http://gitlab.com/secmask/awserver-go?go-get=1: no go-import meta tags
package gitlab.com/secmask/awserver-go: unrecognized import path "gitlab.com/secmask/awserver-go
Yep, it did not see the meta tags because I could not know how to provide login information.
The second try:
Follow https://gist.github.com/shurcooL/6927554. Add config to .gitconfig.
[url "ssh://git#gitlab.com/"]
insteadOf = https://gitlab.com/
$ go get -v gitlab.com/secmask/awserver-go --> not work
$ go get -v gitlab.com/secmask/awserver-go.git --> work but I got src/gitlab.com/secmask/awserer-go.git
Yes it work but with .git extension with my project name, I can rename it to original but do it everytime $ go get is not so good, is there an otherway?
You have one thing to configure. The example is based on GitHub but this shouldn't change the process:
$ git config --global url.git#github.com:.insteadOf https://github.com/
$ cat ~/.gitconfig
[url "git#github.com:"]
insteadOf = https://github.com/
$ go get github.com/private/repo
For Go modules to work (with Go 1.11 or newer), you'll also need to set the GOPRIVATE variable, to avoid using the public servers to fetch the code:
export GOPRIVATE=github.com/private/repo
The proper way is to manually put the repository in the right place. Once the repository is there, you can use go get -u to update the package and go install to install it. A package named
github.com/secmask/awserver-go
goes into
$GOPATH/src/github.com/secmask/awserver-go
The commands you type are:
cd $GOPATH/src/github.com/secmask
git clone git#github.com:secmask/awserver-go.git
I had a problem with go get using private repository on gitlab from our company.
I lost a few minutes trying to find a solution. And I did find this one:
You need to get a private token at:
https://gitlab.mycompany.com/profile/account
Configure you git to add extra header with your private token:
$ git config --global http.extraheader "PRIVATE-TOKEN: YOUR_PRIVATE_TOKEN"
Configure your git to convert requests from http to ssh:
$ git config --global url."git#gitlab.mycompany.com:".insteadOf "https://gitlab.mycompany.com/"
Finally you can use your go get normally:
$ go get gitlab.com/company/private_repo
For people using private GitLabs, here's a snippet that may help: https://gist.github.com/MicahParks/1ba2b19c39d1e5fccc3e892837b10e21
Also pasted below:
Problem
The go command line tool needs to be able to fetch dependencies from your private GitLab, but authenticaiton is required.
This assumes your private GitLab is hosted at privategitlab.company.com.
Environment variables
The following environment variables are recommended:
export GO111MODULE=on
export GOPRIVATE=privategitlab.company.com # this is comma delimited if using multiple private repos
The above lines might fit best in your shell startup, like a ~/.bashrc.
Explanation
GO111MODULE=on tells Golang command line tools you are using modules. I have not tested this with projects not using
Golang modules on a private GitLab.
GOPRIVATE=privategitlab.company.com tells Golang command line tools to not use public internet resources for the hostnames
listed (like the public module proxy).
Get a personal access token from your private GitLab
To future proof these instructions, please follow this guide from the GitLab docs.
I know that the read_api scope is required for Golang command line tools to work, and I may suspect read_repository as
well, but have not confirmed this.
Set up the ~/.netrc
In order for the Golang command line tools to authenticate to GitLab, a ~/.netrc file is best to use.
To create the file if it does not exist, run the following commands:
touch ~/.netrc
chmod 600 ~/.netrc
Now edit the contents of the file to match the following:
machine privategitlab.company.com login USERNAME_HERE password TOKEN_HERE
Where USERNAME_HERE is replaced with your GitLab username and TOKEN_HERE is replaced with the access token aquired in the
previous section.
Common mistakes
Do not set up a global git configuration with something along the lines of this:
git config --global url."git#privategitlab.company.com:".insteadOf "https://privategitlab.company.com"
I beleive at the time of writing this, the SSH git is not fully supported by Golang command line tools and this may cause
conflicts with the ~/.netrc.
Bonus: SSH config file
For regular use of the git tool, not the Golang command line tools, it's convient to have a ~/.ssh/config file set up.
In order to do this, run the following commands:
mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/config
chmod 600 ~/.ssh/config
Please note the permissions on the files and directory above are essentail for SSH to work in it's default configuration on
most Linux systems.
Then, edit the ~/.ssh/config file to match the following:
Host privategitlab.company.com
Hostname privategitlab.company.com
User USERNAME_HERE
IdentityFile ~/.ssh/id_rsa
Please note the spacing in the above file matters and will invalidate the file if it is incorrect.
Where USERNAME_HERE is your GitLab username and ~/.ssh/id_rsa is the path to your SSH private key in your file system.
You've already uploaded its public key to GitLab. Here are some instructions.
All of the above did not work for me. Cloning the repo was working correctly but I was still getting an unrecognized import error.
As it stands for Go v1.13, I found in the doc that we should use the GOPRIVATE env variable like so:
$ GOPRIVATE=github.com/ORGANISATION_OR_USER_NAME go get -u github.com/ORGANISATION_OR_USER_NAME/REPO_NAME
Generate a github oauth token here and export your github token as an environment variable:
export GITHUB_TOKEN=123
Set git config to use the basic auth url:
git config --global url."https://$GITHUB_TOKEN:x-oauth-basic#github.com/".insteadOf "https://github.com/"
Now you can go get your private repo.
If you've already got git using SSH, this answer by Ammar Bandukwala is a simple workaround:
$ go get uses git internally. The following one liners will make git and consequently $ go get clone your package via SSH.
Github:
$ git config --global url."git#github.com:".insteadOf "https://github.com/"
BitBucket:
$ git config --global url."git#bitbucket.org:".insteadOf "https://bitbucket.org/"
I came across .netrc and found it relevant to this.
Create a file ~/.netrc with the following content:
machine github.com
login <github username>
password <github password or Personal access tokens >
Done!
Additionally, for latest GO versions, you might need to add this to the environment variables GOPRIVATE=github.com
(I've added it to my .zshrc)
netrc also makes my development environment setup better as my personal github access for HTTPS is been configured now to be used across the machine (just like my SSH configuration).
Generate GitHub personal access tokens: https://github.com/settings/tokens
See this answer for its use with Git on Windows specifically
Ref: netrc man page
If you want to stick with the SSH authentication, then mask the request to use ssh forcefully
git config --global url."git#github.com:".insteadOf "https://github.com/"
More methods for setting up git access: https://gist.github.com/technoweenie/1072829#gistcomment-2979908
That looks like the GitLab issue 5769.
In GitLab, since the repositories always end in .git, I must specify .git at the end of the repository name to make it work, for example:
import "example.org/myuser/mygorepo.git"
And:
$ go get example.org/myuser/mygorepo.git
Looks like GitHub solves this by appending ".git".
It is supposed to be resolved in “Added support for Go's repository retrieval. #5958”, provided the right meta tags are in place.
Although there is still an issue for Go itself: “cmd/go: go get cannot discover meta tag in HTML5 documents”.
After trying multiple solutions my problem still persisted. The final solution after setting up the ~/.netrc and SSH config file, was to add the following line to my ~/.bash_profile
export GOPRIVATE="github.com/[organization]"
I have created a user specific ssh-config, so my user automatically logs in with the correct credentials and key.
First I needed to generate an key-pair
ssh-keygen -t rsa -b 4096 -C "my#email.here"
and saved it to e.g ~/.ssh/id_my_domain. Note that this is also the keypair (private and public) I've connected to my Github account, so mine is stored in~/.ssh/id_github_com.
I have then created (or altered) a file called ~/.ssh/config with an entry:
Host github.com
HostName github.com
User git
IdentityFile ~/.ssh/id_github_com
On another server, the "ssh-url" is admin#domain.com:username/private-repo.git and the entry for this server would have been:
Host domain.com
HostName domain.com
User admin
IdentityFile ~/.ssh/id_domain_com
Just to clarify that you need ensure that the User, Host and HostName is set correctly.
Now I can just browse into the go path and then go get <package>, e.g go get main where the file main/main.go includes the package (from last example above) domain.com:username/private-repo.git.
For me, the solutions offered by others still gave the following error during go get
git#gl.nimi24.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
What this solution required
As stated by others:
git config --global url."git#github.com:".insteadOf "https://github.com/"
Removing the passphrase from my ./ssh/id_rsa key which was used for authenticating the connection to the repository. This can be done by entering an empty password when prompted as a response to:
ssh-keygen -p
Why this works
This is not a pretty workaround as it is always better to have a passphrase on your private key, but it was causing issues somewhere inside OpenSSH.
go get uses internally git, which uses openssh to open the connection. OpenSSH takes the certs necessary for authentication from .ssh/id_rsa. When executing git commands from the command line an agent can take care of opening the id_rsa file for you so that you do not have to specify the passphrase every time, but when executed in the belly of go get, this did not work somewhy in my case. OpenSSH wants to prompt you then for a password but since it is not possible due to how it was called, it prints to its debug log:
read_passphrase: can't open /dev/tty: No such device or address
And just fails. If you remove the passphrase from the key file, OpenSSH will get to your key without that prompt and it works
This might be caused by Go fetching modules concurrently and opening multiple SSH connections to Github at the same time (as described in this article). This is somewhat supported by the fact that OpenSSH debug log showed the initial connection to the repository succeed, but later tried it again for some reason and this time opted to ask for a passphrase.
However the solution of using SSH connection multiplexing as put forward in the mentioned article did not work for me. For the record, the author suggested adding the collowing conf to the ssh config file for the affected host:
ControlMaster auto
ControlPersist 3600
ControlPath ~/.ssh/%r#%h:%p
But as stated, for me it did not work, maybe I did it wrong
After setting up GOPRIVATE and git config ...
People may still meeting problems like this when fetching private source:
https fetch: Get "https://private/user/repo?go-get=1": EOF
They can't use private repo without .git extension.
The reason is the go tool has no idea about the VCS protocol of this repo, git or svn or any other, unlike github.com or golang.org them are hardcoded into go's source.
Then the go tool will do a https query before fetching your private repo:
https://private/user/repo?go-get=1
If your private repo has no support for https request, please use replace to tell it directly :
require private/user/repo v1.0.0
...
replace private/user/repo => private.server/user/repo.git v1.0.0
https://golang.org/cmd/go/#hdr-Remote_import_paths
first I tried
[url "ssh://git#github.com/"]
insteadOf = https://github.com/
but it didn't worked for my local.
I tried
ssh -t git#github.com
and it shows my ssh is fine.
finally, I fix the problem to tell the go get to consider all as private and use ssh instead of HTTPS.
adding export GOPRIVATE=*
Make sure you remove your previous gitconfigs, I had the same issue.
Previously I executed gitconfig whose token was expired, when you execute the command next time with new token make sure to delete previous one.
For standalone/final repos, an as a quick fix, why don't just to name the module within the go.mod as a package using your company's domain ... ?
module go.yourcompany.tld/the_repo
go.yourcompany.tld don't even have to exist as a valid (sub)domain...
Also, in the same go.mod you can use replacement block/lines to use private repos previously cloned the same way (within a respective folder cloned also in $GOPATH/src/go.yourcompany.tld) (why do we have to depend too much in GitHub?)
Edit
Needless to say that a private repo usually shall be a private repo, typically a standard git repo, right? With that, why not just to git clone and then go get within the cloned folder?
It's Hard Code In Go Get. Not The Git Reason. So Modify Go Source.
Reason:
repoRootForImportDynamic Will Request: https://....go-get
// RepoRootForImportPath analyzes importPath to determine the
// version control system, and code repository to use.
func RepoRootForImportPath(importPath string, mod ModuleMode, security web.SecurityMode) (*RepoRoot, error) {
rr, err := repoRootFromVCSPaths(importPath, security, vcsPaths)
if err == errUnknownSite {
rr, err = repoRootForImportDynamic(importPath, mod, security)
if err != nil {
err = importErrorf(importPath, "unrecognized import path %q: %v", importPath, err)
}
}
if err != nil {
rr1, err1 := repoRootFromVCSPaths(importPath, security, vcsPathsAfterDynamic)
if err1 == nil {
rr = rr1
err = nil
}
}
So add gitlab domain to vcsPaths will ok.
Download go source code:
vi ./src/cmd/go/internal/vcs/vcs.go
Look for code below:
var vcsPaths = []*vcsPath{
// GitHub
{
pathPrefix: "github.com",
regexp: lazyregexp.New(`^(?P<root>github\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/[A-Za-z0-9_.\-]+)*$`),
vcs: "git",
repo: "https://{root}",
check: noVCSSuffix,
},
add Code As Follow,XXXX Is Your Domain:
// GitLab
{
pathPrefix: "gitlab.xxxx.com",
regexp: lazyregexp.New(`^(?P<root>gitlab.xxxx\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/[A-Za-z0-9_.\-]+)*$`),
vcs: "git",
repo: "https://{root}",
check: noVCSSuffix,
},
compile and replace go.

how to change compromised jenkins node "secret"?

When my jenkins slave starts on its node, the command is run locally from that server:
/bin/java -jar /usr/local/jenkins/slave.jar \
-jnlpUrl https://example.com/computer/foo/slave-agent.jnlp \
-secret <big long hex id>
The "big long hex id" found its way into a git repository and is now compromised. How do I tell my jenkins master to change it?
If you delete the Slave from Jenkins, then re-add it, it should have a new ID assigned to it.
I am guessing that this is using the JNLP protocol and not JNLP4. The classes that generate the secrets: JNLPJnlpSlaveAgentProtocol/JnlpAgentReceiver uses a HMAC which uses the hostname as one input and a secret key as the other input. The secret input is fetched from the "DefaultConfidentialStore" which generates and stores a file in $JENKINS_HOME/secrets/. The name for the file in this case is probably: $JENKINS_HOME/secrets/jenkins.slaves.JnlpSlaveAgentProtocol.secret
To get a different result you either need to change the hostname or remove that file (a new one will be auto generated).
Links:
https://javadoc.jenkins-ci.org/jenkins/security/DefaultConfidentialStore.html
https://javadoc.jenkins-ci.org/jenkins/slaves/JnlpAgentReceiver.html

sesu : not found no such file or directory in Jenkins ssh plugin

Trying to connect to the server using Jenkins ssh plugin and executing some commands. It's connected but sesu command is not working. Jenkins is unable to recognize sesu command. It says sesu : not found no such file or directory. When trying with putty, sesu command is working. Jenkins version is 2.7. Please help me on same. Thanks in advance. :)
Use locate sesu command. It will show path of sesu command. Use that one in spite of only sesu. e.g. /opt/CA/AccessControl/bin/sesu
As we can't enter password on Jenkins at run-time. Other option for accessing server is generate ssh-key. Use that key to access server from Jenkins.
Use putty or similar tools to create key.
Firstly, log in to your server using credentials.
Then switch to user who has all access rights by using sesu or other switch user command like sudo su.
Execute below steps after that :
ssh-keygen -t rsa
hit enter for all steps without entering input.
then,
Once key is created, type cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys hit [Enter]
use created rsa private key after all these step in SSH plug and you are able to access unix server from Jenkins and execute commands on that server.

Deploying from Jenkins server to another server: Host key verification failed

I am trying to deploy to another server from Jenkins server, and I can't do it using Jenkins Build script.
When I am on the Jenkins server, I can deploy. For example:
:/var/lib/jenkins/workspace/MyProject$ scp my_file ubuntu#my_address:~/MyProject
Runs perfectly fine; however,
When I specify:
scp my_file ubuntu#my_address:~/MyProject
In "Execute Shell" under build in Jenkins window. I get the following error:
Host key verification failed.
I know that the first time I ran the above command directly on Jenkins server, I was prompted:
The authenticity of host 'my_address (my_address)' can't be established.
ECDSA key fingerprint is cf:4b:58:66:d6:d6:87:35:76:1c:aa:cf:9a:7c:78:cc.
Are you sure you want to continue connecting (yes/no)?
So I had to hit "yes" in order to continue. But since I already directly in the terminal, I don't have to do anything extra.
Second answer to this question: Jenkins Host key verification failed
indicates that, if I understand it correctly.
What am I missing? What can I do to fix my problem?
I got it working, I needed to do two things:
1) I had to use -o StrictHostKeyChecking=no:
scp -v -o StrictHostKeyChecking=no my_file ubuntu#my_address:~/MyProject
instead of:
scp my_file ubuntu#my_address:~/MyProject
2) I needed to copy my id_rsa to /var/lib/jenkins/.ssh
The /var/lib/jenkins/.ssh folder and files inside of it need to be owned by jenkins.
Old question but may be someone would find this useful:
ssh root#jenkinsMaster 'echo "$(ssh-keyscan -t rsa,dsa jenkinsSlave)" >> /root/.ssh/known_hosts'
Make sure that you're first settingup ssh connection to remote host then try to copy the things to remote host.
ssh -o StrictHostKeyChecking=no user#ip_addr
scp file_name user#ip_addr:/home/user

Jenkins Host key verification failed

I have a problem with jenkins, setting "git", shows the following error:
Failed to connect to repository : Command "git ls-remote -h https://person#bitbucket.org/person/projectmarket.git HEAD" returned status code 128:
stdout:
stderr: fatal: Authentication failed
I have tested with ssh:
git#bitbucket.org:person/projectmarket.git
This is error:
Failed to connect to repository : Command "git ls-remote -h git#bitbucket.org:person/projectmarket.git HEAD" returned status code 128:
stdout:
stderr: Host key verification failed.
fatal: The remote end hung up unexpectedly
I've also done these steps with "SSH key".
Login under Jenkins
sudo su jenkins
Copy your github key to Jenkins .ssh folder
cp ~/.ssh/id_rsa_github* /var/lib/jenkins/.ssh/
Rename the keys
mv id_rsa_github id_rsa
mv id_rsa_github.pub id_rsa.pub
but still not working git repository in jenkins.
thanks by help!.
Change to the jenkins user and run the command manually:
git ls-remote -h git#bitbucket.org:person/projectmarket.git HEAD
You will get the standard SSH warning when first connecting to a new host via SSH:
The authenticity of host 'bitbucket.org (207.223.240.181)' can't be established.
RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting (yes/no)?
Type yes and press Enter. The host key for bitbucket.org will now be added to the ~/.ssh/known_hosts file and you won't get this error in Jenkins anymore.
Jenkins is a service account, it doesn't have a shell by design. It is generally accepted that service accounts. shouldn't be able to log in interactively.
To resolve "Jenkins Host key verification failed", do the following steps. I have used mercurial with jenkins.
1)Execute following commands on terminal
$ sudo su -s /bin/bash jenkins
provide password
2)Generate public private key using the following command:
ssh-keygen
you can see output as ::
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/jenkins/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
3)Press Enter --> Do not give any passphrase--> press enter
Key has been generated
4) go to --> cat /var/lib/jenkins/.ssh/id_rsa.pub
5) Copy key from id_rsa.pub
6)Exit from bash
7) ssh#yourrepository
8) vi .ssh/authorized_keys
9) Paste the key
10) exit
11)Manually login to mercurial server
Note: Pls do manually login otherwise jenkins will again give error "host verification failed"
12)once manually done, Now go to Jenkins and give build
Enjoy!!!
Good Luck
Or you can use:
ssh -oStrictHostKeyChecking=no host
This will be insecure (man in the middle attacks) but easiest solution.
The better way to do that is to generate correct mappings between host and ip address, so ssh will not complain:
#!/bin/bash
for domain in "github.com" "bitbucket.org"; do
sed -i "/$domain/d" ~/.ssh/known_hosts
line=$(ssh-keyscan $domain,`nslookup $domain | awk '/^Address: / { print $2 ; exit }'`)
echo $line >> ~/.ssh/known_hosts
done
Excerpt from gist.
I think, that many people didnt recognize, at least available since jenkins 2.361:
btw. No Verification is for sure not the best option.
Had same problem, i fix it like that :
reset permission on id_rsa* only for current user no group no other
chmod o-rwx ~/.ssh/id*
chmod G-rwx ~/.ssh/id*
ls -lart ~/.ssh/
-rw------- 1 jenkins nogroup 398 avril 3 09:34 id_rsa.pub
-rw------- 1 jenkins nogroup 1675 avril 3 09:34 id_rsa
And clear ~/.ssh/know_hosts
Now Connect as jenkins
sudo su jenkins
Try the jenkins commands
git ls-remote -h git#bitbucket.org:user/project.git HEAD
If no problem appears, now jenkins will be able to connect the repo (for me ^^ at least)
As for the workaround (e.g. Windows slave), define the following environment variable in global properties:
GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
Note: If you don't see the option, you probably need EnvInject plugin for it.
login as jenkins using: "sudo su -s /bin/bash jenkins"
git clone the desired repo which causes the key error
it will ask you to add the key by showing Yes/No (enter yes or y)
that's it!
you can now re-run the jenkins job.
I hope you this will fix your issue.
using https://bitbucket.org/YYYY/XX.git
you shoud delete username#
Make sure we are not editing any of the default sshd_config properties to skip the error
Host Verification Failed - Definitely a missing entry of hostname in known_hosts file
Login to the server where the process is failing and do the following:
Sudo to the user running the process
ssh-copy-id destinationuser#destinationhostname
It will prompt like this for the first time, say yes and it will also ask password for the first time:
The authenticity of host 'sample.org (205.214.640.91)' can't be established.
RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting (yes/no)? *yes*
Password prompt ? give password
Now from the server where process is running, do ssh destinationuser#destinationhostname. It should login without a password.
Note: Do not change the default permissions of files in the user's .ssh directory, you will end up with different issues
I ran into this issue and it turned out the problem was that the jenkins service wasn't being run as the jenkins user. So running the commands as the jenkins user worked just fine.
Copy host keys from both bitbucket and github:
ssh root#deployserver 'echo "$(ssh-keyscan -t rsa,dsa bitbucket.org)" >> /root/.ssh/known_hosts'
ssh root#deployserver 'echo "$(ssh-keyscan -t rsa,dsa github.com)" >> /root/.ssh/known_hosts'
Best way you can just use your "git url" in 'https" URL format in the Jenkinsfile or wherever you want.
git url: 'https://github.com/jglick/simple-maven-project-with-tests.git'
SSH
If you are trying it with SSH, then the Host key Verification error can come due to several reasons.Follow these steps to overcome all the reasons.
Set the Environment variable as HOME and provide the address as the root directory of .ssh folder. e.g:- If your .ssh is kept inside Name folder.
C:/Users/Name.
Now make sure that the public SSH key is being provided in the repository link also. Either it is github or bitbucket or any other.
Open git bash. And try cloning the project from the repository. This will help in adding your repository URL in the known_host file, which is being auto created in the .ssh folder.
Now open jenkins and create a new job. Then click on configure.
provide the cloning URL in Source code management under Git. The URL should be start with git#github.com/......... or ssh://proje........
Under the Credential you need to add the username and password of your repository form which you are cloning the project. Select that credential.
And now apply and save the configuration.
Bingo! Start building the project. I hope now you will not get any Host Key verification error!
Try
ssh-keygen -R hostname
-R hostname Removes all keys belonging to hostname from a known_hosts file. This option is useful to delete hashed hosts
Use ssh-keyscan should be much more easier:
ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts
This command will put all required hosts to ~/.ssh/known_hosts. You will need to run this command inside your Jenkins machine. You can also create a job and put that command into the "Execute shell" section of the Configure of that job and then execute the job.
issue is with the /var/lib/jenkins/.ssh/known_hosts. It exists in the first case, but not in the second one. This means you are running either on different system or the second case is somehow jailed in chroot or by other means separated from the rest of the filesystem (this is a good idea for running random code from jenkins).
Next steps are finding out how are the chroots for this user created and modify the known hosts inside this chroot. Or just go other ways of ignoring known hosts, such as ssh-keyscan, StrictHostKeyChecking=no or so.
After ssh-keygen probably one only needs to copy the public key to remote host with:
ssh-copy-id -i ~/.ssh/mykey user#host
There is a safe and (relative easy) way to accomplish this, which should also work if you have separate worker nodes/clouds (like docker/kubernetes).
Adding host keys to Jenkins configuration
First go to a console and execute ssh-keyscan your_git_server.url
Copy the output of that command
Then navigate to https://YOUR_JENKINS_URL/manage/configureSecurity/
Scroll down to Git Host Key Verification Configuration
Paste the output of the command into the window. it should look like this:
Both bitbucket and github have pages about their keys and servers. Read them and ensure that you are adding the proper keys and not some random keys
Getting the ssh-keyscan via your Jenkins installation
If you for some reason do not have ssh-keyscan, you can go to the script console ( https://YOUR_JENKINS_URL/manage/script ) and paste in the following script:
def sout = new StringBuilder(), serr = new StringBuilder()
def proc = 'ssh-keyscan bitbucket.org'.execute()
proc.consumeProcessOutput(sout, serr)
proc.waitForOrKill(1000)
println "copy this to jenkins>\n$sout"
//println "err> $serr"

Resources