I am trying to set git private repo on jenkins server. I have installed git plugin and also github. when I set repo url in jenkins project ui the error is
Failed to connect to repository : Command "git -c core.askpass=true
ls-remote -h git#github.com:repo/project.git HEAD" returned
status code 128:
stdout:
stderr: Permission denied (publickey).
fatal: The remote end hung up unexpectedl
What I have done up to now:
My server user and jenkin user( both are in same server) are different. Though it seems to me these are not related. jenkins user are given all credintial.
In my server under var/lib/jenkins/.ssh(.ssh is created by me) I added ssh key . Public key is added to github repo.
By swithcting user to jenkins i can clone the project by this ssh. So i think there is not any public key adding problem.
I have googled the problem. there are many solutions. I tried most of them. But still no solution. Probably I am missing something.
My repo url is something like this
git#github.com:repo/project.git
If your HOME set in /var/lib/jenkins/ then i hope all the step you have been done successfully :)
Then one thing may be happen for your case. Like when you switch the user by using:
su jenkins
This command means that you switch the user but the home directory will be same as a root's home!
So you need to switch user by confirming the specific user home also switched. TO doing so, you need to follow:
su -s /bin/bash jenkins
Then you need to generate either the ssh public key once again or just update the known host. This will work.
Related Link
It depends on what HOME is set to when Jenkins is running: git will look for the ssh (public and private) keys under $HOME/.ssh.
Simply add a build step with an echo $HOME, and make sure your .ssh is in that folder.
Related
My CI project is dependent on another private repo. So I refer to the document to upload the private key using
➜ travis sshkey --upload ~/.ssh/id_travis_rsa --pro
Updating ssh key for Jeff-Tian/uni-sso with key from /Users/tianjef/.ssh/id_travis_rsa
Current SSH key: key for clone k8s-config
Finger print: 65:25:66:26:4d:5d:9f:ac:25:ba:ea:be:c4:d5:e3:5f
From the above I double checked the finger print, and compares to the github ssh keys:
They are matched.
However, the travis build still fails by:
(https://travis-ci.com/github/Jeff-Tian/uni-sso/builds/161350192)
$ git clone git#github.com:Jeff-Tian/k8s-config.git ${HOME}/k8s-config
Cloning into '/home/travis/k8s-config'...
Warning: Permanently added the RSA host key for IP address '140.82.114.4' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The command "git clone git#github.com:Jeff-Tian/k8s-config.git ${HOME}/k8s-config" failed and exited with 128 during .
And then I check the settings on travis settings, can't find the ssh keys settings pane:
Help:
Where goes wrong? Is it a Travis CI bug?
Seems the ssh keys config is only available for private repos.
The issue here is the main repo is public, but when deploy it, a private repo need to be downloaded. This scenario is not covered by the official document.
The workaround is to switch copying the private repo via https instead of ssh, so no need to upload the ssh keys.
By setting up the GH_TOKEN in the setting, and then write that token to .netrc file. Then copy the private repo using https is working:
.travis.yml:
- echo -e "machine github.com\n login $GH_TOKEN" > ~/.netrc
- git clone https://github.com/Jeff-Tian/k8s-config.git ${HOME}/k8s-config
The following code is an "Execute Shell" build step in Jenkins. The job pulls from a repo which contains a file ranger-policies/policies.json. What I'd like to do is update that file (with a curl command, in this case) and then commit the change to source control and update the remote repo. The job successfully pulls from the remote repo in the "Source Code Management" section of the job configuration page over SSH using SSH keys. However, when the job gets to the "git push origin master" line in the "Execute Shell" step, I get a Permission denied (publickey) error, as if those same SSH keys which allowed me to successfully pull the repo are not available in the "Execute Shell" step when I want to push.
curl -X GET --header "text/json" -H "Content-Type: text/json" -u user:pass "http://my-url.com/exportJson" > ranger-policies/policies.json
git add ranger-policies/policies.json
git commit -m "udpate policies.json with latest ranger policies `echo "$(date +'%Y-%m-%d')"`"
git push origin master
I ended up figuring out how to make it work. The solution involves using the SSH Agent plugin. Here's a step-by-step that describes how I did it, hopefully it helps someone else:
First, create a new pipeline job.
Then, as hinted at in this post from Jenkins' documentation, go to the home screen for your new pipeline job, and click on "Pipeline Syntax." Choose "git: Git" as the "Sample Step, and enter the git repo you want to push to in the "Repository URL" field. Then choose the corresponding valid SSH keys for that repo from the "Credentials dropdown." Everything should look like this:
Grab the value of "credentialsId", highlighted with red in the above screenshot. You'll need it later.
Install the "Workspace Cleanup Plugin" (https://wiki.jenkins.io/display/JENKINS/Workspace+Cleanup+Plugin, optional) and the "SSH Agent Plugin" (https://jenkins.io/doc/pipeline/steps/ssh-agent/, not optional, required for this process to work).
Now go back to your new pipeline job and hit "Configure," which will take you to the screen where you define the job. Drop the following code into the "Pipeline" section ("Definition" should be set to "Pipeline script"): https://gist.github.com/ScottNeaves/5cdce294296437043b24f0f3f0a8f1d8
Drop your "credentialsId" into the appropriate places in the above Jenkinsfile, and fix up the repo names to target the repo you want, and you should be good to go.
Relevant documentation:
https://jenkins.io/doc/pipeline/examples/#push-git-repo
https://gist.github.com/blaisep/eb8aa720b06eff4f095e4b64326961b5#file-jenkins-pipeline-git-cred-md
https://issues.jenkins-ci.org/browse/JENKINS-28335?focusedCommentId=269000&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-269000
As per this gist, you need to set the remote origin url as per:
git remote set-url origin git#github.com:username/your-repository.git
I have a setup of 2 VMs : VM1 with jenkins, VM2 with gitlab
On VM2 I have created a repo with user root with public access http://192.168.0.32/root/sparkjava_hello_world (acccess OK)
and generate the access token
On VM1:
- I installed the gitlab plugin in jenkins
- I copied the public key of user jenkins to authorized_key of user git in VM2 : from user jenkins shell, ssh git#VM2 is OK, no password asked
- I created the gitlab api credential and pasted the access token in it
- I configured the gitlab url in Manage Jenkins -> Configure System menu (it responds ok)
BUT when I setup the git source git#192.168.0.32:root/sparkjava_hello_world.git in my jenkins job, it doesnt work :
Failed to connect to repository : Command "/usr/bin/git ls-remote -h git#192.168.0.32:root/sparkjava_hello_world.git HEAD" returned status code 128:
stdout:
stderr: fatal: 'root/sparkjava_hello_world.git' does not appear to be a git repository
fatal: Could not read from remote repository.
I assume the ssh connection to VM2 is ok, since this is not a connection refused message.
I tried "ssh://git#192.168.0.32:root/sparkjava_hello_world.git" doesnt work either
What did i missed, or did wrong ??
thanks for help :)
Check that in VM2 you do have (as defined by default in a typical gitlab.yml) a /home/git/repositories/root/sparkjava_hello_world.git
Try an interactive ssh session on VM2 (from VM1), and do the ls-remote there:
ssh git#192.168.0.32
git ls-remote /home/git/repositories/root/sparkjava_hello_world.git
For Jenkins, what you need is to use your public key (~/.ssh/id_rsa.pub) as:
a deploy key on the GitLab side
a credential on the Jenkins side (see this tutorial)
Make sure to deploy that deploy key on your GitLab project (project settings/deploy keys), and then your Jenkins will be able to access your GitLab project (using that ssh key as credential).
Note: the normal use of a GitLab user key (like user xxx) in VM1 would be:
to define a user xxx in GitLab
to associate its public key in its user settings/ssh keys (that will modify ~git/.ssh/authorized_keys for you, adding a forced command line (this link is for gitolite, but it applies to gitlab too)
That means an ssh -T git#192.168.0.32 should not open an interactive session, but generate the message:
Welcome to GitLab, xxx
I am trying to connect Jenkins to a private BitBucket repository.
I have a set (both id_rsa and id_rsa.pub) of SSH keys generated at /var/lib/jenkins/.ssh. I've copied the id_rsa.pub key and pasted it in the deploy keys section of my BitBucket repository. Similarly, running the command git ls-remote -h git#bitbucket.org:user/project.git has a valid return and has been added to known_hosts.
What might be causing my Jenkins to not connect in the git Source Code Management section of Jenkins? The error returned is:
Failed to connect to repository : Command "git -c core.askpass=true ls-remote -h https://git#bitbucket.org/user/project.git HEAD" returned status code 128:
stdout:
stderr: remote: Invalid username or password. If you log in via a third party service you must ensure you have an account password set in your account profile.
fatal: Authentication failed for 'https://git#bitbucket.org/user/project.git/'
I also tried ssh -Tv git#bitbucket.org as per this doc from Atlassian regarding troubleshooting SSH issues. The return states at that:
This deploy key has read access to the following repositories:
user/project: Jenkins -- jenkins#ip-xxx-xx-xx-xxx
For the Project Repository field that Jenkins' Git Plugin provides, it seems that one of their examples shown in their home page for that field (specifically, https://git#bitbucket.org/user/project.git) does not work. It might either be a case of plugin version, or how BitBucket might be configured differently than GitHub, of which their documentation uses for examples.
What did work for me however, was using the git address of the following format:
git#bitbucket.org:user/project.git
I've set up Jenkins on my VPS and created a job which is set to execute a shell script containing the following command: ssh -T git#bitbucket.org
It turns out that i get a "Permission denied (publickey)." response because ssh-agent is not started. This can be solved by adding these two lines to the shell script:
eval `ssh-agent -s`
ssh-add ~/.ssh/bitbucket_key
However, I don't like to add these tho lines to every Jenkins item when I need to clone a repository. I would expect that if I log in via ssh to my VPS and change to the Jenkins user to execute the two lines myself, this would no longer be necessary. Unfortunately, this is not the case. I can successfully run ssh -T git#bitbucket.org myself, but the Jenkins job still fails without the two extra lines.
Is there a way to avoid this behavior, i.e. a way in which I only have to start the ssh-agent and add my key to it once instead of every time I want to clone a repository? I cannot imagine that it would be a good practice to start (a new) ssh-agent every time I want to clone and build my code.
Use ssh_config in your ~/.ssh/config. It has simple syntax as you can read in manual page for ssh_config. For your case should be enough
Host bitbucket.org
IdentityFile ~/.ssh/bitbucket_key