My CI project is dependent on another private repo. So I refer to the document to upload the private key using
➜ travis sshkey --upload ~/.ssh/id_travis_rsa --pro
Updating ssh key for Jeff-Tian/uni-sso with key from /Users/tianjef/.ssh/id_travis_rsa
Current SSH key: key for clone k8s-config
Finger print: 65:25:66:26:4d:5d:9f:ac:25:ba:ea:be:c4:d5:e3:5f
From the above I double checked the finger print, and compares to the github ssh keys:
They are matched.
However, the travis build still fails by:
(https://travis-ci.com/github/Jeff-Tian/uni-sso/builds/161350192)
$ git clone git#github.com:Jeff-Tian/k8s-config.git ${HOME}/k8s-config
Cloning into '/home/travis/k8s-config'...
Warning: Permanently added the RSA host key for IP address '140.82.114.4' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The command "git clone git#github.com:Jeff-Tian/k8s-config.git ${HOME}/k8s-config" failed and exited with 128 during .
And then I check the settings on travis settings, can't find the ssh keys settings pane:
Help:
Where goes wrong? Is it a Travis CI bug?
Seems the ssh keys config is only available for private repos.
The issue here is the main repo is public, but when deploy it, a private repo need to be downloaded. This scenario is not covered by the official document.
The workaround is to switch copying the private repo via https instead of ssh, so no need to upload the ssh keys.
By setting up the GH_TOKEN in the setting, and then write that token to .netrc file. Then copy the private repo using https is working:
.travis.yml:
- echo -e "machine github.com\n login $GH_TOKEN" > ~/.netrc
- git clone https://github.com/Jeff-Tian/k8s-config.git ${HOME}/k8s-config
Related
When I attempt a build on my Jenkins job that is configured to checkout from GitLab I am getting below error output (truncated) :
....................
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git#10.777.77.777:root/xxx.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:1001)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1242)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1302)
....................................................
....................................................
at hudson.model.Executor.run(Executor.java:431)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- git#10.777.77.777:root/xxx.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: Load key "/var/lib/jenkins/workspace/gitlabjenkinsdemo#tmp/jenkins-gitclient-ssh1964800292912998995.key": invalid format
Permission denied, please try again.
Permission denied, please try again.
git#10.170.8.204: Permission denied (publickey,password).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2681)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2102)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:86)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:624)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:999)
... 11 more
ERROR: Error fetching remote repo 'origin'
Finished: FAILURE
I am trying to setup a CI/CD pipeline as per this post Jenkins_Gitlab
I expected that the private key I added should enable checkout out of the box (Note : Both Jenkins server and Gitlab server are running on the same Debian instance).
In the ~/.ssh directory I have the following :
root#myservername:~/.ssh# ls -ld
drwx------ 2 root root 4096 Oct 18 15:30 .
In the source code management tab of my Jenkins job I have also added the SSH link that I extracted from my project in Gitlab. However what I am unsure of is the message below that the project is showing in Gitlab :
What am I missing ?
Both Jenkins server and Gitlab server are running on the same Debian instance
Ideally, they would run with their own associated service account, not "root".
But still, even as root, this should work.
Check which private SSH key you have added, and make sure, at least for testing, it wasn't one protected with a passphrase (meaning the private key file does not have a Proc-Type: 4,ENCRYPTED line in it)
That might be a cause for the jenkins-gitclient-ssh1964800292912998995.key": invalid format you have.
(That or you have copied the private key file content with CRLF instead of LF as end-of-line)
(or you have copied a public key, where a private one was expected, or vice-versa)
From the discussion:
the private key had to be registered again in Jenkins (making sure its eol -- end-of-line -- are LF, not CRLF)
the branch needs to be renamed:
My repo had master when I did git init, so had to rename branch to main when I set the GitLab remote
I have a setup of 2 VMs : VM1 with jenkins, VM2 with gitlab
On VM2 I have created a repo with user root with public access http://192.168.0.32/root/sparkjava_hello_world (acccess OK)
and generate the access token
On VM1:
- I installed the gitlab plugin in jenkins
- I copied the public key of user jenkins to authorized_key of user git in VM2 : from user jenkins shell, ssh git#VM2 is OK, no password asked
- I created the gitlab api credential and pasted the access token in it
- I configured the gitlab url in Manage Jenkins -> Configure System menu (it responds ok)
BUT when I setup the git source git#192.168.0.32:root/sparkjava_hello_world.git in my jenkins job, it doesnt work :
Failed to connect to repository : Command "/usr/bin/git ls-remote -h git#192.168.0.32:root/sparkjava_hello_world.git HEAD" returned status code 128:
stdout:
stderr: fatal: 'root/sparkjava_hello_world.git' does not appear to be a git repository
fatal: Could not read from remote repository.
I assume the ssh connection to VM2 is ok, since this is not a connection refused message.
I tried "ssh://git#192.168.0.32:root/sparkjava_hello_world.git" doesnt work either
What did i missed, or did wrong ??
thanks for help :)
Check that in VM2 you do have (as defined by default in a typical gitlab.yml) a /home/git/repositories/root/sparkjava_hello_world.git
Try an interactive ssh session on VM2 (from VM1), and do the ls-remote there:
ssh git#192.168.0.32
git ls-remote /home/git/repositories/root/sparkjava_hello_world.git
For Jenkins, what you need is to use your public key (~/.ssh/id_rsa.pub) as:
a deploy key on the GitLab side
a credential on the Jenkins side (see this tutorial)
Make sure to deploy that deploy key on your GitLab project (project settings/deploy keys), and then your Jenkins will be able to access your GitLab project (using that ssh key as credential).
Note: the normal use of a GitLab user key (like user xxx) in VM1 would be:
to define a user xxx in GitLab
to associate its public key in its user settings/ssh keys (that will modify ~git/.ssh/authorized_keys for you, adding a forced command line (this link is for gitolite, but it applies to gitlab too)
That means an ssh -T git#192.168.0.32 should not open an interactive session, but generate the message:
Welcome to GitLab, xxx
Here is the workflow I want to achieve:
commit code
bitbucket pipeline run test on my public docker image
bitbucket pipeline executes ansible script to deploy on my public docker image
The first 2 steps working fine, but here is the problem:
How/Where should I store my private keys to allow ansible to ssh to my remote server via ssh agent?
I am a bit reluctant to store the private key inside Pipeline env settings, since everyone else has admin access to the repo can see it.
There is a similar question asked here but the answer suggests to setup the keys on docker and use private repo, which it's a bit different to mine.
You can now setup SSH keys under pipeline settings so that you do not need to use environment variables and copy to certain locations in the container. The private key is not shown at all.
Under
Settings -> Pipelines -> SSH keys
You would need to get the public key to the production containers known_hosts file.
I have set up a similar process and used Pipelines environment variables, there is a checkbox to secure the value so you don't need to worry about others viewing it.
The set up is pretty easy:
Base64 encode a private key and store it in an environment variable
in Bitbucket
Commit a "my_known_hosts" file to your codebase that includes
the public SSH key of the remote host.
Then in your bitbucket-pipelines.yml file set up the known_hosts and key:
- mkdir -p ~/.ssh
- cat my_known_hosts >> ~/.ssh/known_hosts
- (umask 077 ; echo $MY_SSH_KEY | base64 --decode > ~/.ssh/id_rsa)
Full documentation is available here https://confluence.atlassian.com/bitbucket/access-remote-hosts-via-ssh-847452940.html
I am trying to connect Jenkins to a private BitBucket repository.
I have a set (both id_rsa and id_rsa.pub) of SSH keys generated at /var/lib/jenkins/.ssh. I've copied the id_rsa.pub key and pasted it in the deploy keys section of my BitBucket repository. Similarly, running the command git ls-remote -h git#bitbucket.org:user/project.git has a valid return and has been added to known_hosts.
What might be causing my Jenkins to not connect in the git Source Code Management section of Jenkins? The error returned is:
Failed to connect to repository : Command "git -c core.askpass=true ls-remote -h https://git#bitbucket.org/user/project.git HEAD" returned status code 128:
stdout:
stderr: remote: Invalid username or password. If you log in via a third party service you must ensure you have an account password set in your account profile.
fatal: Authentication failed for 'https://git#bitbucket.org/user/project.git/'
I also tried ssh -Tv git#bitbucket.org as per this doc from Atlassian regarding troubleshooting SSH issues. The return states at that:
This deploy key has read access to the following repositories:
user/project: Jenkins -- jenkins#ip-xxx-xx-xx-xxx
For the Project Repository field that Jenkins' Git Plugin provides, it seems that one of their examples shown in their home page for that field (specifically, https://git#bitbucket.org/user/project.git) does not work. It might either be a case of plugin version, or how BitBucket might be configured differently than GitHub, of which their documentation uses for examples.
What did work for me however, was using the git address of the following format:
git#bitbucket.org:user/project.git
I am trying to set git private repo on jenkins server. I have installed git plugin and also github. when I set repo url in jenkins project ui the error is
Failed to connect to repository : Command "git -c core.askpass=true
ls-remote -h git#github.com:repo/project.git HEAD" returned
status code 128:
stdout:
stderr: Permission denied (publickey).
fatal: The remote end hung up unexpectedl
What I have done up to now:
My server user and jenkin user( both are in same server) are different. Though it seems to me these are not related. jenkins user are given all credintial.
In my server under var/lib/jenkins/.ssh(.ssh is created by me) I added ssh key . Public key is added to github repo.
By swithcting user to jenkins i can clone the project by this ssh. So i think there is not any public key adding problem.
I have googled the problem. there are many solutions. I tried most of them. But still no solution. Probably I am missing something.
My repo url is something like this
git#github.com:repo/project.git
If your HOME set in /var/lib/jenkins/ then i hope all the step you have been done successfully :)
Then one thing may be happen for your case. Like when you switch the user by using:
su jenkins
This command means that you switch the user but the home directory will be same as a root's home!
So you need to switch user by confirming the specific user home also switched. TO doing so, you need to follow:
su -s /bin/bash jenkins
Then you need to generate either the ssh public key once again or just update the known host. This will work.
Related Link
It depends on what HOME is set to when Jenkins is running: git will look for the ssh (public and private) keys under $HOME/.ssh.
Simply add a build step with an echo $HOME, and make sure your .ssh is in that folder.