not able to pip install from private repo in docker [duplicate] - docker

I'm on Mac Snow Leopard and I just installed git.
I just tried
git clone git#thechaw.com:cakebook.git
but that gives me this error:
Initialized empty Git repository in `/Users/username/Documents/cakebook/.git/`
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
What am I missing?
I've also tried doing ssh-keygen with no passphase but still same error.

If the user has not generated a ssh public/private key pair set before
This info is working on theChaw but can be applied to all other git repositories which support SSH pubkey authentications. (See [gitolite][1], gitlab or github for example.)
First start by setting up your own public/private key pair set. This
can use either DSA or RSA, so basically any key you setup will work.
On most systems you can use ssh-keygen.
First you'll want to cd into your .ssh directory. Open up the terminal and run:
cd ~/.ssh && ssh-keygen
Next you need to copy this to your clipboard.
On OS X run: cat id_rsa.pub | pbcopy
On Linux run: cat id_rsa.pub | xclip
On Windows (via Cygwin/Git Bash) run: cat id_rsa.pub | clip
On Windows (Powershell) run: Get-Content id_rsa.pub | Set-Clipboard (Thx to #orion elenzil)
Add your key to your account via the website.
Finally setup your .gitconfig.
git config --global user.name "bob"
git config --global user.email bob#...
(don't forget to restart your command line to make sure the config is reloaded)
That's it you should be good to clone and checkout.
Further information can be found at https://help.github.com/articles/generating-ssh-keys (thanks to #Lee Whitney)
[1]: https://github.com/sitaramc/gitolite
-
If the user has generated a ssh public/private key pair set before
check which key have been authorized on your github or gitlab account settings
determine which corresponding private key must be associated from your local computer
eval $(ssh-agent -s)
define where the keys are located
ssh-add ~/.ssh/id_rsa

More extensive troubleshooting and even automated fixing can be done with:
ssh -vT git#github.com
Alternatively, according to below comments, we could issue:
ssh -vT git#gitlab.com
or substitute gitlab/github with whatever Git Instance your organisation is running.
Source: https://help.github.com/articles/error-permission-denied-publickey/

This error can happen when you are accessing the SSH URL (Read/Write) instead of Git Read-Only URL but you have no write access to that repo.
Sometimes you just want to clone your own repo, e.g. deploy to a server. In this case you actually only need READ-ONLY access. But since that's your own repo, GitHub may display SSH URL if that's your preference. In this situation, if your remote host's public key is not in your GitHub SSH Keys, your access will be denied, which is expected to happen.
An equivalent case is when you try cloning someone else's repo to which you have no write access with SSH URL.
In a word, if your intent is to clone-only a repo, use HTTPS URL (https://github.com/{user_name}/{project_name}.git) instead of SSH URL (git#github.com:{user_name}/{project_name}.git), which avoids (unnecessary) public key validation.
Update: GitHub is displaying HTTPS as the default protocol now and this move can probably reduce possible misuse of SSH URLs.

The github help link helped me sort out this problem. Looks like the ssh key was not added to the ssh-agent. This is what I ended up doing.
Command 1:
Ensure ssh-agent is enabled. The command starts the ssh-agent in the background:
eval "$(ssh-agent -s)"
Command 2:
Add your SSH key to the ssh-agent:
ssh-add ~/.ssh/id_rsa

Got the same error report.
Fixed with using the HTTPS instead of the SSH protocol. Since I don't want to set "SSH keys" for a test PC.
Change URL to HTTPS when clone:
git clone https://github.com/USERNAME/REPOSITORY.git
My problem is a little bit different: I have the URL set to SSH when adding an existing local repo to remote, by using:
git remote add origin ssh://github.com/USERNAME/REPOSITORY.git
To fix it, reset the URL to HTTPS:
git remote set-url origin https://github.com/USERNAME/REPOSITORY.git
BTW, you may check your URL using the command:
git remote -v
origin https://github.com/USERNAME/REPOSITORY.git (fetch)
origin https://github.com/USERNAME/REPOSITORY.git (push)
Hope this will help some one like me. :D

Another possibility on Windows, which is not covered in any of these answers, and is not covered in the git or github docs on troubleshooting:
git may be using a different openssh executable than you think it is.
I was receiving the Permission denied (public key) error when trying to clone or pull from github and ssh.dev.azure.com, and I'd followed all the instructions and verified that my SSH keys were setup correctly (from SSH's standpoint) using ssh -vT git#github.com and ssh -vT git#ssh.dev.azure.com. And was still getting these errors:
git#github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I eventually figured out that the problem is that Git for Windows, and Windows, both have their own versions of openssh. This is documented here: https://github.com/desktop/desktop/issues/5641
I was relying on the Windows ssh-agent service to store my ssh key passphrases, so git (with it's separate version of openssh) couldn't read my private keys. I consider it a bug that this error message is used - it's misleading.
The fix was:
git config --global core.sshCommand "'C:\Windows\System32\OpenSSH\ssh.exe'"
Or in your ~/.gitconfig:
[core]
sshCommand = 'C:\\Windows\\System32\\OpenSSH\\ssh.exe'
Perhaps this will be fixed in git for Windows soon, but this is the 2nd time I've wasted time on this issue.

I was struggling with the same problem that's what I did and I was able to clone the repo. I followed this procedure for Mac.
First Step: Checking if we already have the public SSH key.
Open Terminal.
Enter ls -al ~/.ssh to see if existing SSH keys are present:
Check the directory list to see if you already have a public SSH key. Default public is one of the following d_dsa.pub, id_ecdsa.pub, id_ed25519.pub, id_rsa.pub.
If you don't find then go to step 2 otherwise follow step 3
Step 2: Generating public SSH key
Open Terminal.
Enter the following command with a valid email address that you use for github ssh-keygen -t rsa -b 4096 -C "your_email#example.com"
You will see the following in your terminal Generating public/private rsa key pair. When it prompts to"Enter a file in which to save the key," press Enter. This accepts the default file location. When it prompts to Enter a file in which to save the key (/Users/you/.ssh/id_rsa): [Press enter] Just press enter again.
At the prompt, "Type a secure passphrase. Enter passphrase (empty for no passphrase): [Type a passphrase]" press enter if you don't want to Enter same passphrase again: [Type passphrase again] press enter again
This will generate id_rsa.pub
Step 3: Adding your SSH key to the ssh-agent
Interminal type eval "$(ssh-agent -s)"
Add your SSH key to the ssh-agent. If you are using an existing SSH key rather than generating a new SSH key, you'll need to replace id_rsa in the command with the name of your existing private key file. Enter this command $ ssh-add -K ~/.ssh/id_rsa
Now copy the SSH key and also add it to you github account
In terminal enter this command with your ssh file name pbcopy < ~/.ssh/id_rsa.pub This will copy the file to your clipboard
Now open you github account Go to Settings > SSH and GPG keys > New SSH key Enter title and paste the key from clipboard and save it. Voila you're done.

This works for me:
ssh-add ~/.ssh/id_rsa

Visual guide (Windows)
1 of 2. Git batch side
1.1. Open git batch (Download her)
1.2. Paste the text below (Change to your GitHub account email)
$ ssh-keygen -t rsa -b 4096 -C "your_email#example.com"
1.3. Press Enter (Accepts the default file location)
1.4. Click Enter Twice (Or set SSH key passphrases - Gitbub passphrases docs)
> Enter passphrase (empty for no passphrase): [Type a passphrase]
> Enter same passphrase again: [Type passphrase again]
1.5. The key generate:
Your identification has been saved in /c/Users/user/.ssh/id_rsa...
1.6. Copy the SSH key to your clipboard.
$ clip < ~/.ssh/id_rsa.pub
2 of 2. Github website user side
Under user setting
SSH and GPG keys => New SSH key:
Paste the code from step 1.6
Done :)
If someone doesn't want to use SSH use HTTPS :
Github docs: https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh

If your problem appears out of the blue recently (the latter half of 2021), it may have been caused by incompatible hash algorithms.
As of this post (Oct 2021), the latest version of Git for windows is 2.33.1 (release note), who has embraced the latest OpenSSH 8.8p1 (release note), who in turn has deprecated SHA-1. Meanwhile, if your remote Git repository still sticks to SHA-1, you'll fail the authentication.
To see whether you could have fallen into this case, check the version of your software by:
ssh -V
git --version
Then you should check the "Potentially-incompatible changes" section of OpenSSH 8.8/8.8p release note.
TL;DR
Solution 1: Enable SHA-1 again by adding this to your ~/.ssh/config file:
Host <remote>
HostkeyAlgorithms +ssh-rsa
PubkeyAcceptedAlgorithms +ssh-rsa
Remember to replace <remote> with the hostname of your remote repository.
Solution 2: Regenerate your key pair using ECDSA or Ed25519, instead of RSA. For example:
ssh-keygen -t ecdsa -C <comment>
Remember to replace <comment> with your own mnemonic phrase. Then, upload the generated public key to your remote repository.
FYI, I encountered this prompt message when accessing Gitee.com, who uses golang.org/x/crypto/ssh on their server and has posted a page on this issue here (in Mandarin).
git#gitee.com: Permission denied (publickey).

Note that (at least for some projects) you must have a github account with an ssh key.
Look at the keys listed in your authentication agent (ssh-add -l)
(if you don't see any, add one of your existing keys with ssh-add /path/to/your/key (eg: ssh-add ~/.ssh/id_rsa))
(if you don't have any keys, first create one. See: http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html or just google ssh-keygen)
To verify that you have a key associated with your github account:
Go to: https://github.com/settings/ssh
You should see at least one key with a hash key matching one of the hashes you saw when you typed ssh-add -l just a minute ago.
If you don't, add one, then try again.

I met the same issue because of I was thought the difference between SSH and HTTPS is
https://github.com/USERNAME/REPOSITORY.git
ssh://github.com/USERNAME/REPOSITORY.git
So I changed from HTTPS to SSH just by changing https:// to ssh:// nothing on the end of the url was changed.
But the truth is:
https://github.com/USERNAME/REPOSITORY.git
git#github.com:USERNAME/REPOSITORY.git
Which means I changed ssh://github.com/USERNAME/REPOSITORY.git to git#github.com:USERNAME/REPOSITORY.git it works.
Stupid error but hope helps someone!

These are the steps I followed in windows 10
Open Git Bash.
Generate Public Key:
ssh-keygen -t rsa -b 4096 -C "youremailaddress#xyz.com"
Copy generated key to the clipboard (works like CTRL+C)
clip < ~/.ssh/id_rsa.pub
Browser, go to Github => Profile=> Settings => SSH and GPG keys => Add Key
Provide the key name and paste clipboard (CTRL+V).
Finally, test your connection (Git bash)
ssh -T git#github.com
Thanks!

Please try this if nothing is worked out
Generate personal Access token (Setting -> Developer settings -> Personal access tokens -> Generate new token)
git remote set-url origin https://<TOEKN>#github.com/USERNAME/REPOSITORY.git
Note: If a password popup comes, try to enter the token only (try twice)

I had a slight different situation, I was logged on to a remote server and was using git on the server, when I ran any git command I got the same message
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
The way I fixed it was by changing the file /etc/ssh_config on my Mac.
from
ForwardAgent no
to
ForwardAgent yes

I had to copy my ssh keys to the root folder.
Google Cloud Compute Engine running Ubuntu 18.04
sudo cp ~/.ssh/* /root/.ssh/

Solution using gh i.e. Github's official CLI
gh installation
brew install gh
gh login or authentication via cli
gh auth login
repo clone
gh repo clone <username or orgname>/<repo-name>
Example: gh repo clone keshavdulal/sample-repo
Rant: I too was bashing my head when git clone suddenly decided not to work anymore and I don't have the patience or brainpower to relearn ssh/public keys/cryptography from scratch just to clone a freaking repo I already have access to. Also surprised no one mentioned gh in the answers yet

Guys this is how it worked for me:
Open terminal and go to user [See attached image]
Open .ssh folder and make sure it doesn't have any file like id_rsa or id_rsa.pub otherwise sometimes it wont properly rewrite files
git --version [Check for git installation and version]
git config --global user.email "your email id"
git config --global user.name "your name"
git config --list [make sure you have set your name & email]
cd ~/.ssh
ssh-keygen, it prompts for saving file, allow it
cat ~/.ssh/id_rsa.pub [Access your public key & copy the key to gerrit settings]
Note: You should not be using the sudo command with Git. If you have a very good reason you must use sudo, then ensure you are using it with every command (it's probably just better to use su to get a shell as root at that point). If you generate SSH keys without sudo and then try to use a command like sudo git push, you won't be using the same keys that you generated

ALWAYS CHECK GITHUB FOR SSH-KEYS GENERATION PROCEDUR, NOT SOME OUTDATED BLOG
https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent
There you can see that keys are generated with:
ssh-keygen -t ed25519 -C "your_email#example.com"
So algorithm is ed25519 not rsa or anything else.

Are you in a corporate environment? Is it possible that your system variables have recently changed? Per this SO answer, ssh keys live at %HOMEDRIVE%%HOMEPATH%\.ssh\id_rsa.pub. So if %HOMEDRIVE% recently changed, git doesn't know where to look for your key, and thus all of the authentication stuff.
Try running ssh -vT git#github.com. Take note of where the identity file is located. For me, that was pointing not to my normal \Users\MyLogin but rather to a network drive, because of a change to environment variables pushed at the network level.
The solution? Since my new %HOMEDRIVE% has the same permissions as my local files, I just moved my .ssh folder there, and called it a day.

I hit this error because I needed to give my present working directory permissions 700:
chmod -R 700 /home/ec2-user/

On Windows, make sure all your apps agree on HOME. Msys will surprisingly NOT do it for you. I had to set an environment variable because ssh and git couldn't seem to agree on where my .ssh directory was.

One of the easiest way
go to terminal-
git push <Git Remote path> --all

The basic GIT instructions did not make a reference to the SSH key stuff. Following some of the links above, I found a git help page that explains, step-by-step, exactly how to do this for various operating systems (the link will detect your OS and redirect, accordingly):
http://help.github.com/set-up-git-redirect/
It walks through everything needed for GITHub and also gives detailed explanations such as "why add a passphrase when creating an RSA key." I figured I'd post it, in case it helps someone else...

The easiest solution to this, when you are trying to push to a repository with a different username is:
git remote set-url origin https://USERNAME#github.com/USERNAME/PROJECTNAME.git

I helped the following:
Open Terminal (Git Bash)
Remove all files in directory .ssh or rename and create new .ssh folder.
To follow in the steps of the instructions:
Generating a new SSH key
Adding your SSH key to the ssh-agent
System: Windows 10.

In addition to Rufinus' reply, the shortcut to copy your ssh key to the clipboard in Windows is:
type id_rsa.pub | clip
Refs:
Print to standard output
Copy command line output to clipboard

If you have more than one key you may need to do
ssh-add private-keyfile

Its pretty straight forward. Type the below command
ssh-keygen -t rsa -b 4096 -C "youremailid#yourdomain.com"
Generate the SSH key. Open the file and copy the contents. Go to GitHub setting page , and click on SSH key . Click on Add new SSH key, and paste the contents here. That's it :) You shouldn't see the issue again.

I deleted node_modules/ package-lock.json and yarn.lock files. Ran npm i again. This resolved the issue for me.

Related

How to properly build a private Go project? [duplicate]

I'm searching for the way to get $ go get work with private repository, after many google try.
The first try:
$ go get -v gitlab.com/secmask/awserver-go
Fetching https://gitlab.com/secmask/awserver-go?go-get=1
https fetch failed.
Fetching http://gitlab.com/secmask/awserver-go?go-get=1
Parsing meta tags from http://gitlab.com/secmask/awserver-go?go-get=1 (status code 200)
import "gitlab.com/secmask/awserver-go": parse http://gitlab.com/secmask/awserver-go?go-get=1: no go-import meta tags
package gitlab.com/secmask/awserver-go: unrecognized import path "gitlab.com/secmask/awserver-go
Yep, it did not see the meta tags because I could not know how to provide login information.
The second try:
Follow https://gist.github.com/shurcooL/6927554. Add config to .gitconfig.
[url "ssh://git#gitlab.com/"]
insteadOf = https://gitlab.com/
$ go get -v gitlab.com/secmask/awserver-go --> not work
$ go get -v gitlab.com/secmask/awserver-go.git --> work but I got src/gitlab.com/secmask/awserer-go.git
Yes it work but with .git extension with my project name, I can rename it to original but do it everytime $ go get is not so good, is there an otherway?
You have one thing to configure. The example is based on GitHub but this shouldn't change the process:
$ git config --global url.git#github.com:.insteadOf https://github.com/
$ cat ~/.gitconfig
[url "git#github.com:"]
insteadOf = https://github.com/
$ go get github.com/private/repo
For Go modules to work (with Go 1.11 or newer), you'll also need to set the GOPRIVATE variable, to avoid using the public servers to fetch the code:
export GOPRIVATE=github.com/private/repo
The proper way is to manually put the repository in the right place. Once the repository is there, you can use go get -u to update the package and go install to install it. A package named
github.com/secmask/awserver-go
goes into
$GOPATH/src/github.com/secmask/awserver-go
The commands you type are:
cd $GOPATH/src/github.com/secmask
git clone git#github.com:secmask/awserver-go.git
I had a problem with go get using private repository on gitlab from our company.
I lost a few minutes trying to find a solution. And I did find this one:
You need to get a private token at:
https://gitlab.mycompany.com/profile/account
Configure you git to add extra header with your private token:
$ git config --global http.extraheader "PRIVATE-TOKEN: YOUR_PRIVATE_TOKEN"
Configure your git to convert requests from http to ssh:
$ git config --global url."git#gitlab.mycompany.com:".insteadOf "https://gitlab.mycompany.com/"
Finally you can use your go get normally:
$ go get gitlab.com/company/private_repo
For people using private GitLabs, here's a snippet that may help: https://gist.github.com/MicahParks/1ba2b19c39d1e5fccc3e892837b10e21
Also pasted below:
Problem
The go command line tool needs to be able to fetch dependencies from your private GitLab, but authenticaiton is required.
This assumes your private GitLab is hosted at privategitlab.company.com.
Environment variables
The following environment variables are recommended:
export GO111MODULE=on
export GOPRIVATE=privategitlab.company.com # this is comma delimited if using multiple private repos
The above lines might fit best in your shell startup, like a ~/.bashrc.
Explanation
GO111MODULE=on tells Golang command line tools you are using modules. I have not tested this with projects not using
Golang modules on a private GitLab.
GOPRIVATE=privategitlab.company.com tells Golang command line tools to not use public internet resources for the hostnames
listed (like the public module proxy).
Get a personal access token from your private GitLab
To future proof these instructions, please follow this guide from the GitLab docs.
I know that the read_api scope is required for Golang command line tools to work, and I may suspect read_repository as
well, but have not confirmed this.
Set up the ~/.netrc
In order for the Golang command line tools to authenticate to GitLab, a ~/.netrc file is best to use.
To create the file if it does not exist, run the following commands:
touch ~/.netrc
chmod 600 ~/.netrc
Now edit the contents of the file to match the following:
machine privategitlab.company.com login USERNAME_HERE password TOKEN_HERE
Where USERNAME_HERE is replaced with your GitLab username and TOKEN_HERE is replaced with the access token aquired in the
previous section.
Common mistakes
Do not set up a global git configuration with something along the lines of this:
git config --global url."git#privategitlab.company.com:".insteadOf "https://privategitlab.company.com"
I beleive at the time of writing this, the SSH git is not fully supported by Golang command line tools and this may cause
conflicts with the ~/.netrc.
Bonus: SSH config file
For regular use of the git tool, not the Golang command line tools, it's convient to have a ~/.ssh/config file set up.
In order to do this, run the following commands:
mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/config
chmod 600 ~/.ssh/config
Please note the permissions on the files and directory above are essentail for SSH to work in it's default configuration on
most Linux systems.
Then, edit the ~/.ssh/config file to match the following:
Host privategitlab.company.com
Hostname privategitlab.company.com
User USERNAME_HERE
IdentityFile ~/.ssh/id_rsa
Please note the spacing in the above file matters and will invalidate the file if it is incorrect.
Where USERNAME_HERE is your GitLab username and ~/.ssh/id_rsa is the path to your SSH private key in your file system.
You've already uploaded its public key to GitLab. Here are some instructions.
All of the above did not work for me. Cloning the repo was working correctly but I was still getting an unrecognized import error.
As it stands for Go v1.13, I found in the doc that we should use the GOPRIVATE env variable like so:
$ GOPRIVATE=github.com/ORGANISATION_OR_USER_NAME go get -u github.com/ORGANISATION_OR_USER_NAME/REPO_NAME
Generate a github oauth token here and export your github token as an environment variable:
export GITHUB_TOKEN=123
Set git config to use the basic auth url:
git config --global url."https://$GITHUB_TOKEN:x-oauth-basic#github.com/".insteadOf "https://github.com/"
Now you can go get your private repo.
If you've already got git using SSH, this answer by Ammar Bandukwala is a simple workaround:
$ go get uses git internally. The following one liners will make git and consequently $ go get clone your package via SSH.
Github:
$ git config --global url."git#github.com:".insteadOf "https://github.com/"
BitBucket:
$ git config --global url."git#bitbucket.org:".insteadOf "https://bitbucket.org/"
I came across .netrc and found it relevant to this.
Create a file ~/.netrc with the following content:
machine github.com
login <github username>
password <github password or Personal access tokens >
Done!
Additionally, for latest GO versions, you might need to add this to the environment variables GOPRIVATE=github.com
(I've added it to my .zshrc)
netrc also makes my development environment setup better as my personal github access for HTTPS is been configured now to be used across the machine (just like my SSH configuration).
Generate GitHub personal access tokens: https://github.com/settings/tokens
See this answer for its use with Git on Windows specifically
Ref: netrc man page
If you want to stick with the SSH authentication, then mask the request to use ssh forcefully
git config --global url."git#github.com:".insteadOf "https://github.com/"
More methods for setting up git access: https://gist.github.com/technoweenie/1072829#gistcomment-2979908
That looks like the GitLab issue 5769.
In GitLab, since the repositories always end in .git, I must specify .git at the end of the repository name to make it work, for example:
import "example.org/myuser/mygorepo.git"
And:
$ go get example.org/myuser/mygorepo.git
Looks like GitHub solves this by appending ".git".
It is supposed to be resolved in “Added support for Go's repository retrieval. #5958”, provided the right meta tags are in place.
Although there is still an issue for Go itself: “cmd/go: go get cannot discover meta tag in HTML5 documents”.
After trying multiple solutions my problem still persisted. The final solution after setting up the ~/.netrc and SSH config file, was to add the following line to my ~/.bash_profile
export GOPRIVATE="github.com/[organization]"
I have created a user specific ssh-config, so my user automatically logs in with the correct credentials and key.
First I needed to generate an key-pair
ssh-keygen -t rsa -b 4096 -C "my#email.here"
and saved it to e.g ~/.ssh/id_my_domain. Note that this is also the keypair (private and public) I've connected to my Github account, so mine is stored in~/.ssh/id_github_com.
I have then created (or altered) a file called ~/.ssh/config with an entry:
Host github.com
HostName github.com
User git
IdentityFile ~/.ssh/id_github_com
On another server, the "ssh-url" is admin#domain.com:username/private-repo.git and the entry for this server would have been:
Host domain.com
HostName domain.com
User admin
IdentityFile ~/.ssh/id_domain_com
Just to clarify that you need ensure that the User, Host and HostName is set correctly.
Now I can just browse into the go path and then go get <package>, e.g go get main where the file main/main.go includes the package (from last example above) domain.com:username/private-repo.git.
For me, the solutions offered by others still gave the following error during go get
git#gl.nimi24.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
What this solution required
As stated by others:
git config --global url."git#github.com:".insteadOf "https://github.com/"
Removing the passphrase from my ./ssh/id_rsa key which was used for authenticating the connection to the repository. This can be done by entering an empty password when prompted as a response to:
ssh-keygen -p
Why this works
This is not a pretty workaround as it is always better to have a passphrase on your private key, but it was causing issues somewhere inside OpenSSH.
go get uses internally git, which uses openssh to open the connection. OpenSSH takes the certs necessary for authentication from .ssh/id_rsa. When executing git commands from the command line an agent can take care of opening the id_rsa file for you so that you do not have to specify the passphrase every time, but when executed in the belly of go get, this did not work somewhy in my case. OpenSSH wants to prompt you then for a password but since it is not possible due to how it was called, it prints to its debug log:
read_passphrase: can't open /dev/tty: No such device or address
And just fails. If you remove the passphrase from the key file, OpenSSH will get to your key without that prompt and it works
This might be caused by Go fetching modules concurrently and opening multiple SSH connections to Github at the same time (as described in this article). This is somewhat supported by the fact that OpenSSH debug log showed the initial connection to the repository succeed, but later tried it again for some reason and this time opted to ask for a passphrase.
However the solution of using SSH connection multiplexing as put forward in the mentioned article did not work for me. For the record, the author suggested adding the collowing conf to the ssh config file for the affected host:
ControlMaster auto
ControlPersist 3600
ControlPath ~/.ssh/%r#%h:%p
But as stated, for me it did not work, maybe I did it wrong
After setting up GOPRIVATE and git config ...
People may still meeting problems like this when fetching private source:
https fetch: Get "https://private/user/repo?go-get=1": EOF
They can't use private repo without .git extension.
The reason is the go tool has no idea about the VCS protocol of this repo, git or svn or any other, unlike github.com or golang.org them are hardcoded into go's source.
Then the go tool will do a https query before fetching your private repo:
https://private/user/repo?go-get=1
If your private repo has no support for https request, please use replace to tell it directly :
require private/user/repo v1.0.0
...
replace private/user/repo => private.server/user/repo.git v1.0.0
https://golang.org/cmd/go/#hdr-Remote_import_paths
first I tried
[url "ssh://git#github.com/"]
insteadOf = https://github.com/
but it didn't worked for my local.
I tried
ssh -t git#github.com
and it shows my ssh is fine.
finally, I fix the problem to tell the go get to consider all as private and use ssh instead of HTTPS.
adding export GOPRIVATE=*
Make sure you remove your previous gitconfigs, I had the same issue.
Previously I executed gitconfig whose token was expired, when you execute the command next time with new token make sure to delete previous one.
For standalone/final repos, an as a quick fix, why don't just to name the module within the go.mod as a package using your company's domain ... ?
module go.yourcompany.tld/the_repo
go.yourcompany.tld don't even have to exist as a valid (sub)domain...
Also, in the same go.mod you can use replacement block/lines to use private repos previously cloned the same way (within a respective folder cloned also in $GOPATH/src/go.yourcompany.tld) (why do we have to depend too much in GitHub?)
Edit
Needless to say that a private repo usually shall be a private repo, typically a standard git repo, right? With that, why not just to git clone and then go get within the cloned folder?
It's Hard Code In Go Get. Not The Git Reason. So Modify Go Source.
Reason:
repoRootForImportDynamic Will Request: https://....go-get
// RepoRootForImportPath analyzes importPath to determine the
// version control system, and code repository to use.
func RepoRootForImportPath(importPath string, mod ModuleMode, security web.SecurityMode) (*RepoRoot, error) {
rr, err := repoRootFromVCSPaths(importPath, security, vcsPaths)
if err == errUnknownSite {
rr, err = repoRootForImportDynamic(importPath, mod, security)
if err != nil {
err = importErrorf(importPath, "unrecognized import path %q: %v", importPath, err)
}
}
if err != nil {
rr1, err1 := repoRootFromVCSPaths(importPath, security, vcsPathsAfterDynamic)
if err1 == nil {
rr = rr1
err = nil
}
}
So add gitlab domain to vcsPaths will ok.
Download go source code:
vi ./src/cmd/go/internal/vcs/vcs.go
Look for code below:
var vcsPaths = []*vcsPath{
// GitHub
{
pathPrefix: "github.com",
regexp: lazyregexp.New(`^(?P<root>github\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/[A-Za-z0-9_.\-]+)*$`),
vcs: "git",
repo: "https://{root}",
check: noVCSSuffix,
},
add Code As Follow,XXXX Is Your Domain:
// GitLab
{
pathPrefix: "gitlab.xxxx.com",
regexp: lazyregexp.New(`^(?P<root>gitlab.xxxx\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/[A-Za-z0-9_.\-]+)*$`),
vcs: "git",
repo: "https://{root}",
check: noVCSSuffix,
},
compile and replace go.

Bitbucket/Github: permission denied public key

when I am trying to clone a rails app repo I have got permission to, I am getting this issue.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Even after adding public key by generating one, I am unable to solve this.
Although I am able to clone using he https method but after making changes, the same error comes while I try to push the code.
Please suggest an answer for this.
First, cd into your .ssh directory. Open up the terminal and run:
cd ~/.ssh && ssh-keygen
Second, you need to copy this to your clipboard:
cat id_rsa.pub | pbcopy # On OSX
cat id_rsa.pub | xclip # On Linux
Third, add your newly generated ssh key to your account via the github/bitbucket website (just paste there).
Next, setup your git config:
git config --global user.name 'your_user_name'
git config --global user.email 'your_email'
Finally, restart your command line to make sure the config is reloaded.
Now, you should be able to clone and push from/to your github repository.
For more information on this, see this github page or this bitbucket page.
When attempting to clone, push, or pull over SSH with Git, you may receive one of these messages if Bitbucket couldn't authenticate with the keys that your SSH agent offered.
Here are the most common reasons why you may see these messages:
You used sudo when attempting the connection
You shouldn't use sudo when cloning, pushing, or pulling because the ssh-agent runs on the user level, not the root level.
Your public key isn't loaded into Bitbucket
To check if your public key is loaded into Bitbucket, do the following:
From Bitbucket, choose Personal settings from your avatar in the lower left.
The Account settings page displays.
Click SSH keys.
The SSH keys page shows a list of any existing keys.
If you don't have any keys listed, you can follow our Set up an SSH key documentation to set one up.
Your key isn't loaded into your SSH agent
If your SSH agent doesn't know to offer Bitbucket a key, the connection fails. You may run into this issue if you've recently restarted your system.
You can refer to this Article for more informations:
https://support.atlassian.com/bitbucket-cloud/docs/troubleshoot-ssh-issues/
Check few things.
Is the generated new key is the one your ssh agent using when trying to ssh to server.
(Your ssh agent might be using a different key than the one you generated)
use this to list currently loaded keys by agent.
ssh-add -L
You properly added public key to your repository hosting location.
The keys corresponding to above 1 and 2 should match.
Please see this article: GitHub: Generating SSH Keys. What happens when you run:
ssh -T git#bitbucket.org
?
You may have added the wrong key to authenticate with.
I faced this error when I created another repository in my local. My ssh-keys were already set up and I was trying to push code through vs code.
The issue got resolved when I git push-ed through git bash like I was doing before.
For bit bucket I think I have tried everything with ssh. I have tried the answer from this stackoverflow question as well. But it doesn't work. So finally I just changed the clone command from SSH to HTTPS and it worked. Only then it asked for password for my account.

ruby on rails git permission denied

I am following this guide to set ruby on rails environment on my Mac El Captain.
I followed upto installing homebrew, ruby latest version 2.2.3 with rbenv. Now, I was setting up git.
Followed up first few commands
git config --global color.ui true
git config --global user.name "YOUR NAME"
git config --global user.email "YOUR#EMAIL.com"
ssh-keygen -t rsa -C "YOUR#EMAIL.com"
Now, I have been asked to save the generated key. I saved it to ~ directory with a name file. I have now two files namely file and file.pub.
I went to this link to copy ssh key. I clicked on Add SSH key option there. Named the key ROR SSH Key.
The key in file.pub looks like
ssh-rss asfjasfhjalsfdhaskfdhalsdfsdf\asdf\as\dg\sa\fasdfas\f\asdf---so on random numbers---adfasdfasfa myemail#gmail.com
and I pasted the key there in github and saved the key.
Then, I went back to terminal and typed the below command.
ssh -T git#github.com
but I didn't received any message saying "Hi excid3! You've successfully authenticated, but GitHub does not provide shell access."
I got a message saying
The authenticity of host 'github.com (192.30.251.130)' can't be established.
RSA key fingerprint is SHA256:nThbg6sdfgdfgsdfgGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,192.30.251.130' (RSA) to the list of known hosts.
Permission denied (publickey).
Here above I have change few characters in SHA256 key. Just for security. Also, I have changed IP address a little bit for the same. But, the idea behind it is same.
Please guide me what's wrong. Thanks.
By default, ssh will look in the ~/.ssh folder for your private keys. Since you saved it in ~ instead, it can't find it.
You can either:
Move the file and file.pub files into ~/.ssh and rename to id_rsa and id_rsa.pub, as OS X will automatically use those files for any ssh command (if you hadn't manually entered a filename, this is where ssh-keygen would have saved them).
Use the ssh-add -K file command to permanently add your key to the OS X Keychain.
Note that GitHub's own instructions say they "strongly suggest keeping the default settings" instead of saving the private/public key somewhere else.

Jenkins Host key verification failed

I have a problem with jenkins, setting "git", shows the following error:
Failed to connect to repository : Command "git ls-remote -h https://person#bitbucket.org/person/projectmarket.git HEAD" returned status code 128:
stdout:
stderr: fatal: Authentication failed
I have tested with ssh:
git#bitbucket.org:person/projectmarket.git
This is error:
Failed to connect to repository : Command "git ls-remote -h git#bitbucket.org:person/projectmarket.git HEAD" returned status code 128:
stdout:
stderr: Host key verification failed.
fatal: The remote end hung up unexpectedly
I've also done these steps with "SSH key".
Login under Jenkins
sudo su jenkins
Copy your github key to Jenkins .ssh folder
cp ~/.ssh/id_rsa_github* /var/lib/jenkins/.ssh/
Rename the keys
mv id_rsa_github id_rsa
mv id_rsa_github.pub id_rsa.pub
but still not working git repository in jenkins.
thanks by help!.
Change to the jenkins user and run the command manually:
git ls-remote -h git#bitbucket.org:person/projectmarket.git HEAD
You will get the standard SSH warning when first connecting to a new host via SSH:
The authenticity of host 'bitbucket.org (207.223.240.181)' can't be established.
RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting (yes/no)?
Type yes and press Enter. The host key for bitbucket.org will now be added to the ~/.ssh/known_hosts file and you won't get this error in Jenkins anymore.
Jenkins is a service account, it doesn't have a shell by design. It is generally accepted that service accounts. shouldn't be able to log in interactively.
To resolve "Jenkins Host key verification failed", do the following steps. I have used mercurial with jenkins.
1)Execute following commands on terminal
$ sudo su -s /bin/bash jenkins
provide password
2)Generate public private key using the following command:
ssh-keygen
you can see output as ::
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/jenkins/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
3)Press Enter --> Do not give any passphrase--> press enter
Key has been generated
4) go to --> cat /var/lib/jenkins/.ssh/id_rsa.pub
5) Copy key from id_rsa.pub
6)Exit from bash
7) ssh#yourrepository
8) vi .ssh/authorized_keys
9) Paste the key
10) exit
11)Manually login to mercurial server
Note: Pls do manually login otherwise jenkins will again give error "host verification failed"
12)once manually done, Now go to Jenkins and give build
Enjoy!!!
Good Luck
Or you can use:
ssh -oStrictHostKeyChecking=no host
This will be insecure (man in the middle attacks) but easiest solution.
The better way to do that is to generate correct mappings between host and ip address, so ssh will not complain:
#!/bin/bash
for domain in "github.com" "bitbucket.org"; do
sed -i "/$domain/d" ~/.ssh/known_hosts
line=$(ssh-keyscan $domain,`nslookup $domain | awk '/^Address: / { print $2 ; exit }'`)
echo $line >> ~/.ssh/known_hosts
done
Excerpt from gist.
I think, that many people didnt recognize, at least available since jenkins 2.361:
btw. No Verification is for sure not the best option.
Had same problem, i fix it like that :
reset permission on id_rsa* only for current user no group no other
chmod o-rwx ~/.ssh/id*
chmod G-rwx ~/.ssh/id*
ls -lart ~/.ssh/
-rw------- 1 jenkins nogroup 398 avril 3 09:34 id_rsa.pub
-rw------- 1 jenkins nogroup 1675 avril 3 09:34 id_rsa
And clear ~/.ssh/know_hosts
Now Connect as jenkins
sudo su jenkins
Try the jenkins commands
git ls-remote -h git#bitbucket.org:user/project.git HEAD
If no problem appears, now jenkins will be able to connect the repo (for me ^^ at least)
As for the workaround (e.g. Windows slave), define the following environment variable in global properties:
GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
Note: If you don't see the option, you probably need EnvInject plugin for it.
login as jenkins using: "sudo su -s /bin/bash jenkins"
git clone the desired repo which causes the key error
it will ask you to add the key by showing Yes/No (enter yes or y)
that's it!
you can now re-run the jenkins job.
I hope you this will fix your issue.
using https://bitbucket.org/YYYY/XX.git
you shoud delete username#
Make sure we are not editing any of the default sshd_config properties to skip the error
Host Verification Failed - Definitely a missing entry of hostname in known_hosts file
Login to the server where the process is failing and do the following:
Sudo to the user running the process
ssh-copy-id destinationuser#destinationhostname
It will prompt like this for the first time, say yes and it will also ask password for the first time:
The authenticity of host 'sample.org (205.214.640.91)' can't be established.
RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting (yes/no)? *yes*
Password prompt ? give password
Now from the server where process is running, do ssh destinationuser#destinationhostname. It should login without a password.
Note: Do not change the default permissions of files in the user's .ssh directory, you will end up with different issues
I ran into this issue and it turned out the problem was that the jenkins service wasn't being run as the jenkins user. So running the commands as the jenkins user worked just fine.
Copy host keys from both bitbucket and github:
ssh root#deployserver 'echo "$(ssh-keyscan -t rsa,dsa bitbucket.org)" >> /root/.ssh/known_hosts'
ssh root#deployserver 'echo "$(ssh-keyscan -t rsa,dsa github.com)" >> /root/.ssh/known_hosts'
Best way you can just use your "git url" in 'https" URL format in the Jenkinsfile or wherever you want.
git url: 'https://github.com/jglick/simple-maven-project-with-tests.git'
SSH
If you are trying it with SSH, then the Host key Verification error can come due to several reasons.Follow these steps to overcome all the reasons.
Set the Environment variable as HOME and provide the address as the root directory of .ssh folder. e.g:- If your .ssh is kept inside Name folder.
C:/Users/Name.
Now make sure that the public SSH key is being provided in the repository link also. Either it is github or bitbucket or any other.
Open git bash. And try cloning the project from the repository. This will help in adding your repository URL in the known_host file, which is being auto created in the .ssh folder.
Now open jenkins and create a new job. Then click on configure.
provide the cloning URL in Source code management under Git. The URL should be start with git#github.com/......... or ssh://proje........
Under the Credential you need to add the username and password of your repository form which you are cloning the project. Select that credential.
And now apply and save the configuration.
Bingo! Start building the project. I hope now you will not get any Host Key verification error!
Try
ssh-keygen -R hostname
-R hostname Removes all keys belonging to hostname from a known_hosts file. This option is useful to delete hashed hosts
Use ssh-keyscan should be much more easier:
ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts
This command will put all required hosts to ~/.ssh/known_hosts. You will need to run this command inside your Jenkins machine. You can also create a job and put that command into the "Execute shell" section of the Configure of that job and then execute the job.
issue is with the /var/lib/jenkins/.ssh/known_hosts. It exists in the first case, but not in the second one. This means you are running either on different system or the second case is somehow jailed in chroot or by other means separated from the rest of the filesystem (this is a good idea for running random code from jenkins).
Next steps are finding out how are the chroots for this user created and modify the known hosts inside this chroot. Or just go other ways of ignoring known hosts, such as ssh-keyscan, StrictHostKeyChecking=no or so.
After ssh-keygen probably one only needs to copy the public key to remote host with:
ssh-copy-id -i ~/.ssh/mykey user#host
There is a safe and (relative easy) way to accomplish this, which should also work if you have separate worker nodes/clouds (like docker/kubernetes).
Adding host keys to Jenkins configuration
First go to a console and execute ssh-keyscan your_git_server.url
Copy the output of that command
Then navigate to https://YOUR_JENKINS_URL/manage/configureSecurity/
Scroll down to Git Host Key Verification Configuration
Paste the output of the command into the window. it should look like this:
Both bitbucket and github have pages about their keys and servers. Read them and ensure that you are adding the proper keys and not some random keys
Getting the ssh-keyscan via your Jenkins installation
If you for some reason do not have ssh-keyscan, you can go to the script console ( https://YOUR_JENKINS_URL/manage/script ) and paste in the following script:
def sout = new StringBuilder(), serr = new StringBuilder()
def proc = 'ssh-keyscan bitbucket.org'.execute()
proc.consumeProcessOutput(sout, serr)
proc.waitForOrKill(1000)
println "copy this to jenkins>\n$sout"
//println "err> $serr"

git push heroku master permission denied

I am following the ruby.railstutorial. I run the command "git push heroku master" and it spits out this error.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I am inside my rails app "/Users/lexi87/rails_projects/first_app". Any solutions?
The best way to avoid such errors is to use one public/private key pair and not an extra key for heroku. This way you (or your system) can't choose a wrong key to login in heroku.
If you get this error, you have done something wrong. Check this site: https://devcenter.heroku.com/articles/keys
If you geht this error, the best way is to remove unnecessary keys and use only one.
If you need more than one key pair on your system, you can set one key for heroku. This is done through the following command:
heroku keys:add
Some help for Windows 7 users with Github Windows client installed:
Even though heroku toolbelt reports it found my git_hub public key and uploaded it, 'git push heroku master' failed. After taking the steps below, it works fine.
Create .ssh folder under your User folder if one does not exist. If
it does, delete all files in it (this assumes you are OK with starting from scratch with ssh keys).
In Windows Explorer, right click the
.ssh folder, and choose Git bash from the context menu. This is installed along with the Github Windows client software.
In the bash window enter
ssh-keygen -t rsa -C "yourname#email.com" When prompted enter a
passphrase (don't lose this).
Close the bash shell window.
From a cmd prompt in your project's root, enter heroku keys:add.
This will find and upload the key you just created from your /.ssh file to
Heroku.
Now you can enter git push heroku master to push you app up to Heroku.
Note: you will need to add your newly generated ssh public key to your Github account when done.
I faced the same issue. In my .ssh folder I had a file called 'Known Hosts'. I kept trying to delete and create new ssh keys it did not work. In the end I just deleted everything in .ssh including 'Known Hosts' and then created a new rsa key using:
ssh-keygen -t rsa
then I added this new key to heroku using:
heroku keys:add
then create a new heroku repo and pushed my app to it:
heroku create
git push heroku master
ssh-keygen -t rsa
Above is optional as you could also link to an existing key. Heroku will prompt to select the key in the next step.
heroku keys:add
Add your newly created key or an existing one. If you are still running into the issue, you will most likely need to add the key to your machine's list of ssh keys by performing the following:
ssh-add ~/.ssh/name_of_your_rsa
and confirm that your ssh has been added
ssh-add -l
This should get you access to push to Heroku's remote repo.
on OSX, I was having experiencing the same issue, I was getting
no such identity: /Users/me/.ssh/yourPrivateKey: No such file or directory
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I tried to create a new key with ssh-keygen -t rsa then adding it with heroku keys:add, but it didn't help.
Then I found a file named config in ~/.ssh/, and inside the file there was:
ServerAliveInterval 300
ServerAliveCountMax 3
host heroku.com
user git
hostname heroku.com
identityfile ~/.ssh/yourPrivateKey
So I changed yourPrivateKey to my private key filename (id_rsa by default) aaand it worked :)
I created a key with
ssh-keygen -t rsa
and used a different filename than id_rsa (in my case heroku). I added the key to heroku with
heroku keys:add
On trying to push my master branch to heroku I always received the following error:
$ git push heroku master
The authenticity of host 'heroku.com
(50.19.85.132)' can't be established. RSA key fingerprint is
8b:48:5e:67:0e:c9:16:47:32:99:87:0c:1f:c8:60:bb. Are you sure you want
to continue connecting (yes/no)? yes Warning: Permanently added
'heroku.com,50.19.85.132' (RSA) to the list of known hosts. Permission
denied (publickey). fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository
exists.
As I noticed git only used my id_rsa key from another ssh-access (you can check that via your git gui: Help -> SSH keys).
I renamed my .ssh directory C:\Users\%username%.ssh to .ssh.bak and copied my heroku private and public key (from the .ssh.bak directory) to a newly created .ssh directory and named it id_rsa (and id_rsa.pub).
Now pushing worked as expected:
git push heroku master
If you are working on Windows, be sure to use git-bash instead of Powershell/Command Prompt.
If you just want to reset your ssh keys:
delete your user's .ssh dir
open git-bash
ssh-keygen -t rsa
heroku keys:add
and then you will be able to git push.

Resources