I'd like to access Google Cloud Storage from my scripts, and I need to automate authentication. By default, gsutil config asks to open a link and type in generated code, and then it writes OAuth token into .boto file.
Google Cloud also supports creating OAuth 2.0 client IDs in "Credentials" page, but I cannot make sense how to plug those credentials (client_id and client_secret) into my .boto file:
{"installed":{"client_id":"677005197220-eim3l5of3m16225qr0m9vquocj6mugt4.apps.googleusercontent.com","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://accounts.google.com/o/oauth2/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_secret":"pFghf5URxxxBFVRsQ1elWbbZ","redirect_uris":["urn:ietf:wg:oauth:2.0:oob","http://localhost"]}}
(Please don't try to use it, as I slightly modified the codes)
I plugged them in .boto file in this way:
[OAuth2]
client_id
="677005197220-eim3l5of3m16225qr0m9vquocj6mugt4.apps.googleusercontent.com"
client_secret ="pFghf5URxxxBFVRsQ1elWbbZ" provider_label = Google
provider_authorization_uri = https://accounts.google.com/o/oauth2/auth
provider_token_uri = https://accounts.google.com/o/oauth2/token
This is how gsutil is failing:
# gsutil ls gs://mybucket/
You are attempting to access protected data with no configured
credentials. Please visit https://cloud.google.com/console#/project
and sign up for an account, and then run the "gsutil config" command
to configure gsutil to use these credentials.
If I run gsutil config I can configure credentials and then it works, but I need to use my client ID and client secret.
Can someone please suggest how to make gsutil work with .boto with client_id and client_secret? Thanks
Here is how you can create a .boto file with access key ID and secret access key.
gsutil config -a
The above commmand will generate a .boto file that you can then use as a sample that you are after.
Related
I have private repo where I am uploading images outside of the docker.
image: example-registry.com:4000/test
I have that defined in my docker-compose file.
How I can provide credentials or API key in order to pull from that repository? Is it possible to do it without executing "docker login" command or it is required to always execute those commands prior the docker-compose command?
I have API key which I am using for example to do the REST API from PowerShell or any other tool.
Can I use that somehow in order to avoid "docker login" command constantly?
Thank you
docker login creates or updates the ~/.docker/config.json file for you. With just the login part, it look likes
{
"auths": {
"https://index.docker.io/v1/": {
"auth": "REDACTED"
}
}
}
There can be many things in this file, here is the doc
So to answer your question, you can avoid the login command by distributing this file instead. Something like:
Create a dedicated token (you shouldn't have multiple usage by token) here https://hub.docker.com/settings/security
Move your current config elsewhere if it does exist mv ~/.docker/config.json /tmp
Execute docker login -u YOUR-ACCOUNT, using the token as password
Copy the generated ~/.docker/config.json that you can then distribute to your server(s). This file is as much a secret as your password , don't make it public!
Move back your current config mv /tmp/config.json ~/.docker/
Having the file as a secret that you distribute doesn't make much difference than inputing the docker login command though, especially if you've some scripting to do that.
Basically, I am trying to create a credential on jenkins via Rest API. Using xml data below:
<?xml version='1.0' encoding='UTF-8'?>
<com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey>
<scope>GLOBAL</scope>
<id>jenkins-github-ssh</id>
<description>jenkins-github-ssh</description>
<username>username</username>
<directEntryPrivateKeySource>
<privateKey>-----BEGIN OPENSSH PRIVATE KEY-----
*****************************************
-----END OPENSSH PRIVATE KEY-----</privateKey>
</directEntryPrivateKeySource>
</com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey>
I can see the credential after calling REST post request. But when I use this credential for a GitHub repository, Jenkins says:
Failed to connect to repository : Command "git ls-remote -h -- git#github.com:***.git HEAD" returned status code 128:
stdout:
stderr: Load key "/tmp/ssh3978703187838467164.key": invalid format
git#github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
But If I update the credential which is created by rest api with same private key above manually. It works. Somehow key is broken while posting. Do you guys have any idea to point me the solution?
Jenkins 2.198 & SSH Credentials Plugin 1.17.3
Thanks
I faced exactly the same problem while pushing private SSH keys to Jenkins by a Python script. I'm using the Requests library to create and update SSH key credential sets in arbitrary credential stores on the Jenkins server.
The general problem is that your XML structure is partially wrong. The tag
<directEntryPrivateKeySource>
must be replaced by
<privateKeySource class="com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey$DirectEntryPrivateKeySource">
Getting the basic XML structure
You can get the correct XML structure by yourself from the Jenkins server when you follow these steps:
Create a SSH key credential item manually. In the example below the key's id is test-sshkey. Let's place it in a credential store which is located in the folder "API-Test" which is a subfolder of "Playground", i.e. Playground/API-Test.
When you click on the newly created credential item in the Jenkins UI its URL should look like this:
https://JENKINS_HOSTNAME/job/Playground/job/API-Test/credentials/store/folder/domain/_/credential/test-sshkey/
Add /config.xml to the URL above so that it looks like this:
https://JENKINS_HOSTNAME/job/Playground/job/API-Test/credentials/store/folder/domain/_/credential/test-sshkey/config.xml
The XML structure returned by the URL in step 3 has almost the structure that we need for using with the Credentials API but is partially incomplete:
<com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey plugin="ssh-credentials#1.18.1">
<id>test-sshkey</id>
<description>DELETE AFTER USE</description>
<username>test</username>
<privateKeySource class="com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey$DirectEntryPrivateKeySource">
<privateKey>
<secret-redacted/>
</privateKey>
</privateKeySource>
</com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey>
Using the Credentials API
Add the tags <scope> and <passphrase> for a valid XML scaffold that you can POST to the Credentials API:
<com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey>
<scope>GLOBAL</scope>
<id>CREDENTIAL_ID</id>
<description>MY_DESCRIPTION</description>
<username>A_USERNAME</username>
<passphrase>OPTIONAL_KEY_PASSWORD</passphrase>
<privateKeySource class="com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey$DirectEntryPrivateKeySource">
<privateKey>YOUR_PRIVATE_SSH_KEY_GOES_HERE</privateKey>
</privateKeySource>
</com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey>
Caveat: If the submitted XML has a wrong structure the REST API of the Credentials Plugin will nevertheless accept it and return a misleading HTTP status code 200!
Now we can use this XML structure to POST it to the API endpoints for creating or updating by cURL or similar tools. We assume that all operations are executed in the credential store of the folder "Playground/API-Test".
The following code example in Python is "dumbed down" completely to focus on the general approach:
def addCredentialSetSshPrivateKey(self, credentialDataObj):
"""
Adds a credential set with a private SSH key to a credential store
credentialDataObj: An instance of a simple DTO
"""
jenkinsRequestUrl = "https://ci-yoda-new.codemanufaktur.com/job/Playground/job/API-Test/credentials/store/folder/domain/_/createCredentials"
authentication = ("jenkins_admin_user", "API-TOKEN_FOR_THE_USER")
completeSamlData = """
<com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey>
<scope>GLOBAL</scope>
<id>{0}</id>
<description>{1}</description>
<username>{2}</username>
<passphrase>{3}</passphrase>
<privateKeySource class="com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey$DirectEntryPrivateKeySource">
<privateKey>{4}</privateKey>
</privateKeySource>
</com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey>
""".format(credentialDataObj.id(), credentialDataObj.description(), credentialDataObj.username(), credentialDataObj.key_passphrase(), credentialDataObj.private_ssh_key())
# When using CSRF protection in Jenkins a API crumb must be included in the actual REST call.
# The following method requests the Jenkins Crumb Issuer for a API crumb and returns a JSON object like this:
# {'_class': 'hudson.security.csrf.DefaultCrumbIssuer', 'crumb': 'a5d36ef09e063322169888f0b81341fe13b4109482a7936bc08c6f9a01badd39', 'crumbRequestField': 'Jenkins-Crumb'}
jsonCrumb = self._requestApiCrumb()
# The actual REST call with headers, XML payload and all other bells and whistles.
remoteSession = requests.Session()
return remoteSession.post(jenkinsRequestUrl, auth = authentication, headers = {"content-type":"application/xml", jsonCrumb['crumbRequestField']:jsonCrumb['crumb']}, data = completeSamlData)
REST endpoint for creating a SSH credential item:
https://JENKINS_HOSTNAME/job/Playground/job/API-Test/credentials/store/folder/domain/_/createCredentials
REST endpoint for updating a SSH credential item:
https://ci-yoda-new.codemanufaktur.com/job/Playground/job/API-Test/credentials/store/folder/domain/_/credential/credential_ci-yoda-new-project-apex_privatekey/config.xml
Apparently in the latter case you just update the config.xml file of an existing credential item.
Also see the user guide for the Credentials Plugin, section REST API, expecially for constructing the correct REST URLs. For requesting the Jenkins crumb issuer with Python see this StackOverflow answer.
Solution tested with:
Jenkins 2.214
Credentials Plugin 2.3.1
SSH Credentials Plugin 1.18.1
For the people who are having the exact same problem;
I've tried uploading it as a file, uploading it with API, using jenkins CLI, etc. Everything I tried has failed. Same issue is alsoposted in here;
https://issues.jenkins.io/browse/JENKINS-60714
So steps that finally worked is explained as follows;
Install and configure the Jenkins Configuration as Code Plugin.
Upload your configuration similar to yaml file below.
You might also want to define the private key content as an environment variable in the Jenkins instance and use it as "${private_key}" instead of pasting it visibly.
jenkins:
systemMessage: "Example of configuring credentials in Jenkins"
credentials:
system:
domainCredentials:
- credentials:
- basicSSHUserPrivateKey:
description: "kro"
id: "kro"
scope: GLOBAL
username: "kro"
privateKeySource:
directEntry:
privateKey: |
-----BEGIN RSA PRIVATE KEY-----
MIIG5AIBAAKCAYEAvuiaIDs+ydzR7Xxo5Owvv+G9/arbqN0YwhaGQQlicJjM4ZvI
..........YOUR KEY.............
53Zg4QmSb1XGKUTXxIeGd27OIvgkwAn7K/cjQsU9t802iYV3tisnfA==
-----END RSA PRIVATE KEY-----
I've been looking at https://github.com/rfdickerson/watson-translation-demo which attempts to show how to do authentication using Facebook OAuth on iOS and with a NodeJS Backend.
The iOS code is straightforward :
let token: String = FBSDKAccessToken.currentAccessToken().tokenString
let fbAccess = FacebookAuthenticationStrategy(
tokenURL: "https://watsonsdkdemo.mybluemix.net/language-translation-service/api/v1/token",
fbToken: token)
translateService = LanguageTranslation(authStrategy: fbAccess)
The problem is that the server/app.js has
var creds = appEnv.getServiceCreds(/facebook-authentication/) || {}
and the manifest.yml has
- services:
- facebook-authentication
But when you cf push the scripts to your Bluemix account you get:
FAILED
Could not find service facebook-authentication to bind to xxxxxxx
The problem is nowhere does the author describe what the 'facebook-authentication" service is.
In the server deployment instructions they have
$ cf login
$ cf create-service speech_to_text standard speech-to-text-service
$ cf create-service text_to_speech standard text-to-speech-service
$ cf create-service language_translation standard language-translation-service
$ cf env
$ cd server
$ cf push
Nothing stating what the facebook-authentication service is.
I apologize that the instructions are incomplete. facebook-authentication is supposed to be a user-provided service. You can create it using the cf create-user-provided-service command:
cf create-user-provided-service facebook-authentication -p "APP_SECRET_GOES_HERE"
The idea is that when you register a Facebook application, they give you an App Secret. Since that secret should not be embedded in the source code, you can set it in the environment variables VCAP by creating a custom service that provides that information for you.
Note, I did not finish actually using the app secret to check the legitimacy of the credentials. Although this is a small security vulnerability for man-in-the-middle-attacks, the facebook token checking still works because in the Facebook App settings I made it so that using the App secret is not required. For now, just create the user service and set the value to anything you would like.
Hope this helps!
I have a installation of IBM Connections 4.5 and the SBTPlayground on my Domino Server. Anyway I want to use it with the Playground on premise. But I can't find the right information for the custom environment. It wants OAuth2 - Consumer Key, OAuth2 - Consumer Secret and so on. So I have found many documentation about all. One of this with the WebSecurityStore but for this I need also actual URL's which I not found.
My first step is to bring a OAuth2 configuration with Greenhouse Connections.
The second step with Connections on premise.
So with this documentations are all not working, the URL's are not working. Or I can't register a app on Greenhouse, or any URL gives the keys back.
http: //heidloff.net/nh/home.nsf/article.xsp?id=12152011034545AMNHECAP.htm
http: //www.xpagescheatsheet.com/cheatsheet.nsf/135E58313968CEEB8825799100478A6F/$FILE/Ni9-CS-SocialTools-8.5x11%20PDF.pdf
http://www-10.lotus.com/ldd/appdevwiki.nsf/xpAPIViewer.xsp?lookupName=API+Reference#action=openDocument&res_title=Step_2_Obtain_authorization_code_sbt&content=apicontent
http://www.openntf.org/Projects/pmt.nsf/DA2F4D351A9F15B28625792D002D1F18/%24file/SocialEnabler111006.pdf
Does anyone have an idea?
#Raphael use these URLS from the sbt.properties file
# Connections OAuth 2.0 Endpoint Parameters
connectionsOA2.url=https://qs.renovations.com:444
connectionsOA2.serviceName=SBTK
connectionsOA2.appId=SBTK
connectionsOA2.consumerKey=SBTK
connectionsOA2.consumerSecret=
connectionsOA2.authorizationURL=https://qs.renovations.com:444/oauth2/endpoint/connectionsProvider/authorize
connectionsOA2.accessTokenURL=https://qs.renovations.com:444/oauth2/endpoint/connectionsProvider/token
connectionsOA2.apiVersion=4.0
Register your oauth keys Using wsadmin.sh
http://www-01.ibm.com/support/knowledgecenter/SSYGQH_4.5.0/admin/admin/r_admin_common_oauth_manage_list.html
Example is https://github.com/OpenNTF/SocialSDK/blob/0f7237b6ff22fed631bde9e4e16ed9744506694c/samples/scripts/oauthQSI.py
import sys
execfile('oauthAdmin.py')
OAuthApplicationRegistrationService.addApplication(sys.argv[0],sys.argv[1],sys.argv[2])
clientSecret = OAuthApplicationRegistrationService.getApplicationById(sys.argv[0]).get('client_secret')
print clientSecret
you can invoke it using a script
#
Parameters
USER=$1
PASSWORD=$2
CLIENTID=$3
APPID=$4
URL=$5
#Starts WSAdmin
cd /local/con/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin/
/local/con/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin/wsadmin.sh -user $USER -password $PASSWORD -lang jython -port 8883 -f /local/qsi/bin/con/oauthQSI.py $CLIENTID $APPID $URL
I am working on a service which requires authentication.
I would like to base the authentication on my Redmine and grant access to registered users which are members in a private project.
The membership I have figured out:
curl -v -u account:secret \
https://myredmine/projects/private/memberships.json
But how to find out if a user can authenticate?
Use /users/current.json:
curl -v -u account:secret \
https://myredmine/users/current.json
It will return 401 if the user fails to login.
Add ?include=memberships to the URL to retrieve a list of associated projects.
I don't think it will work with OpenID though.
Use the built-in API. You can enable it for each user, once you get the key:
// Pseudo-code
api_key = '65454ftfg53543f34g34f23g'
url = "http://www.myredminesite.com/projects/my_project/issues.json?key=" + api_key
You can enable the API key if you log in and click on "My Account", then on the right should be your API access key.project.
There where some issues with older version I think. I run Redmine 2.1.2.stable and that works great.