Which flag for run_flow() will simulate the now deprecated run() - oauth-2.0

I am trying to authenticate my credentials to access the GMail API. Previously I did this using the run() method from OAuth2, and the code credentials = tools.run(flow, STORAGE, http=http) but this is now a deprecated method. I am now using the run_flow() method to authenticate my credentials.
import httplib2
import argparse
from apiclient import errors
from apiclient.discovery import build
from oauth2client import tools
from oauth2client.file import Storage
from oauth2client.client import flow_from_clientsecrets
CLIENT_SECRET_FILE = 'your_client_secret.json'
OAUTH_SCOPE = 'https://www.googleapis.com/auth/gmail.modify'
STORAGE = Storage('gmail.storage')
flow = flow_from_clientsecrets(CLIENT_SECRET_FILE, scope=OAUTH_SCOPE)
http = httplib2.Http()
credentials = STORAGE.get()there are credentials, no reauth is needed
#parser = argparse.ArgumentParser(parents=[tools.argparser])
#flags = parser.parse_args() #Put your arguments in the parenthesis
if credentials is None or credentials.access_token_expired:
credentials = run(flow, STORAGE, http=http)
#credentials = tools.run_flow(flow, STORAGE, flags, http=http)
http = credentials.authorize(http)
gmail_service = build('gmail', 'v1', http=http)
The commented lines are the code that uses run_flow() and not run().
The commented out code gives me the error: run.py: error: unrecognized arguments: AdminTests, AdminTests is not an argument I give to Python.
And when I change the arguments parsed to flags = parser.parse_args(['--noauth_local_webserver']) I get no error, but nothing happens.
Which flag should I use to simulate the run() as closesly as possible and how should I parse it?
Edit: When using the run() method to authenticate my credentials the URL accessed is:
http://localhost:8080/?code=4/myuniqueID (missing my unique ID in the example)

what you need to do for this is pass an empty list of args to the argparser like this
flags = tools.argparser.parse_args(args=[])
credentials = tools.run_flow(flow, storage, flags)

After comparing your code to the source code of OAuth's run and run_flow, it turns out that there is a significant difference between whether you include the http argument.
So,
tools.run(flow, STORAGE, http=http)
can be simulated with,
tools.run_flow(flow, STORAGE, flags, http=http)
but you have,
tools.run_flow(flow, STORAGE, flags)

Related

Adding Google Service Account Credentials by a groovy script

I want to add Google Service Account credentials using a groovy file initializer on init.groovy.d. So far I could define UserName-Password credentials and String credentials. However, the Google API key requires a special credential type: GoogleRobotPrivateKeyCredentials. I could not manage to add that type of credential.
How can I define Google Service Account credentials of type GoogleRobotPrivateKeyCredentials through a groovy script?
It seems there is an answer on that link: https://github.com/jenkinsci/google-oauth-plugin/issues/5
I have changed that slightly, as that answer did not work on the latest Jenkins installation:
import com.cloudbees.plugins.credentials.Credentials
import com.cloudbees.plugins.credentials.CredentialsStore
import com.cloudbees.plugins.credentials.SystemCredentialsProvider
import com.cloudbees.plugins.credentials.domains.Domain
import com.google.jenkins.plugins.credentials.oauth.GoogleRobotPrivateKeyCredentials
import com.google.jenkins.plugins.credentials.oauth.JsonServiceAccountConfig
import hudson.model.FileParameterValue.FileItemImpl
import org.apache.commons.fileupload.FileItem
def jsonFile = new File("/myfilepath/key.json")
def fileItem = new FileItemImpl(jsonFile)
def config = new JsonServiceAccountConfig()
config.setJsonKeyFileUpload(fileItem)
Credentials credentials = new GoogleRobotPrivateKeyCredentials("gpc", config, null)
SystemCredentialsProvider instance = SystemCredentialsProvider.getInstance()
CredentialsStore store = instance.getStore()
store.addCredentials(Domain.global(), credentials)

mock auth0 server in continuous integration environment with no internet

I want to write e2e tests that can run in our CI environment which is tightly locked down and has no internet access.
I am using the auth0 react loginWithRedirect function which will attempt to redirect to the auth0 login on their servers and will timeout on CI.
I am able to intercept the call to /authorize in express:
app.get('/auth0/:simulation_id/authorize', middleware, (req, res) => {
const { client_id, redirect_uri, scope, state } = req.query;
Is it now possible for me to generate a mock oauth token that will be accepted by the #auth0/react client code?
Yes. Using the jsrsasign.js library, where accessKey contains a private key:
function createJwt(username, type, realm, scopes, sessionId, expires_in, audience) {
var oHeader = {
alg: 'RS256',
typ: 'JWT'
};
// Payload
var oPayload = {};
var tNow = jwt.KJUR.jws.IntDate.get('now');
var token_expiry = expires_in;
var tEnd = tNow + token_expiry;
oPayload.sub = username;
oPayload.auditTrackingId = uuidv4();
oPayload.iss = "https://mock.org/identities/v1/" + realm + "/token";
oPayload.tokenName = type;
oPayload.token_type = 'Bearer';
oPayload.authGrantId = !sessionId ? uuidv4() : sessionId;
oPayload.aud = (audience ? audience : "https://mock.org/identities/v1/" + realm + "/token");
oPayload.nbf = tNow;
oPayload.scope = scopes;
// oPayload.auth_time = tNow;
oPayload.realm = '/' + realm;
oPayload.exp = tEnd;
oPayload.expires_in = token_expiry * 100000;
oPayload.iat = tNow;
oPayload.jti = uuidv4();
var sHeader = JSON.stringify(oHeader);
var sPayload = JSON.stringify(oPayload);
var prvKey = accessKey;
var sJWT = jwt.KJUR.jws.JWS.sign("RS256", sHeader, sPayload, prvKey);
return sJWT;
}
Set the audience, subject, scp and any other properties in line with the tokens that auth0 is creating.
You also might need to mock out the jwk endpoints, but I doubt the library will care. If it doesn't work it will be due to certain properties not being available on the token.
You might need to create an id token as well, depending on your setup.
Access key is similar to this:
var accessKey = jwt.KEYUTIL.getKey("-----BEGIN PRIVATE KEY-----\n" +
"MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQC0g+nePEZ6aFiH\n" +
"ccnle/ryqz1lNvloPGuA5b6GeE8MrT7SATxv54zQ2LnvuRp86cd32elL0YAw3GMc\n" +
... snip ...
"Rno3R82z6SAvy9HgIMzfti5NSVgqC9nmhFEs+ChFWuboGotVV99COVJId9S/567n\n" +
"kz90cLENtD/8JTYAhLea5F/PJBJJHSvQT298ZxR4bw1vQ5Bq2FMnLSYuIOQMkQdr\n" +
"Yqt+8gXW0+3kfyb3cCyI+2HKcQ==\n" +
"-----END PRIVATE KEY-----\n");
And keep going til you've mocked out the entire IdP platform :) (I basically did the same thing with OpenAM a few years ago).
In addition to #stringy05's response; if you've got back-end services (e.g. Java, .NET, etc) that are called by your React app and need to do JWT validation, you might need to statically host your own an OpenID Configuration and JSON Web Key Set endpoints in your own environment.
The reason for this is because JWT validation (by back-end services) would still require communication to the external authorization server. With JWT-based auth this is generally limited to a single (up-front or lazy-loaded) request to get the public key for signature validation, but there could also be additional calls e.g. when the service encounters an unknown kid (key ID) on a JWT, as well as a periodical refresh interval (e.g. daily).
For Auth0 OpenID Configuration and JSON Web Key Set endpoints are hosted at the following locations:
OpenID Configuration: https://[tenant-name].[2-letter-iso-country-code].auth0.com/.well-known/openid-configuration
JSON Web Key Set: https://[tenant-name].[2-letter-iso-country-code].auth0.com/.well-known/jwks.json
These are just static JSON documents. You can probably just copy these from one of your existing tenants, modify accordingly, and then host in your own environment. The modifications you'd need to make include:
OpenID Configuration to reference local jwks_uri endpoint.
Update the key information in the JSON Web Key Set endpoint. Note: you can generate an asymetric key pair using OpenSSL, and see here for information about how to extract modulus and exponent from public key (which are required for the JWKS document). Note: in your React app, you'd need to sign the tokens with the private key that you generate.
Once you've done the above, your back-end services can reference the locally hosted OpenID Configuration and will be able to do JWT validation without communicating to an external authorization server.

Twilio - Quick question (Unable to update record)

hope you are doing it right these days.
To summarize my problem, I think this is not working becuase I am using a free Twilio account instead of a paid one. But that's just my beginner theory. Now, the issue:
I have tried an official Twilio tutorial (https://www.twilio.com/blog/automating-ngrok-python-twilio-applications-pyngrok, I shared the link in case someone finds it interesting or needs it), which allows us to automate SMS webhook (sms_url) configuration by using Client (twilio) and pyngrok (ngrok).
def start_ngrok():
from twilio.rest import Client
from pyngrok import ngrok
url = ngrok.connect(5000)
print(' * Tunnel URL:', url)
client = Client()
client.incoming_phone_numbers.list(
phone_number=os.environ.get('TWILIO_PHONE_NUMBER'))[0].update(
sms_url=url + '/bot')
I can't explain all the things that I tried in the last 4 days, with no success. I keep getting the same error:
client.incoming_phone_numbers.list(phone_number=os.environ.get('TWILIO_PHONE_NUMBER'))[0].update(sms_url=url + '/bot')
IndexError: list index out of range
Something is not working with the list, it comes empty, although environment variables are working properly. I will work with just one phone_number, so there no need for list, indeed, so I started to change that line to avoid different errors and ended up with this:
def start_ngrok():
from twilio.rest import Client
from pyngrok import ngrok
url = ngrok.connect(5000)
print(' * Tunnel URL:', url)
client = Client()
client.incoming_phone_numbers("my_number").update(sms_url=str(url) + '/bot')
Then I got the final error that I can't solve by my self:
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/twilio/rest/api/v2010/account/incoming_phone_number/__init__.py", line 442, in update
payload = self._version.update(method='POST', uri=self._uri, data=data, )
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/twilio/base/version.py", line 106, in update
raise self.exception(method, uri, response, 'Unable to update record')
twilio.base.exceptions.TwilioRestException:
HTTP Error Your request was:
POST /Accounts/my_account_SID/IncomingPhoneNumbers/+my_number.json
Twilio returned the following information:
Unable to update record: The requested resource /2010-04-01/Accounts/my_account_SID/IncomingPhoneNumbers/+my_number.json was not found
More information may be available here:
https://www.twilio.com/docs/errors/20404
I tried all different phone numbers combinations/formats: nothing works.
Thanks for your time reading all this!
Looks like something changed since the blog was written or there was a mistake.
Try the below:
The only difference is adding .public_url to the url object. Also allowed a GET to /bot for testing.
from dotenv import load_dotenv
from flask import Flask, request
from twilio.twiml.messaging_response import MessagingResponse
load_dotenv()
app = Flask(__name__)
#app.route('/bot', methods=['POST','GET'])
def bot():
user = request.values.get('From', '')
resp = MessagingResponse()
resp.message(f'Hello, {user}, thank you for your message!')
return str(resp)
def start_ngrok():
from twilio.rest import Client
from pyngrok import ngrok
url = ngrok.connect(5000)
print('This is',url)
print(' * Tunnel URL:', url)
client = Client()
client.incoming_phone_numbers.list(
phone_number=os.environ.get('TWILIO_PHONE_NUMBER'))[0].update(
sms_url=url.public_url + '/bot')
if __name__ == '__main__':
if os.environ.get('WERKZEUG_RUN_MAIN') != 'true':
start_ngrok()
app.run(debug=True)

Cloud Function & OAuth 2.0

I have a issue to use OAuth2.0 on GCP Cloud Function. I use to run this code locally. it works and it open a web browser's page to ask access to my gmail account.
I know that InstalledAppFlow is only use for local application.
SCOPES = ['https://mail.google.com/']
creds = None
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file("credentials.json", SCOPES) # <-- Oauth2.0 credential
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
I then tried to do another way using /tmp repesitory to store the token but still doesn't work and I can't see where is the issue ... Do you have any idea ? thank you so much
SCOPES = ['https://mail.google.com/']
CLIENT_SECRET_FILE = 'credentials.json' #OAuth credentials
APPLICATION_NAME = 'Gmail API Python'
def get_credentials():
store = oauth2client.file.Storage("/tmp/tempcredentials.json")
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
credentials = tools.run_flow(flow, store)
return credentials
As stated in the documentation the temporary folder is just for creating temporary files that will be stored in RAM memory and only available for the instance currently executing your code, thus there's no guarantee of persistence between invocations.
You should check this tutorial as it explains how to authenticate from Cloud Functions into Gmail (you will need more than a single function).

How to send a speech to text request using google_speech1 in Rust?

I am trying to use google_speech1 for Rust, but the documentation provides incomplete examples, which makes it very hard for me, being both new at Rust and at using Google Speech Api, to figure out how to do send a speech to text request.
More specifically, I would like to be able to send a local audio file, indicate the source language and retrieve the transcription.
Here is the closest I could find in the official documentation(https://docs.rs/google-speech1/1.0.8+20181005/google_speech1/struct.SpeechRecognizeCall.html):
use speech1::RecognizeRequest;
// As the method needs a request, you would usually fill it with the desired information
// into the respective structure. Some of the parts shown here might not be applicable !
// Values shown here are possibly random and not representative !
let mut req = RecognizeRequest::default();
// You can configure optional parameters by calling the respective setters at will, and
// execute the final call using `doit()`.
// Values shown here are possibly random and not representative !
let result = hub.speech().recognize(req)
.doit();
UPDATE
Taking a step back, even simple examples provided on the website don't seem to run properly. Here is some sample very basic code:
pub mod speech_api_demo {
extern crate google_speech1 as speech1;
extern crate hyper;
extern crate hyper_rustls;
extern crate yup_oauth2 as oauth2;
use oauth2::{ApplicationSecret, Authenticator, DefaultAuthenticatorDelegate, MemoryStorage};
use speech1::Speech;
use speech1::{Error, Result};
use std::fs::File;
use std::io::Read;
#[derive(Deserialize, Serialize, Default)]
pub struct ConsoleApplicationSecret {
pub web: Option<ApplicationSecret>,
pub installed: Option<ApplicationSecret>,
}
pub fn speech_sample_demo() {
/*
Custom code to generate application secret
*/
let mut file =
File::open("C:\\Users\\YOURNAME\\.google-service-cli\\speech1-secret.json").unwrap();
let mut data = String::new();
file.read_to_string(&mut data).unwrap();
use serde_json as json;
let my_console_secret = json::from_str::<ConsoleApplicationSecret>(&data);
assert!(my_console_secret.is_ok());
let unwrappedConsoleSecret = my_console_secret.unwrap();
assert!(unwrappedConsoleSecret.installed.is_some() && unwrappedConsoleSecret.web.is_none());
let secret: ApplicationSecret = unwrappedConsoleSecret.installed.unwrap();
/*
Custom code to generate application secret - END
*/
// Instantiate the authenticator. It will choose a suitable authentication flow for you,
// unless you replace `None` with the desired Flow.
// Provide your own `AuthenticatorDelegate` to adjust the way it operates and get feedback about
// what's going on. You probably want to bring in your own `TokenStorage` to persist tokens and
// retrieve them from storage.
let auth = Authenticator::new(
&secret,
DefaultAuthenticatorDelegate,
hyper::Client::with_connector(hyper::net::HttpsConnector::new(
hyper_rustls::TlsClient::new(),
)),
<MemoryStorage as Default>::default(),
None,
);
let mut hub = Speech::new(
hyper::Client::with_connector(hyper::net::HttpsConnector::new(
hyper_rustls::TlsClient::new(),
)),
auth,
);
let result = hub.operations().get("name").doit();
match result {
Err(e) => match e {
// The Error enum provides details about what exactly happened.
// You can also just use its `Debug`, `Display` or `Error` traits
Error::HttpError(_)
| Error::MissingAPIKey
| Error::MissingToken(_)
| Error::Cancelled
| Error::UploadSizeLimitExceeded(_, _)
| Error::Failure(_)
| Error::BadRequest(_)
| Error::FieldClash(_)
| Error::JsonDecodeError(_, _) => (println!("{}", e)),
},
Ok(res) => println!("Success: {:?}", res),
}
}
}
Running this code (calling speech_sample_demo) gives the following error:
Token retrieval failed with error: Invalid Scope: 'no description
provided'
I also tried some very ugly code to force the scope into the request, but it did not make any difference. I am having a hard time understanding what this error means. Am I missing something in my request or is it something else getting in the way at the other end? Or maybe that api code library is just broken?
Please also note that client id and client secret provided by default don't work anymore, when I was using those it would say that account is deleted.
I then set up an OAuth 2.0 client and generated the json file which I copied over to default location and then started getting the error above. Maybe it is just me not setting Google Api account properly, but in any case would be great if someone else could try it out to see if I am the only one having those issues.
Once I get over running such a simple request, I have some more code ready to be tested that sends over an audio file, but for now it fails very early on in the process.
The error you get originates from here and means that the OAuth scope you used when generating your credentials file doesn't allow you to access the Google speech API. So the problem is not in your Rust code, but instead in the script you used to generate your OAuth access tokens.
Basically, this means that when you generated your OAuth json file, you requested access to the Google API in a general way, but you didn't say which specific APIs you meant to use. According to this document, you need to request access to the https://www.googleapis.com/auth/cloud-platform scope.
You are missing the flow param to Authenticator. This is how you get the access token. You create an Enum using FlowType.
example:
use oauth2::{ApplicationSecret, Authenticator, DefaultAuthenticatorDelegate, MemoryStorage,FlowType};
let Flo = FlowType::InstalledInteractive;
let auth = Authenticator::new(
&secret,
DefaultAuthenticatorDelegate,
hyper::Client::with_connector(hyper::net::HttpsConnector::new(
hyper_rustls::TlsClient::new(),
)),
<MemoryStorage as Default>::default(),
None,)
See here: https://docs.rs/yup-oauth2/1.0.3/yup_oauth2/enum.FlowType.html
Not exactly easy to figure out.
I made this work via service accounts by doing this
let https = hyper_rustls::HttpsConnectorBuilder::new()
.with_native_roots()
.https_only()
.enable_http1()
.build();
let service_account_key: oauth2::ServiceAccountKey = oauth2::read_service_account_key(
&"PATH_TO_SERVICE_ACCOUNT.json".to_string(),
)
.await
.unwrap();
let auth = oauth2::ServiceAccountAuthenticator::builder(service_account_key)
.build()
.await
.unwrap();
let hub = Speech::new(hyper::Client::builder().build(https), auth);

Resources