I have a requirement that I want to list all the accounts and then write all the credentials in my ~/.aws/credentials file. Fir this I am using boto3 in the following way
import boto3
client = boto3.client('organizations')
response = client.list_accounts(
NextToken='string',
MaxResults=123
)
print(response)
This fails with the following error
botocore.exceptions.ClientError: An error occurred (ExpiredTokenException) when calling the ListAccounts operation: The security token included in the request is expired
The question is , which token is it looking at? And if I want information about all accounts what credentials should I be using in the credentials file or the config file?
You can use boto3 paginators and pages.
Get an organizations object by using an aws configuration profile in the master account:
session = boto3.session.Session(profile_name=master_acct)
client = session.client('sts')
org = session.client('organizations')
Then use the org object to get a paginator.
paginator = org.get_paginator('list_accounts')
page_iterator = paginator.paginate()
Then iterate through every page of accounts.
for page in page_iterator:
for acct in page['Accounts']:
print(acct) # print the account
I'm not sure what you mean about "getting credentials". You can't get someone else's credentials. What you can do is list users, and if you want then list their access keys. That would require you to assume a role in each of the member accounts.
From within the above section, you are already inside a for-loop of each member account. You could do something like this:
id = acct['Id']
role_info = {
'RoleArn': f'arn:aws:iam::{id}:role/OrganizationAccountAccessRole',
'RoleSessionName': id
}
credentials = client.assume_role(**role_info)
member_session = boto3.session.Session(
aws_access_key_id=credentials['Credentials']['AccessKeyId'],
aws_secret_access_key=credentials['Credentials']['SecretAccessKey'],
aws_session_token=credentials['Credentials']['SessionToken'],
region_name='us-east-1'
)
However please note, that the role specified OrganizationAccountAccessRole needs to actually be present in every account, and your user in the master account needs to have the privileges to assume this role.
Once your prerequisites are setup, you will be iterating through every account, and in each account using member_session to access boto3 resources in that account.
Related
Need to obtain the list of all tokens in organization.
Used the token to make a call to https://vssps.dev.azure.com/{organization}/_apis/tokens/pats?api-version=6.1-preview.1
My permission in DevOps are set as the Collection Administrator.
Received response was:
{“$id”:“1”,“innerException”:null,“message”:“The requested operation is not allowed.”,“typeName”:“Microsoft.TeamFoundation.Framework.Server.InvalidAccessException, Microsoft.TeamFoundation.Framework.Server”,“typeKey”:“InvalidAccessException”,“errorCode”:0,“eventId”:3000}
Is there some lack of permissions or do I need to set up something else to get list of tokens in organization?
You don't mention how you get your token, and criteria for authentication flow but I will share my adventure that started similarly yours.
I got your exact error while following this guide: https://learn.microsoft.com/en-gb/azure/devops/organizations/accounts/manage-personal-access-tokens-via-api?view=azure-devops
The token I got from that python code just didn't work.
Then I found this code instead: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/app-aad-token#--username-password-flow-programmatic
While using the same app registration from the link above, I copied my scope and tenantID from the dysfunctional code into this new code, and then go to your app registration --> authentication --> Allow public client flows to yes, see screenshot.
I ran the script after giving the credentials and now the token worked.
Dumping the code for future reference:
# Given the client ID and tenant ID for an app registered in Azure,
# along with an Azure username and password,
# provide an Azure AD access token and a refresh token.
# If the caller is not already signed in to Azure, the caller's
# web browser will prompt the caller to sign in first.
# pip install msal
from msal import PublicClientApplication
import sys
# You can hard-code the registered app's client ID and tenant ID here,
# along with the Azure username and password,
# or you can provide them as command-line arguments to this script.
client_id = '<client-id>'
tenant_id = '<tenant-id>'
username = '<username>'
password = '<password>'
# Do not modify this variable. It represents the programmatic ID for
# Azure Databricks along with the default scope of '/.default'.
scope = [ '2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default' ]
# Check for too few or too many command-line arguments.
if (len(sys.argv) > 1) and (len(sys.argv) != 5):
print("Usage: get-tokens-for-user.py <client ID> <tenant ID> <username> <password>")
exit(1)
# If the registered app's client ID and tenant ID along with the
# Azure username and password are provided as command-line variables,
# set them here.
if len(sys.argv) > 1:
client_id = sys.argv[1]
tenant_id = sys.argv[2]
username = sys.argv[3]
password = sys.argv[4]
app = PublicClientApplication(
client_id = client_id,
authority = "https://login.microsoftonline.com/" + tenant_id
)
acquire_tokens_result = app.acquire_token_by_username_password(
username = username,
password = password,
scopes = scope
)
if 'error' in acquire_tokens_result:
print("Error: " + acquire_tokens_result['error'])
print("Description: " + acquire_tokens_result['error_description'])
else:
print("Access token:\n")
print(acquire_tokens_result['access_token'])
print("\nRefresh token:\n")
print(acquire_tokens_result['refresh_token'])
I'm trying to access Azure Table Storage using the TableClient class, but I want to authenticate using AzureAD credentials via the browser popup.
I have tried 2 approaches and are sure I have things configured correctly in Azure, but I just keep getting
This request is not authorized to perform this operation using this
permission.
Here is test code 1 using MSAL
let app = PublicClientApplicationBuilder.Create("---registered app ID---")
.WithAuthority(AzureCloudInstance.AzurePublic, "---tennant id----" )
.WithDefaultRedirectUri()
.Build()
let! ar = app.AcquireTokenInteractive(["https://storage.azure.com/user_impersonation"]).ExecuteAsync()
let tokenCredential = { new TokenCredential() with
member x.GetTokenAsync(_,_) = task {return AccessToken(ar.AccessToken, ar.ExpiresOn)} |> ValueTask<AccessToken>
member x.GetToken(_,_) = AccessToken(ar.AccessToken, ar.ExpiresOn)}
let tc = new TableClient(Uri("https://--endpoint---.table.core.windows.net/"), "--Table--", tokenCredential)
and test 2 using Azure.Identity
let io = new InteractiveBrowserCredentialOptions(ClientId = "---registered app ID---", RedirectUri = Uri("https://login.microsoftonline.com/common/oauth2/nativeclient"))
let tc = new TableClient(Uri("https://--endpoint---.table.core.windows.net/"), "--Table--", new InteractiveBrowserCredential(io))
I have the app registered in Azure & I have added api permissions for Azure Storage, with Admin consent. My account is a Service Administrator for the tennant so I have full access to the storage account. I have scoured all the docs but just cant see what I'm missing.
To access table data using your Azure AD credentials, your user account should be assigned either Storage Table Data Contributor or Storage Table Data Reader role.
Please assign one of these roles to your user account and re-acquire the token. You should not get the error you are getting then.
I want to build a web application. Clients can use the web application to read their google Adwrods accounts information( campagins or budgets ).
First, I use oath2 get client's refresh_token and access_token.
Using the refresh_token, I can get all adwords id under the client by (https://github.com/googleads/google-ads-ruby)
client = Google::Ads::GoogleAds::GoogleAdsClient.new do |config|
config.client_id = "client_id"
config.client_secret = "client_secret"
config.refresh_token = "refresh_token"
config.login_customer_id = "XXX-XXX-XXXX"
config.developer_token = "XXXXXXXXXXXXXXXX"
end
accessible_customers = client.service.customer.list_accessible_customers().resource_names
When I want to get client Adword account information,
resource_name = client.path.customer("XXXXXXXX")
customer = client.service.customer.get_customer(resource_name: resource_name)
I get "GRPC::Unauthenticated: 16:Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential", but the config file can't let me set access_token.
So, where can i set client's access_token, or which step i missed?
The error is telling you that client has not been set up correctly. It could be a bunch of issues from Google account to wrong information. I would check and make sure all the info you are passing in Google::Ads::GoogleAds::GoogleAdsClient.new is correct.
Also, you only need to pass 'login_customer_id' for manager accounts only, it doesn't sound like you are a manager account.
From https://github.com/googleads/google-ads-ruby/blob/master/google_ads_config.rb
# Required for manager accounts only: Specify the login customer ID used to
# authenticate API calls. This will be the customer ID of the authenticated
# manager account. If you need to use different values for this field, then
# make sure fetch a new copy of the service after each time you change the
# value.
# c.login_customer_id = 'INSERT_LOGIN_CUSTOMER_ID_HERE'
So before showing my code, let me explain what steps I took to 'properly' set up service account environment.
In google developer console, created service account. (this produced Client ID (which is a long number), Service account (xxxxx#xxxx.iam.gserviceaccount.com), and private key which I downloaded in P12.
In Admin console, put the client ID with appropriate scope. In my case the scopes I added is https://www.googleapis.com/auth/admin.directory.group.readonly and https://www.googleapis.com/auth/admin.directory.group.member.readonly.
In my code, correctly set up private key path and other environments.
def getDirectoryService: Directory = {
val httpTransport: HttpTransport = new NetHttpTransport()
val jsonFactory: JacksonFactory = new JacksonFactory()
val credential: GoogleCredential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId("xxxxx#xxxx.iam.gserviceaccount.com")
.setServiceAccountScopes(util.Arrays.asList(DirectoryScopes.ADMIN_DIRECTORY_GROUP_READONLY, DirectoryScopes.ADMIN_DIRECTORY_GROUP_MEMBER_READONLY))
.setServiceAccountUser("admin#domain.com")
.setServiceAccountPrivateKeyFromP12File(
new java.io.File("/pathToKey/privatekey.p12"))
.build()
val service: Directory = new Directory.Builder(httpTransport, jsonFactory, null)
.setHttpRequestInitializer(credential).build()
service
}
And then I attempt to execute something like this:
service.groups().list().execute()
or
service.groups().list("domain.com").execute()
This code would result in,
com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
"error" : "invalid_grant"
}
at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:287)
at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:307)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:384)
at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:489)
at com.google.api.client.auth.oauth2.Credential.intercept(Credential.java:217)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:868)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.company.project.GoogleServiceProvider.getGroups(GoogleServiceProvider.scala:81)
at com.company.project.ProjectHandler.handle(ProjectHandler.scala:110)
at com.company.common.web.DispatcherServlet.service(DispatcherServlet.scala:40)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1174)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1106)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:524)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
What could have I done wrong? I have been searching solution for past two days, and tried many things. One of the solutions I am not still not sure of is ntp syncing (as in how to exactly sync server time to ntp).
Any adivce would be very helpful, thank you!
UPDATE: I also made sure to activate the Admin Directory SDK, and enabled the Domain-Wide Delegation on developer's console.
UPDATE #2: I forgot to mention that, the admin account is not the owner of the project itself. So basically, I am a member of a domain, and I created a project, so I am the only owner of the project and the service account.(I am not the admin). But should an admin be owner of the project and create service account in order for this to work properly???
Ok, my problem was that in setServiceAccountUser I put admin group email address, not the actual user account. Apparently, it doesn't allow putting in group email (alias) address into setServiceAccountUser.
So after putting in an actual user account with admin privilege, it seems to be working.
I still wonder what would be the best practice though. As in, should I create a separate user account with admin privilege just for the project? I definitely don't want to just put in an admin account email address in my code.
Gmail API fails for one domain when retrieving messages with this error:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 OK
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Delegation denied for <user email>",
"reason" : "forbidden"
} ],
"message" : "Delegation denied for <user email>"
}
I am using OAuth 2.0 and Google Apps Domain-Wide delegation of authority to access the user data. The domain has granted data access rights to the application.
Seems like best thing to do is to just always have userId="me" in your requests. That tells the API to just use the authenticated user's mailbox--no need to rely on email addresses.
I had the same issue before, the solution is super tricky, you need to impersonate the person you need to access gmail content first, then use userId='me' to run the query. It works for me.
here is some sample code:
users = # coming from directory service
for user in users:
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
####IMPORTANT######
credentials_delegated = credentials.with_subject(user['primaryEmail'])
gmail_service = build('gmail', 'v1', credentials=credentials_delegated)
results = gmail_service.users().labels().list(userId='me').execute()
labels = results.get('labels', [])
for label in labels:
print(label['name'])
Our users had migrated into a domain and their account had aliases attached to it. We needed to default the SendAs address to one of the imported aliases and want a way to automate it. The Gmail API looked like the solution, but our privileged user with roles to make changes to the accounts was not working - we kept seeing the "Delegation denied for " 403 error.
Here is a PHP example of how we were able to list their SendAs settings.
<?PHP
//
// Description:
// List the user's SendAs addresses.
//
// Documentation:
// https://developers.google.com/gmail/api/v1/reference/users/settings/sendAs
// https://developers.google.com/gmail/api/v1/reference/users/settings/sendAs/list
//
// Local Path:
// /path/to/api/vendor/google/apiclient-services/src/Google/Service/Gmail.php
// /path/to/api/vendor/google/apiclient-services/src/Google/Service/Gmail/Resource/UsersSettingsSendAs.php
//
// Version:
// Google_Client::LIBVER == 2.1.1
//
require_once $API_PATH . '/path/to/google-api-php-client/vendor/autoload.php';
date_default_timezone_set('America/Los_Angeles');
// this is the service account json file used to make api calls within our domain
$serviceAccount = '/path/to/service-account-with-domain-wide-delagation.json';
putenv('GOOGLE_APPLICATION_CREDENTIALS=' . $serviceAccount );
$userKey = 'someuser#my.domain';
// In the Admin Directory API, we may do things like create accounts with
// an account having roles to make changes. With the Gmail API, we cannot
// use those accounts to make changes. Instead, we impersonate
// the user to manage their account.
$impersonateUser = $userKey;
// these are the scope(s) used.
define('SCOPES', implode(' ', array( Google_Service_Gmail::GMAIL_SETTINGS_BASIC ) ) );
$client = new Google_Client();
$client->useApplicationDefaultCredentials(); // loads whats in that json service account file.
$client->setScopes(SCOPES); // adds the scopes
$client->setSubject($impersonateUser); // account authorized to perform operation
$gmailObj = new Google_Service_Gmail($client);
$res = $gmailObj->users_settings_sendAs->listUsersSettingsSendAs($userKey);
print_r($res);
?>
I wanted to access the emails of fresh email id/account but what happened was, the recently created folder with '.credentials' containing a JSON was associated with the previous email id/account which I tried earlier. The access token and other parameters present in JSON are not associated with new email id/account. So, in order make it run you just have to delete the '.credentails' folder and run the program again. Now, the program opens the browser and asks you to give permissions.
To delete the folder containing files in python
import shutil
shutil.rmtree("path of the folder to be deleted")
you may add this at the end of the program
Recently I started exploring Gmail API and I am following the same approach as Guo mentioned. However, it is going to take of time and too many calls when we the number of users or more. After domain wide delegation my expectation was admin id will be able to access the delegated inboxes, but seems like we need to create service for each user.