So before showing my code, let me explain what steps I took to 'properly' set up service account environment.
In google developer console, created service account. (this produced Client ID (which is a long number), Service account (xxxxx#xxxx.iam.gserviceaccount.com), and private key which I downloaded in P12.
In Admin console, put the client ID with appropriate scope. In my case the scopes I added is https://www.googleapis.com/auth/admin.directory.group.readonly and https://www.googleapis.com/auth/admin.directory.group.member.readonly.
In my code, correctly set up private key path and other environments.
def getDirectoryService: Directory = {
val httpTransport: HttpTransport = new NetHttpTransport()
val jsonFactory: JacksonFactory = new JacksonFactory()
val credential: GoogleCredential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId("xxxxx#xxxx.iam.gserviceaccount.com")
.setServiceAccountScopes(util.Arrays.asList(DirectoryScopes.ADMIN_DIRECTORY_GROUP_READONLY, DirectoryScopes.ADMIN_DIRECTORY_GROUP_MEMBER_READONLY))
.setServiceAccountUser("admin#domain.com")
.setServiceAccountPrivateKeyFromP12File(
new java.io.File("/pathToKey/privatekey.p12"))
.build()
val service: Directory = new Directory.Builder(httpTransport, jsonFactory, null)
.setHttpRequestInitializer(credential).build()
service
}
And then I attempt to execute something like this:
service.groups().list().execute()
or
service.groups().list("domain.com").execute()
This code would result in,
com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
"error" : "invalid_grant"
}
at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:287)
at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:307)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:384)
at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:489)
at com.google.api.client.auth.oauth2.Credential.intercept(Credential.java:217)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:868)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.company.project.GoogleServiceProvider.getGroups(GoogleServiceProvider.scala:81)
at com.company.project.ProjectHandler.handle(ProjectHandler.scala:110)
at com.company.common.web.DispatcherServlet.service(DispatcherServlet.scala:40)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1174)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1106)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:524)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
What could have I done wrong? I have been searching solution for past two days, and tried many things. One of the solutions I am not still not sure of is ntp syncing (as in how to exactly sync server time to ntp).
Any adivce would be very helpful, thank you!
UPDATE: I also made sure to activate the Admin Directory SDK, and enabled the Domain-Wide Delegation on developer's console.
UPDATE #2: I forgot to mention that, the admin account is not the owner of the project itself. So basically, I am a member of a domain, and I created a project, so I am the only owner of the project and the service account.(I am not the admin). But should an admin be owner of the project and create service account in order for this to work properly???
Ok, my problem was that in setServiceAccountUser I put admin group email address, not the actual user account. Apparently, it doesn't allow putting in group email (alias) address into setServiceAccountUser.
So after putting in an actual user account with admin privilege, it seems to be working.
I still wonder what would be the best practice though. As in, should I create a separate user account with admin privilege just for the project? I definitely don't want to just put in an admin account email address in my code.
Related
I'm using terraform to write azure policy as code
I found two problems
1 I can't seem to use log analytics workspace that is on a different subscription, within same subscription, it's fine
2 For policies that needs managed identity, I can't seem to assign correct rights to it.
resource "azurerm_policy_assignment" "Enable_Azure_Monitor_for_VMs" {
name = "Enable Azure Monitor for VMs"
scope = data.azurerm_subscription.current.id
policy_definition_id = "/providers/Microsoft.Authorization/policySetDefinitions/55f3eceb-5573-4f18-9695-226972c6d74a"
description = "Enable Azure Monitor for the virtual machines (VMs) in the specified scope (management group, subscription or resource group). Takes Log Analytics workspace as parameter."
display_name = "Enable Azure Monitor for VMs"
location = var.location
metadata = jsonencode(
{
"category" : "General"
})
parameters = jsonencode({
"logAnalytics_1" : {
"value" : var.log_analytics_workspace_ID
}
})
identity {
type = "SystemAssigned"
}
}
resource "azurerm_role_assignment" "vm_policy_msi_assignment" {
scope = azurerm_policy_assignment.Enable_Azure_Monitor_for_VMs.scope
role_definition_name = "Contributor"
principal_id = azurerm_policy_assignment.Enable_Azure_Monitor_for_VMs.identity[0].principal_id
}
for var.log_analytics_workspace_ID, if i use the workspace id that is in the same subscription as the policy, it would work fine. but If I use a workspace ID from a different subscription, after deployment, the workspace field will be blank.
also for
resource "azurerm_role_assignment" "vm_policy_msi_assignment"
, I have already given myself user access management role, but after deployment, "This identity currently has the following permissions:" is still blank?
I got an answer to my own question:)
1 this is not something designed well in Azure, I recon.
MS states "a Managed Identity (MSI) is created for each policy assignment that contains DeployIfNotExists effects in the definitions. The required permission for the target assignment scope is managed automatically. However, if the remediation tasks need to interact with resources outside of the assignment scope, you will need to manually configure the required permissions."
which means, the system generated managed identity which needs access in log analytics workspace in another subscription need to be manually with log analytics workspace contributor rights
Also since you can't user user generated managed ID, you can't pre-populate this.
so if you want to to achieve in terraform, it seems you have to run policy assignment twice, the first time is just to get ID, then manual ( or via script) to assign permission, then run policy assignment again to point to the resource..
2 The ID was actually given the contributor rights, you just have to go into sub RBAC to see it.
I'm trying use a service account with google's api to work with classroom data with the goal of synchronizing our web service for schools with the google classroom data.
I have delegated domain wide authority to the service account and have activated the Google Classroom API. I have downloaded the json Key file used below.
I have added https://www.googleapis.com/auth/classroom.courses to the scope of the service account.
My test code in app/models/g_service.rb:
class GService
require 'google/apis/classroom_v1'
def get_course
authorizer = Google::Auth::ServiceAccountCredentials.make_creds(
json_key_io: File.open('/Users/jose/Downloads/skt1-301603-4a655caa8963.json'),
scope: [ Google::Apis::ClassroomV1::AUTH_CLASSROOM_COURSES ]
)
authorizer.fetch_access_token!
service = Google::Apis::ClassroomV1::ClassroomService.new
service.authorization = authorizer
puts "\n service\n #{service.inspect}"
response = service.get_course( '99999' )
puts "\n response \n#{response.inspect}"
end
end
The results in the console are:
>> GService.new.get_course
service
#<Google::Apis::ClassroomV1::ClassroomService:0x007fe1cff98338 #root_url="https://classroom.googleapis.com/", #base_path="", #upload_path="upload/", #batch_path="batch", #client_options=#<struct Google::Apis::ClientOptions application_name="unknown", application_version="0.0.0", proxy_url=nil, open_timeout_sec=nil, read_timeout_sec=nil, send_timeout_sec=nil, log_http_requests=false, transparent_gzip_decompression=true>, #request_options=#<struct Google::Apis::RequestOptions authorization=#<Google::Auth::ServiceAccountCredentials:0x0xxxxxxx #project_id="sssssssss", #authorization_uri=nil, #token_credential_uri=#<Addressable::URI:0x000000000 URI:https://www.googleapis.com/oauth2/v4/token>, #client_id=nil, #client_secret=nil, #code=nil, #expires_at=2021-01-13 20:56:46 -0800, #issued_at=2021-01-13 19:56:47 -0800, #issuer="xxxxxxx.iam.gserviceaccount.com", #password=nil, #principal=nil, #redirect_uri=nil, #scope=["https://www.googleapis.com/auth/classroom.courses"], #state=nil, #username=nil, #access_type=:offline, #expiry=60, #audience="https://www.googleapis.com/oauth2/v4/token", #signing_key=#<OpenSSL::PKey::RSA:0xxxxxxxxx>, #extension_parameters={}, #additional_parameters={}, #connection_info=nil, #grant_type=nil, #refresh_token=nil, #access_token="-------------------------------------------------------------------------------------->, retries=0, header=nil, normalize_unicode=false, skip_serialization=false, skip_deserialization=false, api_format_version=nil, use_opencensus=true>>
Google::Apis::ClientError: forbidden: The caller does not have permission
It appears everything is working fine until the service.get_course('99999') call.
I've tested this call using the https://developers.google.com/classroom/reference/rest/v1/courses/get online tool and it works fine.
I've poured over the documentation but have been unable to resolve this.
Can anybody please let me know what I am missing?
I'm running rails 3.2 and ruby 2.1
Considering the error you're getting, I think you are not impersonating any account.
The purpose of domain-wide delegation is that the service account can impersonate a regular account in your domain, but in order to do that, you have to specify which account you want to impersonate. Otherwise, you are calling the service account by itself, and it doesn't matter that you've enabled domain-wide delegation for it.
In the Ruby library, you can specify that using the :sub parameter, as shown in the section Preparing to make an authorized API call at the library docs:
authorizer.sub = "<email-address-to-impersonate>"
Note:
Make sure the account you impersonate has access to this course, otherwise you'll get the same error.
Related:
Delegating domain-wide authority to the service account
Google API Server-to-Server Communication not working (Ruby implementation)
I have a requirement that I want to list all the accounts and then write all the credentials in my ~/.aws/credentials file. Fir this I am using boto3 in the following way
import boto3
client = boto3.client('organizations')
response = client.list_accounts(
NextToken='string',
MaxResults=123
)
print(response)
This fails with the following error
botocore.exceptions.ClientError: An error occurred (ExpiredTokenException) when calling the ListAccounts operation: The security token included in the request is expired
The question is , which token is it looking at? And if I want information about all accounts what credentials should I be using in the credentials file or the config file?
You can use boto3 paginators and pages.
Get an organizations object by using an aws configuration profile in the master account:
session = boto3.session.Session(profile_name=master_acct)
client = session.client('sts')
org = session.client('organizations')
Then use the org object to get a paginator.
paginator = org.get_paginator('list_accounts')
page_iterator = paginator.paginate()
Then iterate through every page of accounts.
for page in page_iterator:
for acct in page['Accounts']:
print(acct) # print the account
I'm not sure what you mean about "getting credentials". You can't get someone else's credentials. What you can do is list users, and if you want then list their access keys. That would require you to assume a role in each of the member accounts.
From within the above section, you are already inside a for-loop of each member account. You could do something like this:
id = acct['Id']
role_info = {
'RoleArn': f'arn:aws:iam::{id}:role/OrganizationAccountAccessRole',
'RoleSessionName': id
}
credentials = client.assume_role(**role_info)
member_session = boto3.session.Session(
aws_access_key_id=credentials['Credentials']['AccessKeyId'],
aws_secret_access_key=credentials['Credentials']['SecretAccessKey'],
aws_session_token=credentials['Credentials']['SessionToken'],
region_name='us-east-1'
)
However please note, that the role specified OrganizationAccountAccessRole needs to actually be present in every account, and your user in the master account needs to have the privileges to assume this role.
Once your prerequisites are setup, you will be iterating through every account, and in each account using member_session to access boto3 resources in that account.
We are trying to query shifts in the Microsoft Graph API using a C# app, now that StaffHub got deprecated , in the past we were getting an Unknown Error which looked like a permissions issue.
In the docs I noticed permissions for Schedule.ReadAll and Schedule.ReadWriteAll so I added them to the application permissions in our App Registration in Azure.
Now when we send the request to https://graph.microsoft.com/beta/teams/{teamid}/schedule we get this error:
Microsoft.Graph.ServiceException: 'Code: Forbidden Message: {"error":{"code":"Forbidden","message":"MS-APP-ACTS-AS header needs to be set for application context requests.","details":[],"innererror":{"code":"MissingUserIdHeaderInAppContext"}}}
The documentation says the Schedule permissions are in private preview, are these required for querying a schedule & shifts, and if so, is it possible to request access to the private preview?
I'm in the same situation. It's possible to request private preview access (we have), but I'm guessing that it's primarily granted to Microsoft partners or at least have a connection at Microsoft.
The workaround for me has been getting access on behalf of a user. It does however require the user to enter username and password in order to get an access token, so it might not be a perfect solution for you. But it works. You need to add (and, I believe, grant admin consent for) delegated permissions for this to work, either Group.Read.All or Group.ReadWrite.All.
Edit:
I've got it working now. We have private preview access, so I'm not sure this will help you unless you do too, but as I understand it will be available eventually. Given your question, I presume you already have an access token.
Add MS-APP-ACT-AS as a header with the user ID of the user you want the Graph client to act as.
If you're using the Graph SDK for .NET Core you can just add a header to the authentication provider:
public IAuthenticationProvider GetAuthenticationProviderForActingAsUser(string userId, string accessToken)
{
return new DelegateAuthenticationProvider(
requestMessage =>
{
requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
// Get event times in the current time zone.
requestMessage.Headers.Add("Prefer", "outlook.timezone=\"" + TimeZoneInfo.Local.Id + "\"");
requestMessage.Headers.Add("MS-APP-ACTS-AS", userId);
return Task.CompletedTask;
});
}
Then you call the graph service client:
var authenticationProvider = GetAuthenticationProviderForActingAsUser(userId, accessToken);
var graphClient = new GraphServiceClient(authenticationProvider);
You should then be able to fetch the shifts:
var shifts = await graphClient.Teams[teamId].Schedule.Shifts
.Request()
.AddAsync(shift);
Gmail API fails for one domain when retrieving messages with this error:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 OK
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Delegation denied for <user email>",
"reason" : "forbidden"
} ],
"message" : "Delegation denied for <user email>"
}
I am using OAuth 2.0 and Google Apps Domain-Wide delegation of authority to access the user data. The domain has granted data access rights to the application.
Seems like best thing to do is to just always have userId="me" in your requests. That tells the API to just use the authenticated user's mailbox--no need to rely on email addresses.
I had the same issue before, the solution is super tricky, you need to impersonate the person you need to access gmail content first, then use userId='me' to run the query. It works for me.
here is some sample code:
users = # coming from directory service
for user in users:
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
####IMPORTANT######
credentials_delegated = credentials.with_subject(user['primaryEmail'])
gmail_service = build('gmail', 'v1', credentials=credentials_delegated)
results = gmail_service.users().labels().list(userId='me').execute()
labels = results.get('labels', [])
for label in labels:
print(label['name'])
Our users had migrated into a domain and their account had aliases attached to it. We needed to default the SendAs address to one of the imported aliases and want a way to automate it. The Gmail API looked like the solution, but our privileged user with roles to make changes to the accounts was not working - we kept seeing the "Delegation denied for " 403 error.
Here is a PHP example of how we were able to list their SendAs settings.
<?PHP
//
// Description:
// List the user's SendAs addresses.
//
// Documentation:
// https://developers.google.com/gmail/api/v1/reference/users/settings/sendAs
// https://developers.google.com/gmail/api/v1/reference/users/settings/sendAs/list
//
// Local Path:
// /path/to/api/vendor/google/apiclient-services/src/Google/Service/Gmail.php
// /path/to/api/vendor/google/apiclient-services/src/Google/Service/Gmail/Resource/UsersSettingsSendAs.php
//
// Version:
// Google_Client::LIBVER == 2.1.1
//
require_once $API_PATH . '/path/to/google-api-php-client/vendor/autoload.php';
date_default_timezone_set('America/Los_Angeles');
// this is the service account json file used to make api calls within our domain
$serviceAccount = '/path/to/service-account-with-domain-wide-delagation.json';
putenv('GOOGLE_APPLICATION_CREDENTIALS=' . $serviceAccount );
$userKey = 'someuser#my.domain';
// In the Admin Directory API, we may do things like create accounts with
// an account having roles to make changes. With the Gmail API, we cannot
// use those accounts to make changes. Instead, we impersonate
// the user to manage their account.
$impersonateUser = $userKey;
// these are the scope(s) used.
define('SCOPES', implode(' ', array( Google_Service_Gmail::GMAIL_SETTINGS_BASIC ) ) );
$client = new Google_Client();
$client->useApplicationDefaultCredentials(); // loads whats in that json service account file.
$client->setScopes(SCOPES); // adds the scopes
$client->setSubject($impersonateUser); // account authorized to perform operation
$gmailObj = new Google_Service_Gmail($client);
$res = $gmailObj->users_settings_sendAs->listUsersSettingsSendAs($userKey);
print_r($res);
?>
I wanted to access the emails of fresh email id/account but what happened was, the recently created folder with '.credentials' containing a JSON was associated with the previous email id/account which I tried earlier. The access token and other parameters present in JSON are not associated with new email id/account. So, in order make it run you just have to delete the '.credentails' folder and run the program again. Now, the program opens the browser and asks you to give permissions.
To delete the folder containing files in python
import shutil
shutil.rmtree("path of the folder to be deleted")
you may add this at the end of the program
Recently I started exploring Gmail API and I am following the same approach as Guo mentioned. However, it is going to take of time and too many calls when we the number of users or more. After domain wide delegation my expectation was admin id will be able to access the delegated inboxes, but seems like we need to create service for each user.