User manage Service Account to deploy CloudRun instance - google-cloud-run

I need your help please. I am not able to find out what I am missing. I created user managed SA and provided roles
roles/run.admin
roles/iam.serviceAccountUser
but somehow I am not able to see it when creating service:
I also added impersonation to default compute SA.
I am pushing changes via terraform:
resource "google_service_account" "sa-deployer" {
project = local.project_id
account_id = "${local.env}-sa-deployer-tf"
display_name = "Service Account to deploy CloudRun instance"
}
resource "google_service_account_iam_member" "gce-default-account-iam" {
service_account_id = data.google_compute_default_service_account.default.name
role = "roles/iam.serviceAccountUser"
member = "serviceAccount:${google_service_account.sa-deployer.email}"
depends_on = [
google_service_account.sa-deployer
]
}
resource "google_project_iam_binding" "sa-deployer-run-admin" {
project = local.project_id
role = "roles/run.admin"
members = [
"serviceAccount:${google_service_account.sa-deployer.email}",
]
depends_on = [
google_service_account.sa-deployer
]
}
resource "google_project_iam_binding" "sa-deployer-build-admin" {
project = local.project_id
role = "roles/cloudbuild.builds.builder"
members = [
"serviceAccount:${google_service_account.sa-deployer.email}",
]
depends_on = [
google_service_account.sa-deployer
]
}

The current user must be serviceAccountUser to be able to list the service account on the project.

To allow a user to manage service accounts, grant one of the following roles:
Service Account User (roles/iam.serviceAccountUser): Includes permissions to list service accounts, get details about a service account, and impersonate a service account.
Service Account Admin (roles/iam.serviceAccountAdmin): Includes permissions to list service accounts and get details about a service account. Also includes permissions to create, update, and delete service accounts, and to view or change the IAM policy on a service account.
To learn more about these roles, see Service Accounts roles.
IAM basic roles(roles/viewer, roles/editor) also contain permissions to manage service accounts. You should not grant basic roles in a production environment, but you can grant them in a development or test environment.
For more information refer to the following documentations.
Permissions to manage service accounts.
Listing service accounts.

Related

Azure tableclient in console app using interactive authentication

I'm trying to access Azure Table Storage using the TableClient class, but I want to authenticate using AzureAD credentials via the browser popup.
I have tried 2 approaches and are sure I have things configured correctly in Azure, but I just keep getting
This request is not authorized to perform this operation using this
permission.
Here is test code 1 using MSAL
let app = PublicClientApplicationBuilder.Create("---registered app ID---")
.WithAuthority(AzureCloudInstance.AzurePublic, "---tennant id----" )
.WithDefaultRedirectUri()
.Build()
let! ar = app.AcquireTokenInteractive(["https://storage.azure.com/user_impersonation"]).ExecuteAsync()
let tokenCredential = { new TokenCredential() with
member x.GetTokenAsync(_,_) = task {return AccessToken(ar.AccessToken, ar.ExpiresOn)} |> ValueTask<AccessToken>
member x.GetToken(_,_) = AccessToken(ar.AccessToken, ar.ExpiresOn)}
let tc = new TableClient(Uri("https://--endpoint---.table.core.windows.net/"), "--Table--", tokenCredential)
and test 2 using Azure.Identity
let io = new InteractiveBrowserCredentialOptions(ClientId = "---registered app ID---", RedirectUri = Uri("https://login.microsoftonline.com/common/oauth2/nativeclient"))
let tc = new TableClient(Uri("https://--endpoint---.table.core.windows.net/"), "--Table--", new InteractiveBrowserCredential(io))
I have the app registered in Azure & I have added api permissions for Azure Storage, with Admin consent. My account is a Service Administrator for the tennant so I have full access to the storage account. I have scoured all the docs but just cant see what I'm missing.
To access table data using your Azure AD credentials, your user account should be assigned either Storage Table Data Contributor or Storage Table Data Reader role.
Please assign one of these roles to your user account and re-acquire the token. You should not get the error you are getting then.

can't use log analytics workspace in a different subscription? terraform azurerm policy assignment

I'm using terraform to write azure policy as code
I found two problems
1 I can't seem to use log analytics workspace that is on a different subscription, within same subscription, it's fine
2 For policies that needs managed identity, I can't seem to assign correct rights to it.
resource "azurerm_policy_assignment" "Enable_Azure_Monitor_for_VMs" {
name = "Enable Azure Monitor for VMs"
scope = data.azurerm_subscription.current.id
policy_definition_id = "/providers/Microsoft.Authorization/policySetDefinitions/55f3eceb-5573-4f18-9695-226972c6d74a"
description = "Enable Azure Monitor for the virtual machines (VMs) in the specified scope (management group, subscription or resource group). Takes Log Analytics workspace as parameter."
display_name = "Enable Azure Monitor for VMs"
location = var.location
metadata = jsonencode(
{
"category" : "General"
})
parameters = jsonencode({
"logAnalytics_1" : {
"value" : var.log_analytics_workspace_ID
}
})
identity {
type = "SystemAssigned"
}
}
resource "azurerm_role_assignment" "vm_policy_msi_assignment" {
scope = azurerm_policy_assignment.Enable_Azure_Monitor_for_VMs.scope
role_definition_name = "Contributor"
principal_id = azurerm_policy_assignment.Enable_Azure_Monitor_for_VMs.identity[0].principal_id
}
for var.log_analytics_workspace_ID, if i use the workspace id that is in the same subscription as the policy, it would work fine. but If I use a workspace ID from a different subscription, after deployment, the workspace field will be blank.
also for
resource "azurerm_role_assignment" "vm_policy_msi_assignment"
, I have already given myself user access management role, but after deployment, "This identity currently has the following permissions:" is still blank?
I got an answer to my own question:)
1 this is not something designed well in Azure, I recon.
MS states "a Managed Identity (MSI) is created for each policy assignment that contains DeployIfNotExists effects in the definitions. The required permission for the target assignment scope is managed automatically. However, if the remediation tasks need to interact with resources outside of the assignment scope, you will need to manually configure the required permissions."
which means, the system generated managed identity which needs access in log analytics workspace in another subscription need to be manually with log analytics workspace contributor rights
Also since you can't user user generated managed ID, you can't pre-populate this.
so if you want to to achieve in terraform, it seems you have to run policy assignment twice, the first time is just to get ID, then manual ( or via script) to assign permission, then run policy assignment again to point to the resource..
2 The ID was actually given the contributor rights, you just have to go into sub RBAC to see it.

boto3 list all accounts in an organization

I have a requirement that I want to list all the accounts and then write all the credentials in my ~/.aws/credentials file. Fir this I am using boto3 in the following way
import boto3
client = boto3.client('organizations')
response = client.list_accounts(
NextToken='string',
MaxResults=123
)
print(response)
This fails with the following error
botocore.exceptions.ClientError: An error occurred (ExpiredTokenException) when calling the ListAccounts operation: The security token included in the request is expired
The question is , which token is it looking at? And if I want information about all accounts what credentials should I be using in the credentials file or the config file?
You can use boto3 paginators and pages.
Get an organizations object by using an aws configuration profile in the master account:
session = boto3.session.Session(profile_name=master_acct)
client = session.client('sts')
org = session.client('organizations')
Then use the org object to get a paginator.
paginator = org.get_paginator('list_accounts')
page_iterator = paginator.paginate()
Then iterate through every page of accounts.
for page in page_iterator:
for acct in page['Accounts']:
print(acct) # print the account
I'm not sure what you mean about "getting credentials". You can't get someone else's credentials. What you can do is list users, and if you want then list their access keys. That would require you to assume a role in each of the member accounts.
From within the above section, you are already inside a for-loop of each member account. You could do something like this:
id = acct['Id']
role_info = {
'RoleArn': f'arn:aws:iam::{id}:role/OrganizationAccountAccessRole',
'RoleSessionName': id
}
credentials = client.assume_role(**role_info)
member_session = boto3.session.Session(
aws_access_key_id=credentials['Credentials']['AccessKeyId'],
aws_secret_access_key=credentials['Credentials']['SecretAccessKey'],
aws_session_token=credentials['Credentials']['SessionToken'],
region_name='us-east-1'
)
However please note, that the role specified OrganizationAccountAccessRole needs to actually be present in every account, and your user in the master account needs to have the privileges to assume this role.
Once your prerequisites are setup, you will be iterating through every account, and in each account using member_session to access boto3 resources in that account.

Google Admin Directory API, bad request 400 invalid_grant. (using service account)

So before showing my code, let me explain what steps I took to 'properly' set up service account environment.
In google developer console, created service account. (this produced Client ID (which is a long number), Service account (xxxxx#xxxx.iam.gserviceaccount.com), and private key which I downloaded in P12.
In Admin console, put the client ID with appropriate scope. In my case the scopes I added is https://www.googleapis.com/auth/admin.directory.group.readonly and https://www.googleapis.com/auth/admin.directory.group.member.readonly.
In my code, correctly set up private key path and other environments.
def getDirectoryService: Directory = {
val httpTransport: HttpTransport = new NetHttpTransport()
val jsonFactory: JacksonFactory = new JacksonFactory()
val credential: GoogleCredential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId("xxxxx#xxxx.iam.gserviceaccount.com")
.setServiceAccountScopes(util.Arrays.asList(DirectoryScopes.ADMIN_DIRECTORY_GROUP_READONLY, DirectoryScopes.ADMIN_DIRECTORY_GROUP_MEMBER_READONLY))
.setServiceAccountUser("admin#domain.com")
.setServiceAccountPrivateKeyFromP12File(
new java.io.File("/pathToKey/privatekey.p12"))
.build()
val service: Directory = new Directory.Builder(httpTransport, jsonFactory, null)
.setHttpRequestInitializer(credential).build()
service
}
And then I attempt to execute something like this:
service.groups().list().execute()
or
service.groups().list("domain.com").execute()
This code would result in,
com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
"error" : "invalid_grant"
}
at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:287)
at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:307)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:384)
at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:489)
at com.google.api.client.auth.oauth2.Credential.intercept(Credential.java:217)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:868)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.company.project.GoogleServiceProvider.getGroups(GoogleServiceProvider.scala:81)
at com.company.project.ProjectHandler.handle(ProjectHandler.scala:110)
at com.company.common.web.DispatcherServlet.service(DispatcherServlet.scala:40)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1174)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1106)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:524)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
What could have I done wrong? I have been searching solution for past two days, and tried many things. One of the solutions I am not still not sure of is ntp syncing (as in how to exactly sync server time to ntp).
Any adivce would be very helpful, thank you!
UPDATE: I also made sure to activate the Admin Directory SDK, and enabled the Domain-Wide Delegation on developer's console.
UPDATE #2: I forgot to mention that, the admin account is not the owner of the project itself. So basically, I am a member of a domain, and I created a project, so I am the only owner of the project and the service account.(I am not the admin). But should an admin be owner of the project and create service account in order for this to work properly???
Ok, my problem was that in setServiceAccountUser I put admin group email address, not the actual user account. Apparently, it doesn't allow putting in group email (alias) address into setServiceAccountUser.
So after putting in an actual user account with admin privilege, it seems to be working.
I still wonder what would be the best practice though. As in, should I create a separate user account with admin privilege just for the project? I definitely don't want to just put in an admin account email address in my code.

Gmail API returns 403 error code and "Delegation denied for <user email>"

Gmail API fails for one domain when retrieving messages with this error:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 OK
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Delegation denied for <user email>",
"reason" : "forbidden"
} ],
"message" : "Delegation denied for <user email>"
}
I am using OAuth 2.0 and Google Apps Domain-Wide delegation of authority to access the user data. The domain has granted data access rights to the application.
Seems like best thing to do is to just always have userId="me" in your requests. That tells the API to just use the authenticated user's mailbox--no need to rely on email addresses.
I had the same issue before, the solution is super tricky, you need to impersonate the person you need to access gmail content first, then use userId='me' to run the query. It works for me.
here is some sample code:
users = # coming from directory service
for user in users:
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
####IMPORTANT######
credentials_delegated = credentials.with_subject(user['primaryEmail'])
gmail_service = build('gmail', 'v1', credentials=credentials_delegated)
results = gmail_service.users().labels().list(userId='me').execute()
labels = results.get('labels', [])
for label in labels:
print(label['name'])
Our users had migrated into a domain and their account had aliases attached to it. We needed to default the SendAs address to one of the imported aliases and want a way to automate it. The Gmail API looked like the solution, but our privileged user with roles to make changes to the accounts was not working - we kept seeing the "Delegation denied for " 403 error.
Here is a PHP example of how we were able to list their SendAs settings.
<?PHP
//
// Description:
// List the user's SendAs addresses.
//
// Documentation:
// https://developers.google.com/gmail/api/v1/reference/users/settings/sendAs
// https://developers.google.com/gmail/api/v1/reference/users/settings/sendAs/list
//
// Local Path:
// /path/to/api/vendor/google/apiclient-services/src/Google/Service/Gmail.php
// /path/to/api/vendor/google/apiclient-services/src/Google/Service/Gmail/Resource/UsersSettingsSendAs.php
//
// Version:
// Google_Client::LIBVER == 2.1.1
//
require_once $API_PATH . '/path/to/google-api-php-client/vendor/autoload.php';
date_default_timezone_set('America/Los_Angeles');
// this is the service account json file used to make api calls within our domain
$serviceAccount = '/path/to/service-account-with-domain-wide-delagation.json';
putenv('GOOGLE_APPLICATION_CREDENTIALS=' . $serviceAccount );
$userKey = 'someuser#my.domain';
// In the Admin Directory API, we may do things like create accounts with
// an account having roles to make changes. With the Gmail API, we cannot
// use those accounts to make changes. Instead, we impersonate
// the user to manage their account.
$impersonateUser = $userKey;
// these are the scope(s) used.
define('SCOPES', implode(' ', array( Google_Service_Gmail::GMAIL_SETTINGS_BASIC ) ) );
$client = new Google_Client();
$client->useApplicationDefaultCredentials(); // loads whats in that json service account file.
$client->setScopes(SCOPES); // adds the scopes
$client->setSubject($impersonateUser); // account authorized to perform operation
$gmailObj = new Google_Service_Gmail($client);
$res = $gmailObj->users_settings_sendAs->listUsersSettingsSendAs($userKey);
print_r($res);
?>
I wanted to access the emails of fresh email id/account but what happened was, the recently created folder with '.credentials' containing a JSON was associated with the previous email id/account which I tried earlier. The access token and other parameters present in JSON are not associated with new email id/account. So, in order make it run you just have to delete the '.credentails' folder and run the program again. Now, the program opens the browser and asks you to give permissions.
To delete the folder containing files in python
import shutil
shutil.rmtree("path of the folder to be deleted")
you may add this at the end of the program
Recently I started exploring Gmail API and I am following the same approach as Guo mentioned. However, it is going to take of time and too many calls when we the number of users or more. After domain wide delegation my expectation was admin id will be able to access the delegated inboxes, but seems like we need to create service for each user.

Resources