I have a common azure databricks resources in a shared resource group. For every customer i have different resource groups automated. Based on my design, My notebook need to read some keyvault variables from keyvaults created in other resource groups using databricks scope variables.
Is it possible to create such scope variables. I am finding errors.
Yes, it is possible to connect Azure Databricks secret scope to a keyVault on other resource group.
Here is an example of creating secret scope to a KeyVault from another resource group.
Hope this helps. Do let us know if you any further queries.
Related
How do you manage clientId and clientCredentials of an OIDC(or OAuth2) application's deployment consisting of multiple instances fronted by let's say an L7 load balancer ?
Do you maintain separate clientId/clientCredential for each application instance ?
If yes, how do you manage clientId/clientCredentials for dynamically provisioned instance (E.g. Kubernetes/AWS adding a new application instance in response to a health check failure or a scale-out event)
If you share clientId/clientCredentials across multiple application instances,Isn't that violating the basic rule (i.e. 'secret' is no longer a secret anymore) ?
Also, Compromise of any individual instance by potential attackers, kind of impacts the entire deployment.
If you share clientId/clientCredentials across multiple application
instances,Isn't that violating the basic rule (i.e. 'secret' is no
longer a secret anymore)
Instead of using the secret, you can leverage the Hashicorp Vault which will store and inject the environment variable to the deployment. You can implement encryption at rest and other security options like RBAC on vault UI access.
Yes, secret is base64 encoded not encrypted if you have a large team managing a cluster and RBAC is not set everyone having access to the cluster will be able to decode the secret.
Read more about the has corp vault : https://www.vaultproject.io/
With Kubernetes : https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-raft-deployment-guide?in=vault%2Fkubernetes
Once the vault is set and you have created the secret into it with values you can refer my answer for more details injecting the secret to deployment : https://stackoverflow.com/a/73046067/5525824
I am have a aws cdk project creating stacks that are based on existing an existing kms key and existing subnet.
I need the subnet_id and the kms key arn for an IAM policy.
Since I did not find a way to find out the kms key arn and subnet id at runtime, I started by hardcoding them into the code.
But now I want to deploy the stack into different accounts (for which of the kms key arn and subnet id of course differs).
When I was using just Cloudformation, without aws cdk, I would use the Mapping section of the cloudformation template to map account ids to the needed information:
Mappings:
KMSKeyArn:
<account-id-1>:
ARN: <kms-key1-arn>
<account-id-2>:
ARN: <kms-key2-arn>
What is a good way to do this with AWS CDK?
Should I use CfnMapping?
Or can I somehow know the Account ID at CDK Execution time?
Is there a better way I am missing here?
similar to the aws-cli, the cdk detects its account and region through the environment. See https://yshen4.github.io/infrastructure/AWS/CDK_context.html
I want to use Drive & Sheets API from Python3.
I tried to create a service account in GCP console, but it said You already have credentials that are suitable for this purpose, without telling me what role of GCP makes it suitable.
Now I want to find and add the role to my favorite service account. I do not want to create a new one, nor use the already-suitable ones.
What is the name of the role which allows using Google APIs?
I think this is confusing and poorly explained in Google's documentation but IAM and OAuth scopes are mostly (now) complementary technologies. I consider IAM to refine or provide more granular, account|credential-specific scopes
Scopes came first and IAM came later.
IAM -- actually Cloud IAM -- applies only to Google Cloud Platform (GCP) services.
OAuth scopes exist for all Google's APIs (including those of GCP).
https://developers.google.com/identity/protocols/googlescopes
In your case, Drive and Sheets are part of G Suite and not included in Cloud IAM. There are thus no IAM roles for you to assign to your service account that apply to these services.
For GCP services, it is customary (though seemingly redundant) to use both IAM and scopes.
Often you'll see https://www.googleapis.com/auth/cloud-platform used as something of a catch-all|default scope.
UPDATE: In retrospect, I should add a qualifier to this, OAuth scopes are effectively redundant for service accounts. In the case of a human|user-based OAuth flow where the end-user is delegating authority, the scopes remain relevant as they summarize the permissions for the end-user to review before granting to the code.
My current customer has secrets stored in Blob Storage and we want to propose them to migrate to KeyVault. May I know what are the benefits or storing secrets into KeyVault as compared to Blob?
When I read the documentation, KeyVault uses the HSM to protect the keys and secrets but Blob also uses the encryption which is also secure. so what are the other advantages?
I'd say that in general they look very similar, however I'd say the most important difference between the two would be the authorization model.
Access to a storage account is done by one of the two available connectionstrings/keys. Access to a KeyVault can be assigned directly to users or groups (from AAD) and the access to resources within the Key vault can be configured with more granularity. Next to that it is very easy to limit the type of resources from within azure that may or may not retrieve data from a KeyVault, reducing the attack service.
Storage accounts do have AAD integration currently in preview, but what i gather is that that is mostly focusing on the Azure file share functionality (https://learn.microsoft.com/en-us/azure/storage/files/storage-files-active-directory-overview).
Another nice differentiation is definitely the integrations that are already available when using KeyVault (i.e. Retrieving Azure DevOps secrets directly from a KeyVault or automatically retrieving Certificates for VMs)
FYI, i'm by no means a KeyVault expert but that's just my 2 cents :)
Is it possible to configure an access policy that would allow the job creator (or less preferably anybody with access to the project) access to the created instances, without granting similar access to the rest of the machines in the Google Cloud project (e.g., production machines)?
Thanks again,
G
The access policies that you define for the project are carried over to the instances backing up a job submitted in that project.
Details on the policies and related actions are outlined in the GCE documentation: https://cloud.google.com/compute/docs/access
Cheers,
r