What is the correct and easy way of changing ASC Default PARAMETERS in Azure policy using TF?
As an example Set Monitor SQL Encryption to AuditIfNotExists or any other available value than Disabled.
'ASC Default' is initiative, or so called azurerm_policy_set_definition in TF terms. You just need to assign it using 'azurerm_policy_assignment'.
These links might help:
https://github.com/Azure/azure-policy/blob/master/built-in-policies/policySetDefinitions/Security%20Center/AzureSecurityCenter.json
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/policy_assignment
Example for different initiative:
resource "azurerm_policy_assignment" "audit_k8s_security_restricted_standarts" {
name = "42b8ef37-b724-4e24-bbc8-7a7708edfe00"
scope = local.azure_policy_scope
policy_definition_id = "/providers/Microsoft.Authorization/policySetDefinitions/42b8ef37-b724-4e24-bbc8-7a7708edfe00"
description = "This initiative includes the policies for the Kubernetes cluster pod security restricted standards."
display_name = "Kubernetes cluster pod security restricted standards for Linux-based workloads"
identity { type = "SystemAssigned" }
location = var.primary_location
parameters = <<PARAMETERS
{
"effect": {
"value": "audit"
}
}
PARAMETERS
}
What you need is to place proper policy_definition_id and pass valid parameters. And start small, with one simple param, there are bugs in TF.
Related
I have a simple Terraform configuration to create azure virtual network. When I do plan and then apply, a virtual network is created inside of a resource group as expected. But in addition to this resource group, there is one more created by the name NetworkWatcherRG, and inside of it I see a network watcher.
And the network watcher.
Now when I run the Terraform destroy command, I expect that every thing is cleaned up, all the Resource groups are destroyed. But instead, everything except for the NetworkWatcherRG and the Network Watcher inside of it are destroyed.
Looks like the Network Watcher along with its resource group, is NOT managed by Terraform. What am I missing?
The network watcher is not immediately obvious. Its not reveled immediately. So to see that, you need to go the simplified view of the resource groups. You need to click the Refresh button atleast 5 times(each time with a 2 second time gap) or you have to wait for long time and then click refresh.
So what is this network watcher and is it that Azure is creating it by itself and not managed by Terraform?
My Terraform configuration file is as follows.
# Terraform settings Block
terraform {
required_version = ">= 1.0.0"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.0"
}
}
}
# Provider Block
provider "azurerm" {
features {}
}
# create virtual network
resource "azurerm_virtual_network" "myvnet" {
name = "vivek-1-vnet"
address_space = ["10.0.0.0/16"] # This is a list, it has []. If it has { }, then its a map.
location = azurerm_resource_group.myrg.location
resource_group_name = azurerm_resource_group.myrg.name
tags = { # This is a map. This is {}
"name" = "vivek-1-vnet"
}
}
# Resource-1: Azure Resource Group
resource "azurerm_resource_group" "myrg" {
name = "vivek-vnet-rg"
location = var.resource_group_location
}
variable "resource_group_location" {
default = "centralindia"
description = "Location of the resource group."
}
And finally the commands I use are as follows.
terraform fmt
terraform init
terraform validate
terraform plan -out main.tfplan
terraform apply main.tfplan
terraform plan -destroy -out main.destroy.tfplan
terraform apply main.destroy.tfplan
I read the response from #RahulKumarShaw-MT . I believe the answer and it makes complete sense that terraform won't destroy resources it didn't create (unless someone can demonstrate otherwise). That said, I was able to delete the NetworkWatcherRG group using terraform! What I did to achieve this was I made sure to add a network watcher as one of my declared resources using azurerm_network_watcher (see https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/network_watcher) in the same terraform script where I requested a virtual machine resource in another separate resource group. I think you created a vnet. My script creates a vnet too, and hence why I think Azure concludes that there is a need for a network watcher maybe? I named the first resource group, which contains my network watcher, whatever I wanted; doesn't have to be 'NetworkWatcherRG'. I watched the resource group be created and destroyed successfully with Terraform (using terraform apply and terraform destroy respectively, of course) along with my VM and vnet resources. Anyway, at the end, I refreshed the Azure Portal web page and saw no resource groups or resources in my test subscription. I'm not an Azure expert, but I suspect that if Azure already sees a network watcher present, then it won't create an additional one when terraform created my resources (e.g. - in my case a vm and a vnet), as a watcher will already be present as long as terraform creates that resource first before Azure gets a chance to.
Before applying terraform code i checked in my resource groups with name network watcher resource group for me , by default this resource grpup is created by Azure side.
As Mike-Ubezzi wrote on Microsoft forums:
Network Watcher resources are located in the hidden NetworkWatcherRG
resource group which is created automatically. For example, the NSG
Flow Logs resource is a child resource of Network Watcher and is
enabled in the NetworkWatcherRG.
The Network Watcher resource represents the backend service for
Network Watcher and is fully managed by Azure. Customers do no need to
manage it. Operations like move are not supported on the resource.
However, the resource can be
deleted.
So terraform destroy will only delete the resource created by you(mentioned in .tfstate file).This is the region you won't able to delete the NetworkWatcherRG Resource Group.
I'm using terraform to write azure policy as code
I found two problems
1 I can't seem to use log analytics workspace that is on a different subscription, within same subscription, it's fine
2 For policies that needs managed identity, I can't seem to assign correct rights to it.
resource "azurerm_policy_assignment" "Enable_Azure_Monitor_for_VMs" {
name = "Enable Azure Monitor for VMs"
scope = data.azurerm_subscription.current.id
policy_definition_id = "/providers/Microsoft.Authorization/policySetDefinitions/55f3eceb-5573-4f18-9695-226972c6d74a"
description = "Enable Azure Monitor for the virtual machines (VMs) in the specified scope (management group, subscription or resource group). Takes Log Analytics workspace as parameter."
display_name = "Enable Azure Monitor for VMs"
location = var.location
metadata = jsonencode(
{
"category" : "General"
})
parameters = jsonencode({
"logAnalytics_1" : {
"value" : var.log_analytics_workspace_ID
}
})
identity {
type = "SystemAssigned"
}
}
resource "azurerm_role_assignment" "vm_policy_msi_assignment" {
scope = azurerm_policy_assignment.Enable_Azure_Monitor_for_VMs.scope
role_definition_name = "Contributor"
principal_id = azurerm_policy_assignment.Enable_Azure_Monitor_for_VMs.identity[0].principal_id
}
for var.log_analytics_workspace_ID, if i use the workspace id that is in the same subscription as the policy, it would work fine. but If I use a workspace ID from a different subscription, after deployment, the workspace field will be blank.
also for
resource "azurerm_role_assignment" "vm_policy_msi_assignment"
, I have already given myself user access management role, but after deployment, "This identity currently has the following permissions:" is still blank?
I got an answer to my own question:)
1 this is not something designed well in Azure, I recon.
MS states "a Managed Identity (MSI) is created for each policy assignment that contains DeployIfNotExists effects in the definitions. The required permission for the target assignment scope is managed automatically. However, if the remediation tasks need to interact with resources outside of the assignment scope, you will need to manually configure the required permissions."
which means, the system generated managed identity which needs access in log analytics workspace in another subscription need to be manually with log analytics workspace contributor rights
Also since you can't user user generated managed ID, you can't pre-populate this.
so if you want to to achieve in terraform, it seems you have to run policy assignment twice, the first time is just to get ID, then manual ( or via script) to assign permission, then run policy assignment again to point to the resource..
2 The ID was actually given the contributor rights, you just have to go into sub RBAC to see it.
I'm currently building my choices depending on available agents and its labels in a pipeline:
def loadConfigurations() {
def configurations = [];
def jenkins = Jenkins.instance;
def onlineComputers = jenkins.computers.findAll { it.online };
def availableLabels = onlineComputers.collect {
it.assignedLabels.collect { LabelAtom.escape(it.expression) } }
.flatten().unique(false);
def lineage16Configurations = ['samsung:klte:lineage:16.0'];
if (availableLabels.containsAll(['lineage', '16.0'])) {
configurations.addAll(lineage16Configurations);
}
return configurations;
}
def configurations = loadConfigurations();
pipeline {
agent { label 'master' }
parameters {
choice name: 'CONFIG', choices: configurations, description: 'Configuration containing vendor, device, OS and its version. Each separated by a colon.'
}
//...
Now, lets say all agents are offline, when requesting the remote access API I don't get up-to-date choices cause they're only updated when starting a build. Is there any existing way to retrieve them somehow through remote access API or do I need to write my own plugin which adds a new endpoint for the remote access API?
I've already tried the Active Choices Parameter and the Extended Choice Parameter without success. Both don't display any choices in the API.
I was playing around with the Extensible Choice Parameter and created a Pull Request which exposes the choiceList to the Remote Access API. The API was then returning changed choices without building the job.
I have followed Create an Azure Key Vault-backed secret scope to integrate Databricks with Key Vault and all works ok. Unfortunately this requires manual intervention, which breaks our 'full automated infrastructure' approach. Is there any way to automate this step?
UPDATE: You create a Databricks-backed secret scope using the Databricks CLI (version 0.7.1 and above). Alternatively, you can use the Secrets API.
It does not appear that Azure Key Vault backed secret scope creation has a publicly available API call, unlike the Databricks backed secret scope creation. This is backed by the 'Note' on the secret scopes doc page:
Creating an Azure Key Vault-backed secret scope is supported only in the Azure Databricks UI. You cannot create a scope using the Secrets CLI or API.
A request for the feature you are asking for was made last year, but no ETA was given.
I took a look at the request made by the UI page. While the form data is simple enough, the headers and security measures make programmatic access impractical. If you are dead-set on automating this part, you could use one of those tools which automates the cursor around the screen and clicks things for you.
Now it is possible, but you can't use a service principal token. It must be a user token which hinder automation.
Refer to Microsoft Docs:
https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes#create-an-azure-key-vault-backed-secret-scope-using-the-databricks-cli
You can use Databricks Terraform provider to create secret scope baked by the Azure KeyVault. But because of Azure limitations it should be done by using user’s AAD token (usually using azure cli). Here is the working snippet for creation of the secret scope from existing KeyVault:
terraform {
required_providers {
databricks = {
source = "databrickslabs/databricks"
version = "0.2.9"
}
}
}
provider "azurerm" {
version = "2.33.0"
features {}
}
data "azurerm_databricks_workspace" "example" {
name = var.workspace_name
resource_group_name = var.resource_group
}
provider "databricks" {
azure_workspace_resource_id = data.azurerm_databricks_workspace.example.id
}
data "azurerm_key_vault" "example" {
name = var.keyvault_name
resource_group_name = var.resource_group
}
resource "databricks_secret_scope" "example" {
name = data.azurerm_key_vault.example.name
keyvault_metadata {
resource_id = data.azurerm_key_vault.example.id
dns_name = data.azurerm_key_vault.example.vault_uri
}
}
variable resource_group {
type = string
description = "Resource group to deploy"
}
variable workspace_name {
type = string
description = "The name of DB Workspace"
}
variable keyvault_name {
type = string
description = "The name of DB Workspace"
}
when generating Service Principal in Azure manually, as a result of the operation I'm provided a password.
It's not the case however if I create service principal with Terraform, the password is not among the outputs of this module:
+ azuread_service_principal.k8s_principal
id: <computed>
application_id: "${azuread_application.app.application_id}"
display_name: <computed>
Is there anything I missed? Why does the Terraform behavior differs in the output compared to CLI?
password is required INPUT to the azuread_service_principal_password block. As such, you can generate a random password and export it yourself. Complete Terraform code is something like this:
resource "azuread_application" "app" {
name = "${local.application_name}"
}
# Create Service Principal
resource "azuread_service_principal" "app" {
application_id = "${azuread_application.app.application_id}"
}
resource "random_string" "password" {
length = 32
special = true
}
# Create Service Principal password
resource "azuread_service_principal_password" "app" {
end_date = "2299-12-30T23:00:00Z" # Forever
service_principal_id = "${azuread_service_principal.app.id}"
value = "${random_string.password.result}"
}
output "sp_password" {
value = "${azuread_service_principal_password.app.value}"
sensitive = true
}
to who using newer version of Terraform, you don't need to preset the password, following code is working fine:
resource "azuread_service_principal_password" "auth_pwd" {
service_principal_id = azuread_service_principal.auth.id
}
output "auth_client_secret" {
value = azuread_service_principal_password.auth_pwd.value
description = "output password"
sensitive = true
}
then you can run the following cli to retrieve the password:
terraform output -raw auth_client_secret
tested on terraform 1.0.10, hashicorp/azuread provider 2.11
In the terraform document, the azuread_service_principal block only defines the Argument application_id and Attributes id, display_name, So you only could see these resources. Also, the azuread_service_principal_password block allows you to export the Key ID for the Service Principal Password. You still could not see the real password.
In the Azure CLI az ad sp create-for-rbac has an optional parameter --Password. So you could see the password output.