Terraform CustomScriptExtension reports unable to download files - terraform-provider-azure

Question

G'day,
I am running a Terraform extension - CustomScriptExtension. However it reports the error that it is unable to load the files.
resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-ad-ext-install-addom" {
count = "${var.count_ad_vm}"
name = "${var.ad_base_hostname}${format("%02d",count.index+1)}-CSE"
location = "${azurerm_resource_group.tf-rg-grpprd-core.location}"
resource_group_name = "${azurerm_resource_group.tf-rg-grpprd-core.name}"
virtual_machine_name = "${var.ad_base_hostname}${format("%02d",count.index+1)}"
publisher = "${var.extension_publisher_ad}"
type = "${var.extension_type_customerscriptextension}"
auto_upgrade_minor_version = "true"
type_handler_version = "${var.extension_version_customscriptextension}"
depends_on = ["azurerm_virtual_machine.tf-vm-grpprd-ad"]
settings = <<SETTINGS
{
"fileUris": ["https://blackbeltteam.visualstudio.com/blackbeltteam/_git/groupsprod/scripts/Install_AD_Components.ps1"],
"commandToExecute": "powershell.exe pwd",
"commandToExecute": "powershell.exe ls",
"commandToExecute": "powershell.exe -ExecutionPolicy unrestricted -NoProfile -NonInteractive -File Install_AD_Components.ps1"
}
SETTINGS
}
I get the error message as below:
2019-07-31T01:17:20.0860444Z [1m[31mError: [0m[0m[1mCode="VMExtensionProvisioningError" Message="VM has reported a failure when processing extension 'grpprdad02-CSE'. Error message: \"Failed to download all specified files. Exiting. Error Message: The remote server returned an error: (404) Not Found.\"."[0m
2019-07-31T01:17:20.0860808Z
2019-07-31T01:17:20.0861339Z [0m on ad.tf line 250, in resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-ad-ext-install-addom":
2019-07-31T01:17:20.0861901Z 250: resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-ad-ext-install-addom" [4m{[0m
2019-07-31T01:17:20.0862336Z [0m
2019-07-31T01:17:20.0862897Z [0m[0m
2019-07-31T01:17:20.0894295Z [31m
2019-07-31T01:17:20.0895064Z [1m[31mError: [0m[0m[1mCode="VMExtensionProvisioningError" Message="VM has reported a failure when processing extension 'grpprdad01-CSE'. Error message: \"Failed to download all specified files. Exiting. Error Message: The remote server returned an error: (404) Not Found.\"."[0m
2019-07-31T01:17:20.0895600Z
2019-07-31T01:17:20.0896889Z [0m on ad.tf line 250, in resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-ad-ext-install-addom":
2019-07-31T01:17:20.0897664Z 250: resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-ad-ext-install-addom" [4m{[0m
2019-07-31T01:17:20.0898223Z [0m
2019-07-31T01:17:20.0898695Z [0m[0m
2019-07-31T01:17:20.0943083Z [31m
2019-07-31T01:17:20.0944168Z [1m[31mError: [0m[0m[1mcompute.VirtualMachineExtensionsClient#CreateOrUpdate: Failure sending request: StatusCode=404 -- Original Error: Code="ParentResourceNotFound" Message="Can not perform requested operation on nested resource. Parent resource 'grpprdaos01' not found."[0m
2019-07-31T01:17:20.0944773Z
2019-07-31T01:17:20.0945288Z [0m on aos.tf line 240, in resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-aos-ext-join-ad":
2019-07-31T01:17:20.0945858Z 240: resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-aos-ext-join-ad" [4m{[0m
2019-07-31T01:17:20.0946979Z [0m
2019-07-31T01:17:20.0947787Z [0m[0m
2019-07-31T01:17:20.0990513Z [31m
2019-07-31T01:17:20.0991014Z [1m[31mError: [0m[0m[1mcompute.VirtualMachineExtensionsClient#CreateOrUpdate: Failure sending request: StatusCode=404 -- Original Error: Code="ParentResourceNotFound" Message="Can not perform requested operation on nested resource. Parent resource 'grpprdaos02' not found."[0m
2019-07-31T01:17:20.0991108Z
2019-07-31T01:17:20.1002906Z [0m on aos.tf line 240, in resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-aos-ext-join-ad":
2019-07-31T01:17:20.1003501Z 240: resource "azurerm_virtual_machine_extension" "tf-vm-grpprd-aos-ext-join-ad" [4m{[0m
It does not indicate which command it failed to execute, where it is running etc, how can I understand what is causing the grief.

Answer 1

The problem might be the resource is not found when you access the URI https://blackbeltteam.visualstudio.com/blackbeltteam/_git/groupsprod/scripts/Install_AD_Components.ps1. The script location can be anywhere, as long as the VM can route to that endpoint, such as GitHub or an internal file server.
You could ensure the resources could be retrieved by the file Uri in the SETTINGS from your VM. If you need to download a script externally then additional firewall and Network Security Group ports need to be opened.
For more references:
Custom Script Extension for Windows
Bootstrapping Azure VMs with Terraform

Answer 2

I was hitting the same issue and eventually found it was due to the file name in blob storage having higher case letters in it! Hopefully this will help someone else avoid the same issue.