The document describes how to attach an ACR to existing AKS cluster, https://learn.microsoft.com/en-us/azure/aks/cluster-container-registry-integration
How do I attach a second ACR to my AKS cluster?
The ACR has a different subscription.
attach the first ACR:
az account set --subscription acr-subscription
$ACR_ID = $(az acr show --name $ACRName --resource-group $ACRResourceGroupName --query "id" --output tsv)
az account set --subscription aks-subscription
az aks update --name cluster-name --resource-group cluster-resource --attach-acr $ACR_ID
Based on this StackOverFLow question, this should work as long as your ID is in the right format
echo $(az acr show --name $name --resource-group $resourcegroup --query "id" --output tsv)
returns the right format and value.
the code associated with --attach-acr simply add the acrpull IAM role to the ACR resource using the identity_profile ID associated to the kubernetes cluster.
https://github.com/Azure/azure-cli/blob/a9fe6167381b53dac19a3007f726adf27b64f28b/src/azure-cli/azure/cli/command_modules/acs/custom.py#L559
there is no where in the code where it checks if another ACR was previous attached to the AKS resource. Therefore, it should be able to set the role assignment directly for multiple instances.
this should work properly
az account set --subscription acr-subscription
$ACR_ID = $(az acr show --name $ACRName --resource-group $ACRResourceGroupName --query "id" --output tsv)
az account set --subscription acr2-subscription
$ACR2_ID = $(az acr show --name $ACR2Name --resource-group $ACR2ResourceGroupName --query "id" --output tsv)
az account set --subscription aks-subscription
az aks update --name cluster-name --resource-group cluster-resource --attach-acr $ACR_ID
az aks update --name cluster-name --resource-group cluster-resource --attach-acr $ACR2_ID
Related
I'm building deployment pipeline using Google Cloud Build and store the Docker image in GCR. I planned to restart the GCE instance group on the latest Cloud Build step so the GCE can run the latest docker image by add docker pull gcr.io/my-project/my-image in the GCE instance template startup script. The problem is I can't authorize the docker to pull image from GCR. I've read the 4 GCR authentication method but all of them required us to login manually from the browser. Also at this stage I can't upload the service account key since I need to provision and maintain the infrastructure fully from code (Terraform), no Google Cloud console. So how do we authenticate docker as a machine?
If the instance doesn't have gcloud installed, you can use the Metadata service to acquire an access token and use that to login to GCR using Docker.
I've not used this to login to GCR using Docker but it should work. I use this format to access Google Cloud services from an instance startup script:
echo "Getting token from metadata"
ENDPOINT="http://metadata.google.internal/computeMetadata/v1"
ACCOUNT="default" # Replace with Service Account Email (!)
TOKEN=$(\
curl \
--silent \
--header "Metadata-Flavor: Google" \
http://${ENDPOINT}/instance/service-accounts/${ACCOUNT}/token)
echo "Extract access token"
ACCESS=$(\
echo ${TOKEN} \
| grep --extended-regexp --only-matching "(ya29.[0-9a-zA-Z._-]*)")
echo "Login to Docker"
HOST="https://gcr.io" # Or ...
printf ${ACCESS} \
| docker login -u oauth2accesstoken \
--password-stdin ${HOST}
You can grant IAM privileges or scopes to the service account attached to your GCE instance, then run the following command:
gcloud auth print-access-token | docker login -u oauth2accesstoken \
--password-stdin https://HOSTNAME
That will authenticate against the registry and be able to push and pull images.
I am using az keyvault secret list to get secrets from my Azure key vault. Its help says:
Arguments
--maxresults : Maximum number of results to return in a page. If not
specified, the service will return up to 25 results.
It is not possible to set --maxresults any higher than 25. The help says "in a page", but I can find no explanation of how to get the next page.
Is it possible to list more than the top 25 secrets using this tool?
We cannot get more than 25 Secret lists by using the --maxresults in the CLI command.
Please find the below workaround:
If we specify the --maxresults more than 25 the cli returns the below result.
Az keyvault secret list --vault-name <your keyvault name> --maxresults 30
If you want to get all the Secrets in a specific key vault you have to use the below command without using --maxresults.
Az keyvault secret list --vault-name <your keyvault name>
Or
If you want it to achieve programmatically need to write a script with the REST API or some language library directly. Refer here
To get all the secrets with name and value via azure cli in Mac, you can use the below script:
sh keyvault-list.sh keyvaultname
#!/usr/bin/env bash
keyvaultEntries=($(az keyvault secret list --vault-name $1 --query "[*].{name:name}" -o tsv))
for i in "${keyvaultEntries[#]}"
do
# do whatever on "$i" here
echo "$i"::"$(az keyvault secret show --name $i --vault-name $1 -o tsv --query value)"
done
How can i add Auth key from Service Account for (GCP-> Container Registry) to docker daemon.json?
Normally i write url and user:pass in base64 in docker daemon.json and docker can do pull from private registry.
How about GCP Container registry? I generated a json key and it works.
docker login -u _json_key --password-stdin https://gcr.io < credentials.json
I can login to GCP Container Registry and pull the image from it but how can i add this Key to docker daemon.json So that the docker automatically makes a pull from private repo.
Thanks.
Seems that you already choose your authentication metthod:
Choosing an authentication method
gcloud credential helper
Standalone docker credential helper
Access Token
JSON key file
Regarding JSON Key File, Use the following guidelines to limit access to your container images:
Create dedicated service accounts that are only used to interact with Container Registry.\
Grant the specific role for the least amount of access that the service account requires.\
Follow best practices for managing credentials.
To create a new service account and a service account key for use with Container Registry repositories only:
Create a new service account that will interact with Container Registry.
You can run the following commands using Cloud SDK on your local machine, or in Cloud Shell.
a. Create the service account. Replace NAME with a name for the service account.
gcloud iam service-accounts create NAME
b. Grant a role to the service account. Replace PROJECT_ID with your project ID and ROLE with the appropriate Cloud Storage role for the service account.
gcloud projects add-iam-policy-binding PROJECT_ID --member "serviceAccount:NAME#PROJECT_ID.iam.gserviceaccount.com" --role "roles/ROLE"
Obtain a key for the service account that will interact with Container Registry.
You can run the following command using Cloud SDK on your local machine, or in Cloud Shell.The instructions on this page use the file name keyfile.json for the key file.
gcloud iam service-accounts keys create keyfile.json --iam-account [NAME]#[PROJECT_ID].iam.gserviceaccount.com
Verify that permissions are correctly configured for the service account. If you are using the Compute Engine service account, you must correctly configure both permissions and access scopes.
Use the service account key as your password to authenticate with Docker.
Username is _json_key (NOT the name of your service account)
keyfile.json is the service account key you created
for example:
cat keyfile.json | docker login -u _json_key --password-stdin https://HOSTNAME
where HOSTNAME is gcr.io, us.gcr.io, eu.gcr.io, or asia.gcr.io.
Or, for older Docker clients which don't support --password-stdin:
docker login -u _json_key -p "$(cat keyfile.json)" https://HOSTNAME
where HOSTNAME is gcr.io, us.gcr.io, eu.gcr.io, or asia.gcr.io.
My release pipeline runs successfully and creates a container in Azure Kubernetes, however when I view in azure Portal>Kubernetes service> Insights screen, it shows a failure.
It fails to pull the image from my private container repository with error message 'ImagePullBackOff'
I did a kubectl describe on the pod and got below error message:
Failed to pull image "myexampleacr.azurecr.io/myacr:13": [rpc error: code = Unknown desc = Error response from daemon: Get https://myexampleacr.azurecr.io/v2/myacr/manifests/53: unauthorized: authentication required.
Below is a brief background on my setup:
I am using Kubernetes secret to access the containers in private container registry.
I generated the Kubernetes secret using clientId and password(secret) from the Service Principle that my DevOps team created.
.
The command used to generate kubernetes secret:
kubectl create secret docker-registry acr-auth --docker-server --docker-username --docker-password --docker-email
I then updated my deployment.yaml with imagePullSecrets: name:acr-auth
After this, I ran my deployment and release pipeline both ran successfully, but they show failure in the kubernetes service with error message 'ImagePullBackOff' error.
Any help will be much appreciated.
As the error shows it required authentication. As I see from your description, the possible reason is that your team does not assign the ACR role to the service principal that your team creates, or you use the wrong service principal. So you need to check two things:
If the service principal you use has the right permission of the ACR.
If the Kubernetes secret was created right in the Kubernetes service.
The way to check if the service principal has the right permission of the ACR is that pull an image in the ACR after you log in with the service principal in docker server. Also, as the comment said, you need to make sure the command is right as below:
kubectl create secret docker-registry acr-auth --docker-server myexampleacr.azurecr.io --docker-username clientId --docker-password password --docker-email yourEmail
Additional, there is a little possibility that you use the wrong image with tag. By the way, check it out.
I had the same error, and I realised that the service principal is expired.
To check the expiration date of your service principal and update your AKS cluster with the new credentials, fallow the following steps:
NOTE: You need the Azure CLI version 2.0.65 or later installed and configured.
1- Get the Client ID of your cluster using the az aks show command.
az aks show --resource-group YOUR_AKS_RESOURCE_GROUP_NAME --name YOUR_AKS_CLUSTER_NAME --query "servicePrincipalProfile.clientId"
2- Check the expiration date of your service principal.
az ad sp credential list --id YOUR_CLIENT_ID --query "[].endDate" -o tsv
If the service principal is expired then, to reset the existing service principal credential fallow the following steps:
1- Reset the credentials using az ad sp credential reset command.
az ad sp credential reset --name YOUR_CLIENT_ID --query password -o tsv
2- Update your AKS cluster with the new service principal credentials.
az aks update-credentials --resource-group YOUR_AKS_RESOURCE_GROUP_NAME --name YOUR_AKS_CLUSTER_NAME --reset-service-principal --service-principal YOUR_CLIENT_ID --client-secret YOUR_NEW_PASSWORD
Source: https://learn.microsoft.com/en-us/azure/aks/update-credentials
It's odd, maybe it shows an old deployment which you didn't delete. It may also be these; incorrect credientials, acr may not be up, image name or tag is wrong. You can also go with aks-acr native authentication and never use a secret: https://learn.microsoft.com/en-gb/azure/container-registry/container-registry-auth-aks
In my case the problem was that my --docker-password had an special character and I was not escaping it using quotes (i.e. --docker-password 'myPwd$')
You can check your password is correct my executing this command:
kubectl get secret < SECRET > -n < NAMESPACE> --output="jsonpath={.data..dockerconfigjson}" | base64 --decode
Reference: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
I am trying to bring up K8s Cluster in Azure. This is the error I am getting:
az aks create --resource-group upf-infra-ResourceGroup --name upf-infra-K8sCluster-1 --node-count 1 --generate-ssh-keys
Deployment failed. Correlation ID: d90bed78-075f-4a07-81c0-271dac75e0ca. PutControlPlane error
Include kubernetes version when creating your cluster.
Ex: az aks create --resource-group --name --node-count 1 --generate-ssh-keys --kubernetes-version 1.8.7
https://github.com/Azure/AKS/issues/284