Creating a deployment in Redhat Openshift using a specific USER id - docker

Redhat Openshift autumatically creates a range of user ids that can be used in a given namespace, e.g.
$ oc describe namespace xyz-123
Name: xyz-123
Labels: <none>
Annotations: xx.com/accountID: id-testcluster-account
xx.com/type: System
openshift.io/sa.scc.mcs: s0:c25,c20
openshift.io/sa.scc.supplemental-groups: 1000640000/10000
openshift.io/sa.scc.uid-range: 1000640000/10000
Here is the problem:
While creating docker image, I am setting USER id in Dockerfile:
USER 1001121001:1001121001
I am specifying runAsUser in Helm charts to deploy this image:
runAsUser : 1001121001
When I try to create the deployment, the deployment fails. Because the user ID 1001121001 does not fall in the range above i.e. [1000640000, 1000640000+10000].
The deployment error:
$ oc get deployment abc-123 -n xyz-123 -o yaml
....
....
message: 'pods "abc-123-7f8fc74765-" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.runAsUser: Invalid value: 1000321001: must be in the ranges: [1000660000, 1000669999]]'
....
....
Tried options - 1:
Using anyuid works as described here : https://www.openshift.com/blog/a-guide-to-openshift-and-uids
But the document says:
"Upon closer inspection of the “anyuid” SCC, it is clear that any user and any group can be used by the Pod launched by a ServiceAccount with access to the “anyuid” SCC. The “RunAsAny” strategy is effectively skipping the default OpenShift restrictions and authorization allowing the Pod to choose any ID."
Hence, I dont want to use this anyuid optiuon.
Tried option-2:
After creating a namespace get the range allowed for that namespace and select an id (say 1000660000) from that range and use that while deploying by setting that id for runAsUser: 1000660000.
All files/folders in the docker image will have the ownership/permissions set to USER 1001121001 and the container started with the id 1000660000 and hence there are issues running the container due to read/write/execute permissions.
To overcome this I need to give o+rwx permissions for all the files, which is risky.
Is there any other way to specify a USER in Dockerfile and use the same USER id during deployment in Redhat Openshift?
$ oc version
Client Version: 4.6.9
Server Version: 4.6.9
Kubernetes Version: v1.19.0+7070803
Solution:
The suggestion from Ritesh worked.
Created the namespace specifying the UID range and covering the specific USER ID. Then created the deployment in this namespace:
Created a namespace with predefined user id range (covering the specific USER id 1001121001) before deploying into the namespace.
apiVersion: v1
kind: Namespace
metadata:
name: xyz-123
annotations:
annotations:
openshift.io/sa.scc.mcs: 's0:c26,c5'
openshift.io/sa.scc.supplemental-groups: 1001120001/10000
openshift.io/sa.scc.uid-range: 1001120001/10000

If you are creating namespace while doing deployment or before then can use following option. Using this you can use runAsUser : 1001121001 (or any other user)
define the yaml file
apiVersion: v1
kind: Namespace
metadata:
name: dspx-dummy-runtimeid
annotations:
openshift.io/sa.scc.mcs: <>
openshift.io/sa.scc.supplemental-groups: <>
openshift.io/sa.scc.uid-range: <>
Use kubectl apply -f <namespace.yaml> or oc apply -f <namespace.yaml>.

Related

Kubernete CreateContainerConfigError

I'm trying to deploy an app on k8s but I keep getting the following error
NAME READY STATUS RESTARTS AGE
pod_name 1/2 CreateContainerConfigError 0 26m
If I try to see what went wrong using kubectl describe pod pod_name I get the following error: Error from server (NotFound): pods "pod_name" not found
You didn't include the command that generated the output shown, so it's hard to tell. Perhaps you're looking at different namespaces?
One of the parameters key in the file was misspelling making the deploy fail. Unfortunately, the error message was not helpful...
CreateContainerConfigError means kubernetes is unable to find the configmap you have provided for volume.Make sure both pod and configmap are deployed in same namespace.
Do cross verify the name of configmap you have created and configmap name you have specified in pod volume definition is same.
the message Error from server (NotFound): pods "pod_name" not found is very clear for me that you deployed your pod in a different namespace
In your deployment yaml file check the value of namespace and execute the command
kubectl describe pod pod_name -n namespace_from_yaml_file

AKS | How to integrate VerticalPodAutoscaler

I am trying to implement an example application for VerticalPodAutoscaler (VPA) and got this error
error: unable to recognize "foo.yaml": no matches for kind "VerticalPodAutoscaler" in version "autoscaling.k8s.io/v1beta2"
Source code refered: https://medium.com/infrastructure-adventures/vertical-pod-autoscaler-deep-dive-limitations-and-real-world-examples-9195f8422724
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: bar
spec:
targetRef:
apiVersion: "apps/v1"
kind: Deployment
name: foo
updatePolicy:
updateMode: "Off"
I also tried combinations of v1, vXbetaY but nothing was working.
Debugging Done:
I tried to search specific examples for Azure AKS VPA but did not find any relevant documentation.
I did this kubectl api-resources | grep autoscaling and ONLY HorizontalPodAutoscaler is present in this list
Any thing that I am missing for getting VPA working on AKS?
well, since its a custom resource, you first need to install it. https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler#installation

Openshift: User "system:serviceaccount:<project>:jenkins" can't create PV : RBAC: clusterrole.rbac.authorization.k8s.io "create" not found

Openshift/okd version: 3.11
I'm using jenkins-ephemeral app from the openshift catalog and using a buildconfig to create a pipeline. Reference: https://docs.okd.io/3.11/dev_guide/dev_tutorials/openshift_pipeline.html
When i start the pipeline, in one the stage of jenkins it needs to create a persistent volume, at that point im getting the following error:
Error from server (Forbidden): persistentvolumes is forbidden: User "system:serviceaccount:pipelineproject:jenkins" cannot create persistentvolumes at the cluster scope: RBAC: clusterrole.rbac.authorization.k8s.io "create" not found
I have tried giving the cluster-create role to service account jenkins with following command, still im getting the same error.
oc adm policy add-cluster-role-to-user create system:serviceaccount:pipelineproject:jenkins
Creating a PersistentVolume is typically something that you should not be manually doing. You should ideally be relying on PersistentVolumeClaims. PersistentVolumeClaims are namespaced resources, that your service account should be able to create with the edit Role.
$ oc project pipelineproject
$ oc policy add-role-to-user edit -z jenkins
However, if it's required that you interact with PersistentVolume objects directly, there is a storage-admin ClusterRole that should be able to give your ServiceAccount the necessary permissions.
$ oc project pipelineproject
$ oc adm policy add-cluster-role-to-user storage-admin -z jenkins

How to get resolved sha digest for all images within Kubernetes yaml?

Docker image tags are mutable, in that image:latest and image:1.0 can both point to image#sha256:....., but when version 1.1 is released, image:latest stored within a registry can be pointed to an image with a different sha digest. Pulling an image with a particular tag now does not mean that an identical image will be pulled next time.
If a Kubernetes YAMl resource definition refers to an image by tag (not by digest), is there a means of determining what sha digest each image will actually resolve to, before the resource definition is deployed? Is this functionality supported using kustomize or kubectl?
Use case is wanting to determine what has actually been deployed in one environment before deploying to another (I'd like to take a hash of the resolved resource definition and could then use this to understand whether image:1.0 to be deployed to PROD refers to the same image:1.0 that was deployed to UAT).
Are there any tools that can be used to support this functionality?
For example, given the following YAML, is there a way of replacing all images with their resolved digests?
apiVersion: v1
kind: Pod
metadata:
name: example
spec:
containers:
- name: image1
image: image1:1.1
command:
- /bin/sh -c some command
- name: image2
image: image2:2.2
command:
- /bin/sh -c some other command
To get something like this:
apiVersion: v1
kind: Pod
metadata:
name: example
spec:
containers:
- name: image1
image: image1#sha:....
command:
- /bin/sh -c some command
- name: image2
image: image2#sha:....
command:
- /bin/sh -c some other command
I'd like to be able to do something like pipe yaml (that might come from cat, kustomize or kubectl ... --dry-run) through a tool and then pass to kubectl apply -f:
cat mydeployment.yaml | some-tool | kubectl apply -f -
EDIT:
The background to this is the need to be able to prove to auditors/regulators that what is about to be deployed to one env (PROD) is exactly what has been successfully deployed to another env (UAT). I'd like to use normal tags in the deployment template and at the time of deploying to UAT, take a snapshot of the template with the tags replaced with the digests of the resolved images. That snapshot will be what is deployed (via kubectl or similar). When deploying to PROD, that same snapshot will be used.
This tool is supporting exactly what you need...
kbld: https://get-kbld.io/
Resolves name-tag pair reference (nginx:1.17) into digest reference
(index.docker.io/library/nginx#sha256:2539d4344...)
Looks integrates quite well with templating tools like Kustomize or even Helm
You can all the containers used info with this command. This will list all namespaces, with pod names, with container image name and sha256 of the image.
kubectl get pods --all-namespaces -o=jsonpath='{range .items[*]}{"\n"}{.metadata.namespace}{","}{.metadata.name}{","}{range .status.containerStatuses[*]}{.image}{", "}{.imageID}{", "}{end}{end}' | sort
is there a means of determining what sha digest each image will actually resolve to, before the resource definition is deployed?
No, and in the case you describe, it can vary by node. The Deployment will create some number of Pods, each Pod will get scheduled on some Node, and the Kubelet there will only pull the image if it doesn’t have something with that tag already. If you have two replicas, and you’ve changed the image a tag points to, then on node A it could use the older image that was already there, but on node B where there isn’t an image, it will pull and get the newer version.
The best practice here is to avoid changing the image a tag points to. Give each build coming out of your CI system a unique tag (a datestamp or source control commit ID for example) and use that in your Kubernetes object specifications. That avoids this problem entirely.
A workaround is to set
imagePullPolicy: Always
in your pod specs, which will force the node to pull a newer version, but this is unnecessary overhead in most cases.
Here's another on - k8s-digester from google folks. It's a bit different in a sense than the main focus is on cluster-side changes(via Adm Controller) even though client-side KRM functions seems to also be possible.
Overall, kbld seems to be more about development experience and adoption with your cli/CICD/orchestration, while k8s-digester is more about administration on the cluster side.

Argo artifact passing cant save output

I am trying to run the artifact passing example on Argoproj. However, I am getting the following error:
failed to save outputs: verify serviceaccount platform:default has necessary privileges
This error is appearing in the first step (generate-artifact) itself.
Selecting the generate-artifact component and clicking YAML gives following line highlighted
Nothing appears on clicking LOGS.
I need to understand the correct sequence of steps in running the YAML file so that this error does not appear and artifacts are passed. Could not find much resources on this issue other than this page where the issue is discussed on argo repository.
All pods in a workflow run with the service account specified in workflow.spec.serviceAccountName, or if omitted, the default service account of the workflow's namespace.
Here the default service account of that namespace doesn't seem to be given any roles by default.
Try granting a role to the “default” service account in a namespace:
kubectl create rolebinding argo-default-binding \
--clusterrole=cluster-admin \
--serviceaccount=platform:default \
--namespace=platform
Since the default service account now gets all access via the 'cluster-admin' role, the example should work now.

Resources