PodSecurityPolicy (PSP) is hard to get right in the first attempt. There has never been a situation when I haven’t banged my head to get it working on the cluster. It is a frustrating experience, but it is one of the essential security features of Kubernetes. Some applications have started shipping the PSP configs with their helm charts, but if a helm chart does not ship a PSP config, it must be handcrafted by the cluster-admin to make the application work reliably in the cluster.
This post will define a nomenclature for PSP types, how a cluster-admin can create a PSP that is allowed for all workloads, and how a developer can generate PSP from scratch. This post won’t try to define steps on achieving above but will give guidelines and point you into the right direction for securing your cluster with PSPs.
Some PSPs also mutate the pod specification besides allowing or disallowing pods. The policy order of Kubernetes documentation states the following two ways a PSP is chosen for a pod:
- If a PSP allows the pod specification as is without a mutation, then that PSP is used.
- If the above condition fails, then the fist PSP is chosen from an allowed-PSP list, and the pod is mutated accordingly.
So from the above policy order, we can infer that we will need two sets of PSPs in a cluster:
- Application-Specific PSP: This ensures that the pod passes as it is without a mutation. The application developers have to ensure that the PSP config they ship with the deployment manifests should match exactly otherwise their application could either be disallowed from working in a PSP enabled cluster or runs at reduced privileges (due to mutation) enforced by cluster-wide PSP.
- Cluster-wide PSP: The cluster should have a “restrictive” policy which comes into the picture as per the second point in the policy order. Such a restrictive PSP should be allowed to all the workloads.
These are the general steps I take when crafting the PSP that is application-specific:
- Lockdown the application with the help of pod’s security context. Explore each option in the security context and decide what is suitable for the application.
- Ensure that the application is working fine in the above-restricted settings previously unknown to the application.
- Start from privileged PSP (see below) and keep restricting it as per the application pod specification.
apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: privileged annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*' spec: privileged: true allowPrivilegeEscalation: true allowedCapabilities: - '*' volumes: - '*' hostNetwork: true hostPorts: - min: 0 max: 65535 hostIPC: true hostPID: true runAsUser: rule: 'RunAsAny' seLinux: rule: 'RunAsAny' supplementalGroups: rule: 'RunAsAny' fsGroup: rule: 'RunAsAny'
Once the above conditions are satisfied, it does not matter what you name it, because this PSP will be given first priority over any other allowed PSP since it enables the pod created by your application to pass precisely without any mutation.
In the above gif, you can see that the wall has a hole (silhouette) of Bugs Bunny, think of it as if the Kubernetes cluster only allows PSP with bunny properties lets call it
bunny-psp. Now the Elmer Fudd tries to enter through it but cannot since no PSP allows his properties to pass through. So the cluster-admin has to either create a PSP that matches his profile, and he can pass through the wall without hitting it.
These PSPs apply to the workloads that don’t ship their own PSP manifests. You would want your clusters to have a catch-all PSP config. Such a generalised PSP should ideally be restrictive in nature. The PSP should have everything locked down like no root user, no privilege escalation using
privileged field, all the host namespace fields are disallowed, etc.
apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: aa-restricted annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default,runtime/default' apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default' seccomp.security.alpha.kubernetes.io/defaultProfileName: 'runtime/default' apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default' spec: privileged: false # Required to prevent escalations to root. allowPrivilegeEscalation: false # This is redundant with non-root + disallow privilege escalation, # but we can provide it for defense in depth. requiredDropCapabilities: - ALL # Allow core volume types. volumes: - 'configMap' - 'emptyDir' - 'projected' - 'secret' - 'downwardAPI' # Assume that persistentVolumes set up by the cluster admin are safe to use. - 'persistentVolumeClaim' hostNetwork: false hostIPC: false hostPID: false runAsUser: # Require the container to run without root privileges. rule: 'MustRunAsNonRoot' seLinux: # This policy assumes the nodes are using AppArmor rather than SELinux. rule: 'RunAsAny' supplementalGroups: rule: 'MustRunAs' ranges: # Forbid adding the root group. - min: 1 max: 65535 fsGroup: rule: 'MustRunAs' ranges: # Forbid adding the root group. - min: 1 max: 65535 readOnlyRootFilesystem: false
Such a policy is allowed to every authenticated user (including service accounts) via the group
system:authenticated. In the
ClusterRoleBinding, when you provide
subjects list the aforementioned group as one of them and the
ClusterRole mentioned under the
roleRef is bound to each and every authenticated user on the cluster.
- kind: Group apiGroup: rbac.authorization.k8s.io name: system:authenticated
Allowing this policy to every authenticated user (even workload users a.k.a. service accounts) is not enough. You also need to make sure that this policy is on the top of the alphabetically sorted list. Ensure that the policy name starts with
aa, so it is always on the top of the list.
A generic restrictive PSP that mutates the pods looks like above, cheese cutting/moulding machine. It is ensuring that whatever comes out is of fixed mould, in case of Kubernetes the pod has fixed security restrictions.
I hope this blog gives you some mental models of understanding how PSPs works in general. Although PSP is being discussed in upstream for replacing with something robust like the OPA Gatekeeper, it is
not official yet. It is not clear when will that materialise, but until then, PSP is the saviour we have.
If you are still confused about this, please reach out to me I will happy to explain.
EDIT: The PSP is officially deprecated, I was wrong before to mention it is not deprecated yet. This was pointed out to me on a Reddit thread. According to this Kubernetes PR, PSP will be deprecated in Kubernetes
v1.21 and removed entirely in