Pod security standards restricted defaults

[sheidkamp]

Description

Update Helm templates to allow all containers' securityContexts to be defined in order to support Pod Security Standards.

Provide a flag to default all containers to a securityContext that has the minimal changes needed to conform to restricted policy.

Helm Changes

• Add a new template to render container securityContexts and apply appropriate defaults.
• Add a securityContext for each container that did not have one defined.
• Add globals.podSecurityStandards to manage data related to Pod Security Standards
  • Add globals.podSecurityStandards.container to manage container-level configuration
    • Add global.podSecurityStandards.container.enableRestrictedContainerDefaults - uses restricted compliant defaults for container
    • Add global.podSecurityStandards.container.defaultSeccompProfileType - defines default seccompProfileType to use for containers.

The default securityContext when global.podSecurityStandards.container.enableRestrictedContainerDefaults is enabled is equivalent to:

securityContext:
  allowPrivilegeEscalation: false
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault
  capabilities:
    drop:
    - ALL

There is no variation allowed for these values, except seccompProfile.type can be Localhost instead of RuntimeDefault. That value can be configured with global.podSecurityStandards.container.defaultSeccompProfileType. capabilities.add can also be set to

Context

Users ran into this while attempted to run Gloo Edge with a Restricted Pod Security Policy

Testing steps

Manual validation

Default behavior

With a kubernetes environment created, gloo not installed, and $VERSION defined:

• create gloo-system namespace and label as warn=restricted

kubectl create namespace gloo-system
kubectl label --overwrite namespace gloo-system pod-security.kubernetes.io/warn=restricted

• build and package helm charts:

make generate-helm-files build-test-chart -B

• Install gloo:

helm install gloo _test/gloo-${VERSION}.tgz --namespace gloo-system

You will see output that looks like:

W0516 12:46:38.559070   10276 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "certgen" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "certgen" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "certgen" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
W0516 12:46:43.713909   10276 warnings.go:70] would violate PodSecurity "restricted:latest": seccompProfile (pod or container "gateway-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
W0516 12:46:43.715139   10276 warnings.go:70] would violate PodSecurity "restricted:latest": seccompProfile (pod or container "discovery" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
W0516 12:46:43.715403   10276 warnings.go:70] would violate PodSecurity "restricted:latest": seccompProfile (pod or container "gloo" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
W0516 12:46:43.719802   10276 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "kubectl" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "kubectl" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "kubectl" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
W0516 12:46:44.012042   10276 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "kubectl" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "kubectl" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "kubectl" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
NAME: gloo
LAST DEPLOYED: Thu May 16 12:46:36 2024
NAMESPACE: gloo-system
STATUS: deployed
REVISION: 1
TEST SUITE: None

Restricted Compliant behavior

• Uninstall Gloo Edge:

glooctl uninstall

• Install Gloo Edge with default restricted container securityContexts

helm install gloo _test/gloo-${VERSION}.tgz --namespace gloo-system --set global.podSecurityStandards.container.enableRestrictedContainerDefaults=true

The warnings no longer appear.

• The namespace can be labelled to enforce the Restricted Security Standard:

k label --overwrite namespace gloo-system pod-security.kubernetes.io/enforce=restricted

And Gloo Edge can be successfully uninstalled and reinstalled.

Automated testing

Additionally, unit tests have been added/expanded, and the global.podSecurityStandards.container.enableRestrictedContainerDefaults=true has been set on the kube2e tests to ensure that functionality works with these standards applied.

The test server used for the kube2e tests uses an image that runs as root, so for the moment we are not applying the labels to the namespace in the e2e tests, instead relying on the helm flag to enforce compliance for Edge components. This can revisited if deemed worth the time and effort.

Notes for reviewers

• Check Helm template parameter names and documentation
• Look for any missed containers

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works

[bewebi]

I haven't done the manual testing, but overall this passes the eye test

I'm hoping that indentation can be more consistent and/or that there can be an explanation for why it needs to be different in certain cases

[sheidkamp]

Updated with PR feedback

[bewebi]

LGTM

I also did some of the local validation