Kubernetes API Audit Ingestion

Kubernetes API audit ingestion lets the platform answer "which user did this?" for workloads. It collects the kube-apiserver audit stream so that pod create, update, and delete actions are attributed to the identity that performed them. The identity appears as the Pod Creator in File Activity and as a User node in the Data Security Posture topology.

Combined with storage audit (PowerScale, ObjectScale, and Trino), it traces a file or object operation back through the pod and the PVC or bucket to a person.

This page is the end-to-end setup and concept reference. To turn ingestion on and manage the secret, use Settings > K8s Audit. See K8s Audit settings. This page covers the concepts and the cluster-side configuration, which is where setup most often stops.

Pod Creator and Workload Owner

The product shows two complementary identities for a workload, from two independent sources.

Column	Source	Answers	Typical value
Pod Creator	Kubernetes API audit (this feature)	Which identity authored the workload behind this pod?	The person who created or changed the Deployment, StatefulSet, or Job (including the cluster admin). An automation identity when a GitOps tool, CI system, or operator service account deployed it.
Workload Owner	NVIDIA RunAI correlation	Which person submitted the training job?	The data scientist behind the workload, even when a controller created the pod.

The two columns are not redundant. A RunAI training job is submitted by a data scientist (Workload Owner), but the pod is usually created by the service account of the RunAI scheduler (Pod Creator). Seeing both lets you distinguish who launched the work from the identity the platform used to create the pod.

For workloads that do not use RunAI, there is no Workload Owner. Pod Creator is the only user attribution, so without Kubernetes API audit these pods trace back to nobody.

Both columns are filled on every File Activity search (PowerScale and ObjectScale). Both also feed the User nodes of the Data Security Posture topology. The topology uses the RunAI user first and falls back to the Pod Creator from Kubernetes audit. It excludes system:* identities as noise.

How events flow

1. The kube-apiserver emits an audit event for every API request that matches the audit policy.
2. The kube-apiserver sends batches of events to the console at https://<console>/api/k8s-audit/webhook/{clusterId}. The shared bearer secret authenticates the request.
3. The console stores the events in ClickHouse.
4. At each inventory scan, the console resolves the root workload of each pod from its owner references. The root workload is a Deployment, StatefulSet, DaemonSet, Job, CronJob, or a bare pod. The console attaches the audit attribution of that workload to the pod. One lookup serves all replicas of a workload.

How attribution works

• Only new activity is captured. The console captures only activity that occurs after ingestion is configured. A workload that already existed has an empty Pod Creator until it is next updated, patched, or deleted.
• All modifying actions count. Pod Creator covers create, update, patch, and delete. For a workload that predates the webhook, or that a GitOps tool only patches, the identity that last changed it is the available answer.
• Attribution is stored. The resolved attribution is kept after the source audit events age out of their retention window. Storage-audit rows also get a best-effort fill-in at search time.
• Controller-created pods are attributed to the person. A controller (for example a Deployment's ReplicaSet or a Job) names the pods it creates. The console resolves the pod's root workload and attributes the pod to the person who authored that workload. For a Deployment created by alice, the Pod Creator is alice.
• Automation is a valid answer. When software created the workload (ArgoCD, CI, or another operator service account), the Pod Creator is that automation identity, marked as automation. This is a complete answer, not a gap.
• A blank Pod Creator means not attributed. Either ingestion is off, or the workload predates ingestion and has no create, modify, or delete event in the window.

Ingest mode

Webhook (push) is the only supported ingest mode. The kube-apiserver sends audit batches directly to the console. It requires self-managed clusters (RKE2, kubeadm, or k3s) where you control the API server flags.

warning

Managed control planes (EKS, AKS, and GKE) are not supported. Cloud providers do not expose the --audit-webhook-config-file flag that webhook delivery requires.

Set up webhook mode

Setup has four parts, performed in order: enable the receiver on the console, then write the audit policy, write the webhook kubeconfig, and set the API server flags on the control-plane node. It needs control-plane access (root on the API server node). This is not a Kubernetes RBAC permission. The audit policy and the webhook configuration are static kube-apiserver flags and files on the node, not cluster resources, so no Role or ClusterRole governs them. If your organization restricts control-plane access, the team that owns the control plane must make this change.

The read-only superna-k8security service account that is used for inventory scanning is unrelated and grants no audit permissions.

Enable the receiver and generate a secret

Complete this step on the console first. The next three steps are performed on the control-plane node.

1. Open Settings > K8s Audit.
2. Turn on Enable.
3. Click Generate secret and copy the value. It is shown only once.
4. Copy the per-cluster Webhook URL: https://<console>/api/k8s-audit/webhook/{clusterId}. The {clusterId} is a short identifier that you choose for the cluster, using lowercase letters, digits, ., and -. It is added to every event so that you can tell clusters apart.

warning

HTTPS is required. The kube-apiserver client sends the bearer token only over https:// (or loopback). If the console is served over plain http://, put an HTTPS terminator in front of it and point the kubeconfig at the https:// endpoint. Otherwise the token is dropped and every delivery is rejected as unauthenticated.

Write the audit policy file

On the control-plane node, create the audit policy file that decides which events are captured, for example at /etc/kubernetes/audit-policy.yaml. Use the recommended Superna audit policy. Its coverage balances security value against exposure of secret data:

Resources	Captured
Pods	Full request body (service account, image, security context, and volumes).
Pod subresources (exec, attach, portforward, proxy)	Metadata. These are operator-initiated sessions.
Workload controllers (Deployment, StatefulSet, DaemonSet, Job, CronJob)	Metadata.
RBAC (ServiceAccount, Role, RoleBinding, ClusterRole, ClusterRoleBinding)	Request body, because the body is the policy.
Secrets and ConfigMaps	Metadata only. Request bodies are not captured, so secret material never reaches the audit store.
Nodes, Namespaces, PVCs, PVs, and COSI resources	Metadata.
Everything else	Read-only verbs (get, list, watch) are dropped because of their volume.

warning

If you write your own policy, capture deployments, statefulsets, daemonsets, jobs, and cronjobs for create, update, patch, and delete, at Metadata level or higher. With a pods-only policy, Pod Creator shows only the controller service account and never the person.

Write the webhook kubeconfig

On the control-plane node, create a small kubeconfig, for example at /etc/kubernetes/audit-webhook.kubeconfig. It points the API server at the console and carries the secret from the previous step as a bearer token. Use the Webhook URL that you copied earlier as the server: value, including the console port if it is not the default HTTPS port.

apiVersion: v1
kind: Config
clusters:
  - name: aisecurity
    cluster:
      server: https://<console>/api/k8s-audit/webhook/<clusterId>
      insecure-skip-tls-verify: true   # testing only, use a real CA in production
users:
  - name: aisecurity
    user:
      token: <the secret you generated>
current-context: default
contexts:
  - context: { cluster: aisecurity, user: aisecurity }
    name: default

Set the API server flags and restart

Add these flags to the kube-apiserver. The paths must match the files that you created in the previous two steps. On kubeadm, add the flags to the static pod manifest, and also mount the policy file and the kubeconfig into the pod as hostPath volumes so that the API server can read them. On RKE2 and k3s, add the flags to config.yaml.

--audit-policy-file=/etc/kubernetes/audit-policy.yaml
--audit-webhook-config-file=/etc/kubernetes/audit-webhook.kubeconfig
--audit-webhook-mode=batch
--audit-webhook-batch-max-wait=5s

Then restart the API server. On kubeadm, the API server restarts automatically when the manifest changes. On RKE2 and k3s, restart the server service.

Batch mode is important. If the console is briefly unreachable, the API server is never blocked, because events are dropped instead of queued indefinitely. On busy clusters, you can tune the throttle with --audit-webhook-batch-max-size=400 and --audit-webhook-truncate-enabled=true.

Retention and alarms

• Retention — audit events are kept for the Kubernetes audit retention window. The default is 365 days. Configure it in Settings > Advanced Settings > ClickHouse retention > K8s audit events (days).

Verify that it works

1. In Settings > K8s Audit, wait for the receiving-health badge to change to Receiving events (green). The badge shows the last event time, the 24-hour and total counts, and how many distinct clusters delivered events.
2. Create or delete a pod on the configured cluster, then run an inventory scan.
3. Open File Activity or the Data Security Posture topology and confirm that the pod shows a Pod Creator. For a pod that you created by hand, it is your own identity. For a workload, it can be the controller or service account that created it.
4. Open File Activity > User Actions and select a user. The Kubernetes API actions of that user appear in the audit stream. This is the user-centric view of the same data.

Troubleshooting

• The badge never leaves "no events" — the cluster side is not configured, the API server was not restarted, the console URL is not reachable from the control plane, or the bearer token does not match. Rotating the secret in Settings invalidates the old one, so configure the cluster again with the new secret.
• The API server logs "the server has asked for the client to provide credentials" — the webhook URL uses http://. Change the kubeconfig server: value to an https:// endpoint so that the token is sent.
• Events arrive but the Pod Creator of a pod is empty — attribution applies only to new activity. The pod was created before ingestion started, or by a filtered system:* identity. New pods, and the next update or delete of an existing pod, are attributed.