Skip to main content
Migration Notice
We're migrating documentation from the old portal into this one. Some things may look a little different or out of place in the meantime — we know, and we're working to get it right. If something's unclear or doesn't look right, let us know.

K8s Audit

The K8s Audit tab turns on and configures Kubernetes API audit ingestion. The console collects the Kubernetes API server audit stream so that pod create, update, and delete actions are attributed to the user who performed them.

That identity appears as the Pod Creator in File Activity search rows and as the user node in the Data Security Posture topology. It lets you trace a storage event back through the pod to a person.

Setup Overview​

To turn on attribution, complete these tasks in order:

  1. Enable ingestion on this tab.
  2. Generate the webhook secret.
  3. Choose the webhook URL for each cluster.
  4. Configure the cluster API server to send audit events.
  5. Check Receiving Health to confirm that events arrive.

How It Works​

The Kubernetes API server produces an audit event for every API request. When ingestion is enabled, the API server pushes batches of audit events to a per-cluster webhook on the console, authenticated with a shared bearer secret. At each inventory scan, the console records the acting user on the pod.

Attribution applies only to actions that occur after ingestion is configured. A pod that already existed shows an empty Pod Creator until it is next updated or deleted.

Pod user columns​

The product shows two columns that answer "who". This feature provides the first.

  • Pod Creator — the identity that issued the pod-create API call. It can be a person, or a controller or service account when the pod was created indirectly, for example a Deployment's ReplicaSet controller or a RunAI scheduler service account.
  • Workload Owner — the user who submitted the RunAI job. This is the person behind a training workload even when a controller created the pod.

Together they distinguish who launched the job from the identity the platform used to create the pod.

Enable Ingestion​

  • Enable audit ingestion — the master switch. When it is off, the console rejects every delivery with a 401 response.
  • Ingest mode — Webhook (push) is the supported mode. The API server posts audit batches directly to the console. It requires a self-managed cluster, such as RKE2, kubeadm, or k3s, where you control the API server flags. Managed control planes such as EKS, AKS, and GKE cannot set the API server webhook flags and are not supported.

Webhook Secret​

The webhook secret is the shared bearer token that the API server sends as Authorization: Bearer <secret>. It is stored AES-256 encrypted and shown masked (********).

  • Generate secret — creates a strong random token and stores it encrypted. Copy the revealed value immediately and paste it into the audit-webhook kubeconfig of the API server.
  • Reveal — shows the stored secret on demand. This action is logged.
  • Rotate — replaces the secret. The old token stops working immediately, and the cluster stops ingesting until you update the API server with the new value.

Webhook URL​

Each source cluster has its own webhook URL, in this form:

/api/k8s-audit/webhook/{clusterId}

{clusterId} is a short identifier that you choose for the cluster, using lowercase letters, digits, ., and -. The console stamps it on every event so that you can tell clusters apart in search and topology.

Point the API server's --audit-webhook-config-file at this URL. The tab shows suggested URLs for each registered cluster.

Configure the Cluster​

Before you start, enable ingestion, generate the webhook secret, and note the webhook URL for the cluster. The webhook receives events only after the API server is configured to send them. This step most often blocks ingestion, and it requires control-plane access (root on the API server node).

  1. Write an audit policy file on the control-plane node. It defines which events to capture.
  2. Write an audit-webhook kubeconfig. Set server: to the webhook URL and set the user's token: to the generated secret.
  3. Add the API server flags --audit-policy-file, --audit-webhook-config-file, and --audit-webhook-mode=batch, then restart the API server.

For the recommended audit policy, the webhook kubeconfig, the exact flags for kubeadm, RKE2, and k3s, and the reasons Pod Creator can be empty, see Kubernetes API Audit Ingestion.

warning

Capture workloads at all mutating verbs, not only pods. Pod Creator resolves to the person who authored the pod's root workload (Deployment, StatefulSet, DaemonSet, Job, or CronJob), because controllers name pods with generateName while people name workloads.

The audit policy must capture deployments, statefulsets, daemonsets, jobs, and cronjobs for create, update, patch, and delete, at Metadata level or higher. create alone is not enough. A workload that existed before ingestion, or one that a GitOps tool only patches, is attributed to whoever last modified it. With a pods-only policy, Pod Creator shows only the controller service account, never the person.

The recommended policy already includes these resources. Keep them if you write your own policy.

warning

The bearer token requires HTTPS. The Kubernetes API server client attaches the bearer token only over https:// or loopback. If the console is served over plain http://, put an HTTPS terminator in front of it and point the kubeconfig at the https:// address. Otherwise the token is silently dropped and every delivery is rejected as unauthenticated.

Receiving Health​

The status card shows whether events are arriving.

StatusColorMeaning
Receiving eventsGreenAt least one event in the last 24 hours.
Enabled — no events yet or No events in last 24hAmberIngestion is enabled and configured, but no events arrive. Usually the cluster side is not configured, the API server cannot reach the console, or the bearer token does not match.
Secret not configured or DisabledGreyIngestion is off or has no secret.

The card also shows the time of the last event, the event counts for the last 24 hours and in total, and the number of distinct clusters that have delivered events.

Retention and Alarms​

  • Retention of stored audit events is set under Settings > Advanced Settings > ClickHouse retention > K8s audit events (days).

Troubleshooting​

SymptomWhat to check
The badge stays on "no events"Confirm that the cluster side is configured and the API server was restarted. Check the API server logs for audit webhook errors. Verify that the bearer token matches, because rotating a secret invalidates the old one. Confirm that the API server can reach the webhook URL.
The API server logs "the server has asked for the client to provide credentials"The webhook uses http://. Change the kubeconfig server: to an https:// endpoint so the token is sent.
Events arrive but Pod Creator is empty for a podAttribution applies only to actions after ingestion started, and the pod was created earlier. New pods, and the next update or delete of an existing pod, are attributed.