Skip to main content
Migration Notice
We're migrating documentation from the old portal into this one. Some things may look a little different or out of place in the meantime — we know, and we're working to get it right. If something's unclear or doesn't look right, let us know.

ClickHouse Health

The ClickHouse Health page provides deep diagnostics for the analytics database that stores all audit events, copy job metrics, and ingest telemetry. Use it to investigate database performance issues, stuck mutations, and storage growth.

Overview​

ClickHouse is the authoritative store for audit event data. This page shows internals that application-level status checks do not show: mutation state, merge pressure, active queries, and per-table storage metrics. Review it regularly, particularly after you run data wipe operations.

The page refreshes every 15 seconds by default. Pause in the header freezes the view while you investigate, and Resume restarts the refresh.

Summary cards​

The header shows the ClickHouse version and how long the server has run since its last restart. The summary cards below it give a quick snapshot of database health.

CardMeaning
Memory (Resident)Resident memory of the ClickHouse process, with the tracked and virtual memory underneath
Active QueriesQueries currently executing, with the number of active merges and delayed inserts underneath. The card turns red when inserts are being delayed.
ConnectionsOpen client connections, split into TCP and HTTP
Mutation HealthPending mutations, with the total and failed mutation counts underneath
Merge PressureThe highest number of active parts in any single partition, with the number of tables merging now underneath

Below the cards, a band shows cumulative rates since the server started: queries, inserts and selects per second, average query and insert time, inserted rows and MB per second, and the error rate.

Table Storage & Parts Health​

Each table is listed with these metrics:

MetricMeaning
RowsTotal rows stored in the table
CompressedDisk space used after compression
RatioCompression ratio. A higher ratio means the table's data compresses well.
PartsNumber of active data parts the table is split into
DeepParts that have been merged many times
ModifiedWhen the table was last written to

ClickHouse merges parts in the background. A very high part count means many small inserts arrived faster than ClickHouse can merge them. This is a warning sign, but it usually resolves itself once the insert rate drops.

Mutations​

This panel lists the 20 most recent ALTER TABLE mutations, pending ones first. The Status column shows:

  • Done (green) — the mutation completed successfully.
  • Pending (red) — the mutation is still running.
  • Failed (orange) — the mutation reported a failure.

When any mutation is pending, a warning banner appears at the top of the page.

warning

A completed mutation that stays registered in the system re-applies to all new data inserted into that table. If a completed mutation is not cleaned up, issue a KILL MUTATION command through the ClickHouse HTTP API immediately.

Slow Query Log​

The Recent Queries — Last 1 Hour (slowest first) panel lists the 50 slowest queries that the application ran in the last hour. Click a row to expand the full SQL. Use the panel to find inefficient query patterns or missing filters that cause full-table scans.

note

On a Kubernetes (Helm) deployment the slow-query log is off by default to conserve disk space. On the VMware appliance (OVA) it is on, and entries are kept for 14 days. When it is disabled, the panel shows a "query log disabled" badge and an explanation instead of a table. The Currently Running Queries panel remains the live view of what is running now. The panel fills automatically if query log retention is later enabled on the ClickHouse server.

Currently Running Queries​

This panel lists the queries running now, with the elapsed time and the user that submitted each query. Use it to find runaway queries that consume disproportionate resources. A query that has run for an unexpected length of time can be killed through the ClickHouse HTTP API.

Workflows​

Check for stuck mutations after a data wipe​

  1. Open the Mutations panel.
  2. Look for any mutation in a state other than Done.
  3. If a completed mutation is still present, issue a KILL MUTATION for it through the ClickHouse HTTP API.
  4. Refresh the page to confirm the mutation is removed.

Investigate a table with slow inserts​

  1. Open the Table Storage & Parts Health panel.
  2. Find the table and check its part count.
  3. If the part count is very high, check the Merge Pressure card and panel.
  4. If merges are active on that table, the table is catching up on merges. Inserts stay slow until the merges complete.

Find the cause of high ClickHouse CPU usage​

  1. Open the Currently Running Queries panel.
  2. Look for queries with a high elapsed time that belong to the application user.
  3. Expand the Recent Queries — Last 1 Hour (slowest first) panel to read the full SQL of any suspicious query. This panel requires the slow-query log to be enabled.

Tips​

  • After you run any wipe or delete operation from the application, return to this page and check the Mutations panel. Completed mutations that are not killed re-apply silently to every new row inserted into the affected table and corrupt future data.
  • High merge pressure with many parts on a table usually resolves within minutes once insert pressure eases. It needs manual intervention only if it persists for hours.
  • Use Pause when you need to read a slow-query entry or examine a mutation without the panel refreshing. Select Resume when you are done.