Database

The Database tab provides operational tuning and sizing for the two databases the system uses: ClickHouse (analytics and audit store) and PostgreSQL (operational state). It tunes an already-connected system. It does not configure the database connections.

Overview

• ClickHouse — a column-oriented analytics database. It holds every audit event ingested from PowerScale and ObjectScale/ECS devices, plus the high-volume security data: pipeline logs, Kubernetes audit events, and analyzer results. It is the authoritative record for file activity and posture history.
• PostgreSQL — the application database. It holds operational state: jobs, settings, ingest offsets, agent registrations, and the Kubernetes topology.

The two stores are independent. ClickHouse audit data is not affected by the application database.

note

Both databases are provisioned and connected by the deployment, and their schemas are created on first boot. You do not enter hosts, ports, or credentials.

Ingest and Retention

• Ingest Batch Size — the number of records sent in each ClickHouse insert during audit-log ingest for PowerScale and ECS SSH audit events. Default: 100,000, which covers one complete ECS audit file in a single insert. For PowerScale, any value above 1,000 works correctly. The minimum recommended value is 1,000.
• Apply Retention — immediately deletes audit events older than the configured retention period from ClickHouse. The retention period itself is set on the Advanced tab (see Retention Settings).

tip

ClickHouse processes the deletion asynchronously. Allow a few minutes after you apply retention before you check row counts.

K8 Inventory Snapshots

Snapshot Frequency sets how often the system captures K8 inventory snapshots (pods, PVCs, and mounts) after each K8 scan.

Option	Effect
Every Scan	Maximum visibility
Hourly	Reduces data volume in environments with frequent scans
Daily	Minimal storage overhead

Snapshots power the Lifecycle Watch page, and the time-travel and deleted-pod detection on Data Security Posture and File Activity.

Retention Settings

The retention period for the highest-volume ClickHouse data is set on the Advanced tab under ClickHouse retention. A saved value applies immediately without a restart. ClickHouse drops rows older than the window on its next merge pass.

Setting	Data covered	Default	Range
Storage audit events (months)	File create, write, and delete events from PowerScale and ObjectScale. 0 means no expiry.	12	0–60
Pipeline logs (days)	Diagnostics logs shown in the Pipeline Diagnostics flyout	7	1–30
K8s audit events (days)	Kubernetes API audit events	365	30–1095
Analyzer results (months)	Output of the linguistic coherence (LC), classification, and object PII analyzers	12	3–36

Lower Analyzer results if you are short on disk space. Raise it to meet legal or eDiscovery retention requirements.

Console Resource and Database Sizing

The Console Resource & Database Sizing sub-tab provides host-resource estimates and tuning recommendations for ClickHouse and PostgreSQL. It has no Save button. Each database has its own Apply button that writes the settings and records the selected tier.

Console Resource and Host Sizing

This card estimates the minimum host resources when the console, ClickHouse, and PostgreSQL share one host. Enter the Number of Agents and Number of Buckets Managed. The card calculates CPU, RAM, and disk for each component and for the combined total, plus a recommended JVM heap size.

ClickHouse Recommended Parameters

A table lists recommended values for ten ClickHouse server and MergeTree parameters, each with a short rationale.

Tier	Target scale
Small	About 1 billion rows, 16 GB RAM host
Medium	About 5 billion rows, 32 GB RAM host
Large	About 10 billion rows, 64 GB RAM host

The parameters are max_server_memory_usage, max_memory_usage, mark_cache_size, uncompressed_cache_size, background_pool_size, background_schedule_pool_size, max_bytes_to_merge_at_max_space_in_pool, parts_to_delay_insert, parts_to_throw_insert, and merge_tree_max_rows_per_part.

A storage estimate card shows the disk usage per tier for audit events, using the compression ratio measured on the live system (about 13.2x). A copyable ClickHouse query lets you read your current compression ratios and row counts at any time.

Apply ClickHouse Table Configuration

Select a tier and click Apply to ClickHouse Tables. Three parameters take effect immediately, without a ClickHouse restart:

• parts_to_delay_insert
• parts_to_throw_insert
• max_bytes_to_merge_at_max_space_in_pool

The other seven parameters require a change to the ClickHouse config.xml file and a ClickHouse restart. A copyable config.xml snippet for the selected tier is shown on screen. A badge in the card header shows the tier that was last applied.

PostgreSQL Recommended Parameters

A table lists recommended PostgreSQL parameters for the write-heavy, small-connection-pool workload of this application, including shared_buffers and effective_cache_size.

Tier	Target host
Small	4–8 GB RAM
Standard	16–32 GB RAM

Apply PostgreSQL Configuration

Select a tier and click Apply. The settings that do not need a restart take effect immediately. shared_buffers, max_connections, and wal_buffers take effect only after PostgreSQL restarts.

note

Applying PostgreSQL settings requires the application database user to have superuser privileges. With table-level privileges only, Apply returns a permission-denied error.

Workflows

Apply an audit retention policy

1. Set the window in Settings > Advanced Settings > ClickHouse retention > Storage audit events (months).
2. Save the Advanced tab to apply the new retention to the stored data over time. To delete out-of-window events immediately, click Apply Retention on this tab.
3. After Apply Retention, wait for the confirmation message.
4. Verify that the ClickHouse row count dropped as expected.

Apply a sizing profile

1. Open the Console Resource & Database Sizing sub-tab.
2. In the ClickHouse or PostgreSQL card, choose the tier that matches your host.
3. Click Apply to ClickHouse Tables (ClickHouse) or Apply (PostgreSQL).

Parameters that need a restart take effect only after the database restarts, even though the badge shows the tier as applied.