Vault Data Sync Dashboard

Operational Mode panel

At the top of the Vault Data Sync Dashboard is the Operational Mode panel. It shows whether the system is in Preparation Mode or Vault Mode Active.

The same mode appears as a badge in the AirGap for ObjectScale page header, next to the AWS SDK version, on every Cyber Vault tab:

Badge	Meaning
Vault Mode (blue)	At least one protection group is in Vault Mode. (N/M groups) is added when only some groups are.
Preparation Mode (amber)	No protection group is in Vault Mode.

Hover over the badge to see the group count. The badge refreshes every 30 seconds, so entering or exiting Vault Mode shows up in the header without reloading the page.

Preparation Mode

Protection groups are controlled from the console. The console sends each sync's copy jobs to the copy agents. This is the normal operating state before a vault network is isolated.

There is no automatic incremental schedule in this mode. A sync starts only when you click one of these:

• Sync Group Now
• A bucket's Sync All
• A window's Run Sync, Resume, or Re-sync

The panel subtitle still reads "Console scheduler manages all protection groups".

Vault Mode Active

The vault agent decides when vault-mode groups sync. At each scheduled run it opens the airgap, asks the console to start the sync, and closes the airgap when the sync is done. As in Preparation Mode, the copy itself runs on the copy agents.

You cannot start syncs by hand for these groups.

The panel shows:

• How many protection groups are under vault agent control.
• A View groups button to expand the per-group mode list.
• An Exit Vault Mode button to return the groups to Preparation Mode.

Switch to Vault Mode

Click Switch to Vault Mode to hand the sync schedule to a vault agent. From then on the vault agent starts each sync, not an operator.

You must select an ONLINE vault agent that has a registered ML-KEM-1024 post-quantum key. All protection groups in Preparation Mode are converted at the same time.

Console Topology

When you select a vault agent, a Console Topology callout shows that agent's saved console-location setting:

Topology	Behavior
OUT-OF-VAULT (default)	The console sits outside the vault, on the production management network. The link between the copy agents and the console is severed during closed-airgap periods. Copy agents buffer heartbeats locally and replay them after the vault agent's next open cycle restores reachability.
IN-VAULT	The console shares the vault-side network with this agent's copy agents. Copy agents send heartbeats to the console directly and continuously. Heartbeat buffering, spool, and replay are never used. Use only after you consult support.

This is a read-only confirmation, not a choice you make at activation. The topology is a physical and network fact about the deployment, and it does not change between activate and deactivate cycles.

If the setting needs correction, click Change. It opens this vault agent's config panel on the Vault Agents > Dashboard tab, expanded inline.

Exit Vault Mode

Click Exit Vault Mode to reverse the Vault Mode activation and return the groups to Preparation Mode.

In the dialog, Vault Agent to Exit picks whose groups return:

• Leave it at All vault agents (system-wide exit) to return all groups.
• Select one agent to return only that agent's protection groups. Other agents stay in Vault Mode.

Before you confirm, the dialog lists every action that takes effect:

• The vault-mode protection groups return to Preparation Mode. Their incremental syncs then run only when started by hand (Sync Group Now, Sync All, or a window's buttons).
• The vault agent's scheduling authority is revoked. It no longer drives sync windows or airgap cycles for these groups.
• Vault agents remain registered. Their ML-KEM-1024 keypairs, Quantum Trust Domain device assignments, and delivered credentials are fully preserved.
• Target auth ownership returns to the console. Service account provisioning and key rotation for the vault ObjectScale devices are unblocked on the Target Auth Service Accounts tab.
• Device credential statuses (QUEUED, DELIVERED) in Vault Device Management are unaffected. No passwords are invalidated.
• Vault agent heartbeats continue. Agents stay connected and keep reporting status.

You can re-activate Vault Mode at any time from this panel.

Overview

The Vault Data Sync Dashboard gives you a live, unified view of every protection group and its incremental sync windows.

For each group you can see which sync windows (slots, 6 hours by default) are pending, actively syncing, completed, or failed. You can also drill into the object-level changelist for any slot.

This is the primary operational view for monitoring the incremental copy pipeline that keeps vault-side buckets in sync with the source.

Dashboard tabs

• Sync Windows: the protection groups and their sync windows, described below.
• Active Vault Syncs: protection groups in Vault Mode and the jobs their vault agent started.
• Settings: the global switches for incremental sync.

Settings tab

The Settings tab holds these controls:

Setting	Options
Incremental Sync	Enable or Disable
Smart Skip	Enable or Disable
Planned Sync Time	Random within window, or Window start
Sync Time Limit	1–168 hours (default 24)

See the Incremental Sync help for what Incremental Sync and Smart Skip each do.

Planned Sync Time

Planned Sync Time controls when, inside each protection group's schedule window, the vault agent syncs.

• Random within window (default): a different, unpredictable moment every window. An outsider cannot know in advance when the airgap opens.
• Window start: the start of the window, so the time is predictable.

The vault agent starts each group's sync within 30 seconds of its planned time, whether or not a heartbeat is due. The heartbeat interval does not delay syncs. Several groups can sync at the same time, and the vault stays open until the last one finishes.

The agent also:

• Syncs once per window. Failed windows are still retried.
• Keeps a window that crosses midnight open until it ends.
• Follows the policy's run calendar (First Sync date + k × Repeat Every N days). There is no catch-up the day after a missed or failed run day. See the Sync Schedules help.

Once a window's sync has started, the Agent: Today HH:MM badge next to each vault-mode group adds · syncing since, · synced, · failed, or · skipped with the time.

The setting reaches each vault agent on its next heartbeat. It appears as a HEARTBEAT_CONFIG record vaultRandomPlannedSyncTime=true|false in the Quantum Trust Protocol Audit Log. It applies from the next window. A time already picked for the open window is kept.

tip

Prefer Random unless you need a predictable time. Window start makes groups that share a schedule policy start together and gives up the unpredictability.

Sync Time Limit

Sync Time Limit (1–168 hours, default 24) is how long the vault agent follows a sync's jobs. Enter the value and click Save.

The value reaches each vault agent on its next heartbeat as a HEARTBEAT_CONFIG record vaultSyncTimeoutHours=<n>. It applies to syncs that start after that.

If the jobs are still running after the limit:

1. The agent reports it with alarm CV-116 — CyberVault sync exceeded time limit. It also writes a SYNC_TIMEOUT record, with the group and job IDs, to the Quantum Trust Protocol Audit Log.
2. The agent closes the group's network and releases the vault. The jobs are not stopped. They keep running on the copy agent. With a real airgap, copies still in progress while the vault is closed fail and are retried later.
3. At its next heartbeat, the agent finds the jobs still running, re-adopts them, and opens the vault again. If they are still running one more limit later, the same sequence repeats.

CV-116 clears itself within about a minute after all the jobs it lists have finished (completed, failed, or canceled). You can also clear it by hand.

While a CV-116 alarm is active, no further CV-116 is raised, for this group or any other. The SYNC_TIMEOUT records in the QTP Audit Log list every occurrence.

Use CV-116 to decide whether to wait or cancel the jobs.

View modes

Live Mode (default)

Live Mode shows the current state across all groups and their configured buckets (Live View, trailing 30 days).

Data comes from the in-memory group cache with no ClickHouse query, so it updates very quickly. When any bucket is actively syncing, the view refreshes every 30 seconds with a countdown timer.

Week View

Use the Previous week arrow or the calendar picker (Pick a week start date) to review an earlier 7-day period. The period is shown as a date range such as Apr 1 – Apr 7. Click Live to return to the live view.

Expand a bucket to see its slots grouped by day, newest first. Load earlier week goes back another week.

Expanding a slot fetches the object-level changelist (creates, modifies, deletes) from ClickHouse.

Sync window slots

Each row represents one sync window (slot) of a bucket, 6 hours by default. The live view and the week view label the statuses slightly differently.

Status (live view / week view)	Meaning
Open · closes … / Open · …	The window is still open and changes are still being collected.
Ready	The window is closed and has changes. No sync has run for this slot yet.
Waiting for ingest	The window is held back until the log ingest catches up with it. Hover for the reason.
Syncing / In Progress	A copy job is running for this slot.
Done / Synced N	All objects in the changelist were copied successfully. In the week view, N is the number of objects copied.
Partial / Partial (N fail)	Some objects were copied. Either some failed or were skipped, or the job was canceled while objects were still copying.
Failed	The sync job hit an error, or was canceled before any objects copied.
Skipped	An operator skipped the window manually.
Abandoned	Failed 3 times in a run (CV-114). Sent again at every scheduled run day. Skip it if it can never succeed.
No activity	No changes in this window, so there is nothing to sync.

Slot actions

In the week view (yesterday and earlier), each slot row has action buttons that depend on its state.

The live view has no per-window buttons. Use a bucket's Sync All or Sync Group Now there. A live window row only links to its job.

Action	Description
Run Sync	Triggers a new incremental copy job for a Ready or Failed slot.
Resume	Resumes a partial sync from where it left off.
Re-sync	Runs a synced slot's changelist again. With Smart Skip on, objects whose ETag matches their last successful copy are skipped. Turn Smart Skip off on the Settings tab for a full re-copy.
Skip	Marks the slot as skipped without running a copy job. Available for slots that already have a sync record and are not synced or skipped.
Job #N	Opens the job detail view for the most recent copy job on this slot.

Manual sync buttons in Vault Mode

For a protection group in Vault Mode, the vault agent decides when the group syncs. It opens the airgap, runs the sync, and closes it again. The console does not start incremental jobs for these groups by hand.

Sync Group Now, a bucket's Sync All, and a slot's Run Sync, Resume, and Re-sync stay visible but are disabled. Hover over one to see this message:

Cannot trigger an incremental job manually while the group is in Vault Mode — the vault agent schedules its syncs.

The console enforces the same rule, so the API and the MCP sync_group_now tool also refuse it. Skip still works because it does not start a job.

• To sync a vault-mode group earlier than its schedule, change its sync schedule.
• To run manual jobs again, use Exit Vault Mode.

Failed slots, retries, and disabled copy policies

Automatic retries in Vault Mode

In Vault Mode the vault agent retries failed work by itself.

1. When a sync ends with slots in Failed or Partial, the same sync retries them about a minute later and keeps the vault open.
2. A retry re-sends only failed or partial slots. It never picks up new pending slots.
3. Each slot gets at most 3 failed attempts in a run. The first attempt counts.
4. After that, the agent gives the slot up for this run. The slot becomes Abandoned and alarm CV-114 is raised. Clear the alarm manually.

A failure is never retried on a later day by itself. However, abandoned slots are sent again at every scheduled run day, with 3 new attempts. Once the cause is fixed, nothing else is needed.

For a slot that can never succeed, for example because its objects no longer exist at the source, click Skip. The slot is not sent again, and its changes are never copied to the vault.

As a safety net, the console tells the agent on each check-in whether any enabled copy policy of its groups still has a failed or partial slot. The agent then starts a retry-only sync for its groups whose schedule window is open.

Disabled copy policies

Disabled copy policies are left out of all of this. A failed or partial slot on a policy you have disabled behaves as follows:

• It does not start retry cycles, so new slots of every protection group on that vault agent keep syncing on their normal schedule.
• It does not count toward the 3 attempts and is never abandoned (no CV-114).
• It keeps its Failed or Partial status. Enable the policy again and the slot is retried normally.

A copy that is still running when you disable its policy keeps the vault open until it finishes.

Object-level drill-down

In the week view, expand a slot to see the full changelist: the objects created, modified, or deleted in that window.

The first 1,000 objects are shown immediately. Use Load all X objects to fetch the complete list from ClickHouse.

Protection groups and buckets

The dashboard is organized by protection group. Each group contains one or more bucket configs (source and vault bucket pairs).

Expand a group to see its buckets. Use the per-group search box to filter by bucket config name.

Auto-refresh

When any group has at least one actively syncing bucket, the view refreshes automatically every 30 seconds. The countdown is displayed next to the load time badge.

Refreshes stop when no syncing activity is detected.