TPS Rate Limiting Settings

Overview

The TPS Rate Limiting settings page configures the global throughput governor for all copy agents. When enabled, the governor distributes a configurable transactions-per-second budget across all agents with active jobs, preventing any single deployment from overwhelming a source or vault ObjectScale cluster at high copy concurrency.

Enable TPS Rate Limiting

Toggle this on to activate the governor. When off, agents run at full speed with no coordinated budget.

Global TPS Limit

The total TPS budget shared equally across all agents that have active jobs. Enter the exact transactions-per-second value you want — for example, enter 5000 for a 5,000 TPS budget, or 1500 for a fine-grained 1,500 TPS limit. The field accepts any whole number. The UI echoes the value back (e.g. "= 1,500 TPS total budget") so you can confirm the intended limit.

The default is 1,000 TPS, and the minimum accepted value is 100 TPS. If you enter a value below 100, it is automatically raised to 100 when the field loses focus and again when the settings are saved.

Count as Transactions

Controls which S3 operation types are counted against the budget.

Operation	Default	Notes
LIST	On	Typically frequent during inventory scans
GET	No effect	Never charged
PUT	On	Every object write to the vault target
DELETE	No effect	Never charged

Multipart objects always count as a fixed charge of 11 transactions per object, and single-part objects count as 1, regardless of this setting. The limit caps the copy rate, not the source's real request rate.

Alarm Threshold

A system alarm is raised when the total utilization across all agents exceeds this percentage of the global limit. Default is 95%.

For example, with a 5,000 TPS limit and a 95% threshold, an alarm fires when actual TPS exceeds 4,750. This gives operators time to investigate before the limit causes throttling impact.

The alarm automatically clears when utilization drops back below the threshold.

Saving

Changes take effect after clicking Save Settings. There is no restart required — the governor reads the new limit from the settings store on the next 30-second distribution cycle.

---

How TPS distribution works

The TPS governor lives on the console and divides your Global TPS Limit among the agents that are doing work, refreshing once every 30 seconds. Each agent then meters its own S3 operations against the credit budget the console has handed it. Here is what happens behind the scenes — useful when you are sizing the limit, picking a per-agent expectation, or interpreting the Cyber Vault > System Performance (TPS Monitor) page.

The 30-second window

Every 30 seconds the console runs a single coordination pass. In that pass it:

1. Reads how many TPS each agent actually used in the previous window
2. Decides each agent's new allocation for the next window
3. Sends the new allocation to each agent inside its next heartbeat response
4. The agent loads the new credit budget into its local meter and starts spending

Within a single 30-second window the per-agent budget is fixed. Operations carry on as long as credits remain. If an agent exhausts its credits before the window ends, copy threads pause briefly until the next window's refill arrives.

Equal share, then demand-aware redistribution

The split is based on active agents only — an agent counts as active if it currently has an active copy job AND it heartbeated within the last 90 seconds. Idle agents (no active job) get no share — they are also not subject to TPS limiting until a job starts on them.

Step 1 — equal base split:

baseAllocation = globalTpsLimit / numberOfActiveAgents

Step 2 — identify hungry vs under-utilized agents:

Category	Condition	Treatment
Hungry	Used ≥ 90 % of its current allocation	Eligible to receive a bonus from the pool
Under-utilized	Used < 80 % of its current allocation AND a hungry agent exists	Gives back 50 % of its unused quota to a redistribution pool
Coasting	Utilization between 80 % and 90 %, or all agents are under-utilized	Keeps the base allocation as-is

Step 3 — redistribute the pool to hungry agents:

bonusPerHungryAgent = pool / numberOfHungryAgents

The key safety property: under-utilized agents are only asked to give back quota if a hungry agent exists to receive it. If every agent is under-utilized, all agents keep the full base allocation so they can burst freely.

Smoothing — why allocations change gradually

A pure recompute on every 30-second cycle would cause agent allocations to swing dramatically on small utilization changes. To prevent this, the new allocation is mixed with the previous allocation using a fixed weighting:

newAllocatedTps = 0.4 × newlyComputed + 0.6 × previousAllocatedTps

That means 40 % of the change is applied per cycle. Practical effect: it takes roughly 5 cycles (≈ 2.5 minutes) for an agent's allocation to fully settle to a new value after a major shift in load. This deliberate stickiness is what keeps the system stable when, for example, one agent finishes a job and goes idle — the surrounding agents do not immediately get a huge spike, the redistribution flows through smoothly.

Safety cap

If rounding ever causes the sum of all agent allocations to exceed globalTpsLimit, every agent's allocation is proportionally scaled down so the total stays within budget. You will see this only with very small global limits (e.g. 1,000 TPS spread across 20 agents).

What happens on the agent side

The console hands the agent a credit budget equal to allocatedTps × 30 (one window's worth). The agent stores this in a counter and decrements it on every S3 operation:

Operation	Counts as
LIST (one request)	1
GET	No effect (never charged)
PUT (single-part copy)	1
Multipart copy	11, a fixed charge once per object
DELETE	No effect (never charged)

If the counter goes below zero, the operation is rolled back and the worker thread waits for the next refill (polled every 50 milliseconds).

Worked example

Setting: Global TPS Limit = 5,000, 4 active agents.

Cycle 1 — first allocation, no usage history. Equal split:

Agent	Allocated TPS	Actual TPS	Utilization
agent-A	1,250	—	(new)
agent-B	1,250	—	(new)
agent-C	1,250	—	(new)
agent-D	1,250	—	(new)

Cycle 2 — after one window of activity:

Agent	Previous allocation	Used in window	Utilization
agent-A	1,250	1,200	96 % → hungry
agent-B	1,250	400	32 % → under-utilized
agent-C	1,250	1,150	92 % → hungry
agent-D	1,250	800	64 % → under-utilized

Console computes the new split:

• Hungry: agent-A, agent-C
• Pool returned by under-utilized:
  • agent-B returns: 1,250 × (1 − 0.32) × 0.5 = 425
  • agent-D returns: 1,250 × (1 − 0.64) × 0.5 = 225
  • Pool total = 650
• Bonus per hungry agent: 650 / 2 = 325
• Computed allocations:
  • agent-A: 1,250 + 325 = 1,575
  • agent-B: 1,250 − 425 = 825
  • agent-C: 1,250 + 325 = 1,575
  • agent-D: 1,250 − 225 = 1,025

Smoothing applied (40 % new + 60 % previous):

Agent	Computed	Smoothed for Cycle 2
agent-A	1,575	0.4 × 1,575 + 0.6 × 1,250 = 1,380
agent-B	825	0.4 × 825 + 0.6 × 1,250 = 1,080
agent-C	1,575	0.4 × 1,575 + 0.6 × 1,250 = 1,380
agent-D	1,025	0.4 × 1,025 + 0.6 × 1,250 = 1,160

Total = 5,000 (still equal to the global limit; no safety cap needed). Agent-A and agent-C now get more headroom; agent-B and agent-D have ceded some of theirs. If utilization continues at the same shape over multiple cycles, allocations gradually converge toward the computed split — the smoothing simply spreads the change across roughly 2.5 minutes rather than letting it whipsaw between cycles.

Tuning guidance

• The Global TPS Limit should reflect the steady-state limit your source storage can sustain without returning HTTP 503 ("Please reduce your request rate"). If you do not know that number, start from the agreed replication budget, not the vendor's rated limit, because production also serves live applications. Lower the limit if you see 503-driven auto-pauses.

• Higher is usually better than lower as long as you stay below the source's documented limit. A budget set well below what the source could absorb does not provide any extra safety — the source's own rate limiting is more granular than ours.

• Watch Cyber Vault > System Performance > TPS Monitor. It is shown only while TPS limiting is enabled. In the agent table, the Efficiency bar compares each agent's actual TPS with its allocation:

  • Red at 95 % or more.
  • Amber at 80 % or more.
  • Green below 80 %.

  The governor treats an agent at 90 % or more as hungry and gives it more of the budget. An agent below 80 % gives part of its unused budget back. If agents keep moving between these ranges, the 2.5-minute smoothing window is working as designed and the system is converging.

• An alarm at 95 % of the global limit is the recommended default. If the alarm fires repeatedly, you have hit the budget — either raise the limit (if your source has headroom) or reduce concurrent copy work.