Ingest Offsets
The Ingest Offsets page is a debug view of the byte-level read position for every audit log file that the ingest pipeline tracks. It shows how far into each log file the system has read and processed. You can also reset the positions of all files on one node or device to force a re-read.
Where: the page is not in the menu. Open an audit ingest job on System Operations > System Jobs, then click View ingest offsets (or View all N files) in its Node / File Details section. You can also browse to https://<console>/ingest-offsets.
Overview
The Jobs page shows a 24-hour summary of the same data. The ingest pipeline stores its position in each log file as a byte offset. After a restart, or when it continues an interrupted ingest, the pipeline reads from the stored offset instead of from the start of the file. This page shows the stored offsets and their metadata. Use it to diagnose ingest gaps, verify progress, or re-read the files of a node or device.
Offset table
| Column | Description |
|---|---|
| Device | The ECS or PowerScale device that the log file belongs to. |
| Type | The device type, ECS or PowerScale, and whether the file is a rotated, compressed log (gz) or the current log (active). For both device types, the pipeline connects over SSH and reads the log files over SFTP. |
| File | The name of the log file on the remote device. |
| Offset | The number of bytes read and processed from the file. A value of zero means that the file has not been read yet or that the offset was cleared. |
| Records | The number of log events parsed from the file so far. |
| Status | The state of the file in the pipeline. See Status values. |
| Last Updated | When the offset record was last written. Use it to compare progress between checks. |
Status values
| Status | Meaning |
|---|---|
| IN_PROGRESS | The file is being read, or was partially read and is not marked complete. |
| COMPLETE | The file is fully processed. |
| CLEARED | The offset was reset manually. The file is re-read on the next ingest run. |
Filters
Each column has its own filter control at the top of the table.
- Device and File accept free-text partial matches.
- Type and Status are dropdowns.
- Filters apply instantly and can be combined.
- The row count badge in the header shows how many records match the current filters.
Sorting
Click a column header to sort the table by that column. Click it again to reverse the order.
Infinite scroll
Rows load in batches of 50 as you scroll down. In environments with many tracked files, scrolling to the bottom of the list loads more rows, so there are no pagination controls.
Clear offset
Each row has a Clear action. It resets the byte position of every file on that row's node or device to zero, not only the file on that row, so the ingest pipeline re-reads all of those files from the beginning on its next run. Click Clear, then confirm with Yes. Read the warning under Tips first.
Workflows
Verify that a log file is being processed
- Type the file name or device name in the filter bar to narrow the list.
- Check the Status column. IN_PROGRESS with a non-zero byte offset confirms that the file is being read.
- Check Records to confirm that events are being parsed.
Force a re-read of log files after a parse error
- Filter the table to find the file.
- Click Clear on that row.
- Click Yes to confirm.
The next ingest run for that node or device re-reads all of its files from byte zero.
Check whether ingest has stalled on a device
- Filter by device name to show all files for the device.
- Look for files in IN_PROGRESS with a non-zero offset that has not changed since your last check.
- Compare with System Monitor > Audit Ingest Lag to see the lag for that device.
Tips
Clearing offsets reprocesses every file of that node or device from the beginning. If events from those files are already in ClickHouse, duplicate audit events are created. Use Clear only when duplicates are acceptable, or when you plan to wipe and re-ingest the ClickHouse data.
- If a file is stuck in IN_PROGRESS with no progress, ingest can be stalled at the network level. Check System Monitor > Endpoint Reachability to verify that the device is reachable.
- After an unexpected backend restart, the stored offset can be slightly behind the actual read position. The pipeline can re-read a small number of events after a restart, and those events are stored twice.