Skip to main content
Migration Notice
We're migrating documentation from the old portal into this one. Some things may look a little different or out of place in the meantime — we know, and we're working to get it right. If something's unclear or doesn't look right, let us know.
Version: 2.15.0

Archiving Aging Data with a File List for Data Orchestration for Dell

Introduction​

Golden Copy folder definitions normally archive an entire path. For a cost-driven cleanup use case — periodically finding and archiving only the specific files that qualify as cold or aging, based on criteria such as last-modified date, last-accessed date, size, or owner — Golden Copy can instead take a flat list of absolute file paths as the input to an archive job, using the --uploads-files flag. This lets a separate file inventory or search tool (for example, Superna Search & Recover) identify exactly which files should move to lower-cost object storage, rather than moving an entire folder tree regardless of age.

When to Use This​

  • Reducing primary storage costs by archiving only genuinely aged or cold files, rather than an entire path.
  • Periodic bulk cleanup cycles (for example, quarterly) rather than a continuous sync.
  • Selecting files by criteria other than path alone — age, size, owner, or group — where a search or reporting tool can produce the matching file list.

Prerequisites​

  • A Golden Copy folder definition already configured with a cluster and path — the folder definition must exist, but does not need to have archived any data yet. See Adding Folders with the CLI.
  • A tool or script capable of producing a flat file listing absolute file paths (one path per line, no other columns) — for example, a search tool's CSV export trimmed down to a single path column.

Workflow: Archive by File List​

  1. Identify aging data. Search for files matching your criteria (for example, modified or last-accessed date, size, or owner). The search path used should match the base path configured on the Golden Copy folder definition.

  2. Export the results and reduce to a path list. Export the search results to a CSV file, then edit it to remove all columns except the absolute file path (/ifs/...), with one file per line.

  3. Copy the file list to Golden Copy node 1. Use a tool such as WinSCP to copy the file to node 1 (for example, into /opt/superna/var/csvimport), authenticating as the ecaadmin user.

  4. Run the archive job against the file list:

    searchctl archivedfolders archive --id <folder ID> --uploads-files /opt/superna/var/csvimport/<file>.csv

  5. Monitor the job as with any other archive job — see Monitoring Jobs.

Archive-in-Place vs. Staging Area Patterns​

Two patterns are commonly used with file-list archiving:

  • Archive in place: files are archived directly from their existing location using the absolute paths in the file list. The object store preserves the same path the file had on the source file system. This is the simpler of the two patterns, but recall does not restore data to its original location — recalled data lands in the recall staging path, preserving the original absolute path underneath it.
  • Staging area: before archiving, files matching the age/size criteria are moved (for example, with an administrator-run script) into a dedicated staging path such as /ifs/archivestaging, preserving each file's relative path underneath the staging root. A folder definition configured against the staging path in copy mode then archives everything under it. This pattern keeps the bulk-cleanup activity isolated from the rest of the file system, making it straightforward to verify the copy succeeded (for example, with searchctl archivedfolders export --id <job name>) before removing the staged copies from the source.

In either pattern, removing the original files from the source file system after the archive job completes is an optional, separate administrative step — it is not performed automatically by Golden Copy.

Recalling File-List Archived Data​

Because file-list archiving preserves the original (or staging) absolute path in the object store, recall can target a specific subdirectory rather than the whole folder definition:

searchctl archivedfolders recall --id <folder ID> --subdir <path-to-recall>

Once the recall job completes, the data is available under the recall staging path (see Recalling Data from Object Storage), where it can be shared directly (for example, via a new SMB share) or copied back into its original location.

See Also​

  • Configuration Guide — Folder definitions, job flags, scheduling, and recall.
  • Bulk Data Loading — Physical-device bulk loading and recall for very large data sets.
  • Archive Engine Guide — End-user self-service archiving and age-based policy archiving, as an alternative to file-list-driven archiving.