Skip to main content
Migration Notice
We're migrating documentation from the old portal into this one. Some things may look a little different or out of place in the meantime — we know, and we're working to get it right. If something's unclear or doesn't look right, let us know.
Version: 2.15.0

Configuration Guide for Data Orchestration for Dell

Introduction​

This guide covers day-to-day configuration and operation of Data Orchestration for Dell (Superna Golden Copy): defining folders for archiving, choosing storage targets, scheduling copy and recall jobs, monitoring job performance, and using the Data Orchestration GUI, VM operations commands, and Cloud Browser for Data Mobility.

All CLI commands (searchctl) must be run as the ecaadmin user from Data Orchestration node 1.

Adding Folders with the GUI​

  1. Enable GUI login for a local user (SSH to node 1 as ecaadmin):

    searchctl settings admins add --name ecaadmin --local

  2. Log in to https://<node1 IP> as ecaadmin (default password 3y3gl4ss).

  3. Click the Folders icon in the left panel.

  4. Click + to add a new folder definition and follow the wizard.

  5. Once the folder is added, use the built-in test feature to create and delete a test object, verifying the target credentials and permissions.

note

The values you enter in the GUI wizard follow the same rules as the CLI flags documented below. Not all combinations of inputs are valid — invalid combinations are rejected by the appliance.

Adding Folders with the CLI​

Folders can be added, modified, or removed with searchctl archivedfolders.

Storage Target Examples​

Dell ECS:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://x.x.x.x:9021 --bucket <bucket name> --cloudtype ecs --endpoint-ips x.x.x.x,y.y.y.y,k.k.k.k

  • --isilon is the cluster name (not the IP address) — retrieve it with searchctl isilons list.

  • --endpoint-ips enables load-balancing with health checks across the listed ECS data node IPs.

  • A dedicated View should be created for S3 storage without other protocols enabled, and a dedicated service account/secret and access key should be used.

  • To enable round-robin load balancing across ECS nodes, add to /opt/superna/eca/eca-env-common.conf on node 1 and restart the cluster (ecactl cluster down / ecactl cluster up):

    export ARCHIVE_ENDPOINTS_ROUND_ROBIN=true

Amazon AWS:

searchctl archivedfolders add --isilon prod-cluster --folder /ifs/data/policy1/aws --accesskey <accesskey> --secretkey <secretkey> --endpoint https://s3.ca-central-1.amazonaws.com --region ca-central-1 --bucket mybucketname --cloudtype aws
  • --region is mandatory for AWS.
  • The endpoint must use the region-encoded URL matching the --region value.

Google Cloud Storage:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --secretkey /home/ecaadmin/<service account key>.json --bucket mybucketname --cloudtype gcs

Copy the GCS service account JSON key to node 1 (for example, via WinSCP into /home/ecaadmin) before running this command.

Azure Blob Storage:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --secretkey <storage account key> --endpoint blob.core.windows.net --container gc1 --accesskey <storage account name> --cloudtype azure
  • The storage account name is used as the access key with Azure.
  • Retrieve the access keys from the Azure console.
  • --container (not --bucket) is required for Azure.

Cohesity (S3-compatible):

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://x.x.x.x:3000 --bucket <bucket name> --cloudtype other

BackBlaze (S3-compatible):

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://s3.us-west-000.backblazeb2.com --bucket gctest --cloudtype other
note

For any of the storage targets above, the exact URL/port may vary per your provider or bucket configuration — confirm the endpoint with your storage provider before running the command.

Full archivedfolders add Flag Reference​

Key flags for searchctl archivedfolders add:

  • --path / --folder — the PowerScale path to copy or sync, for example /ifs/data/projectx.
  • --force — bypasses connectivity checks; used only when adding an AWS Snowball device.
  • --tier — sets the storage tier (requires Advanced or Backup Bundle license, and release 1.1.6+). Default is Standard for AWS, Cool for Azure. Azure tier options: hot, cool, archive. AWS tier options: standard (default), standard_IA, glacier, deep_archive (glacier_deeparchive), Glacier_IR (instant access).
  • --cloudtype {aws,ecs,azure,other,gcs,blackhole} — target storage type. Use other for S3-compatible targets using AWS v4 signatures, or otherv2 for AWS v2 signatures.
  • --region — required for AWS.
  • --bucket — required for all storage targets except Azure (which uses --container).
  • --container — required for Azure only.
  • --endpoint — required for ECS, AWS, and Azure (for Azure, must include the storage account name, for example blob.core.windows.net).
  • --secretkey / --accesskey — required credentials for all storage targets (for Azure, --accesskey is the storage account name).
  • --skip-s3-file-exists {true,false} — defaults to false (checks whether a file already exists in the bucket before copying). Set to true to skip this check and always perform a full copy, useful for avoiding LIST/GET API charges on very large file counts.
  • --recyclebucket <trashbucket> — bucket used to store deleted files detected during sync mode operations, useful with a bucket TTL policy.
  • --prefix <prefix> — inserts a prefix into the storage bucket path (<bucket_root>/<cluster_name>/[prefix]/ifs/), useful for a temporary copy without disturbing the existing path.
  • --endpoint-ips — list or range of IPs to load-balance across (ECS or other multi-endpoint targets).
  • --meta-prefix — changes the metadata HTTP header prefix (default x-amz-meta-) for S3 targets requiring a custom prefix.
  • --incremental-schedule, --full-archive-schedule, --archive-data-audit-schedule — cron expressions for scheduled jobs (see Scheduling Jobs below).
  • --storeFileHistory — tracks a file's movement history for traceability with Smart Archiver/incremental workflows.
  • --paused — pauses all jobs on the folder until unpaused.
  • --customMetaData — extracts metadata from supported file types for content-aware backup and search (requires the Pipeline license; not supported with Archive Engine workflows).
  • --includes / --excludes — glob patterns to include or exclude files/folders from archiving (release 1.1.4+). --excludes only applies within files matched by --includes.

Overview of Archive Job Types​

  • Full Archive — tree-walks the folder, skips files that already exist on the target (by last-modified date), and copies new or modified files. Files deleted from the source are not removed from the target. Running a full archive repeatedly behaves like an incremental job and can catch missing data.
  • Incremental Archive — enabled with a schedule on the folder. Uses the PowerScale change list and snapshots to detect created, modified, and deleted files. By default, deleted files are not removed from the target (see below to change this).

Sync Deleted Files During Incremental Jobs​

By default, incremental jobs do not delete files from the S3 target when they are deleted from the source file system. To change this globally, edit /opt/superna/eca/eca-env-common.conf on node 1:

export ARCHIVE_INCREMENTAL_IGNORE_DELETES=false

Save the file, then shut down and restart the cluster:

ecactl cluster down
ecactl cluster up

Concurrent Job Prioritization (Full vs. Incremental)​

By default, incremental jobs are prioritized over full archive jobs when both run concurrently, which can slow or stall full archive progress until the incremental backlog clears. To prioritize full archive jobs instead, edit /opt/superna/eca/eca-env-common.conf on node 1:

export ARCHIVE_FULL_TOPIC_REGEX="archivecontent-*"
export ARCHIVE_INCREMENTAL_TOPIC_REGEX="nomatch"

Save, then restart the cluster (ecactl cluster down / ecactl cluster up).

Starting Archive Jobs​

Start a full or incremental archive job:

searchctl archivedfolders archive --id <folder ID>

Useful flags:

  • --incremental — runs an on-demand snapshot-based changelist detection since the last incremental job (requires 1.1.4+).
  • --follow — moves directly to a live monitor view of the job's progress (requires 1.1.4+).
  • --auto-rerun — automatically queues a new job to retry all failed copies (requires 1.1.4+).
  • --recursive {true,false} — recursive copy of all data under the path (default when omitted).
  • --subdir <SUBDIR> — copies only a subfolder under the archived path.
  • --s3update — audits the S3 bucket against the file system, fixing differences and deleting files in the bucket that no longer exist on the file system. This is a slow operation and should be used deliberately, not as a routine copy validation step.
  • --snapshot <snapshotName> — uses an existing snapshot as the archive source instead of creating a new one, useful for archiving a specific point-in-time snapshot before deleting it to reclaim space.
  • --uploads-files <path to file> — archives only the files listed in a flat file (one absolute path per line, no other columns), instead of the entire folder tree. This is useful for archiving a curated list of files identified by another tool — for example, a file inventory or search report exported to CSV and trimmed down to a single path column. See File List Archiving for the full workflow.

Before starting a job, validate target permissions:

searchctl archivedfolders test --id <folder ID>

This validates PowerScale-to-S3 connectivity, file upload, upload verification, and file deletion from the target.

Scheduling Jobs​

Scheduling requires release 1.1.4 build 178 or later. Schedule times are evaluated in UTC. Cron string syntax follows standard cron format (see crontab.guru for reference).

Add an incremental schedule (examples):

# Every hour
searchctl archivedfolders add <other parameters> --incremental-schedule "0 * * * *"

# Every 2 hours
searchctl archivedfolders add <other parameters> --incremental-schedule "0 */2 * * *"

# Every 6 hours
searchctl archivedfolders add <other parameters> --incremental-schedule "0 */6 * * *"

# Once a day at midnight
searchctl archivedfolders add <other parameters> --incremental-schedule "0 0 * * *"

Modify an existing folder's schedule:

searchctl archivedfolders modify --id <folder id> <other parameters> --incremental-schedule "0 0 * * *"
searchctl archivedfolders modify --id <folder id> <other parameters> --archive-data-audit-schedule "0 0 * * *"

Disable a schedule:

searchctl archivedfolders modify --id <folder-id> --full-archive "NEVER" --incremental-schedule "NEVER"

Add a data audit schedule (compares the file system to the target, adding missing data, removing deleted data, and updating changed files — can take hours; suggested weekly, on Sunday):

searchctl archivedfolders add <other parameters> --archive-data-audit-schedule "0 0 * * 0"

Monitoring Jobs​

  • List running jobs: searchctl jobs running
  • Follow a running job: searchctl jobs view --id <job id> --follow
  • View job history: searchctl jobs history (filter with --folderid, --type, --tail <n>, or export with --output json)
  • Cancel a job: searchctl jobs cancel --id <job id>
  • Show summary statistics for all jobs, folders, or nodes: searchctl jobs summary [--nodes | --folders | --jobs]
  • View active file copies in progress on each node: searchctl jobs activecopies
  • Get processing rates for a folder with a running job: searchctl stats --folder <folder ID>
  • View error reasons for a failed job: searchctl archivedfolders error --id <job name> --count 25 --tail

Auto Progress Emails​

Enable a notification channel with SMTP first (see the appliance's cluster operations documentation for notification group setup), then:

searchctl archivedfolders notifications addgroup --isilon <host> --group <notification-group-name>
searchctl archivedfolders notifications modify --isilon <cluster name> --enabled true
searchctl archivedfolders notifications list
searchctl archivedfolders notifications removegroup --group <notification-group-name>

The default reporting interval is 24 hours. To change it (this example sets 5 minutes):

searchctl schedules modify --id JOBS_SUMMARY --schedule "*/5 * * * *"
searchctl schedules list

Managing Folders​

searchctl archivedfolders list
searchctl archivedfolders list --verbose

Modify an existing folder (--id is required; common modifiable flags include --cloudtype, --region, --tier, --bucket, --container, --endpoint, --secretkey, --cluster-name):

searchctl archivedfolders modify --id <folder ID> [--cloudtype {aws,ecs,azure,gcs,other}] [--region <region>] [--tier <tier>] [--bucket <bucket>]

Modify the cluster's service account or password:

searchctl isilons modify --name <cluster name> [--ip <ip>] [--user <username>] [--update-password] [--recall-sourcepath <path>]

Pause all scheduled jobs on a folder, and re-run only failed files or a failed tree walk, using the --paused flag on add/modify and the appropriate rerun options on archive/archivedfolders error as documented above.

Recalling Data from Object Storage​

Recall reads data back from the S3 target to the file system, staged under /ifs/goldencopy/recall on the source cluster.

Prerequisites:

  • The recall NFS export configured during installation must be in place (see the Installation Guide).
  • The folder must be in sync mode, or the copy job must have run more than once to detect changes, if version-aware recall is needed.
searchctl archivedfolders recall --id <folder ID>
info

On a folder with an active archive job, recall jobs are blocked until the archive job completes.

The Data Orchestration GUI​

Log in to the appliance GUI at https://<node 1 IP> after enabling a local admin user (see Adding Folders with the GUI, above). The GUI includes:

  • Archived Folders tab — create, delete, pause/resume, and test folder definitions. Selecting a folder and clicking the trash icon deletes it; the pause/play buttons pause or resume its schedule; the test button submits a test object upload/delete job to validate connectivity and permissions.
  • Cluster Management tab — lists PowerScale/Isilon clusters added to the appliance, with the option to remove one.
  • Archive Jobs tab — view active and historical jobs. Expand a running job to see live progress; expand a historical job to see data copied, errored, or skipped. Folder Errors and Active Copies views (release 1.1.9+) show copy errors per folder and real-time active file copies per node.
  • Cloud Browser (release 1.1.7+) — browse object storage for configured folders without a third-party S3 browser tool, without exposing access/secret keys outside the appliance. Only appliance administrators can use the Cloud Browser. To browse: log in as ecaadmin, open Cloud Browser, filter by target type, select a bucket/container, and navigate the folder tree. A version history link shows all versions of an object (requires a version-enabled bucket).
  • System Monitoring Health Dashboard tab — real-time and historical health metrics for all appliance components, including CPU, memory, and network transmit/receive per container. Network bandwidth per node is shown at the bottom of the dashboard (not real-time — use a tool like nload on the node for real-time bandwidth).

Recalling Files from the Cloud Browser GUI​

Requires the Advanced license (GCA) and release 1.1.9+.

searchctl licenses add --path /home/ecaadmin/<zip filename>
searchctl isilons license --name <cluster name> --applications GCA
searchctl isilons list

In the GUI: open the Cloud Browser tab, browse to the folder, and click the download/cloud icon to start a recall job for all data under that path.

Downloading Single Files with Cloud Browser​

Requires release 1.1.11+. Administrators (ecaadmin) can see all data across all S3 targets; users logging in with user@domain (AD) format see only data their SMB share permissions allow. Clicking the download icon issues a temporary, encrypted, time-limited S3 credential redirect — the file downloads directly to the user's PC without exposing credentials.

VM Operations​

Common ecactl/searchctl operations for managing the appliance cluster itself:

  • Start/stop the cluster:

    ecactl cluster down
    ecactl cluster up

  • Export/import appliance configuration (release 1.1.9+) — protects cluster and folder definitions, including encrypted credentials, useful before a factory reset:

    searchctl settings config export
    searchctl settings config importconfig

  • Wipe configuration while retaining clusters and folder definitions (release 1.1.9+):

    ecactl cluster rediscover

  • Remove a cluster node from active copy operations (for maintenance or performance reasons, without stopping other jobs):

    searchctl isilons modify --name <cluster name> --isilon-ips x.x.x.x y.y.y.y

    Provide the list of IPs that should remain in use, omitting the node you want to stop sending copy jobs to.

Cloud Browser for Data Mobility​

Cloud Browser for Data Mobility (CBDM) is a self-service portal that lets end users browse cloud object storage and move, copy, or sync selected cloud data into their own SMB shares, with IT retaining centralized monitoring, SMB/S3 permission enforcement, and an audit trail of all data movement.

Prerequisites:

  • Pipeline license subscription
  • Advanced license key
  • Per-seat license for each user of the workflow
  • AWS S3 service (as the source cloud)

Enable and Configure CBDM​

  1. On node 1, SSH as ecaadmin, edit /opt/superna/eca/eca-env-common.conf, and add:

    export ENABLE_CONTENT_CREATOR=true

  2. Confirm the Pipeline NFS mount for data recall is configured (see the Installation Guide). The NFS mount path must match the --recall-sourcepath value used below.

  3. Set the cluster's recall data location:

    searchctl isilons modify --name <cluster name> --recall-sourcepath /ifs/fromcloud

  4. Create the staging and trash S3 buckets, and at least one SMB share for testing, named with the mandatory gcdm- prefix (lowercase, case-sensitive), for example gcdm-movie-A mapped to /ifs/fromcloud/movie-a. Only SMB shares beginning with gcdm- are shown in the CBDM GUI.

  5. Configure the pipeline folder definition:

    searchctl archivedfolders add --isilon gcsource --folder /ifs/fromcloud --accesskey <accesskey> --secretkey <secretkey> --endpoint s3.ca-central-1.amazonaws.com --region ca-central-1 --bucket gc-cbdm-staging --cloudtype aws --source-path gcsource/pipeline --recall-from-sourcepath --recyclebucket gc-cbdm-trash --trash-after-recall

End-User CBDM Workflow​

  1. The user logs in to CBDM as an AD user at https://<node 1 IP>.
  2. In their profile, the user adds their S3 access keys, endpoint, and an email address for job notifications, then logs out and back in.
  3. The user opens the Cloud Browser for Data Mobility workflow icon, selects a cloud source and bucket, browses the bucket, and adds files to a job.
  4. The user runs the job, selecting an SMB share as the on-premises destination, and submits it. Progress can be monitored in the GUI, and the user receives an email when the job completes.
  5. Copied or moved files appear on the user's mounted SMB share under a folder referencing the source S3 bucket name.

Administrators can monitor all user job activity from the History tab of the CBDM workflow icon.

See Also​

  • Installation Guide – Appliance deployment, source cluster preparation, and licensing.
  • Integration Guide – Integrations with other Superna products and external automation.
  • API Guide – GraphQL API for automating archive, recall, and job monitoring operations.