Skip to main content
Migration Notice
We're migrating documentation from the old portal into this one. Some things may look a little different or out of place in the meantime — we know, and we're working to get it right. If something's unclear or doesn't look right, let us know.
Version: 2.15.0

Configuration Guide for Data Orchestration for Dell

Introduction​

This guide covers day-to-day configuration and operation of Data Orchestration for Dell (Superna Golden Copy): defining folders for archiving, choosing storage targets, scheduling copy and recall jobs, monitoring job performance, and using the Data Orchestration GUI, VM operations commands, and Cloud Browser for Data Mobility.

All CLI commands (searchctl) must be run as the ecaadmin user from Data Orchestration node 1.

Adding Folders with the GUI​

  1. Enable GUI login for a local user (SSH to node 1 as ecaadmin):

    searchctl settings admins add --name ecaadmin --local

  2. Log in to https://<node1 IP> as ecaadmin (default password 3y3gl4ss).

  3. Click the Folders icon in the left panel.

  4. Click + to add a new folder definition and follow the wizard.

  5. Once the folder is added, use the built-in test feature to create and delete a test object, verifying the target credentials and permissions.

note

The values you enter in the GUI wizard follow the same rules as the CLI flags documented below. Not all combinations of inputs are valid — invalid combinations are rejected by the appliance.

Adding Folders with the CLI​

Folders can be added, modified, or removed with searchctl archivedfolders.

Storage Target Examples​

Dell ECS:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://x.x.x.x:9021 --bucket <bucket name> --cloudtype ecs --endpoint-ips x.x.x.x,y.y.y.y,k.k.k.k

  • --isilon is the cluster name (not the IP address) — retrieve it with searchctl isilons list.

  • --endpoint-ips enables load-balancing with health checks across the listed ECS data node IPs.

  • A dedicated View should be created for S3 storage without other protocols enabled, and a dedicated service account/secret and access key should be used.

  • To enable round-robin load balancing across ECS nodes, add to /opt/superna/eca/eca-env-common.conf on node 1 and restart the cluster (ecactl cluster down / ecactl cluster up):

    export ARCHIVE_ENDPOINTS_ROUND_ROBIN=true

Amazon AWS:

searchctl archivedfolders add --isilon prod-cluster --folder /ifs/data/policy1/aws --accesskey <accesskey> --secretkey <secretkey> --endpoint https://s3.ca-central-1.amazonaws.com --region ca-central-1 --bucket mybucketname --cloudtype aws
  • --region is mandatory for AWS.
  • The endpoint must use the region-encoded URL matching the --region value.

Google Cloud Storage:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --secretkey /home/ecaadmin/<service account key>.json --bucket mybucketname --cloudtype gcs

Copy the GCS service account JSON key to node 1 (for example, via WinSCP into /home/ecaadmin) before running this command.

Azure Blob Storage:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --secretkey <storage account key> --endpoint blob.core.windows.net --container gc1 --accesskey <storage account name> --cloudtype azure
  • The storage account name is used as the access key with Azure.
  • Retrieve the access keys from the Azure console.
  • --container (not --bucket) is required for Azure.

Cohesity (S3-compatible):

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://x.x.x.x:3000 --bucket <bucket name> --cloudtype other

BackBlaze (S3-compatible):

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://s3.us-west-000.backblazeb2.com --bucket gctest --cloudtype other

Scality (open source):

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey accessKey1 --secretkey verySecretKey1 --endpoint http://x.x.x.x:8000 --region us-east-1 --bucket bucket --cloudtype other

Create a folder on the target device; it becomes the bucket name at the root of the target device.

Wasabi:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://s3.wasabisys.com --bucket gctest --cloudtype other

IBM Cloud Object Storage and BackBlaze S3:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://<endpoint> --bucket gctest --cloudtype other

Cloudian:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://s3-us-east.internal.superna.net --bucket gctest --region us-east --cloudtype aws

The bucket URL must resolve to the endpoint IP address using the URL access method. For example, gctest.s3-us-east.internal.superna.net must resolve in DNS on the Golden Copy VMs.

Pure Storage FlashBlade:

searchctl archivedfolders add --isilon <cluster name> --folder <path> --accesskey <accesskey> --secretkey <secretkey> --endpoint https://<dns name> --bucket <bucket name> --region us-east-1 --cloudtype aws

Weka S3:

searchctl archivedfolders add --isilon <cluster name> --folder <path> --accesskey <accesskey> --secretkey <secretkey> --endpoint https://<dns name>:9000 --bucket <bucket name> --cloudtype other

Hitachi Vantara HCP CS:

searchctl archivedfolders add --isilon <cluster name> --folder <path> --accesskey <accesskey> --secretkey <secretkey> --endpoint https://<dns name> --bucket <bucket name> --region us-east-1 --cloudtype aws

Dell ObjectScale:

searchctl archivedfolders add --isilon <cluster name> --folder <path> --accesskey <accesskey> --secretkey <secretkey> --endpoint https://c.c.c.c:443 --bucket <bucket name> --region us-east-1 --cloudtype ecs

Quantum ActiveScale:

searchctl archivedfolders add --isilon <cluster name> --folder '<path>' --accesskey <accesskey> --secretkey <secretkey> --endpoint https://c.c.c.c:443 --bucket <bucket name> --region us-east-1 --cloudtype other

Vast Data:

searchctl archivedfolders add --isilon gcsource --folder /ifs/archive --accesskey <accesskey> --secretkey <secretkey> --endpoint https://c.c.c.c --bucket <bucket name> --region us-east-1 --cloudtype aws

NetApp ONTAP:

searchctl archivedfolders add --isilon <cluster name> --folder '<path>' --accesskey <accesskey> --secretkey <secretkey> --endpoint https://c.c.c.c --bucket <bucket name> --region us-east-1 --cloudtype aws
note

For any of the storage targets above, the exact URL/port may vary per your provider or bucket configuration — confirm the endpoint with your storage provider before running the command.

Full archivedfolders add Flag Reference​

Key flags for searchctl archivedfolders add:

  • --path / --folder — the PowerScale path to copy or sync, for example /ifs/data/projectx.
  • --force — bypasses connectivity checks; used only when adding an AWS Snowball device.
  • --tier — sets the storage tier (requires Advanced or Backup Bundle license, and release 1.1.6+). Default is Standard for AWS, Cool for Azure. Azure tier options: hot, cool, archive. AWS tier options: standard (default), standard_IA, glacier, deep_archive (glacier_deeparchive), Glacier_IR (instant access).
  • --cloudtype {aws,ecs,azure,other,gcs,blackhole} — target storage type. Use other for S3-compatible targets using AWS v4 signatures, or otherv2 for AWS v2 signatures.
  • --region — required for AWS.
  • --bucket — required for all storage targets except Azure (which uses --container).
  • --container — required for Azure only.
  • --endpoint — required for ECS, AWS, and Azure (for Azure, must include the storage account name, for example blob.core.windows.net).
  • --secretkey / --accesskey — required credentials for all storage targets (for Azure, --accesskey is the storage account name).
  • --skip-s3-file-exists {true,false} — defaults to false (checks whether a file already exists in the bucket before copying). Set to true to skip this check and always perform a full copy, useful for avoiding LIST/GET API charges on very large file counts.
  • --recyclebucket <trashbucket> — bucket used to store deleted files detected during sync mode operations, useful with a bucket TTL policy.
  • --prefix <prefix> — inserts a prefix into the storage bucket path (<bucket_root>/<cluster_name>/[prefix]/ifs/), useful for a temporary copy without disturbing the existing path.
  • --endpoint-ips — list or range of IPs to load-balance across (ECS or other multi-endpoint targets).
  • --meta-prefix — changes the metadata HTTP header prefix (default x-amz-meta-) for S3 targets requiring a custom prefix.
  • --incremental-schedule, --full-archive-schedule, --archive-data-audit-schedule — cron expressions for scheduled jobs (see Scheduling Jobs below).
  • --storeFileHistory — tracks a file's movement history for traceability with Smart Archiver/incremental workflows.
  • --paused — pauses all jobs on the folder until unpaused.
  • --customMetaData — extracts metadata from supported file types for content-aware backup and search (requires the Pipeline license; not supported with Archive Engine workflows).
  • --includes / --excludes — glob patterns to include or exclude files/folders from archiving (release 1.1.4+). --excludes only applies within files matched by --includes.

Overview of Archive Job Types​

  • Full Archive — tree-walks the folder, skips files that already exist on the target (by last-modified date), and copies new or modified files. Files deleted from the source are not removed from the target. Running a full archive repeatedly behaves like an incremental job and can catch missing data.
  • Incremental Archive — enabled with a schedule on the folder. Uses the PowerScale change list and snapshots to detect created, modified, and deleted files. By default, deleted files are not removed from the target (see below to change this).

Sync Deleted Files During Incremental Jobs​

By default, incremental jobs do not delete files from the S3 target when they are deleted from the source file system. To change this globally, edit /opt/superna/eca/eca-env-common.conf on node 1:

export ARCHIVE_INCREMENTAL_IGNORE_DELETES=false

Save the file, then shut down and restart the cluster:

ecactl cluster down
ecactl cluster up

Concurrent Job Prioritization (Full vs. Incremental)​

By default, incremental jobs are prioritized over full archive jobs when both run concurrently, which can slow or stall full archive progress until the incremental backlog clears. To prioritize full archive jobs instead, edit /opt/superna/eca/eca-env-common.conf on node 1:

export ARCHIVE_FULL_TOPIC_REGEX="archivecontent-*"
export ARCHIVE_INCREMENTAL_TOPIC_REGEX="nomatch"

Save, then restart the cluster (ecactl cluster down / ecactl cluster up).

Starting Archive Jobs​

Start a full or incremental archive job:

searchctl archivedfolders archive --id <folder ID>

Useful flags:

  • --incremental — runs an on-demand snapshot-based changelist detection since the last incremental job (requires 1.1.4+).
  • --follow — moves directly to a live monitor view of the job's progress (requires 1.1.4+).
  • --auto-rerun — automatically queues a new job to retry all failed copies (requires 1.1.4+).
  • --recursive {true,false} — recursive copy of all data under the path (default when omitted).
  • --subdir <SUBDIR> — copies only a subfolder under the archived path.
  • --s3update — audits the S3 bucket against the file system, fixing differences and deleting files in the bucket that no longer exist on the file system. This is a slow operation and should be used deliberately, not as a routine copy validation step.
  • --snapshot <snapshotName> — uses an existing snapshot as the archive source instead of creating a new one, useful for archiving a specific point-in-time snapshot before deleting it to reclaim space.
  • --uploads-files <path to file> — archives only the files listed in a flat file (one absolute path per line, no other columns), instead of the entire folder tree. This is useful for archiving a curated list of files identified by another tool — for example, a file inventory or search report exported to CSV and trimmed down to a single path column. See File List Archiving for the full workflow.

Before starting a job, validate target permissions:

searchctl archivedfolders test --id <folder ID>

This validates PowerScale-to-S3 connectivity, file upload, upload verification, and file deletion from the target.

Scheduling Jobs​

Scheduling requires release 1.1.4 build 178 or later. Schedule times are evaluated in UTC. Cron string syntax follows standard cron format (see crontab.guru for reference).

Add an incremental schedule (examples):

# Every hour
searchctl archivedfolders add <other parameters> --incremental-schedule "0 * * * *"

# Every 2 hours
searchctl archivedfolders add <other parameters> --incremental-schedule "0 */2 * * *"

# Every 6 hours
searchctl archivedfolders add <other parameters> --incremental-schedule "0 */6 * * *"

# Once a day at midnight
searchctl archivedfolders add <other parameters> --incremental-schedule "0 0 * * *"

Modify an existing folder's schedule:

searchctl archivedfolders modify --id <folder id> <other parameters> --incremental-schedule "0 0 * * *"
searchctl archivedfolders modify --id <folder id> <other parameters> --archive-data-audit-schedule "0 0 * * *"

Disable a schedule:

searchctl archivedfolders modify --id <folder-id> --full-archive "NEVER" --incremental-schedule "NEVER"

Add a data audit schedule (compares the file system to the target, adding missing data, removing deleted data, and updating changed files — can take hours; suggested weekly, on Sunday):

searchctl archivedfolders add <other parameters> --archive-data-audit-schedule "0 0 * * 0"

Monitoring Jobs​

  • List running jobs: searchctl jobs running
  • Follow a running job: searchctl jobs view --id <job id> --follow
  • View job history: searchctl jobs history (filter with --folderid, --type, --tail <n>, or export with --output json)
  • Cancel a job: searchctl jobs cancel --id <job id>
  • Show summary statistics for all jobs, folders, or nodes: searchctl jobs summary [--nodes | --folders | --jobs]
  • View active file copies in progress on each node: searchctl jobs activecopies
  • Get processing rates for a folder with a running job: searchctl stats --folder <folder ID>
  • View error reasons for a failed job: searchctl archivedfolders error --id <job name> --count 25 --tail

Auto Progress Emails​

Enable a notification channel with SMTP first (see the appliance's cluster operations documentation for notification group setup), then:

searchctl archivedfolders notifications addgroup --isilon <host> --group <notification-group-name>
searchctl archivedfolders notifications modify --isilon <cluster name> --enabled true
searchctl archivedfolders notifications list
searchctl archivedfolders notifications removegroup --group <notification-group-name>

The default reporting interval is 24 hours. To change it (this example sets 5 minutes):

searchctl schedules modify --id JOBS_SUMMARY --schedule "*/5 * * * *"
searchctl schedules list

Managing Folders​

searchctl archivedfolders list
searchctl archivedfolders list --verbose

Modify an existing folder (--id is required; common modifiable flags include --cloudtype, --region, --tier, --bucket, --container, --endpoint, --secretkey, --cluster-name):

searchctl archivedfolders modify --id <folder ID> [--cloudtype {aws,ecs,azure,gcs,other}] [--region <region>] [--tier <tier>] [--bucket <bucket>]

Modify the cluster's service account or password:

searchctl isilons modify --name <cluster name> [--ip <ip>] [--user <username>] [--update-password] [--recall-sourcepath <path>]

Pause all scheduled jobs on a folder, and re-run only failed files or a failed tree walk, using the --paused flag on add/modify and the appropriate rerun options on archive/archivedfolders error as documented above.

Recalling Data from Object Storage​

Recall reads data back from the S3 target to the file system, staged under /ifs/goldencopy/recall on the source cluster.

Prerequisites:

  • The recall NFS export configured during installation must be in place (see the Installation Guide).
  • The folder must be in sync mode, or the copy job must have run more than once to detect changes, if version-aware recall is needed.
searchctl archivedfolders recall --id <folder ID>
info

On a folder with an active archive job, recall jobs are blocked until the archive job completes.

The Data Orchestration GUI​

Log in to the appliance GUI at https://<node 1 IP> after enabling a local admin user (see Adding Folders with the GUI, above). The GUI includes:

  • Archived Folders tab — create, delete, pause/resume, and test folder definitions. Selecting a folder and clicking the trash icon deletes it; the pause/play buttons pause or resume its schedule; the test button submits a test object upload/delete job to validate connectivity and permissions.
  • Cluster Management tab — lists PowerScale/Isilon clusters added to the appliance, with the option to remove one.
  • Archive Jobs tab — view active and historical jobs. Expand a running job to see live progress; expand a historical job to see data copied, errored, or skipped. Folder Errors and Active Copies views (release 1.1.9+) show copy errors per folder and real-time active file copies per node.
  • Cloud Browser (release 1.1.7+) — browse object storage for configured folders without a third-party S3 browser tool, without exposing access/secret keys outside the appliance. Only appliance administrators can use the Cloud Browser. To browse: log in as ecaadmin, open Cloud Browser, filter by target type, select a bucket/container, and navigate the folder tree. A version history link shows all versions of an object (requires a version-enabled bucket).
  • System Monitoring Health Dashboard tab — real-time and historical health metrics for all appliance components, including CPU, memory, and network transmit/receive per container. Network bandwidth per node is shown at the bottom of the dashboard (not real-time — use a tool like nload on the node for real-time bandwidth).

Recalling Files from the Cloud Browser GUI​

Requires the Advanced license (GCA) and release 1.1.9+.

searchctl licenses add --path /home/ecaadmin/<zip filename>
searchctl isilons license --name <cluster name> --applications GCA
searchctl isilons list

In the GUI: open the Cloud Browser tab, browse to the folder, and click the download/cloud icon to start a recall job for all data under that path.

Downloading Single Files with Cloud Browser​

Requires release 1.1.11+. Administrators (ecaadmin) can see all data across all S3 targets; users logging in with user@domain (AD) format see only data their SMB share permissions allow. Clicking the download icon issues a temporary, encrypted, time-limited S3 credential redirect — the file downloads directly to the user's PC without exposing credentials.

VM Operations​

Common ecactl/searchctl operations for managing the appliance cluster itself:

  • Start/stop the cluster:

    ecactl cluster down
    ecactl cluster up

  • Export/import appliance configuration (release 1.1.9+) — protects cluster and folder definitions, including encrypted credentials, useful before a factory reset:

    searchctl settings config export
    searchctl settings config importconfig

  • Wipe configuration while retaining clusters and folder definitions (release 1.1.9+):

    ecactl cluster rediscover

  • Remove a cluster node from active copy operations (for maintenance or performance reasons, without stopping other jobs):

    searchctl isilons modify --name <cluster name> --isilon-ips x.x.x.x y.y.y.y

    Provide the list of IPs that should remain in use, omitting the node you want to stop sending copy jobs to.

Change the IP Address of a PowerScale Cluster​

To change the IP address of a cluster that has already been added to the appliance, run the following command. NAME is the cluster name reported by searchctl isilons list, and IP is a node IP in the System zone, in an IP pool with dynamic mode enabled:

searchctl isilons modify --name NAME --ip IP

Change TLS Security Settings for Clusters with Older Security Algorithms​

In some cases, the TLS security settings used to connect to clusters or other devices must be changed, for example when connecting to clusters that do not support the highest security algorithms.

  1. Edit the Java security file in /opt/superna/eca/conf/java with vim and make the required changes.

  2. Restart the isilongateway container to apply the settings:

    ecactl cluster containers restart isilongateway

Enable PhoneHome Support​

The PhoneHome feature enables remote log collection for faster support of cluster issues.

  1. Log in to node 1 and run the following command to start the registration process and listen for remote log upload requests:

    ecactl phonehome now

    Registration succeeds if the appliance has a valid license key and the required monitoring service URLs are allowed through the firewall.

  2. If an Internet proxy is required, run sudo -s (enter the admin password), type yast, and enter the proxy configuration under the Network Services → Proxy menu, then save.

  3. To disable PhoneHome:

    ecactl phonehome stop

  4. To upload logs directly to support:

    ecactl phonehome logsupload

Collect Support Logs​

  1. Collect the logs and create a support zip. The --tail option sets how many lines are collected for certain logs and reduces the file size. A value of 5000 is suggested for support:

    ecactl cluster loggather --tail 5000

  2. Run ecactl version and record the appliance ID. The appliance ID is required to upload support logs.

  3. Download the support zip from https://x.x.x.x/downloads/loggather/ (where x.x.x.x is the IP address of node 1).

  4. Upload the support zip to Superna Support with your support request.

Back Up and Restore the Cluster Configuration​

Create the backup and store it off the appliance to protect the cluster configuration.

  1. Create a backup. The results are stored in /opt/superna/var/search/downloads/archive/:

    ecactl cluster backup

  2. To download the backup, browse to https://x.x.x.x/downloads/ (where x.x.x.x is the IP address of node 1) and log in with the ecaadmin WebUI user and the password set during installation.

  3. To restore, the cluster must be shut down first:

    ecactl cluster down
    ecactl cluster restore --path /opt/superna/var/search/downloads/archive/<name of backup file>.zip

Automated Backups to External Storage​

The appliance creates a daily backup automatically and keeps 7 backups without any configuration. The automatic backups are stored in /opt/superna/var/search/downloads/archive/auto. To store them outside the appliance:

  1. Create an NFS export on PowerScale to store the backups with read/write permissions, for example /ifs/goldencopy/dailybackup.

  2. Edit /etc/fstab to mount the export, using an entry like the following, where FQDN is the DNS name used to connect to the cluster:

    FQDN:/ifs/goldencopy/dailybackup nfs nfsvers=3 0 0

  3. Run mount -a to mount the exports in fstab, and run mount to check that the mount succeeded.

Check for Alarms​

To check overall system events, run the following command. The data returned includes the error code, severity, time of the event, subsystem involved, and a description. Depending on the event type, additional fields may be populated.

searchctl notifications list

Configure Alarm Notifications​

The alarm system supports SMTP and syslog channels. Groups of recipients can be added to a channel, and a suppression feature allows rate limiting events or disabling them. The searchctl notifications command supports the operations list, channels, groups, recipients, and suppkeys.

Quick Start SMTP Configuration​

  1. Create a notification channel. For an anonymous SMTP connection:

    searchctl notifications channel addsmtp --host x.x.x.x --port 25 --sender fromemail@domain.com --tlsEnabled false

    For an authenticated connection, add --user supernacorp. The command prompts for the password. The --user option is required only for authenticated SMTP connections.

  2. Test the channel. The --id value is the channel ID returned by searchctl notifications channels list:

    searchctl notifications channels testsmtp --id A5tNdkgIT --recipient fromemail@domain.com

    The response returns diagnostic information from the test message.

  3. Optionally, create a notification group. A group named default already exists and is set to the INFO level, so create a group only if you want to set the severity filter to a higher level. A severity of MAJOR sends only MAJOR and CRITICAL alarms:

    searchctl notifications groups add --name alerts --severity MAJOR

  4. Get the channel ID with searchctl notifications channel list, then add recipients to the group. This example assumes the default group:

    searchctl notifications recipients addsmtp --group default --channel xxxxxxx --to targetemail@example.com

Set Up a Syslog Channel​

  1. Create the notification channel:

    searchctl notifications channels addsyslog --logger SYSLOG

  2. Edit /opt/superna/var/notification/log4j2.xml and change the host value to the IP address of the syslog server that should receive events:

    <Syslog name="SupernaSyslog" format="RFC5424" facility="LOCAL0" host="172.25.8.25" port="5140" protocol="UDP" appName="EyeglassSearch" messageId="SystemEvent" id="Event" connectTimeoutMillis="10000" newLine="true" mdcId="mdc" includeMDC="true" enterpriseNumber="18060"> </Syslog>

  3. Send a test syslog entry to verify communications. Get the channel ID by running searchctl notifications channels list:

    searchctl notifications channels testsyslog --id MMhjYPd0e

Manage Channels​

  • List channels: searchctl notifications channel list --verbose
  • Remove a channel: searchctl notifications channels remove --id xxxxxx (use the list command to get the ID)

Manage Notification Groups​

A notification group is a channel plus an alarm severity and recipients, so different emails can receive different severity alerts. The severity is the minimum alarm level sent on the channel: CRITICAL, MAJOR, MINOR, WARNING, INFO, or TRACE.

searchctl notifications groups add --name INFO --severity INFO
searchctl notifications groups list
searchctl notifications groups remove --name xxxx

Manage Recipients​

searchctl notifications recipients addsmtp --group NAME --channel xxxx --to email@example.com
searchctl notifications recipients addsyslog --group NAME --channel xxxx
searchctl notifications recipients list --group xxxxx --verbose
searchctl notifications recipients remove --id <id> --group <group>

Use the list commands to get the channel, recipient, and group IDs.

Manage Notification Suppression​

Use this optional configuration to rate limit a specific alarm code for a specific recipient group, so the group receives the alarm only once per hour, once per day, or never. This is useful for recurring notifications.

searchctl notifications suppkeys add --group GROUP --code CODE --frequency {ONE_PER_HOUR,ONE_PER_DAY,NEVER}
searchctl notifications suppkeys list --group name
searchctl notifications suppkeys remove --group GROUP --id ID

Use the list command for the group to get the suppression key ID.

Simulated Data Upload Mode​

Simulated data upload mode mocks any archive operation with random data, which removes the PowerScale NFS read operations from the archive path so the result can be compared with a real archive. It is intended primarily for use by Superna Support rather than customer administrators.

  • Available in release search-1.1.12-22196 and later.
  • This mode does not completely remove PowerScale from the test, because the REST API is still used. To remove PowerScale and Golden Copy completely and measure direct file uploads from a node to AWS, use a standalone AWS client application.
  • Every archive operation uses simulated data while the mode is on. Make sure no scheduled archive operation runs during the test.
  • The entire test dataset is uploaded to the cloud, so consider the bandwidth consumed. If the simulated mode is FILE, be careful with the number of different file sizes and the total size of the test dataset.

To run an archive test with simulated data:

  1. Configure an archive folder for the test dataset, for example /ifs/data/testDataset, with folder ID d40ac260.

  2. Populate the test dataset folder with your test files.

  3. Configure the ARCHIVE_SIMULATED_DATA_MODE and ARCHIVE_SIMULATED_DATA_MODE_MAX_SIZE variables.

  4. Restart the Golden Copy cluster (or the AW nodes).

  5. Run the archive:

    searchctl archivedfolder archive --id d40ac260 --follow

  6. When the test is finished, remove ARCHIVE_SIMULATED_DATA_MODE and ARCHIVE_SIMULATED_DATA_MODE_MAX_SIZE and restart the cluster.

The preferred mode is ARRAY. In this mode, a single array of size ARCHIVE_SIMULATED_DATA_MODE_MAX_SIZE filled with random data is created in memory when each AW node starts. The FILE mode creates a temporary file with random data on the fly for every size requested in archive operations. A file is created only if it does not exist yet, in the /tmp/simulated-data/ folder. For example, a test dataset with files of 1 KB and 10 KB results in 1024.dat and 10240.dat in /tmp/simulated-data/.

Cloud Browser for Data Mobility​

Cloud Browser for Data Mobility (CBDM) is a self-service portal that lets end users browse cloud object storage and move, copy, or sync selected cloud data into their own SMB shares, with IT retaining centralized monitoring, SMB/S3 permission enforcement, and an audit trail of all data movement.

Prerequisites:

  • Pipeline license subscription
  • Advanced license key
  • Per-seat license for each user of the workflow
  • AWS S3 service (as the source cloud)

Enable and Configure CBDM​

  1. On node 1, SSH as ecaadmin, edit /opt/superna/eca/eca-env-common.conf, and add:

    export ENABLE_CONTENT_CREATOR=true

  2. Confirm the Pipeline NFS mount for data recall is configured (see the Installation Guide). The NFS mount path must match the --recall-sourcepath value used below.

  3. Set the cluster's recall data location:

    searchctl isilons modify --name <cluster name> --recall-sourcepath /ifs/fromcloud

  4. Create the staging and trash S3 buckets, and at least one SMB share for testing, named with the mandatory gcdm- prefix (lowercase, case-sensitive), for example gcdm-movie-A mapped to /ifs/fromcloud/movie-a. Only SMB shares beginning with gcdm- are shown in the CBDM GUI.

  5. Configure the pipeline folder definition:

    searchctl archivedfolders add --isilon gcsource --folder /ifs/fromcloud --accesskey <accesskey> --secretkey <secretkey> --endpoint s3.ca-central-1.amazonaws.com --region ca-central-1 --bucket gc-cbdm-staging --cloudtype aws --source-path gcsource/pipeline --recall-from-sourcepath --recyclebucket gc-cbdm-trash --trash-after-recall

End-User CBDM Workflow​

  1. The user logs in to CBDM as an AD user at https://<node 1 IP>.
  2. In their profile, the user adds their S3 access keys, endpoint, and an email address for job notifications, then logs out and back in.
  3. The user opens the Cloud Browser for Data Mobility workflow icon, selects a cloud source and bucket, browses the bucket, and adds files to a job.
  4. The user runs the job, selecting an SMB share as the on-premises destination, and submits it. Progress can be monitored in the GUI, and the user receives an email when the job completes.
  5. Copied or moved files appear on the user's mounted SMB share under a folder referencing the source S3 bucket name.

Administrators can monitor all user job activity from the History tab of the CBDM workflow icon.

See Also​

  • Installation Guide – Appliance deployment, source cluster preparation, and licensing.
  • Integration Guide – Integrations with other Superna products and external automation.
  • API Guide – GraphQL API for automating archive, recall, and job monitoring operations.