Skip to main content
Migration Notice
We're migrating documentation from the old portal into this one. Some things may look a little different or out of place in the meantime — we know, and we're working to get it right. If something's unclear or doesn't look right, let us know.
Version: 2.15.0

Azure Blob Rehydration for Burst Compute for Data Orchestration for Dell

Introduction​

Golden Copy can sync PowerScale file data to an Azure Blob container for backup or cost-tiering. When that data needs to be accessed by a production workload in Azure — for example, a burst compute or rendering scenario — it can be rehydrated from the Blob container into an Azure File Share, which supports SMB access. This keeps backup data in the lower-cost Blob container while production workloads read and write against an Azure File Share.

caution

This pattern uses the third-party AzCopy command-line utility and sample PowerShell/batch scripts to move data and reapply ACLs. This sample code is provided as a solution guide only, with no coverage under a support contract and no express or implied warranty. AzCopy itself is a Microsoft tool and is not supported by Superna.

Why AzCopy​

AzCopy can copy objects directly from an Azure Blob container to an Azure File Share without downloading them to a local machine first, taking advantage of the performance of Azure Storage accounts and Azure Virtual Network rather than the bandwidth of the machine running the copy. There are no bandwidth charges when both storage accounts are in the same region.

Configure Golden Copy for Rehydration​

To avoid the ACL metadata objects that Golden Copy stores in the bucket colliding with actual folder names once data is rehydrated, configure a prefix string for those objects before archiving:

  1. SSH to Golden Copy node 1 as ecaadmin.

  2. Edit /opt/superna/eca/eca-env-common.conf and add:

    export ARCHIVE_S3_PREFIX="igls-"

  3. Save and exit, then restart the cluster:

    ecactl cluster down
    ecactl cluster up

AzCopy Setup​

  1. Download AzCopy V10 (the currently supported version) and unzip the executable to a working directory on the Windows machine that will run the copy and the ACL script.
  2. Authenticate to both the Blob container and the File Share using a Shared Access Signature (SAS) token, generated per-container/share from Azure Storage Explorer:
    • For the Blob container used as the Golden Copy target, a Read & List SAS is generally sufficient.
    • For the destination Azure File Share, grant Read, Create, Write, Delete, and List permissions.
    • Record the SAS URI for each — it is only displayed once when generated.
note

SAS tokens have a start and expiry date/time. Once a token expires, generate a new one and update any scripts referencing it.

Copying Data with AzCopy​

Two AzCopy commands are relevant:

  • azcopy copy — copies source data to a destination; used for a full copy.
  • azcopy sync — replicates the source to the destination based on last-modified time, skipping files that are already current at the destination; used for incremental copy.

Full copy example:

azcopy copy "https://<storage-account-name>.blob.core.windows.net/<blob-container-name>?<blob-container-SAS-token>" "https://<storage-account-name>.file.core.windows.net/<file-share-name>?<file-share-SAS-token>" --recursive=true

Incremental sync example (with delete propagation):

azcopy sync "https://<storage-account-name>.blob.core.windows.net/<blob-container-name>?<blob-container-SAS-token>" "https://<storage-account-name>.file.core.windows.net/<file-share-name>?<file-share-SAS-token>" --recursive=true --delete-destination=true

Reapplying ACLs After Rehydration​

Golden Copy stores each file's ACL information as a separate S3 object using the ARCHIVE_S3_PREFIX configured above (for example, igls-<filename>). After rehydrating data to the Azure File Share, a sample PowerShell script can read these ACL objects (through an SMB mount of the File Share) and reapply the equivalent NTFS permissions to the corresponding files using icacls.

The script maps Golden Copy's internal access-right and inheritance-flag values to icacls permission strings — for example, dir_gen_all to Full Control (F), dir_gen_read to Read (R), and object_inherit/container_inherit to the corresponding (OI)/(CI) inheritance flags — and skips deny-type entries, applying only allow-type entries.

Scheduling the Rehydration Workflow​

Keep the Blob Container Current​

Configure a Golden Copy incremental schedule on the archived folder so the Blob container stays current with source changes. This can be set from the Golden Copy GUI (Archived Folders → edit the folder → Schedules → Incremental Archive Schedule) or the CLI — see Scheduling Jobs.

Automate the AzCopy Sync and ACL Script​

A batch file combining the azcopy sync command and the ACL PowerShell script can be scheduled with Windows Task Scheduler on the same Windows machine used for the AzCopy setup, to run on a recurring interval (for example, hourly or daily) so the File Share stays rehydrated with recent Blob changes.

note

SAS URIs may contain special characters (such as %); escape these characters when embedding a SAS URI in a batch file.

See Also​