Version: 2.10.0

Airgap for ECS

Introduction

The add-on solution for Data Security for ECS offers maximum data protection with an automated cyber vault. This solution provides Enterprise Airgap capabilities, ensuring secure data isolation and automated synchronization for your ECS environment. It fully integrates with existing ECS infrastructure, simplifying compliance with the NIST cybersecurity framework.

The Enterprise Airgap for ECS automates data synchronization using a containerized ECS Sync, providing high throughput and scalability. The Vault agent manages job scheduling directly, eliminating the need for complex out-of-band setups.

Data Security Compliance with NIST Key Framework Attributes

Framework Attribute	How Data Security Complies	Compliance Status
Identify	Threat identified by user name and IP address	Compliant
Protect	Stops the threat with user lockout in real time	Compliant
Detect	User behavior-based, tripwire, and well-known extension detection	Compliant
Respond	Alerting email, syslog, and automated snapshot creation	Compliant
Recover	Object level tracking and recovery from bucket versions	Compliant

Key Features

S3 to S3 Airgap Support: Supports secure replication between S3 buckets.
CAS to CAS Airgap Support: Support for Content Addressable Storage (CAS) replication.
Inside-the-Vault Automation: Provides seamless automation capabilities within the vault environment.
Enterprise Airgap: Hardened solution with in-band management, offering complete automation from a VM inside the vault.
Smart Airgap Technology: Synchronizes data only when safe, reducing risks. Scheduling is managed by the Vault agent for reliable control.
Per S3 Bucket Level Replication: Supports granular replication at the individual bucket level.
Data Immutability: Uses ECS object lock and bucket versioning to protect data from unauthorized modifications.
Rapid Recovery: The ECS cluster in the vault provides an immutable data copy at petabyte scale, ensuring integrity during recovery.
Many-to-One Protection: Allows you to consolidate multiple source ECS clusters into a single ECS Vault cluster.
IAM User Lockout: Locks out IAM users to prevent unauthorized actions.
Job History and Reports: Access detailed job history and airgap reports through the user interface. Reports can also be emailed on a schedule.
In-Band Management and Alerts: Monitor Vault storage capacity, hardware, and space availability with managed alerts.

Pre-Requisites

The following table outlines the different requirements for AirGap to work correctly on ECS.

Category	Details
Server Requirements	Dell Server model PowerEdge R450 with VMware 7.x, RAID 1 SSD 800GB, 128GB RAM, dual 10G NIC, dual socket (48 threads), 128GB RAM (Cart #3000111987102). Support Options: - 3-year support NBD OR - Pro-deploy plus
Vault ECS	Sized to the quantity of data needed for vault protection.
Optional Equipment	Production Network: - Ethernet switch or Firewall to connect Vault Agent Host. Vault Network: - Ethernet switch to connect Vault Agent Host to private network connected to the Vault ECS cluster.
Load Balancer	Progress Kemp LoadMaster. Installation service required.
Dell Services	- Physical installation of Dell Server host. - ECS Vault cluster + Hardening configuration.
Superna Services	- Airgap Design and Implementation Service. - Ransomware Defender for ECS Installation, Configuration, and Knowledge Transfer.
Scalability	1, 3, or 6 Vault Agent VMs in a cluster for high availability and high-throughput object-to-object async copying.
Software	- Ransomware Defender for ECS. - Enterprise Airgap.

Performance Considerations

Get Up-to-Date Performance Estimates

For the latest performance metrics, refer to ECSSync's performance page: General Performance Metrics.
Recommendations and Best Practices
- Use five job definitions per Airgap ECSSync instance for optimal performance.
- Throughput rates per ECSSync instance are detailed in the link above. Enterprise Airgap can scale from 3 to 99 instances as needed.

Load Balancer Support

If you have a load balancer in front of your ECS, configure its IP during the "Add Managed Device" step. To access this option, navigate to Eyeglass Menu -> Add Managed Device -> ECS. This links the load balancer's IP to your ECS and ensures that airgap jobs managed by the vault agent connect to the correct ECS cluster.

add_device

Deployment Diagram

Configuration

Follow the Installation Guide for Ransomware Defender for ECS
Ransomware Defender for ECS.
Deploy Dell Vault Hardware
Deploy the Dell Vault hardware with VMware pre-installed.
Deploy Enterprise Vault Agent ECA Cluster
Deploy the Enterprise Vault Agent ECA cluster to the vault hardware. Refer to the guide for details.
Configure Firewall Settings
Configure the firewall as specified in the port table in the "Firewall Requirements" section.
Install Enterprise Airgap License Key Install the Enterprise Airgap license key in Eyeglass using the License Manager.
- Access License Manager
  - Go to Eyeglass Main Menu -> License Management.
  - Navigate to: License Management > Manage Licenses > Browse.
  - Upload the zipped file downloaded in the previous step.
  tip
  For more information, see Register Superna License
Set Up Eyeglass IP Address and API Token on Vault Agent
- Set Up Eyeglass
  - Open the main menu in Eyeglass and select "Eyeglass REST API."
  - Create a new token, name it "vault," and copy the token. Use this value where "yyyyyy" is required.
  tip
  for more information regarding "Eyeglass REST API", see "Token Generation and Authentication".
- Configure Vault Agent
  - Log in to the vault agent as ecaadmin.
  - Open Configuration File
    - Run the following command to open the configuration file:
      nano /opt/superna/eca/eca-env-common.conf
  - Add Eyeglass Location
    - Add the Eyeglass location:
      export EYEGLASS_LOCATION=x.x.x.x
  - Add Eyeglass API Token
    - Add the Eyeglass API token:
      export EYEGLASS_API_TOKEN=yyyyyy
  - Enable Object Services for Enterprise Airgap
    - Enable Object Services for Enterprise Airgap:
      export ECS_SYNC_CFG=true
  - Save and Exit Configuration
    - Save and exit the configuration file using control + x.
  - Restart ECA Cluster
    - Restart the ECA cluster:
      ecactl cluster down ecactl cluster up
Follow These Steps to Configure the Vault Agent
- Add ECS Clusters to the Vault Agent
  - Run the following command to add ECS clusters to the vault agent:
    ecactl ecs add --host x.x.x.x --user <user>
  - Enter the IP address of the management interface and the service account user. Refer to the Eyeglass Service Account Minimum Privileges for details.
  - Repeat the process for each managed production and vault ECS cluster.
  - Verify Configuration
    - To verify the added clusters, run:
      ecactl ecs list
- One-Time ECS Sync Instance Configuration
  - Access the ECS Sync UI:
    https://x.x.x.x/ecssyncui
  - Login Credentials
    - Log in with ecaadmin and the default password 3y3gl4ss.
  - Configure Path and Email Address
    - Click the Config tab and enter the config path /opt/emc/ecs-sync/config and provide an email address
    note
    This email will not be used for any alerting but is required as an input
  - Update Sync Settings
    - Uncheck the option labeled "Auto Archive: automatically archive completed syncs."
  - Configure ECS Sync Jobs
    - Access ECS Sync UI
      - Access the ECS Sync UI:
        
        https://x.x.x.x/ecssyncui
        
        info
        Each vault agent has an ecssync container, enter the ip address of each instance to add jobs to the copy engine
      - Login Credentials
        
        Log in using ecaadmin and the default password 3y3gl4ss.
        
        note
        Always change the default password. Follow the steps provided here for details.
      - Add Jobs
        
        See detailed steps below for adding jobs in the "How to Create ECSSync Jobs" section.
      - Save Jobs
        
        Save the job configurations after making changes.
  - Add ECSSync Jobs to Vault Agent
    Add ECSSync jobs to the vault agent for management and integration with Eyeglass.
    - List Configured Jobs
      ecactl ecssync listjobs
      This command lists all configured jobs.
    - Add Job to Managed List
      ecactl ecssync addjob --job xxxx
    info
    Replace xxxx with the job name from the list. This adds the job to the managed job list.
  - Push ECS Job Definitions to Eyeglass
    - Register Jobs
      - Push all defined and added jobs to the vault agent for management, monitoring, and scheduling in Eyeglass.
        
        ecactl ecssync updatejobs
  - Log in to Eyeglass
    - Open Airgap Icon
    - Access ECS Sync Config Tab
      - The newly added jobs should appear automatically and show a status of "Not Scheduled."
    - Enable Ransomware Defender Airgap Control
      - Click the checkbox to enable Ransomware Defender Smart Airgap control and set the schedule for each ECS Sync job listed.
    - Save Changes
      - Click "Save" after making any changes to a policy.
      - Set the schedule and save
      - Repeat for each policy that displays not scheduled.
- Open Jobs Icon to Enable the Job
  - Click the jobs icon to enable the job.
  - The default state is disabled, and no sync jobs will run until enabled.
- Verify Vault Agent Schedule Change
  - Log in to the vault agent.
  - Run the command to list schedules:
    ecactl ecssync schedules
  - The schedule should be displayed for each policy configuration.
- Test a Sync Job
  - Start a Job from Vault Agent CLI
    - Run the following command to force start a job:
      ecactl ecssync startjob --job xxxx
      Replace xxxx with the name of the policy configured, which you can verify by running:
      ecactl ecssync checkjobs
    - Verify Job Status in ECS Sync GUI
      - Once the sync job has been started from the command line, you can verify the status directly in the ECS Sync GUI.
      - Navigate to the "Status" tab in the ECS Sync UI.
      - Look for the "Active Operations" section, where you will see the job listed along with its progress details.
      - The job should display a progress bar indicating the completion percentage. It will show "100% Complete" once the job finishes successfully.
      - Verify that there are no errors reported under the job entry. The "ETA" and "data transfer rate" will also be displayed for additional context.
    note
    Do not archive jobs. They are managed by the Vault Agent, and archiving them manually can disrupt job management.
    - List All Jobs
      - List all configured and active jobs in the Vault Agent for monitoring and review.
Done

How to Create ECSSync Jobs

Log in to each instance of ECSSync on each Vault Agent VM by using https://x.x.x.x/ecssyncui/. (Each Vault VM hosts an instance of ECSSync to balance the workloads.)
Select the Status Tab and click on the New Sync button.

Log in to each instance of ECSSync on each Vault Agent VM by using https://x.x.x.x/ecssyncui/. (Each Vault VM hosts an instance of ECSSync to balance the workloads.)
Select the Status Tab and click on the New Sync button.

IMPORTANT

The job name is mandatory and must be unique across all instances of ECSSync
Select the same ECS for source and destination.
Selecting the "Include Versions" checkbox will increase copy time significantly.
The Key Prefix field allows you to select a location within a bucket.
The source and target host IP address should specify the load balancer DNS name or ip address for the source production cluster and the load balancer in the secure network for the Vault ECS.

To enable high performance multipart copies for large objects, click the Hide Advanced button. This will accelerate the copy with multiple threads per object. (This is always recommended for larger objects.)
Click the start button to save the configuration and start the copy process.

info

If the Vault is open, the job can begin successfully. If the Vault is closed, the job will fail, and can be scheduled by Eyeglass for a later time.

Airgap Operations

Monitoring Vault ECS Cluster Alarms and Free Space

Log in to the Eyeglass VM.
Review alarms retrieved from the vault cluster:
- Click on the Alarms icon on the Desktop of the Eyeglass Web UI.

Monitoring ECSync Job Success and Failures and Policy Sync Job Reports

Log in to Eyeglass.
Click on the Airgap icon.
Click on Airgap Reports.
Click the Open button to view the reports.
- Note: Reports are also emailed.

Monitoring Job History

Log in to Eyeglass.
Click on the Airgap icon.
Click on Job History.
Click each job to view details.

Vault Agent CLI Commands for Airgap

Adding ECS and Vault Clusters to the Vault Agent

Add an ECS Cluster

Use this command to add an ECS production cluster:

ecaclt ecs add --host <host> --user <user>

<host>: Management IP address of the ECS.
<user>: Service account created on the ECS.

List ECS Clusters

View all ECS clusters currently registered:

ecaclt ecs list

Delete an ECS Cluster

Remove an ECS cluster from the Vault Agent:

ecaclt ecs delete

Retrieve ECS Alerts

Forward alerts from all registered ECS clusters to Eyeglass:

ecactl ecs pushalerts

Managing ECS Sync Jobs, Schedules, Logs, and Updates

Starting a Job

Run an ECS Sync job immediately on the instance that owns it:

ecaclt ecssync startjob --job <job_name>

<job_name>: The name of the ECS Sync job.

Checking Job Status

Retrieve the status and summary of all ECS Sync jobs across instances:

ecaclt ecssync checkjobs

Viewing Job Schedules

Display the schedules configured in Eyeglass:

ecaclt ecssync schedules

Pushing Support Logs

Send Vault Agent logs to Eyeglass for support or troubleshooting:

ecaclt ecssync pushvaultagentlogs

Updating Eyeglass Jobs

Push ECS Sync jobs created in the GUI to Eyeglass for management and scheduling:

ecaclt ecssync updatejobs

Performance Table for CAS Files

File Count/Size	Time	Rate (Size per Second)	Rate (Objects per Second)	Threads Allocated	CPU Usage (%) During Copy	Memory Usage (%) During Copy	Number of CPUs Used
787,095 files (8B each)	3h 23m 57s	73.42 KB/s	62 files/s	1	ecssync: 74.58%, mariadb: 34.35%, others as below	ecssync: 44.47%, mariadb: 17.97%, others as below	1

CPU and Memory Graphs During Copy

CPU Usage Graph:

cpu_usage_during_copy

Memory Usage Graph:

memory_usage_during_copy

Introduction​

Data Security Compliance with NIST Key Framework Attributes​

Key Features​

Pre-Requisites​

Performance Considerations​

Load Balancer Support​

Deployment Diagram​

Configuration​

How to Create ECSSync Jobs​

Airgap Operations​

Monitoring Vault ECS Cluster Alarms and Free Space​

Monitoring ECSync Job Success and Failures and Policy Sync Job Reports​

Monitoring Job History​

Vault Agent CLI Commands for Airgap​

Adding ECS and Vault Clusters to the Vault Agent​

Add an ECS Cluster​

List ECS Clusters​

Delete an ECS Cluster​

Retrieve ECS Alerts​

Managing ECS Sync Jobs, Schedules, Logs, and Updates​

Starting a Job​

Checking Job Status​

Viewing Job Schedules​

Pushing Support Logs​

Updating Eyeglass Jobs​

Performance Table for CAS Files​

CPU and Memory Graphs During Copy​

See Also​