Troubleshooting for AirGap for Dell

Introduction

This guide provides steps to open and close maintenance windows and how to stop replication in emergencies. It also covers solutions to common problems such as job failures due to active ransomware events.

Troubleshooting Procedures

Opening and Closing Maintenance Window - Enterprise AirGap

To manage the maintenance window for Enterprise AirGap, follow these steps:

Vault Control mode

The steps below apply to Eyeglass-controlled scheduling. If Vault Control mode is enabled, igls airgap vaultaccessrequest is blocked and won't open the vault — ecactl airgap openvault --interval <interval_in_minutes> from the Vault Agent is the only way to open it in that mode. See Changes in Vault-Controlled Scheduling for the full list of what else changes under Vault Control mode.

1. Preparation:

   • Ensure that the environment variable EYEGLASS_OPEN_VAULT_ENABLED=true is set on the Vault Agent VM by checking /opt/superna/eca/eca-env-common.conf.
   • Collect the Vault Agent ID (EVA_ID) from either /opt/superna/eca/eca-env-common.conf or from /opt/superna/sca/data/airgap/AirGapVaultAgents.json.

2. Opening the Maintenance Window:

   • In Eyeglass-controlled scheduling, you can open the vault from either side, and the two commands behave differently:
     • From Eyeglass:
       igls airgap vaultaccessrequest --interval=<DURATION> --vault=<EVA_ID>.
       Here, <DURATION> is in minutes. This only requests access — it does not open the vault immediately. The request is picked up, and the vault opened, on the next synchronization between Eyeglass and the Vault Agent.
     • From the Vault Agent:
       ecactl airgap openvault --interval <minutes>.
       This opens the vault immediately. It is the same command used under Vault Control mode, where it is the only option.

3. Check Vault Access Requests:

   • To view requests still pending on Eyeglass:
     igls airgap vaultaccessview [--vault=<EVA_ID>].
     This only reflects pending requests — once a request lands on the Vault Agent and the vault opens, the value resets to 0 and no longer confirms anything here.
   • To confirm the vault is actually open and see how much time is left, run the following on the Vault Agent:
     ecactl airgap checkopen.
     If you need more time before the window closes, request it again — either igls airgap vaultaccessrequest --interval=<DURATION> --vault=<EVA_ID> from Eyeglass, or ecactl airgap openvault --interval <minutes> from the Vault Agent.

4. Closing the Maintenance Window:

   • igls airgap vaultaccesscancel --vault=<EVA_ID> only cancels a pending request that has not yet landed on the Vault Agent. Once the request has landed and the vault is open, this command can no longer close it.
   • To force-close a vault that is already open, run the following on the Vault Agent:
     ecactl airgap closevault.
     Any jobs currently running are likely to fail when the vault closes this way, so use it carefully, and only when no job is running.
   • In practice, the window closes either when you close it from the Vault Agent, or when the requested interval expires on its own.

Opening and Closing Maintenance Window - Basic AirGap

note

Basic AirGap means a policy with Managed by Vault Agent disabled (see AirGap Jobs) — the same Virtual AirGap functionality covered in Licensing, not the retired Airgap Basic product edition (deprecated August 1, 2022).

For Basic AirGap, follow these steps to open and close the maintenance window:

1. Opening the Maintenance Window:

   • Collect the job name (JOB_NAME) from AirGap > Job History or from /opt/superna/sca/data/airgap/syncAirGap.json.
   • You will also need the policy name (POLICY) and the source name (SOURCE_NAME) from the Isilon source.
   • Run one of the following commands to open the maintenance window:
     • Using job name:
       igls airgap connect --job=<JOB_NAME> --timeout=<DURATION>[m|h]
     • Using policy and source:
       igls airgap connect --policy=<POLICY> --source=<SOURCE_NAME> --timeout=<DURATION>[m|h].

2. Closing the Maintenance Window:

   • To close the window, run the respective command:
     • Using job name:
       igls airgap disconnect --job=<JOB_NAME>.
     • Using policy and source:
       igls airgap disconnect --policy=<POLICY> --source=<SOURCE_NAME>.

Stop AirGap Replication in an Emergency

In case of an emergency where AirGap replication must be stopped to protect the replicated data, follow these steps:

1. Disable AirGap Replication:

   • SSH into Eyeglass as the admin user.
   • Run the command:
     igls airgap disable.
     This command quickly disables all AirGap policies, halting any syncing activity.

2. Re-enable AirGap Replication:

   • When the situation is resolved, re-enable the replication by running:
     igls airgap enable.

Common Issues

AirGap Job Failed Due to Active RSW Events

AirGap jobs might fail to start if there are active Ransomware events. These events can block AirGap jobs from proceeding.

Troubleshooting Steps:

1. Identify the Failure Point:

   • Check the AirGap > Job History to determine at what step the AirGap job failed.

2. Resolve Active RSW Events:

   • Review any active RSW events and resolve them.

3. Retry the AirGap Job:

   • Once the RSW events are resolved, either wait for the next scheduled job or manually start the job.

AirGap Job Failed After Default Timer Ran Out

Eyeglass has a timeout setting, airgapJobTimeout, which stops the AirGap SIQ job if it exceeds the specified time. By default, this is set to 240 minutes (4 hours). If jobs are failing due to this limit, you can extend the timeout.

1. Find the Current Timeout Value
   Check the file /opt/superna/sca/data/system.xml to find the current airgapJobTimeout setting.

2. Increase the Time Until Timeout

   Follow these steps to modify the timeout value:

   a. Switch to the root user:

   sudo su

   b. Open the file with a text editor:

   vi /opt/superna/sca/data/system.xml

   c. Locate the line with <airgapJobTimeout>240</airgapJobTimeout>, replace 240 with the desired timeout in minutes, then save the file.

   d. Restart the service to apply changes:

   systemctl restart sca

This procedure will update the timeout value, allowing longer AirGap jobs to complete without failure.