Eyeglass Warm Standby Direct Sync Guide

Overview

This procedure protects the production Eyeglass appliance itself using a second Eyeglass appliance, kept in sync via a scheduled backup copy. This second appliance can take over operations under either of these conditions:

1. Controlled failover — deliberately switching the active appliance from one data center to another.
2. Uncontrolled failover protection — the second appliance has a near-real-time synced copy of all policies and configuration data (shares, exports, quotas) needed to complete a failover to the surviving cluster.

caution

If declaring an uncontrolled failover, the active appliance must be at the site where the data will become active. If the site with the active appliance is the one that failed, use the Warm Standby procedure below instead — the appliance should be co-located with the data that will be writable.

Definitions

• Active Appliance — responsible for syncing configuration data; the primary appliance for all failover operations.
• Warm Standby Appliance — the second appliance, kept synced from the active appliance's backups.

Operating Considerations

• After a planned, controlled failover, generate a fresh backup from the active appliance once the backup completes — this reflects the current DR state post-failover. The daily backup/sync process alone won't capture this immediately.
• Best practice:
  • Enable phone-home and ask support to enable daily off-site appliance backup. Backups are retained for 14 days before automatic deletion.
  • After a planned failover, push an off-site backup directly to support: select the generated Support backup, then choose "Upload selected file directly Superna support."
  • If phone-home is disabled or a firewall/proxy blocks direct upload, download the newly created support backup from the active appliance and upload it via the support portal instead.
  • Additionally, download the support backup file and store a copy at the site opposite the active appliance, and document that location internally.
  • The daily sync backup (below) is typically up to 24 hours old and is usually the backup needed in most recovery scenarios.

Deploy the Second Appliance and Prepare for Sync

Deploy a second Eyeglass appliance following the standard installation guide.

Configure Keyless SSH on the Active Appliance

1. On the warm standby appliance, set a root password:

   sudo -s
   passwd
2. On the active appliance, log in as admin via SSH:

   sudo -s
   cd /root
   ssh-keygen
   (accept the defaults for all prompts)
3. Copy the key to the warm standby appliance:

   ssh-copy-id -i /root/.ssh/id_rsa.pub root@x.x.x.x
   Replace x.x.x.x with the warm standby appliance's IP. Accept the SSH fingerprint prompt and enter the root password set above.
4. Verify keyless SSH works:

   whoami
   ssh root@x.x.x.x
   If no password prompt appears, it's configured correctly.

Configure Scheduled Cron Sync from Active to Warm Standby

1. On the warm standby appliance, set up the backup directory permissions:

   sudo mkdir -p /opt/superna/var/backup/
   sudo setfacl -m u:admin:rwx /opt/superna/var/backup/
2. On the active appliance, test copying backup files:

   sudo -s
   rsync -auv -e "ssh -i ~/.ssh/id_rsa" --delete -og --chown=sca:users /opt/superna/var/backup/ root@x.x.x.x:/opt/superna/var/backup/
   (replace x.x.x.x with the warm standby appliance's IP)
3. Create the sync script on the active appliance:

   sudo -s
   nano /root/warmstandby.sh
   Paste (replacing x.x.x.x):

   rsync -auv -e "ssh -i ~/.ssh/id_rsa" --delete -og --chown=sca:users /opt/superna/var/backup/ root@x.x.x.x:/opt/superna/var/backup/
   Save with Ctrl+X, Y, then:

   chmod 777 /root/warmstandby.sh
4. Optional: build the version with failure detection and email alerting — see Script with Failure Detection and Email Alerting below before continuing to the next step.
5. Schedule the script to run daily via cron on the active appliance:

   sudo -s
   cd /etc/cron.d
   echo "0 8 * * * root /usr/bin/timeout 6h /root/warmstandby.sh" > iglsstandby
   systemctl restart cron
6. Verify the copy is running: after the next scheduled run, check the warm standby appliance for correctly dated files:

   ls -ls /opt/superna/var/backup/

Script with Failure Detection and Email Alerting

note

This requires Postfix email configured on Eyeglass — see Custom Email/Webhook Routing for setup, then return here to finish the script.

1. Log in as admin via SSH.
2. Edit the script:

   nano /home/admin/warmstandby.sh
3. Paste the following, replacing x.x.x.x with the warm standby appliance's IP and email@example.com with the group address that should receive sync-failure alerts:

   if ! rsync -auv -e "ssh -i ~/.ssh/id_rsa" --delete -og --chown=sca:users /opt/superna/var/backup/ root@x.x.x.x:/opt/superna/var/backup/
   then
     echo Sync failed
     echo sending error email
     mail -s "Sync Failed to standby Eyeglass Appliance" -r email@example.com < /dev/null
     exit
   fi
   echo sync was successful
   mail -s "Sync was successful to standby Eyeglass Appliance" -r email@example.com < /dev/null
   exit
4. Save with Ctrl+X, Y, then:

   chmod 777 /home/admin/warmstandby.sh

Restore the Warm Standby Appliance to Become Active

note

Requires release 2.5.6 or later.

1. SSH to the warm standby appliance as admin.

2. Run the restore command:

   • Auto-detect the most recent backup:

     igls app restore /opt/superna/var/backup/
     This detects the most recent backup and prompts for confirmation before proceeding. You'll be prompted for the admin password again to elevate to root.
   • Use a specific backup file:

     igls app restore /opt/superna/var/backup/<name_of_backup.zip>

3. Confirm the restore when prompted:

   • Testing only — answer No to exit without restoring. This tests the procedure without making the standby appliance active.

   • Production switch — answer Yes to proceed. Monitor the command until it completes; it may take 15–20 seconds before the web UI becomes available.

     caution

     The original active appliance must be powered off before switching to the warm standby. Never run two appliances against the same clusters — this is unsupported and can cause conflicts. After the restore, verify the warm standby appliance is up and reachable before proceeding.

4. Once restored, log in to the GUI to start a failover job following your normal failover procedure.

Appliance Switch Test Procedure

1. Deploy both appliances as described above.
2. Power off the active appliance VM.
3. Follow the restore procedure above to switch to the warm standby appliance.
4. Log in to the GUI, open the Jobs icon and Running Jobs tab to confirm configuration sync jobs are running, and check the DR Dashboard after at least 15 minutes for a current readiness view.
5. To revert back to the original active appliance:
   1. Log in to the warm standby appliance.
   2. Factory-reset it:

      sudo /opt/superna/sbin/reset.sh
      This deletes the database and removes all managed clusters. If prompted for which databases to reset, select All.
   3. Power the original active appliance VM back on.
   4. Log in and verify jobs and the DR Dashboard.