ECA Cluster Operations

Introduction

This page covers day-to-day operational procedures for the Eyeglass Clustered Agent (ECA) cluster: monitoring ECA services from Eyeglass, common cluster maintenance tasks, the core ecactl CLI commands, and the health-check steps to run before opening a support case.

Monitor ECA Services from Eyeglass

Eyeglass can communicate with multiple ECA endpoints (Ransomware Defender, Easy Auditor). Each endpoint has a unique API token, generated from the Eyeglass REST API window, which the ECA uses to authenticate back to Eyeglass on startup.

Once connected, the ECA:

• Sends a heartbeat.
• Notifies Eyeglass of any detected threats.
• Periodically sends statistics on processed events.
• Periodically polls Eyeglass for updated detection thresholds and Ignore List settings.

The Manage Services icon in the Eyeglass UI lists every registered ECA node and the health of each process running on it.

Service States

• Active — has checked in with a recent heartbeat.
• In-Active — has failed to heartbeat and is no longer processing.

Health States

• Up — running, with uptime shown in days.
• Down — not running.

warning

The Delete icon next to a service registration removes that registration entirely. Don't use it unless directed by support.

Cluster Operational Procedures

All commands below are run over SSH as the ecaadmin user on ECA node 1 (the master node), unless noted otherwise.

Cluster shutdown or restart

1. ecactl cluster down — wait until all nodes report down.
2. SSH into each node individually, elevate with sudo -s, and run shutdown.

Restarting the OS does not automatically bring the ECA cluster back up — follow the startup steps below after reboot.

Cluster startup

1. ecactl cluster up
2. Verify the boot messages show the user tables and signal table exist (confirms the connection to the analytics database).
3. ecactl cluster status to verify all containers and tables are present.

Change an ECA node's IP address

1. ecactl cluster down and wait for a full shutdown.

2. sudo -s, then use your OS network configuration tool to change the interface IP. Repeat on every node in the cluster, then verify connectivity with ping and nslookup.

3. On node 1, edit /opt/superna/eca/eca-env-common.conf and update the node IP variables to match the new addresses:

   export ECA_LOCATION_NODE_1=x.x.x.x
   export ECA_LOCATION_NODE_2=x.x.x.x
   export ECA_LOCATION_NODE_3=x.x.x.x

4. Update the audit/NFS export permissions on the storage platform so the new ECA node IPs are allowed to mount the audit data — see the platform-specific audit setup steps in the Qumulo Installation Guide or VAST Installation Guide.

5. ecactl cluster exec sudo rm -rf /opt/superna/mnt/zk-ramdisk/* && sudo systemctl restart docker

6. ecactl cluster up and verify the boot messages look as expected.

7. On the Eyeglass appliance, check /etc/hosts and confirm the new ECA IP addresses are present — correct the file if needed.

8. Open the Eyeglass Manage Services window and confirm all ECA nodes show Active and Green. Remove any leftover entries still showing the old IP addresses.

Change the ECA management tool password

The ECA cluster's built-in management tool UIs (Hbase, Kafka, Spark), reachable from Manage Services, are protected with basic auth. The default login is ecaadmin / 3y3gl4ss — change this after deployment.

ecactl cluster exec "htpasswd -b /opt/superna/eca/conf/nginx/.htpasswd ecaadmin <new-password>"

The new password takes effect immediately on all nodes.

Restart a single ECA node or recover from a host crash

Use this if a single ECA node was restarted or its host crashed and it needs to rejoin the cluster. Don't use it otherwise unless directed by support.

1. On the master node: ecactl cluster refresh — re-integrates the node and checks database table access on all nodes.
2. ecactl db shell, then status — confirm no dead servers are listed.
3. In the Eyeglass UI, check Manage Services and confirm all nodes are green.

Replace the self-signed certificate on the ECA nginx proxy

Run on ECA node 1:

cd /opt/superna/eca/conf/nginx
mv nginx.crt nginx.crt.bak
mv nginx.key nginx.key.bak
openssl req -new -x509 -sha256 -newkey rsa:2048 -nodes -keyout nginx.key -days 365 -out nginx.crt \
  -subj "/CN=<hostname>" -addext "subjectAltName=DNS:<hostname>,IP:<eca-node-ip>"
ecactl cluster push-config
ecactl cluster services restart --container nginx --all

ECA CLI Command Reference

Command	Function
ecactl cluster up / down / status / refresh	Bring the cluster up or down across all nodes, check status and database connectivity, or re-integrate a node that needs to rejoin the cluster.
ecactl cluster remove-docker-networks	Removes all docker networks so ecactl cluster up recreates them — use after a docker subnet change.
ecactl cluster down --hard	Forces a shutdown without a clean database shutdown. Only use if directed by support or if a normal down hangs.
ecactl cluster up --clean	Cleans up Zookeeper data during cluster up.
ecactl cluster down --bg	Faster, parallelized shutdown for upgrades (2.5.7+).
ecactl stats	Live container memory and CPU statistics.
ecactl containers ps	Lists all running containers.
ecactl logs --follow iglssvc	Tails a service's logs for debugging (also works for rmq, fastanalysis, and other service names).
ecactl cluster exec <command>	Runs a single command across all ECA nodes from the master node.

Health Check Before Opening a Support Case

Run these checks before contacting support, or when asked for a health status update — this is especially useful at dark sites with no outbound connectivity for automated diagnostics.

1. Log in to the Eyeglass CLI as admin and run the Eyeglass log parser report; review it and attach it to the support case.
2. Open the Eyeglass UI and check Manage Services:
   • Confirm each registered ECA node shows Active under Remote Services.
   • Everything under the Health section (Running Containers) should be Green/OK. The most common containers to check are turboaudit, fastanalysis, and evtarchive.
   • Check Remote Services → Validation for time skew or database scanning issues.
3. Check Security Guard for the most recent event results — an ERROR state indicates an unhealthy ECA environment. If needed, open the failed log from Ransomware Defender → Security Guard in the Eyeglass Web UI.
4. Check event rates for turboaudit and evtarchive — the Sent/Received rate graphs should show activity above 0 events all the way to the right edge. All-zero activity on either graph is a problem worth flagging.
5. Check ECA node disk space, CPU, and memory usage if asked.
6. If any ECA containers show INACTIVE, note the node number and container names, then SSH into that node and collect the Eyeglass version, total RAM, and OS version for support.
7. If any replication jobs show ERROR, expand the entry and capture the error message for support.
8. Check Active Alarms for anything ECA-related.

For deeper component-level checks, the ECA management UIs are reachable from Manage Services:

• Hbase-master: http://<ECA_NODE1_IP>:16010 — check for offline or failed regions.
• Kafka-manager: http://<ECA_NODE1_IP>/kafkahq (if not already running, start it with ecactl containers up -d kafkahq then ecactl containers start kafkahq).
• Spark-master: reachable from the same Manage Services links — useful when Easy Auditor reports processing issues.