Version: 2.12.1

Automated Installation

The automated installation service provides a streamlined approach to deploying the threat hunting module with simplified configuration and reduced manual steps.

Before You Begin

Before starting the automated installation, review the system requirements and gather the necessary configuration details in the Before You Begin section.

Installation Process

Configure ECA Nodes for Kafka Connection

Configure firewall rules to allow the threat hunting module host to connect to ECA Kafka services.

Connect to ECA node 1 using SSH with the ecaadmin user credentials.
Stop the cluster:
```
ecactl cluster down
```

Open the firewall configuration file:

nano /opt/superna/eca/scripts/eca_iptables.sh

Add the following line under rule #5. Replace x.x.x.x with your threat hunting module host IP address:

# Allow Kafka access from threat hunting module
sudo /usr/sbin/iptables -A $CHAIN -p tcp --dport 9092 -s x.x.x.x -j ACCEPT

Save the file and exit the editor (Ctrl+X in nano).
Apply the firewall changes by restarting the cluster:
```
ecactl cluster up
```
Confirm the firewall rule exists on all ECA nodes in your cluster.

warning

The following steps must be performed on the ML machine.

warning

If zypper fails to install packages, run these commands

# See repos (note the repo name and number)
sudo zypper lr -u
# Disable the CD/DVD repo by NAME (preferred)
sudo zypper mr -d 'openSUSE-Leap-15.6-1'
# or disable by NUMBER if it's, say, 1
sudo zypper mr -d 1
# Refresh again
sudo zypper ref

Set Up the Repository

Install Git package manager on your system:

sudo zypper ref
sudo zypper --non-interactive install git

Clone the threat hunting module repository:
```
git clone https://github.com/supernadev/chart-seed-ml.git
```
note
You will be prompted for authentication credentials. Contact your system administrator for the required username and access token.

Navigate to the repository and check out the target release version:

export RELEASE="1.1.0"
cd chart-seed-ml
git checkout delivery/v$RELEASE

Configure Installation

Update the installer_vars.yaml file with your environment-specific values before running the installation script. This configuration file contains all the parameters needed for your specific deployment environment.

Configuration File Structure

The installer_vars.yaml file is structured into logical sections, each containing related configuration parameters. All parameters include detailed comments explaining their purpose and how to obtain the correct values.

File Location:

chart-seed-ml/
├── installer.sh
├── installer_vars.yaml          # ← Main configuration file
└── .installation/               # Generated during installation (temporary files)

Required Configuration Steps

Navigate to the configuration file:
```
   nano installer_vars.yaml
```

Update the following required sections with your environment-specific values:

ML Module Configuration

# ========================================
# ML Module Configuration
# ========================================
ml_module:
  # IP address where the ML module services will be accessible
  # This should be the IP of your VM
  ip: "YOUR_ML_MODULE_IP"

Required Information:

ML Module IP: The IP address of the OpenSUSE server/VM where the ML module will be installed
How to obtain: Contact your network administrator or check the server's network configuration with ip addr show or ifconfig

Eyeglass Integration Configuration

# ========================================
# Eyeglass Integration Configuration
# ========================================
eyeglass:
  # Eyeglass server IP address for integration
  ip: "YOUR_EYEGLASS_IP"
  # Public key for Eyeglass authentication (value of the eca node file /opt/superna/eca/data/common/.secure/rsa/isilon.pub)
  public_key: "YOUR_ECA_PUBLIC_KEY"
  # Sera token from Eyeglass UI - Integrations - Api Tokens
  token: "YOUR_EYEGLASS_TOKEN"

Required Information:

Eyeglass IP: IP address of your Eyeglass server. You can get it with ip addr show or ifconfig
Public Key: Content of /opt/superna/eca/data/common/.secure/rsa/isilon.pub file from ECA node
Sera Token: Access Eyeglass web UI → Integrations → Api Tokens to generate the sera token

Release and Version Configuration

# ========================================
# Release and Version Configuration
# ========================================
# Version of SEED ML platform to install
release: "1.1.0"

Update to the version to be installed.

Database Configuration

# ========================================
# Database Configuration
# ========================================
# ClickHouse database password for analytics data
clickhouse_pass: "Clickhouse.Change-me.2025"
# PostgreSQL admin password for application data
postgres_admin_pass: "Postgres.Change-me.2025"
# Apache Superset admin user password for dashboards
superset_admin_user_pass: "Superset.Change-me.2025"
# Superset secret key for session encryption
superset_secret_key: "Change-it-2024"

Security Requirements:

Change all default passwords - Never use the example passwords in production
Use strong passwords with mixed case, numbers, and special characters
Minimum 12 characters recommended
Store passwords securely and restrict access to configuration file

Kafka Cluster Configuration

# ========================================
# Kafka Cluster Configuration
# ========================================
# List of Kafka brokers. Corresponds to the ips and names of all eca clusters.
# To get this info, login to eca node 1 and run: ecactl cluster exec docker network ls
kafka:
  - ip: "YOUR_ECA_NODE1_IP"
    port: 9092
    hostname: "YOUR_ECA_NODE1_HOSTNAME"
  - ip: "YOUR_ECA_NODE2_IP"
    port: 9092
    hostname: "YOUR_ECA_NODE2_HOSTNAME"

Required Information for Each Kafka Node:

IP Address: Network IP address of the Kafka broker
Port: Kafka broker port (standard: 9092)
Hostname: Fully qualified domain name of the Kafka node

How to obtain Kafka brokers configuration:

Connect to ECA Node: SSH to ECA node 1 as ecaadmin
Get kafka brokers information: Run: ecactl cluster exec docker exec kafka cat ./config/server.properties | grep listeners= to get kafka brokers ip and hostname

note

You will see something like

ecaadmin@jmseca1-1:~> ecactl cluster exec docker exec kafka cat ./config/server.properties | grep listeners=
#advertised.listeners=PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://kafka.node1.jmseca1.eca.local:9092
Connection to 10.152.1.148 closed.
# advertised.listeners=PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://kafka.node2.jmseca1.eca.local:9092
Connection to 10.152.1.149 closed.
# advertised.listeners=PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://kafka.node3.jmseca1.eca.local:9092
Connection to 10.152.1.150 closed.

Example with multiple nodes:

kafka:
  - ip: "10.152.1.148"
    port: 9092
    hostname: "kafka.node1.jmseca1.eca.local"
  - ip: "10.152.1.149"
    port: 9092
    hostname: "kafka.node2.jmseca1.eca.local"

Network Requirements:

Ensure firewall rules allow ML module to connect to port 9092 on all Kafka nodes. This is done in the first step of this manual.
Verify DNS resolution for Kafka hostnames from ML module server

Training Scheduler Configuration

warning

Pay attention to the cron expression format, which must include seconds, so it has 6 elements

# ========================================
# Training Scheduler Configuration
# ========================================
# Cron-based scheduling for ML training jobs
# Format: "<cron_expression>:<cluster_name>:<path>:<model_name>"
#
# Cron format: "second minute hour day month day_of_week"
# Examples:
#   "0 0 2 * * 6" = Every Saturday at 02:00:00
#   "0 30 14 * * 1-5" = Every weekday at 14:30:00
#
# Support team: Modify these schedules as needed for your environment
training_scheduler:
- "0 0 0 * * 6:isi-mock-127:/ifs/data/aaa/AccessZoneA:data-exfil-rwmr" # 00:00 Saturday
- "0 0 1 * * 6:isi-mock-127:/ifs/data/aaa/AccessZoneA:data-exfil-rwmw" # 01:00 Saturday
- "0 0 2 * * 6:isi-mock-127:/ifs/data/aaa/AccessZoneA:data-exfil-ml1" # 02:00 Saturday
- "0 0 3 * * 6:isi-mock-127:/ifs/data/aaa/AccessZoneA:data-exfil-ml2" # 03:00 Saturday

Format Explanation:

Cron Expression: "0 0 0 * * 6" = Every Saturday at midnight (00:00:00)
Cluster Name: Isilon cluster name (e.g., isi-mock-127)

note

How to obtain This information is available in the Inventory Icon listing the the cluster name

Path: The full path to the area in the file system that you want to monitor for Data Exfiltration attempts
Model Name: Identifier for the training job type (e.g., data-exfil-rwmr). If you set this value empty then all 4 models will be trained.

Cron Format Details:

Field Allowed values Special characters
Second 0-59 * / , -
Minute 0-59 * / , -
Hour 0-23 * / , -
Day of month 1-31 * / , - ?
Month 1-12 or JAN-DEC * / , -
Day of week 0-6 or SUN-SAT * / , - ?

Common Cron Examples:
"0 0 2 * * 6" = Every Saturday at 02:00:00
"0 30 14 * * 1-5" = Every weekday at 14:30:00
"0 0 */6 * * *" = Every 6 hours
"0 0 0 1 * *" = First day of every month at midnight

Field	Allowed values	Special characters
Second	0-59	`*` `/` `,` `-`
Minute	0-59	`*` `/` `,` `-`
Hour	0-23	`*` `/` `,` `-`
Day of month	1-31	`*` `/` `,` `-` `?`
Month	1-12 or JAN-DEC	`*` `/` `,` `-`
Day of week	0-6 or SUN-SAT	`*` `/` `,` `-` `?`

Security Requirements

Never use default passwords in production
Change all example passwords before installation
Use strong passwords with mixed case, numbers, and special characters
Minimum 12 characters recommended
Never commit passwords to version control

Save file and exit

Execute Installation

Make the installer script executable:
```
chmod +x installer.sh
```
Execute the automated installation:
```
./installer.sh
```
Installation Prompts and Responses

During the installation, you'll be prompted for a few key pieces of information. Follow the recommended responses to ensure a smooth setup.
- Helm Repositories This adds all required Helm repositories for the ML platform components
```
Do you want to add the necessary Helm repositories? (Hit Enter):
```
  Recommended Response: Press Enter (default: Y) or type Y.
- AWS Credentials Prompts The credentials will be provided by Services
  
  If AWS credentials are not set as environment variables, you'll be prompted:
```
Enter AWS Access Key ID:
```
  Response: Enter the AWS Access Key ID
```
Enter AWS Secret Access Key:
```
  Response: Enter the AWS Secret Access Key
  
  note
  The credentials will be provided by Services
- Configuration Validation The script will automatically validate your installer_vars.yaml file. If any required fields are missing or invalid, you'll see errors like:
```
ERROR: Missing required field: ml_module.ip in installer_vars.yaml
```
  Response: Stop the installation (Ctrl+C), fix the configuration file, and restart the installation.
Installation Progress Indicators:

During installation, you'll see progress messages like:
- "Running full installation..."
- "Installing k3s..."
- "Installing helm"
- "Adding repositories needed and updating them"
- "Building dependencies for clickhouse"
- "Installing AWS CLI..."
Confirm all pods are running successfully:
```
kubectl -n seed-ml get pods
```

Access Superset Dashboard

warning

Importing these dashboards is optional for the module's core functionality, but it is highly recommended as they contain valuable tools for analyzing various threat huntings.

After installation completion, access the Superset dashboard using your threat hunting module IP address:

https://<THREAT_HUNTING_MODULE_IP>:30443/

Replace <THREAT_HUNTING_MODULE_IP> with your actual threat hunting module server IP address: ip addr show.

Import Dashboards

In the Superset dashboard, locate and select the import function in the upper-right corner of the interface.

Copy the dashboard configuration files from the installation directory to your local machine:

scp <user>@<THREAT_HUNTING_MODULE_IP>:<installation-path>/chart-seed-ml/charts/superset/resources/*.zip /local/destination/

Upload the dashboard files through the Superset import interface. Enter the database passwords you configured during installation when prompted.
Configure the ML_Model_Instances dataset with your environment-specific settings:
1. Open Dataset Configuration
  
  Navigate to Datasets > ML_Model_Instances
2. Update IP Address
  
  Update the IP address field with your threat hunting module IP address
3. Configure Service URL
  
  Modify the service URL to match your environment:
```
https://<THREAD_HUNTING_MODULE_IP>:31443/prod/ml-data-exfiltration/
```

Enable Threat Hunting Module in Eyeglass

Use SSH to connect to the SCA VM with administrator credentials.
Open the system configuration file:
```
nano /opt/superna/sca/system.xml
```
Locate the <th_enabled> parameter and set its value to true:
```
<th_enabled>true</th_enabled>
```
Apply the configuration changes by restarting the SCA service:
```
systemctl restart sca
```

Installation Modes

Full Installation

Run a complete installation of the threat hunting module with all components and dependencies:

./installer.sh --install

This installation mode deploys:

Complete ML model training infrastructure
ClickHouse analytical database with optimized configurations
PostgreSQL database for metadata and user management
Superset dashboard interface with pre-configured threat hunting visualizations
Kafka consumer services for real-time event processing
Kubernetes deployment configurations for scalability and reliability

Full Installation Timeline:

Initial setup and validation: 5-10 minutes
Container image downloads: 15-30 minutes (depending on network speed)
Database initialization and configuration: 10-15 minutes
Service deployment and health checks: 5-10 minutes
Total Expected Duration: 45-75 minutes

Update Training Schedule Only

Update ML training schedules and configurations without reinstalling the complete system:

./installer.sh --training-jobs-scheduler

Use this update mode to:

Modify existing training schedules after initial deployment
Add new data sources or training paths
Adjust training frequency based on operational requirements
Test new model configurations without full system reinstallation

Training Schedule Update Timeline:

Configuration validation: 1-2 minutes
Schedule deployment: 2-3 minutes
Service restart and verification: 1-2 minutes
Total Expected Duration: 5-10 minutes

Post-Installation Verification

After any installation mode completion, verify system functionality:

# Check all pods are running
kubectl -n seed-ml get pods

# Verify service connectivity
kubectl -n seed-ml get services

# Test dashboard accessibility
curl -k https://<threat-hunting-ip>:30443/health

# Check training job schedules
kubectl -n seed-ml get cronjobs

# View logs for services
kubectl -n seed-ml logs -f -l component=service-pipelines

Most important component labels for viewing logs:

service-pipelines
service-inventory
service-ml-core
ml-data-exfiltration

Examples:

# View service-inventory logs
kubectl -n seed-ml logs -f -l component=service-inventory

# View service-ml-core logs
kubectl -n seed-ml logs -f -l component=service-ml-core

# View ml-data-exfiltration logs
kubectl -n seed-ml logs -f -l component=ml-data-exfiltration

Troubleshooting Automated Installation

For troubleshooting guidance specific to automated installation, including common configuration issues and diagnostic commands, see the Automated Installation Troubleshooting section in the main troubleshooting guide.

Installation Process​

Configure ECA Nodes for Kafka Connection​

Set Up the Repository​

Configure Installation​

Configuration File Structure​

Required Configuration Steps​

ML Module Configuration​

Eyeglass Integration Configuration​

Release and Version Configuration​

Database Configuration​

Kafka Cluster Configuration​

Training Scheduler Configuration​

Execute Installation​

Access Superset Dashboard​

Import Dashboards​

Enable Threat Hunting Module in Eyeglass​

Installation Modes​

Full Installation​

Update Training Schedule Only​

Installation Process

Configure ECA Nodes for Kafka Connection

Set Up the Repository

Configure Installation

Configuration File Structure

Required Configuration Steps

ML Module Configuration

Eyeglass Integration Configuration

Release and Version Configuration

Database Configuration

Kafka Cluster Configuration

Training Scheduler Configuration

Execute Installation

Access Superset Dashboard

Import Dashboards

Enable Threat Hunting Module in Eyeglass

Installation Modes

Full Installation

Update Training Schedule Only