Running Multiple ECS-Sync Jobs

Introduction

ECS-Sync UI supports up to ten active jobs at a time. To run more than 10 jobs while maintaining reliability, vault agent provides three features:

• Automatic Job Archiving: Vault agent removes completed jobs from ecssync to free slots
• Job Queueing: Schedule more than 10 jobs at once. Ten jobs run concurrently while others wait in queue for available slots
• Job Cloning: Duplicate existing jobs quickly with new credentials or buckets for faster setup

Automatic Job Archiving

When a job finishes (started manually with ecactl ecssync start or by schedule), vault agent archives it. This frees one ecssync slot for new jobs.

Execution Workflow

When you start a job manually or on schedule:

1. Open Network Connection
   Opens connection if not already open for other operations

2. Verify RSW Events
   Checks for any Ransomware Defender events

3. Start ECSSync Job
   Runs the job and waits for completion

4. Archive Job
   Removes job from ecssync UI (new step with this feature)

5. Close Network Connection
   Closes connection if no other operations are active

Inspect Results

View job status and execution details:

# View all job statuses
ecactl ecssync listjobs

# Track specific job details
ecactl ecssync statusjob --job <jobName>

Job Queueing

Queue Behavior

When you request a job start (manually or by schedule), vault agent adds it to the queue immediately. A background process continuously checks the queue. When a UI slot becomes available, it processes the oldest queued job using the standard workflow.

Jobs remain in queue when no slots are available.

Time Limit

Jobs wait in queue for up to 24 hours by default.

Configure this limit using the environment variable ECS_SYNC_MAX_TIME_WAIT_IN_QUEUE.

Duplicate Prevention

Each job can appear only once in the queue. When you attempt to launch a job already in queue:

• The start command fails with: "Job was not started, it was already in the waiting list."
• Scheduled launches raise the alarm ECS_SYNC_JOBS_QUEUE_BACKING_UP in Eyeglass

Queue Persistence

Vault agent writes the queue to Zookeeper every 5 seconds (configure interval with ECS_SYNC_ZK_SAVE_QUEUE_INTERVAL). This preserves queue state across vault agent restarts.

Queue Management Commands

Purpose	Command
Show queued jobs	ecactl ecssync queuedjobs
Clear the queue	ecactl ecssync queuedjobs --reset true

Job Cloning

Create a new job by duplicating an existing one. Override only the source and destination details:

ecactl ecssync clonejob \
  --job <existingJobName> \
  --newname <newJobName> \
  --sourcebucket <srcBucket> \
  --destbucket <destBucket> \
  --sourceusername <srcUser> \
  --sourceuserkey <srcKey> \
  --destusername <destUser> \
  --destuserkey <destKey> \
  --sourceconnstr <CAS_source_connection_str> \
  --destconnstr <CAS_dest_connection_str>

The cloned job appears immediately in listjobs. Run it like any other job.

Command Reference

Action	Command
View jobs	ecactl ecssync listjobs
View queued jobs	ecactl ecssync queuedjobs
Reset queue	ecactl ecssync queuedjobs --reset true
Clone existing job	ecactl ecssync clonejob ...