-
Type:
Task
-
Status: Open (View Workflow)
-
Priority:
Normal
-
Resolution: Unresolved
-
Component/s: None
-
Labels:
Group reductions are currently executed synchronously on the actor thread, blocking ingestion and further reductions. This ticket proposes a structural change to run group reductions asynchronously.
Each visit group reduction would be executed as an independent thread-pool job, with its own Pipeline, Butler run, and SeparablePipelineExecutor, ensuring no concurrent writes to the same run. Configuration overrides would be applied per job, and a lock would be added around collection-chain updates to avoid race conditions.
This would allow ingestion to continue while reductions are running and enable multiple group reductions to execute concurrently providing a significant speed-up, given the large number of available CPU on pfsa-usr03
- relates to
-
INSTRM-2244 Move drpActor in Hilo to pfsa-mst01 node
-
- Open
-