Skip to content

Guide: Mongoose server Threading Model

This guide explains how Mongoose server uses threads to run your event processors, services, event sources, and the built‑in scheduler. It focuses on where your code executes, how agents communicate, and how to design components that are correct and performant under the single‑threaded processor model.

You’ll learn:

  • Agent groups and their threads (processor groups, worker/service groups, scheduler)
  • Single‑thread rule for processors and why it matters
  • How event sources publish to processors (queues and wrapping)
  • Lifecycle timing (init → start → startComplete → stop → tearDown)
  • Where callbacks execute (processor thread vs service thread vs scheduler thread)
  • How to safely hand work across threads (EventToQueuePublisher, ScheduledTriggerNode)

References in this repository:

1) Big picture

Mongoose server composes several “agents,” each typically running on its own thread:

  • One or more Processor Agent Groups (each contains one or more StaticEventProcessor instances)
  • One or more Service Agent Groups (each hosts zero or more worker services)
  • One Scheduler Agent (the deadline wheel)
  • Optional Agent‑hosted event sources (e.g., FileEventSource)

Event data is handed between agents through Agrona queues managed by EventFlowManager and EventToQueuePublisher. Control requests (add or stop a processor, deliver a service) reach each processor group through its own ordered control mailbox, which the group drains before it dispatches events. Each processor runs single‑threaded on its agent thread — your processor code is not concurrently executed by multiple threads.

Component/thread overview

The runtime diagram separates application delivery, group control and helper I/O. A feed/group queue may serve several processors; assigning two processors to the same group does not give each its own thread or necessarily its own feed queue.

Scheduler expiry actions run on the scheduler thread. To change processor state, publish into its group or use ScheduledTriggerNode; do not run the processor directly from the scheduler.

2) Lifecycle and when things start

Lifecycle participants expose different callbacks; they do not all follow one identical sequence on one thread. Server startup coordinates service and source preparation, starts group runners and waits for startup progress. The group's admitted adds build and start business processors on its agent thread. Start failures at boot follow the configured boot failure policy, which differs from the default isolation of a later dynamic add.

The group checks control inputs before normal event delivery. A call made inside a handler or lifecycle callback queues follow-on control work; it must not synchronously wait for that work to run on the same group.

Startup and shutdown are fenced against one another. A shutdown deadline can expire while startup is still inside user code: the server then reports incomplete shutdown rather than pretending startup was cancelled. See control requests and shutdown.

3) Event dispatch path

Agent‑hosted event sources (e.g., FileEventSource) typically publish on their own thread. The data is mapped/wrapped by EventToQueuePublisher and offered to one or more target queues. Processors consume from those queues on their own agent thread.

sequenceDiagram
    participant SRC as FileEventSource <br>(service agent thread)
    participant PUB as EventToQueuePublisher
    participant Q as OneToOneConcurrentArrayQueue
    participant P as StaticEventProcessor <br>(processor thread)

    SRC->>PUB: publish(line)
    PUB->>PUB: map + wrap (strategy)
    PUB->>Q: offer(item)
    Note right of Q: Cross-thread boundary
    P->>Q: poll
    Q-->>P: item
    P->>P: invoke handler(s)

Key details:

  • EventWrapStrategy controls whether items are wrapped in NamedFeedEvent or sent raw.
  • If cacheEventLog is enabled in the publisher, events are recorded and replayed on startComplete().
  • EventToQueuePublisher bounds spin (~10 ms) on a full queue, then drops the item for that queue, warns and counts it (droppedCount()). Publishes on one publisher are serialised on a lock held through that spin, so a slow consumer delays every publisher of the same feed; see Publisher serialisation and slow subscribers.

4) Where does my code run?

  • Processor event handlers (e.g., ObjectEventHandlerNode#handleEvent) run on the Processor Agent Group thread.
  • Agent‑hosted services and agent‑hosted event sources run their doWork() on the Service Agent Group thread that hosts them.
  • Scheduler expiry actions run on the Scheduler Agent thread.
  • Admin commands registered within a processor context are delivered into that processor’s queue and executed on its thread, as an event cycle of that processor with its own audit record (details); otherwise they execute on the caller thread.

Implications:

  • Inside a processor, your state is single‑threaded — no external synchronization is required for state mutated only by that processor.
  • To update a processor from another thread (service, scheduler), publish an event into the processor’s queue. Don’t directly mutate processor state from other threads.

Calling server control methods from agent threads

Adding or stopping a processor, delivering a service change, and starting an audit capture use group control requests. The server registry changes outside the group; each group decides its deliveries on its own agent thread, before it dispatches events.

  • An agent thread never waits for a control reply. Called from a processor's handler, its start() or stop(), a @ServiceRegistered callback or an agent-hosted service, addEventProcessor, stopProcessor, registerService and removeService queue the request and return at once. From an ordinary thread they wait up to controlWaitMillis (default 10 s) for the outcome. This skips reply waiting, not arbitrary synchronous service preparation or plugin callbacks. Capture start/stop have their own waits in the capture contract.
  • Do not block on a reply stage from an agent thread (join(), get()). Use the ...Async continuation forms. A dependent attached after the stage has completed runs on the attaching thread.
  • stop() from an agent thread is handed off to a mongoose-server-stop thread.

The full rules, per caller and per method, are in Control-plane contracts: Threading rules.

5) Moving work between threads safely

Common patterns to marshal work back into a processor context:

  • Event publication from services/sources using EventToQueuePublisher (via an EventSource or an admin command routed to a processor queue).
  • Use ScheduledTriggerNode to schedule a new event cycle into the processor after a delay; the trigger causes fireNewEventCycle() to run on the processor thread.

Example using ScheduledTriggerNode:

public class MySchedulerAwareHandler extends ObjectEventHandlerNode {
    private final ScheduledTriggerNode trigger = new ScheduledTriggerNode();

    // After wiring, call this from handleEvent or during start
    void scheduleTick(long millis) {
        trigger.triggerAfterDelay(millis); // later causes a new event cycle on this processor thread
    }
}

6) Best practices

  • Keep Agent#doWork() non‑blocking; do small chunks and return 0 when idle.
  • Don’t call blocking IO from processor event handlers; offload to a worker service and publish results back.
  • For periodic tasks, prefer the scheduler (rescheduling pattern) over sleeping in doWork().
  • Guard verbose logging in hot paths with log.isLoggable(...) or cached flags.
  • If you need pre‑start replay, enable publisher caching in your source’s start() and call dispatchCachedEventLog() on startComplete().
  • Use meaningful agent group names and idle strategies in MongooseServerConfig for observability and performance tuning.

7) Glossary of threads

  • Processor Agent Group thread: runs one or more StaticEventProcessor instances (single‑threaded event handling).
  • Service Agent Group thread: runs worker services and agent‑hosted event sources (Agent#doWork).
  • Scheduler Agent thread: runs DeadWheelScheduler and executes expiry Runnables.

9) Optional: Core pinning for agent threads

Mongoose server supports best-effort CPU core pinning for agent threads. This can help reduce context switches and improve tail latency on systems where CPU affinity is desirable.

Key points:

  • Configure per-agent core pinning using MongooseServerConfig’s agent Threads: ThreadConfig has an optional coreId field (zero-based CPU index).
  • Pinning is applied inside the agent thread itself during start (onStart) for both processor and service agent groups:
  • Mongoose uses a lightweight helper CoreAffinity that attempts to pin via reflection to OpenHFT’s Affinity library if present; otherwise it logs and no-ops.

Configure via fluent builder:

import com.telamin.mongoose.config.MongooseServerConfig;
import com.telamin.mongoose.config.ThreadConfig;
import org.agrona.concurrent.BusySpinIdleStrategy;

MongooseServerConfig mongooseServerConfig = MongooseServerConfig.builder()
    // Configure processor agent group thread
    .addThread(ThreadConfig.builder()
        .agentName("processor-agent")
        .idleStrategy(new BusySpinIdleStrategy())
        .coreId(0) // pin to CPU core 0 (zero-based index)
        .build())
    // Configure service agent group thread
    .addThread(ThreadConfig.builder()
        .agentName("service-agent")
        .coreId(2) // pin to CPU core 2
        .build())
    // ... add groups/feeds/services as usual
    .build();

Runtime behavior:

  • When an agent group thread starts, the server resolves the configured core for that agent via MongooseServer.resolveCoreIdForAgentName and calls CoreAffinity.pinCurrentThreadToCore(coreId). If no coreId is configured, nothing is done.
  • If OpenHFT’s Affinity is not on the classpath, pinning is skipped with an info log.

Optional dependency for pinning:

  • To enable actual OS-level pinning, add the test/runtime dependency on OpenHFT Affinity in your project. See this repository’s POM for an example test-scoped optional dependency: pom.xml (artifact net.openhft:affinity).
  • A simple optional test that exercises pinning via reflection is provided here: CoreAffinityOptionalTest.

Notes:

  • Core IDs are zero-based and depend on your OS/CPU topology.
  • Pinning can reduce scheduling variation but does not guarantee deterministic timing; benchmark your workload.

10) Idle strategies and host environment impact

Agent threads in Mongoose server run a continuous doWork() loop governed by an IdleStrategy. The chosen strategy has significant effects on latency, CPU utilization, power/thermal behavior, and how the OS or container schedules your application. Choose deliberately per agent group.

Common strategies (from org.agrona.concurrent):

  • BusySpinIdleStrategy — tight loop, no yielding. Avoids idle wake-up delay at the cost of constant CPU use; measure latency on the target host.
  • YieldingIdleStrategy — yields the thread when idle. Lower CPU pressure than pure busy spin; more scheduler friendly.
  • BackoffIdleStrategy — progressive spin/yield/park sequence. Good latency/CPU balance for mixed workloads.
  • SleepingIdleStrategy — sleeps for a configured period when idle. Very low CPU during idle, higher wake-up latency.
  • NoOpIdleStrategy — do not idle (return immediately). Use only when an outer loop handles idling.

Effects on the host environment:

  • CPU utilization and heat
    • Busy spin holds a CPU core at near 100% even when idle. Expect higher power draw, fan noise (laptops), and thermal throttling risk under sustained load.
    • Yield/backoff reduce average CPU when idle; sleeping minimizes it.
  • Latency and jitter
    • Busy spin avoids park/unpark delay, but tail latency still depends on scheduling, callbacks and host load.
    • Yield/backoff introduce small, scheduler-dependent jitter; sleeping introduces millisecond-level wake-up delays unless tuned very carefully.
  • OS scheduler interaction
    • Busy spin competes aggressively for CPU time; on oversubscribed hosts it can starve other processes or itself be preempted, causing unpredictable jitter. Consider core pinning and setting process/thread priority where appropriate.
    • Yielding strategies cooperate with the scheduler and play nicer alongside other workloads.
  • Containers and CPU quotas (cgroups/Kubernetes)
    • With CPU limits, busy spin will quickly consume its quota and be throttled, causing bursty latency (periods of fast processing followed by enforced idle). Backoff or sleep strategies tend to produce smoother performance under quotas.
    • If you request fewer CPUs than busy-spinning threads, expect contention and throttling. Align thread count with requested CPU shares/limits.
  • Power management and scaling
    • On laptops/cloud VMs, busy spin may prevent CPUs from entering low-power states and can inhibit turbo frequency boosts for other cores. Sleeping/backoff allow deeper C-states and can reduce cost in autoscaling setups.
  • GC and background services
    • Fully busy-spun cores can reduce headroom for GC, JIT compilation, or sidecar processes. Yield/backoff free slices for these activities especially on shared nodes.

Guidance: choosing an idle strategy per agent

  • Latency-critical processors (market data, control loops): Prefer BusySpinIdleStrategy on dedicated cores or pinned threads. Benchmark p99.9 and tail behavior.
  • Mixed IO/compute services: Use BackoffIdleStrategy to balance responsiveness with CPU efficiency.
  • Mostly idle background workers or development: Use YieldingIdleStrategy or SleepingIdleStrategy to conserve CPU.
  • Under CPU limits/containers: Start with BackoffIdleStrategy; avoid pure busy spin unless cores are dedicated and limits are configured to match.

Configuration examples

YAML-style (conceptual):

agentThreads:
  - agentName: market-data
    idleStrategy: !!org.agrona.concurrent.BusySpinIdleStrategy {}
  - agentName: background-workers
    idleStrategy: !!org.agrona.concurrent.BackoffIdleStrategy { }

Java builder:

import com.telamin.mongoose.config.ThreadConfig;
import org.agrona.concurrent.BackoffIdleStrategy;
import org.agrona.concurrent.BusySpinIdleStrategy;

mongooseServerConfig = mongooseServerConfig.toBuilder()
    .addThread(ThreadConfig.builder()
        .agentName("market-data")
        .idleStrategy(new BusySpinIdleStrategy())
        .build())
    .addThread(ThreadConfig.builder()
        .agentName("background-workers")
        .idleStrategy(new BackoffIdleStrategy())
        .build())
    .build();

Observability and testing

  • Record CPU usage and latency histograms per agent group when tuning. Validate behavior under the same CPU quotas and node types as production.
  • Watch for throttling metrics in Kubernetes (cfs_throttled_*); heavy busy spin under limits will cause periodic stalls.
  • Consider pairing critical busy-spinning agents with core pinning (see section 9) to reduce interference and context switches.