Command Routing and Prioritization Patterns at Scale

Introduction

As systems grow in complexity, services begin receiving thousands to millions of commands from APIs, background jobs, integrations, and event-driven workflows. Handling all commands equally leads to queue congestion, slow responses, and system-wide delays.

This article explains how to design a scalable command routing and prioritization model where each command is routed to an appropriate execution channel and processed with system-wide fairness, SLAs, isolation, and predictable load behavior.

Understanding Commands at Scale

A command is an instruction to perform an operation, for example:

CreateInvoice
ReserveStockLine
ApproveRepairOrder
ProcessPayment
GeneratePickTicket

When scale increases, challenges appear:

Commands arriving faster than they can be processed
Certain commands needing higher priority (e.g., user-facing actions)
Heavy commands blocking small lightweight commands
Retry storms causing overload
No isolation between domains

A structured routing model solves these challenges.

High-Level Architecture

             +---------------------------+
             |  API Gateways / Services  |
             +---------------------------+
                         |
                         v
               +--------------------+
               |  Command Router    |
               +--------------------+
                  |     |       |
                  v     v       v
      +-----------+ +-----------+ +-----------+
      | High-Pri  | | Medium    | | Low-Pri   |
      | Channel   | | Channel   | | Channel   |
      +-----------+ +-----------+ +-----------+
          |             |              |
          v             v              v
   +-------------+ +-------------+ +-------------+
   | Executors   | | Executors   | | Executors   |
   +-------------+ +-------------+ +-------------+

The command router determines:

Which queue/channel the command should go into
What priority level the command should hold
Whether the command should load-balance or go to a sticky executor

Command Routing Patterns

1. Header-Based Routing

Routing based on:

CommandName
TenantId
UserType (Admin vs Customer)
SLA requirement
Metadata

Example rule

IF command == "GenerateInvoice" AND tenantTier == "Gold"
ROUTE TO HighPriorityChannel

2. Domain-Based Routing

Each domain owns its own executor pool:

Inventory Commands  → Inventory Executor
Repair Commands     → Repair Executor
Invoice Commands    → Billing Executor

Benefits

Isolation
Independent scaling
Fault containment

3. Cost-Based Routing

Compute cost based on CPU/RAM/IO:

Lightweight commands → FastLane Channel
Medium commands → General Channel
Heavy commands → Batch Channel

Example

ReserveStockLine → Lightweight
GenerateMonthlyReport → Heavy

Prioritization Patterns

1. Static Priority Tiering

Levels

P0 → Critical (user-facing)
P1 → High (SLA-driven operations)
P2 → Medium (background domain tasks)
P3 → Low (offline, async generation)

2. Dynamic Priority Escalation

If a command waits too long:

If WaitTime > Threshold
    Increase Priority

Useful for:

Long queues
Tenant fairness
Preventing starvation

3. Tenant-Aware Prioritization

If you have subscription-based tiers:

Gold tenants → priority boost
Silver tenants → normal
Bronze tenants → lowest

4. Resource-Aware Throttling

Prevent overload by slowing low-priority channels when CPU/DB load is high.

Workflow Diagram: Command Ingestion to Execution

+-------------+
|  Incoming   |
|  Command    |
+-------------+
       |
       v
+------------------------+
| Validate & Enrich      |
+------------------------+
       |
       v
+------------------------+
| Command Classification |
| (Cost, Domain, SLA)    |
+------------------------+
       |
       v
+----------------------------+
| Routing Decision Engine    |
+----------------------------+
       |
       |-------- High Priority Queue
       |-------- Medium Priority Queue
       |-------- Low Priority Queue
       v
+-----------------------------+
| Executor Pools per Channel  |
+-----------------------------+

Sequence Diagram: Command Routing Lifecycle

User/API -> CommandRouter: SubmitCommand()
CommandRouter -> MetadataService: FetchTenantTier()
CommandRouter -> CostAnalyzer: ComputeCommandCost()
CommandRouter -> PriorityEngine: DeterminePriority()
PriorityEngine -> CommandRouter: PriorityLevel
CommandRouter -> QueueService: Enqueue(priorityChannel)
QueueService -> Executor: DeliverCommand()
Executor -> DomainLogic: Process()
DomainLogic -> EventBus: PublishDomainEvents()

Implementation Blueprint (C# / .NET)

Command Envelope Model

public class CommandEnvelope
{
    public string CommandName { get; set; }
    public string TenantId { get; set; }
    public int Priority { get; set; }
    public Dictionary<string, object> Metadata { get; set; }
    public object Payload { get; set; }
}

Routing Rules Engine

public int DeterminePriority(CommandEnvelope cmd)
{
    if (cmd.Metadata["Tier"].ToString() == "Gold")
        return 0;

    if (cmd.CommandName.Contains("Generate"))
        return 3;

    return 1;
}

Patterns for Large Scale Stability

1. Dead-Letter Routing

When commands fail repeatedly:

Send to DeadLetterQueue with reason + original payload

2. Retry with Backoff

Avoid retry storms:

Retry 3 times → exponential backoff → DLQ

3. Predictive Scaling

Use metrics:

Queue depth
Command latency
Executor CPU

to autoscale executor pods.

Observability and Monitoring

Required Metrics

Queue length per channel
Average wait time per priority
Reject rate due to overload
Slow executor detection
End-to-end latency

Required Traces

traceId propagation inside commands
span for router decision

Best Practices

Never mix synchronous real-time commands with low-priority background commands
Always provide an SLA per priority tier
Keep routing rules declarative (YAML/JSON-based)
Use idempotency keys for retry-proof command execution
Test routing logic using simulation workload

Conclusion

Command routing and prioritization is essential for building predictable, resilient, and cost-efficient systems at scale. With proper routing channels, dynamic SLA-based prioritization, and domain isolation, your system can handle millions of commands without congestion or user-visible delays.