Introduction
As systems grow in complexity, services begin receiving thousands to millions of commands from APIs, background jobs, integrations, and event-driven workflows. Handling all commands equally leads to queue congestion, slow responses, and system-wide delays.
This article explains how to design a scalable command routing and prioritization model where each command is routed to an appropriate execution channel and processed with system-wide fairness, SLAs, isolation, and predictable load behavior.
Understanding Commands at Scale
A command is an instruction to perform an operation, for example:
CreateInvoice
ReserveStockLine
ApproveRepairOrder
ProcessPayment
GeneratePickTicket
When scale increases, challenges appear:
Commands arriving faster than they can be processed
Certain commands needing higher priority (e.g., user-facing actions)
Heavy commands blocking small lightweight commands
Retry storms causing overload
No isolation between domains
A structured routing model solves these challenges.
High-Level Architecture
+---------------------------+
| API Gateways / Services |
+---------------------------+
|
v
+--------------------+
| Command Router |
+--------------------+
| | |
v v v
+-----------+ +-----------+ +-----------+
| High-Pri | | Medium | | Low-Pri |
| Channel | | Channel | | Channel |
+-----------+ +-----------+ +-----------+
| | |
v v v
+-------------+ +-------------+ +-------------+
| Executors | | Executors | | Executors |
+-------------+ +-------------+ +-------------+
The command router determines:
Which queue/channel the command should go into
What priority level the command should hold
Whether the command should load-balance or go to a sticky executor
Command Routing Patterns
1. Header-Based Routing
Routing based on:
Example rule
IF command == "GenerateInvoice" AND tenantTier == "Gold"
ROUTE TO HighPriorityChannel
2. Domain-Based Routing
Each domain owns its own executor pool:
Inventory Commands → Inventory Executor
Repair Commands → Repair Executor
Invoice Commands → Billing Executor
Benefits
Isolation
Independent scaling
Fault containment
3. Cost-Based Routing
Compute cost based on CPU/RAM/IO:
Lightweight commands → FastLane Channel
Medium commands → General Channel
Heavy commands → Batch Channel
Example
ReserveStockLine → Lightweight
GenerateMonthlyReport → Heavy
Prioritization Patterns
1. Static Priority Tiering
Levels
P0 → Critical (user-facing)
P1 → High (SLA-driven operations)
P2 → Medium (background domain tasks)
P3 → Low (offline, async generation)
2. Dynamic Priority Escalation
If a command waits too long:
If WaitTime > Threshold
Increase Priority
Useful for:
Long queues
Tenant fairness
Preventing starvation
3. Tenant-Aware Prioritization
If you have subscription-based tiers:
Gold tenants → priority boost
Silver tenants → normal
Bronze tenants → lowest
4. Resource-Aware Throttling
Prevent overload by slowing low-priority channels when CPU/DB load is high.
Workflow Diagram: Command Ingestion to Execution
+-------------+
| Incoming |
| Command |
+-------------+
|
v
+------------------------+
| Validate & Enrich |
+------------------------+
|
v
+------------------------+
| Command Classification |
| (Cost, Domain, SLA) |
+------------------------+
|
v
+----------------------------+
| Routing Decision Engine |
+----------------------------+
|
|-------- High Priority Queue
|-------- Medium Priority Queue
|-------- Low Priority Queue
v
+-----------------------------+
| Executor Pools per Channel |
+-----------------------------+
Sequence Diagram: Command Routing Lifecycle
User/API -> CommandRouter: SubmitCommand()
CommandRouter -> MetadataService: FetchTenantTier()
CommandRouter -> CostAnalyzer: ComputeCommandCost()
CommandRouter -> PriorityEngine: DeterminePriority()
PriorityEngine -> CommandRouter: PriorityLevel
CommandRouter -> QueueService: Enqueue(priorityChannel)
QueueService -> Executor: DeliverCommand()
Executor -> DomainLogic: Process()
DomainLogic -> EventBus: PublishDomainEvents()
Implementation Blueprint (C# / .NET)
Command Envelope Model
public class CommandEnvelope
{
public string CommandName { get; set; }
public string TenantId { get; set; }
public int Priority { get; set; }
public Dictionary<string, object> Metadata { get; set; }
public object Payload { get; set; }
}
Routing Rules Engine
public int DeterminePriority(CommandEnvelope cmd)
{
if (cmd.Metadata["Tier"].ToString() == "Gold")
return 0;
if (cmd.CommandName.Contains("Generate"))
return 3;
return 1;
}
Patterns for Large Scale Stability
1. Dead-Letter Routing
When commands fail repeatedly:
Send to DeadLetterQueue with reason + original payload
2. Retry with Backoff
Avoid retry storms:
Retry 3 times → exponential backoff → DLQ
3. Predictive Scaling
Use metrics:
Queue depth
Command latency
Executor CPU
to autoscale executor pods.
Observability and Monitoring
Required Metrics
Required Traces
Best Practices
Never mix synchronous real-time commands with low-priority background commands
Always provide an SLA per priority tier
Keep routing rules declarative (YAML/JSON-based)
Use idempotency keys for retry-proof command execution
Test routing logic using simulation workload
Conclusion
Command routing and prioritization is essential for building predictable, resilient, and cost-efficient systems at scale. With proper routing channels, dynamic SLA-based prioritization, and domain isolation, your system can handle millions of commands without congestion or user-visible delays.