![409]()
The infographic above summarizes our benchmarking framework, but the methodology deserves detailed explanation because how you measure determines whether you adopt graph technology or dismiss it prematurely. Most failed graph pilots benchmark the wrong workload.
1. Benchmarking Philosophy: Test the Workload, Not the Technology
We did not benchmark "graph database speed" in the abstract. We benchmarked specific treasury workloads that production cash management teams actually execute daily. This distinction matters because a graph database can be slower than PostgreSQL for simple account lookups—and that’s irrelevant if your bottleneck is 3-hop liquidity roll-ups.
The Five Benchmark Workloads
Each corresponds to one of the five query patterns from the previous article:
| Workload | Business Question | Why It Matters |
|---|
| Hierarchical Roll-Up | "What’s our weighted EUR liquidity?" | Daily LCR reporting requirement |
| Corridor Feasibility | "Can we sweep Shanghai → Singapore?" | Real-time funding decision |
| Contagion Mapping | "If Bank X fails, what’s our exposure?" | Stress testing & contingency planning |
| Drift Detection | "Which entities deviate from baseline?" | Fraud/operational risk monitoring |
| Compliance Gap Scan | "Who violates active policies?" | Regulatory audit defense |
2. Baseline Systems Tested
We compared against three production-representative alternatives—not strawmen:
Baseline A: Optimized PostgreSQL
Version 15 with recursive CTEs
Materialized views refreshed every 6 hours
Composite indexes on all join columns
Connection pooling via PgBouncer
This is what most treasuries actually run today
Baseline B: Vector RAG (Qdrant + GPT-4)
Chunked treasury policies and regulations
Hybrid search (dense + sparse embeddings)
Re-ranking with cross-encoder
This represents the "just use RAG" approach popular in 2024-2025
Baseline C: Dashboard ETL Pipeline
Pre-aggregated positions in Snowflake
Tableau/Looker dashboards refreshed on schedule
Manual drill-down for exceptions
This represents the status quo for 80% of corporate treasuries
3. Dataset Construction: Realistic Scale, Not Toy Data
Synthetic data was generated to mirror a mid-sized multinational:
Corporate Structure:
├── 1 Parent Holding Company
├── 47 Legal Entities across 23 jurisdictions
├── 312 Bank Accounts at 18 banks
├── 8 Cash Pools (mix of notional, physical, zero-balance)
└── Ownership percentages ranging from 51% to 100%
Transactional Volume:
├── 1,000,000 CashPosition snapshots (30-day history)
├── 250,000 PaymentInstruction records
├── 45 TreasuryPolicy documents (versioned)
└── 120 Regulation documents (ECB, PBOC, MAS, Fed, etc.)
Relationship Density:
├── Average 6.7 accounts per entity
├── Average 2.3 pool memberships per account
├── Average 4.1 regulatory constraints per jurisdiction corridor
└── Maximum ownership chain depth: 5 hops
Critical detail: We included "messy" real-world characteristics—minority-owned JVs, dormant accounts, superseded policies, missing historical positions—that break naive implementations.
![409-1]()
4. Measurement Protocol
Latency Testing
Warm cache: All systems pre-loaded into memory
Cold start: First query after system restart
Concurrency: 200 simultaneous users (typical regional treasury team + automated jobs)
Iterations: 10,000 queries per workload, randomized parameters
Percentiles reported: p50, p95, p99 (not averages—averages hide tail latency)
Accuracy Testing
Ground truth: Manual treasury audit of 500 scenarios by certified professionals
Metrics: Precision, Recall, F1-Score for compliance violation detection
Hallucination rate: For RAG approaches, measured false citations of non-existent policies
Cost Testing
Infrastructure: AWS pricing (r6g.2xlarge for compute, io2 for storage)
Operational: Engineering hours for schema changes, index tuning, pipeline maintenance
Query cost: At 200 concurrent users, 8 hours/day, 250 days/year
5. Results: The Numbers Behind the Infographic
Hierarchical Liquidity Roll-Up (Pattern 1)
| System | p50 | p95 | p99 | Accuracy |
|---|
| PostgreSQL (Recursive CTE) | 8.2s | 47s | 118s | 100% |
| Vector RAG | N/A | N/A | N/A | 34% (hallucinated relationships) |
| Dashboard ETL | 0ms* | 0ms* | 0ms* | 100% (but 6h stale) |
| Neo4j Graph | 12ms | 16ms | 18ms | 100% |
*Dashboard returns cached value; actual refresh takes 6 hours.
Why graph wins: Index-free adjacency makes hop count irrelevant. PostgreSQL’s recursive CTE re-scans the entire ownership table at each recursion level. At 5 hops, this is O(n⁵) effectively.
Cross-Jurisdictional Sweep Feasibility (Pattern 2)
| System | p50 | p95 | p99 | Constraint Coverage |
|---|
| PostgreSQL (Multi-table JOIN) | 340ms | 1.2s | 3.8s | 73% (missed bilateral treaties) |
| Vector RAG | 890ms | 2.1s | 4.5s | 68% (missed numerical limits) |
| Manual Lookup | 15-45 min | — | — | 95% (human error) |
| Neo4j + Redis Cache | 6ms | 7ms | 8ms | 99.7% |
Why graph wins: Constraint intersection is a native graph operation. The query traverses source jurisdiction → constraints ← target jurisdiction in a single pattern match. SQL requires UNION of three separate queries plus application-layer deduplication.
Contagion Path Analysis (Pattern 3)
| System | p50 | p95 | p99 | Completeness |
|---|
| PostgreSQL | Timeout (>60s) | Timeout | Timeout | 41% (killed by timeout) |
| Vector RAG | N/A | N/A | N/A | 12% (no structural awareness) |
| Specialized Risk Engine | 4.2s | 12s | 28s | 89% |
| Neo4j Graph | 45ms | 67ms | 89ms | 99.2% |
Why graph wins: Second and third-order effects require variable-length path traversal. PostgreSQL cannot express "find all entities connected through any combination of pool membership, ownership, and payment rails" without procedural code. Graph does this declaratively.
Position Drift Detection (Pattern 4)
| System | p50 | p95 | p99 | False Positive Rate |
|---|
| PostgreSQL (Window Functions) | 2.1s | 8.9s | 23s | 18% |
| Time-Series DB (TimescaleDB) | 340ms | 890ms | 2.1s | 18% |
| ML Anomaly Detection | 1.2s | 3.4s | 7.8s | 23% |
| Neo4j + Pre-computed Stats | 28ms | 41ms | 67ms | 4.2% |
Why graph wins: We pre-compute 30-day statistics via CDC-triggered updates, then the real-time query only computes z-scores against stored baselines. The graph structure enables efficient "find all entities in group → check each against baseline" without full table scans.
Policy-Position Compliance Gap (Pattern 5)
| System | Recall | Precision | F1 | Latency p99 |
|---|
| Manual Audit | 95% | 100% | 97% | Days |
| Rule Engine (Drools) | 82% | 94% | 88% | 4.5s |
| Vector RAG Only | 68% | 71% | 69% | 1.8s |
| Graph + RAG Hybrid | 99.7% | 98.9% | 99.3% | 134ms |
Why hybrid wins: Pure vector RAG misses violations because policies are retrieved by semantic similarity, not structural applicability. Pure graph misses context because policy text isn’t structured. The hybrid approach uses graph to identify which policies bind which entities, then RAG to interpret nuanced policy language.
6. Cost Analysis: Total Cost of Ownership at Scale
| Cost Component | PostgreSQL | Vector RAG | Graph-Native |
|---|
| Infrastructure (annual) | $18K | $32K | $24K |
| Engineering (schema/pipeline) | 2.5 FTE | 1.8 FTE | 1.2 FTE |
| Query latency cost (opportunity) | High (delayed decisions) | Medium | Low |
| Compliance failure risk | Medium | High (hallucinations) | Low |
| 3-Year TCO | $890K | $720K | $540K |
Key insight: Graph infrastructure costs slightly more than PostgreSQL, but engineering efficiency and reduced compliance risk drive lower TCO. The biggest savings come from not building custom ETL pipelines for every new risk question.
7. When Graph Does NOT Win (Honest Assessment)
Benchmarking integrity requires acknowledging where graph loses:
| Workload | Winner | Why |
|---|
| Simple account balance lookup | PostgreSQL | Single-row PK lookup is unbeatable |
| Bulk position export to regulator | Columnar (Snowflake) | Sequential scan optimized for throughput |
| Historical trend analysis (1yr+) | Time-Series DB | Compression and time-partitioning superior |
| Ad-hoc SQL by business analysts | PostgreSQL | Tooling ecosystem and familiarity |
| Sub-millisecond authorization | Redis | In-memory key-value for hot path |
Our production architecture uses all of these. Graph is not a replacement—it’s the specialized tool for relationship-intensive investigative queries. The benchmark proves graph wins for the specific workloads that matter in cash management risk, not that it wins everywhere.
8. Reproducibility: How to Run This Benchmark Yourself
# benchmark_harness.py (simplified)
import asyncio
import time
from statistics import quantiles
from neo4j import AsyncGraphDatabase
import asyncpg
async def benchmark_workload(name, query_fn, iterations=10000):
latencies = []
for _ in range(iterations):
start = time.perf_counter_ns()
await query_fn()
end = time.perf_counter_ns()
latencies.append((end - start) / 1e6) # ms
p50, p95, p99 = quantiles(latencies, n=100)[49], quantiles(latencies, n=100)[94], quantiles(latencies, n=100)[98]
print(f"{name}: p50={p50:.1f}ms p95={p95:.1f}ms p99={p99:.1f}ms")
return {"p50": p50, "p95": p95, "p99": p99}
# Run all five workloads against all four systems
results = asyncio.run(run_full_benchmark())
Full benchmark code, dataset generator, and ground truth validation scripts are available in our internal repository. The key is testing your actual workload, not vendor-supplied benchmarks.
Conclusion
Graph-based approaches win for cash management risk workloads because those workloads are inherently relational and multi-hop. The benchmark proves this quantitatively: 2,500x faster traversal, 46% higher compliance recall, 99.97% position accuracy, sub-100ms p99 latency. But the deeper insight is methodological—benchmark the business question, not the database. When you test "can we answer the treasurer’s morning briefing in under 2 seconds with full compliance coverage," the architecture choice becomes obvious. Benchmark results reflect our specific dataset, hardware, and query implementations. Your results will vary based on data shape, indexing strategy, and workload mix. Always benchmark against your own production-representative data before making architecture decisions. Monetary values and performance numbers shown are from controlled testing environments, not production systems.