Building a Streaming, Low-Allocation Excel Writer in .NET

Introduction

If you've ever worked with Excel generation in .NET, you've probably encountered the same pain point: generating a simple .xlsx file allocates tens or even hundreds of megabytes of memory. For a 10,000-row dataset with four string columns, popular libraries like EPPlus or ClosedXML routinely allocate 30–100 MB. This problem only compounds when you're serving file downloads in a web API under concurrent load.

This article walks through the key techniques I used to build a zero-dependency Excel writer that stays under 1 MB of allocation for the same 10,000-row workload and completes 8–10× faster. We'll cover OOXML internals, streaming strategies, IBufferWriter<T> integration, and real-world benchmark comparisons.

Why OOXML?

An .xlsx file is just a ZIP archive containing XML files. The structure is well-defined:

• [Content_Types].xml – Content type declarations

• _rels/.rels – Package-level relationships

• xl/workbook.xml – Workbook definition (sheet names)

• xl/_rels/workbook.xml.rels – Workbook relationships

• xl/styles.xml – Cell styles

• xl/sharedStrings.xml – Shared string table (optional)

• xl/worksheets/sheet1.xml – Sheet data

• xl/theme/theme1.xml – Theme definition (optional)

This means we don't need any third-party library to generate a valid .xlsx file. We just need to:

1. Write XML parts to a ZIP stream

2. Handle the OPC (Open Packaging Convention) relationship structure

3. Manage cell data, styles, and shared strings efficiently

Technique 1: Direct-to-Stream Writing

Most Excel libraries build an in-memory DOM first, then serialize it. ClosedXML, for example, builds an entire XLWorkbook object graph with IXLRow, IXLCell, styles, formulas, etc. — all in memory — before writing.

A streaming approach flips this: write XML directly to the output stream as data arrives. No intermediate object model.

Here's the core idea in simplified form:

// Traditional DOM approach: everything lives in memory
var workbook = new XLWorkbook();
var sheet = workbook.Worksheets.Add("Sheet1");
sheet.Cell(1, 1).Value = "Header";
// ... add 10,000 rows
workbook.SaveAs("output.xlsx");  // only now does serialization happen

// Streaming approach: write as we go
using var zip = new ZipArchive(stream, mode);
var sheetEntry = zip.CreateEntry("xl/worksheets/sheet1.xml");
using var sw = new StreamWriter(sheetEntry.Open());
sw.Write("<?xml version=\"1.0\"?>");
sw.Write("<worksheet xmlns=\"...\"><sheetData>");
foreach (var row in data)
{
    // Write row XML directly — no intermediate objects
    sw.Write($"<row><c t=\"inlineStr\"><is><t>{Escape(row.Name)}</t></is></c></row>");
}
sw.Write("</sheetData></worksheet>");

This pattern eliminates the entire in-memory object graph. The tradeoff is that you lose random access (no modifying cell B4 after writing row 100), but for export scenarios, you never need it anyway.

Technique 2: IBufferWriter<byte> for Low-Allocation Paths

.NET has a lesser-known gem: System.Buffers.IBufferWriter<byte>. It lets you write bytes directly into rented pooled buffers, avoiding intermediate byte[] allocations and copies.

Here's how it works in practice:

// Traditional: allocates a byte[]
var ms = new MemoryStream();
Xlsx.Write(ms, data);
byte[] result = ms.ToArray();  // another allocation + copy

// IBufferWriter: writes to pooled buffers
var writer = new ArrayBufferWriter<byte>();
Xlsx.Write(writer, data);
// writer.WrittenSpan is a ReadOnlySpan<byte> over rented memory
// No intermediate allocations

The IBufferWriter<byte> path is the primary low-allocation hot path. It chains through ZipArchive → DeflateStream → XML writer, all using IBufferWriter<byte> under the hood, eliminating allocations at every layer.

This is also the technique that System.Text.Json uses under the hood for its high-performance serialization — the same pattern applies to XML generation.

Technique 3: Inline Strings vs. Shared Strings Table (SST)

OOXML supports two ways to store string values:

Inline strings (t="inlineStr"): The string value lives directly inside the cell element. No shared strings file needed. This is optimal when strings are mostly unique.

Shared Strings Table (t="s" with index): Strings are stored once in xl/sharedStrings.xml, and cells reference them by index. This is optimal when strings are highly repeated (e.g., status values, categories).

Most libraries default to SST (or build it unconditionally), but SST is wasteful when strings are unique — you double the memory: once for the SST, once for the cell references.

My approach uses a heuristic: pre-scan the first 64 rows. If string deduplication ratio is below 70%, stay with inline strings. Only switch to SST when it actually saves space. This is similar to what Excel itself does when saving files.

// Auto-detect based on data characteristics
Xlsx.ToBytes(data, p => p.WithAutoSst(true));

// Benchmarks on 10k rows with 16 unique strings (96% dedup):
// SST path:  2,804 μs, 955 KB  — dedup saves XML space
// Inline:    2,933 μs, 979 KB  — inline avoids SST bookkeeping
// For unique strings, inline is both faster and smaller

Technique 4: Fluent Export Profiles

Rather than a verbose configuration object, a fluent API keeps the code readable and the allocation minimal:

// Fluent profile — no intermediate configuration objects allocated
var bytes = Xlsx.ToBytes(orders, p => p
    .Sheet("Orders")
    .Column(x => x.OrderNo, c => c.WithName("Order #").WithWidth(30))
    .Column(x => x.Amount,  c => c.WithFormat("0.00").WithFontColor("#0000FF"))
    .Ignore(x => x.InternalId)
    .WithFreezeHeader());

// Traditional (ClosedXML) — configuration mixed with data manipulation
using var wb = new XLWorkbook();
var ws = wb.Worksheets.Add("Orders");
ws.Cell(1, 1).Value = "Order #";  // mutating a DOM object
ws.Cell(1, 1).Style.Font.Bold = true;
// ...

The Action<ExportProfile<T>> callback is compiled into a delegate once (via Expression.Compile or source-gen fast paths with [XlsxExportable]), and the configuration is applied during streaming without allocating intermediate objects.

Technique 5: IAsyncEnumerable<T> for True Streaming

For web API scenarios where data comes from an async source (database, gRPC, event stream), IAsyncEnumerable<T> enables true end-to-end streaming:

// Database → stream → HTTP response — no materialized list
app.MapGet("/export/orders", async (HttpContext ctx, AppDbContext db) =>
{
    ctx.Response.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
    await Xlsx.WriteAsync(
        ctx.Response.Body,
        ReadOrdersAsync(db),  // IAsyncEnumerable<Order>
        p => p.Sheet("Orders"));
});

async IAsyncEnumerable<Order> ReadOrdersAsync(AppDbContext db)
{
    await foreach (var order in db.Orders.AsAsyncEnumerable()
        .Where(o => o.Status == OrderStatus.Active))
        yield return order;
}

This means the HTTP response body stream is written concurrently with the database query. At no point is the full dataset materialized in memory. Contrast this with:

// Traditional: load all data, then write
var orders = await db.Orders.Where(...).ToListAsync();  // materialized in memory
ClosedXML...SaveAs(stream);                              // writes from memory

The API is symmetric — reading also uses IAsyncEnumerable:

await foreach (var order in Xlsx.ReadAsync<Order>(stream))
{
    await db.Orders.AddAsync(order);
}

Note: IAsyncEnumerable reading streams rows as they're parsed from the ZIP/XWL, but the actual XML cell parsing is synchronous CPU work. The async benefit is primarily on the I/O side.

Real-World Benchmarks