Benchmarking Local Speech-to-Text with Foundry Local and C#

Introduction

Speech-to-text sounds simple until you try to use it in a real application.

A user speaks into a microphone, the application captures the audio, a model processes it, and text comes back. But the experience can change significantly depending on where that processing happens.

If speech is sent to a remote service, network latency becomes part of the experience. If the model runs locally, the application can avoid that network round trip, but now CPU, GPU, memory, model size, and audio processing all matter.

This makes local speech-to-text an interesting option for .NET applications that need lower latency, better privacy, or offline capabilities.

In this article, we will look at how to build a simple local speech-to-text workflow with C#, what to measure when benchmarking it, and which metrics actually matter when deciding whether local inference makes sense for an application.

What Is Local Speech-to-Text?

Traditional cloud-based speech recognition looks roughly like this:

Microphone
    ↓
Application
    ↓
Network
    ↓
Speech-to-Text Service
    ↓
Network
    ↓
Text

There are several components involved in the total response time.

With local inference, the architecture changes:

Microphone
    ↓
C# Application
    ↓
Local Speech Model
    ↓
Text

The network is removed from the inference path.

That can be useful for applications such as:

Desktop assistants
Meeting transcription
Voice-enabled developer tools
Offline applications
Private enterprise applications
Local accessibility tools
Internal applications processing sensitive conversations

However, removing the network does not automatically make the application faster.

The model still needs to process the audio, and the hardware running the model becomes an important part of the equation.

What Should You Benchmark?

A speech-to-text benchmark should measure more than just whether the final text is correct.

At minimum, measure these areas:

Metric	What it tells you
Startup time	How quickly the model becomes usable
First-token latency	How quickly transcription begins
Total transcription time	Time required to process the complete audio
Real-time factor	Whether processing keeps up with speech
CPU usage	CPU cost of inference
Memory usage	RAM consumed by the application and model
GPU usage	Hardware acceleration utilization
Accuracy	Quality of the generated transcript
Model load time	Cost of initializing the model

One particularly useful metric is the real-time factor (RTF).

If a 60-second audio file takes 30 seconds to process:

RTF = 30 / 60
    = 0.5

An RTF below 1 means the system processes audio faster than real time.

If the same audio takes 90 seconds:

RTF = 90 / 60
    = 1.5

The system cannot keep up with real-time speech.

Why Hardware Matters

The same speech model can behave very differently on different machines.

For example:

Machine A
CPU only
8 GB RAM

Machine B
Modern CPU
16 GB RAM

Machine C
Dedicated GPU
32 GB RAM

The model is the same, but inference performance may be very different.

This is why reporting only one benchmark number is not particularly useful.

A better benchmark report looks like:

Model:       Local speech model
Audio:       5 minutes
Hardware:    CPU
Processing:  82 seconds
RTF:         0.27
Memory:      2.8 GB

Then repeat the test using hardware acceleration.

Preparing the C# Application

A simple .NET application can be used as the benchmark harness.

The basic responsibilities are:

Load Model
   ↓
Load Audio
   ↓
Start Timer
   ↓
Run Transcription
   ↓
Stop Timer
   ↓
Record Metrics

Keep the benchmark application separate from the production application.

That makes it easier to run the same workload repeatedly.

For example:

public sealed record TranscriptionBenchmarkResult(
    TimeSpan AudioDuration,
    TimeSpan ProcessingTime,
    double RealTimeFactor,
    long AllocatedBytes);

The result object gives us a consistent way to collect benchmark information.

Measuring Processing Time

Use Stopwatch rather than relying on wall-clock timestamps.

var stopwatch = Stopwatch.StartNew();

var transcript = await transcriber.TranscribeAsync(audio);

stopwatch.Stop();

Console.WriteLine(
    $"Processing time: {stopwatch.Elapsed}");

For benchmarking, run the same test multiple times.

A single run can be affected by:

Background applications
Model loading
Disk caching
CPU scheduling
Thermal throttling
Memory pressure

A more useful benchmark might run the same audio 5–10 times.

Separate Model Loading From Inference

This is an important detail.

Suppose the first run takes 12 seconds:

Model loading:    8 seconds
Inference:        4 seconds
Total:           12 seconds

The next run might take:

Model loading:    0 seconds
Inference:        4 seconds
Total:             4 seconds

If you report only the first result, you may conclude that transcription takes 12 seconds.

That would not accurately represent steady-state inference.

Measure both:

Cold Start
    ↓
Model Loading
    ↓
Inference

Warm Run
    ↓
Inference Only

Both numbers are useful.

Cold-start performance matters for short-lived applications.

Warm performance matters for applications that keep the model loaded.

Measuring Real-Time Factor

RTF is one of the easiest ways to compare speech-to-text performance.

double rtf =
    processingTime.TotalSeconds /
    audioDuration.TotalSeconds;

For example:

Audio duration:     300 seconds
Processing time:     75 seconds

RTF = 75 / 300
    = 0.25

That means five minutes of audio was processed in 75 seconds.

For an interactive application, this is much more useful than simply saying "the model is fast."

Benchmark Different Audio Durations

Do not benchmark only a short audio file.

A useful test set might contain:

30 seconds
1 minute
5 minutes
15 minutes
30 minutes

This helps reveal whether performance remains consistent as the audio gets longer.

For example:

Audio	Processing	RTF
30 sec	8 sec	0.27
1 min	16 sec	0.27
5 min	81 sec	0.27
15 min	260 sec	0.29

The numbers don't need to scale perfectly.

The important thing is to identify whether performance degrades significantly with longer workloads.

Measuring Memory

Local models can consume substantial memory.

In .NET, you can capture basic process memory information:

using System.Diagnostics;

var process = Process.GetCurrentProcess();

Console.WriteLine(
    $"Memory: {process.WorkingSet64 / 1024 / 1024} MB");

Take measurements at multiple points:

Application startup
      ↓
Before model loading
      ↓
After model loading
      ↓
During transcription
      ↓
After transcription

This helps identify whether the model itself is responsible for the memory increase or whether the application is allocating additional buffers during transcription.

CPU and GPU Usage

CPU utilization is especially important when running inference locally.

A benchmark should record whether the machine is:

CPU-bound
GPU-bound
Memory-bound

A CPU-only configuration may be sufficient for background transcription.

An interactive voice application may require hardware acceleration to achieve the desired response time.

The benchmark should therefore compare the execution modes supported by the local runtime.

For example:

CPU
GPU
NPU

Not every machine will support every option, so the benchmark should report the actual execution provider used.

Accuracy Matters Too

A fast transcription engine is not useful if the transcript quality is poor.

Consider:

"Deploy the application to staging."

being transcribed as:

"Deploy the application to aging."

For general conversation, that may be acceptable.

For technical applications, it could cause problems.

Technical vocabulary is particularly important when benchmarking developer-focused speech recognition.

Include test recordings containing:

.NET
ASP.NET Core
Kubernetes
PostgreSQL
OpenTelemetry
API gateway
microservices

Then compare the resulting transcript against a known reference transcript.

Word Error Rate

One common metric for speech recognition is Word Error Rate (WER).

The basic formula is:

WER = (Substitutions + Deletions + Insertions)
      / Number of Reference Words

For example, suppose the reference is:

Deploy the application to staging

and the model produces:

Deploy the application to testing

There is one substitution.

With five reference words:

WER = 1 / 5
    = 20%

WER is useful for comparing models or configurations, but it should not be treated as the only measure of quality.

For some applications, preserving technical terms may matter more than general word-level accuracy.

Test Different Speaking Conditions

A realistic benchmark should include more than a clean studio recording.

Test scenarios such as:

Quiet Environment

Single speaker
Minimal background noise
Normal speaking speed

Noisy Environment

Office noise
Keyboard sounds
Background conversation

Fast Speech

Rapid technical explanation

Multiple Speakers

Two or more people
Changing speakers

Technical Vocabulary

C# and .NET terminology
Product names
Acronyms
Code-related words

This produces a much more realistic view of the model.

A Simple Benchmark Harness

The benchmark can be structured around a reusable method:

public async Task<TranscriptionBenchmarkResult> RunAsync(
    string audioFile)
{
    var audioDuration = GetAudioDuration(audioFile);

    var stopwatch = Stopwatch.StartNew();

    var transcript =
        await transcriber.TranscribeAsync(audioFile);

    stopwatch.Stop();

    var rtf =
        stopwatch.Elapsed.TotalSeconds /
        audioDuration.TotalSeconds;

    return new TranscriptionBenchmarkResult(
        audioDuration,
        stopwatch.Elapsed,
        rtf,
        GC.GetTotalAllocatedBytes());
}

The exact transcription API depends on the local speech runtime being used.

The important part is the benchmark structure, not the API call itself.

Warm-Up Runs Matter

Before collecting measurements, perform one or more warm-up runs.

For example:

Run 0 → Warm-up
Run 1 → Measurement
Run 2 → Measurement
Run 3 → Measurement
Run 4 → Measurement
Run 5 → Measurement

Then calculate:

Minimum
Average
Median
Maximum

The median is particularly useful when background system activity creates occasional outliers.

Don't Benchmark on a Busy Machine

Local inference uses the same hardware as everything else running on the computer.

If the benchmark machine is simultaneously running:

Visual Studio
Browser
Docker
Database
Teams
Background builds

the results may not be representative.

For repeatable tests:

Close unnecessary applications.
Use the same machine for comparisons.
Use the same model.
Use the same audio files.
Keep audio format consistent.
Run multiple iterations.
Record the hardware configuration.

Compare Local and Remote Processing Carefully

It can be tempting to compare:

Local processing = 4 seconds
Cloud processing = 6 seconds

and conclude that local inference wins.

That comparison is incomplete.

The remote measurement may include:

Network upload
Queueing
Server processing
Network download

The local measurement includes:

Model loading
Local preprocessing
Inference

A better comparison measures the complete user experience.

For example:

User speaks
   ↓
Audio captured
   ↓
Processing
   ↓
Text available

Measure the time from the beginning of the user interaction until usable text is available.

Common Benchmarking Mistakes

Measuring Only One Run

One run is not a benchmark.

Including Model Loading Without Reporting It

Cold-start and warm inference are different scenarios.

Measuring Only Processing Speed

Accuracy matters just as much.

Ignoring Hardware

Local AI performance is strongly influenced by the machine.

Using Only Clean Audio

Real users do not always speak in perfect recording environments.

Testing Only Short Audio

Long-running workloads can expose memory and performance problems.

Reporting Only Average Latency

Averages can hide outliers. Include median and maximum values where possible.

Best Practices

Separate cold-start and warm-run measurements.
Use the same audio files for every comparison.
Run multiple iterations.
Report median, average, minimum, and maximum where useful.
Measure real-time factor.
Record CPU and memory usage.
Compare supported hardware acceleration modes.
Include noisy and technical speech in the test set.
Measure transcription accuracy alongside performance.
Record the exact model and runtime configuration.
Test both short and long audio.
Keep benchmark machines consistent.
Measure end-to-end latency for interactive applications.
Treat benchmark results as workload-specific rather than universal performance claims.

FAQs

Is local speech-to-text always faster than cloud speech recognition?

No. Local inference removes network overhead, but model size and hardware can make local processing slower.

What is a good real-time factor?

An RTF below 1 means the system processes audio faster than real time. The acceptable value depends on the application. Interactive transcription generally needs much lower latency than offline batch processing.

Should I benchmark CPU and GPU separately?

Yes, when both execution modes are available. The comparison can show whether hardware acceleration provides a meaningful improvement for your workload.

Is accuracy more important than latency?

It depends on the application. A live assistant may prioritize latency, while legal or medical transcription may place much greater emphasis on accuracy.

Should model loading time be included?

Report it separately. Cold-start time matters for short-lived applications, while warm inference time matters for applications that keep the model loaded.

Conclusion

Local speech-to-text can be a practical choice for .NET applications that need low network dependency, better control over data, or offline processing. But the decision should come from measurements rather than assumptions. By benchmarking cold-start time, warm inference, real-time factor, memory, hardware utilization, and transcription accuracy using the same workloads, developers can get a realistic picture of how a local speech model will behave on their target machines. The most useful benchmark is not the one with the lowest single latency number; it is the one that reflects the actual workload and environment in which the application will run.
:::