How to Build an AI Coding Assistant with ASP.NET Core and Local LLMs

Introduction

AI coding assistants have transformed the way developers write, review, and understand code. These tools can generate code snippets, explain complex logic, identify bugs, and even suggest improvements in real time.

While cloud-based AI services are popular, many organizations prefer running AI models locally for reasons such as privacy, security, compliance, and cost control. This is where Local Large Language Models (LLMs) become valuable.

By combining ASP.NET Core with local LLMs, developers can create intelligent coding assistants that run entirely within their own infrastructure without sending source code to external services.

In this article, you'll learn how to build a simple AI coding assistant using ASP.NET Core and a local LLM, understand the architecture involved, and explore best practices for production-ready implementations.

What Is an AI Coding Assistant?

An AI coding assistant is an application that helps developers perform programming-related tasks.

Common capabilities include:

Code generation
Code explanation
Bug detection
Refactoring suggestions
Documentation generation
Unit test creation
API development assistance

For example, a developer might ask:

Create a C# method to calculate factorial.

The assistant can respond with:

public static long Factorial(int number)
{
    if (number <= 1)
        return 1;

    return number * Factorial(number - 1);
}

Instead of manually searching documentation, developers can interact directly with the assistant.

Why Use Local LLMs?

Local LLMs provide several advantages over cloud-hosted models.

Data Privacy

Source code never leaves the organization's environment.

Reduced Dependency

Applications continue working even when external AI services are unavailable.

Cost Control

No per-request API charges.

Customization

Models can be fine-tuned for company-specific coding standards and workflows.

Compliance

Helps organizations meet regulatory and security requirements.

Popular local LLM options include:

Llama
Mistral
DeepSeek
Qwen
Phi

These models can be served locally using tools such as Ollama.

Solution Architecture

A simple AI coding assistant architecture might look like this:

Developer
    |
    v
ASP.NET Core Web API
    |
    v
Local LLM Server
    |
    v
Generated Response

Workflow:

Developer submits a coding question.
ASP.NET Core receives the request.
The application sends the prompt to the local model.
The model generates a response.
ASP.NET Core returns the result to the user.

Setting Up the ASP.NET Core Project

Create a new Web API project.

dotnet new webapi -n CodingAssistant
cd CodingAssistant

Run the project.

dotnet run

The application will expose REST endpoints that communicate with the local LLM.

Creating the Request Model

Create a request model.

public class PromptRequest
{
    public string Prompt { get; set; } = string.Empty;
}

This model will store the developer's question.

Example:

{
  "prompt": "Explain dependency injection in ASP.NET Core"
}

Creating the Response Model

public class PromptResponse
{
    public string Response { get; set; } = string.Empty;
}

This object returns the generated answer.

Building the AI Service

Create a service responsible for communicating with the local model.

public class AiAssistantService
{
    private readonly HttpClient _httpClient;

    public AiAssistantService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<string> GenerateResponseAsync(string prompt)
    {
        var payload = new
        {
            model = "llama3",
            prompt = prompt,
            stream = false
        };

        var response =
            await _httpClient.PostAsJsonAsync(
                "/api/generate",
                payload);

        response.EnsureSuccessStatusCode();

        return await response.Content.ReadAsStringAsync();
    }
}

This service sends prompts to the local LLM server.

Registering the Service

In Program.cs:

builder.Services.AddHttpClient<AiAssistantService>(
    client =>
    {
        client.BaseAddress =
            new Uri("http://localhost:11434");
    });

This configuration points to the local model endpoint.

Creating the API Controller

Now create a controller for handling requests.

[ApiController]
[Route("api/assistant")]
public class AssistantController : ControllerBase
{
    private readonly AiAssistantService _service;

    public AssistantController(
        AiAssistantService service)
    {
        _service = service;
    }

    [HttpPost]
    public async Task<IActionResult> Ask(
        PromptRequest request)
    {
        var response =
            await _service.GenerateResponseAsync(
                request.Prompt);

        return Ok(response);
    }
}

The endpoint accepts prompts and returns AI-generated answers.

Testing the Assistant

Send a request:

POST /api/assistant

Request Body:

{
  "prompt": "Generate a C# method to reverse a string"
}

Example Response:

public static string Reverse(string value)
{
    return new string(
        value.Reverse().ToArray());
}

The coding assistant can now answer programming questions.

Enhancing the Assistant

A basic implementation is useful, but real-world assistants require additional capabilities.

Code Explanation

Developers can submit existing code.

Example:

Explain this LINQ query.

The assistant analyzes and describes the logic.

Bug Detection

Example:

Find issues in this ASP.NET Core service.

The model identifies potential problems.

Documentation Generation

Example:

Generate XML documentation comments.

The assistant automatically creates documentation.

Unit Test Generation

Example:

Create xUnit tests for this service.

The model generates test cases.

Adding Conversation Context

Maintaining conversation history improves response quality.

Without context:

User:
Explain dependency injection.

Follow-up:

User:
Give me an example.

The model may not understand what "it" refers to.

With context:

User:
Explain dependency injection.

Assistant:
Explanation...

User:
Give me an example.

The model can generate more accurate responses.

Conversation history can be stored:

In memory
Redis
SQL Server
PostgreSQL

Security Considerations

When working with source code, security is critical.

Validate User Input

Always sanitize incoming requests.

if(string.IsNullOrWhiteSpace(request.Prompt))
{
    return BadRequest();
}

Limit Prompt Size

Prevent abuse by restricting input length.

if(request.Prompt.Length > 5000)
{
    return BadRequest("Prompt too large.");
}

Authenticate Users

Protect endpoints using:

JWT Authentication
OAuth
OpenID Connect

Protect Sensitive Code

Avoid exposing proprietary information to unauthorized users.

Best Practices

Use Focused System Prompts

Define the assistant's role clearly.

Example:

You are an expert ASP.NET Core developer.
Provide accurate and secure coding guidance.

Log Requests

Track:

Prompt volume
Errors
Response times
Usage trends

Cache Common Responses

Reduce model workload for frequently asked questions.

Implement Rate Limiting

Protect the application from excessive usage.

ASP.NET Core provides built-in support for rate limiting.

Monitor Performance

Measure:

Response latency
Token usage
Memory consumption
CPU utilization

These metrics help maintain a responsive user experience.

Real-World Use Cases

Organizations use AI coding assistants for:

Developer Productivity

Accelerate development workflows.

Code Reviews

Identify bugs and code quality issues.

Onboarding

Help new developers understand existing codebases.

Documentation

Generate and maintain technical documentation.

Internal Knowledge Systems

Answer organization-specific development questions.

Conclusion

Building an AI coding assistant with ASP.NET Core and local LLMs allows organizations to take advantage of AI-powered development workflows while maintaining control over their data and infrastructure. By hosting models locally, teams can improve privacy, reduce dependency on external services, and customize the assistant to meet their specific requirements.

Using ASP.NET Core as the foundation makes it easy to expose APIs, manage authentication, implement monitoring, and integrate with enterprise systems. As your solution evolves, you can extend it with conversation memory, code analysis tools, vector databases, and agent-based workflows to create a more capable and intelligent development assistant.

Whether you're creating an internal productivity tool or a full-featured coding platform, ASP.NET Core and local LLMs provide a powerful combination for building secure and scalable AI-driven developer experiences.