Introduction
AI coding assistants have transformed the way developers write, review, and understand code. These tools can generate code snippets, explain complex logic, identify bugs, and even suggest improvements in real time.
While cloud-based AI services are popular, many organizations prefer running AI models locally for reasons such as privacy, security, compliance, and cost control. This is where Local Large Language Models (LLMs) become valuable.
By combining ASP.NET Core with local LLMs, developers can create intelligent coding assistants that run entirely within their own infrastructure without sending source code to external services.
In this article, you'll learn how to build a simple AI coding assistant using ASP.NET Core and a local LLM, understand the architecture involved, and explore best practices for production-ready implementations.
What Is an AI Coding Assistant?
An AI coding assistant is an application that helps developers perform programming-related tasks.
Common capabilities include:
For example, a developer might ask:
Create a C# method to calculate factorial.
The assistant can respond with:
public static long Factorial(int number)
{
if (number <= 1)
return 1;
return number * Factorial(number - 1);
}
Instead of manually searching documentation, developers can interact directly with the assistant.
Why Use Local LLMs?
Local LLMs provide several advantages over cloud-hosted models.
Data Privacy
Source code never leaves the organization's environment.
Reduced Dependency
Applications continue working even when external AI services are unavailable.
Cost Control
No per-request API charges.
Customization
Models can be fine-tuned for company-specific coding standards and workflows.
Compliance
Helps organizations meet regulatory and security requirements.
Popular local LLM options include:
Llama
Mistral
DeepSeek
Qwen
Phi
These models can be served locally using tools such as Ollama.
Solution Architecture
A simple AI coding assistant architecture might look like this:
Developer
|
v
ASP.NET Core Web API
|
v
Local LLM Server
|
v
Generated Response
Workflow:
Developer submits a coding question.
ASP.NET Core receives the request.
The application sends the prompt to the local model.
The model generates a response.
ASP.NET Core returns the result to the user.
Setting Up the ASP.NET Core Project
Create a new Web API project.
dotnet new webapi -n CodingAssistant
cd CodingAssistant
Run the project.
dotnet run
The application will expose REST endpoints that communicate with the local LLM.
Creating the Request Model
Create a request model.
public class PromptRequest
{
public string Prompt { get; set; } = string.Empty;
}
This model will store the developer's question.
Example:
{
"prompt": "Explain dependency injection in ASP.NET Core"
}
Creating the Response Model
public class PromptResponse
{
public string Response { get; set; } = string.Empty;
}
This object returns the generated answer.
Building the AI Service
Create a service responsible for communicating with the local model.
public class AiAssistantService
{
private readonly HttpClient _httpClient;
public AiAssistantService(HttpClient httpClient)
{
_httpClient = httpClient;
}
public async Task<string> GenerateResponseAsync(string prompt)
{
var payload = new
{
model = "llama3",
prompt = prompt,
stream = false
};
var response =
await _httpClient.PostAsJsonAsync(
"/api/generate",
payload);
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
This service sends prompts to the local LLM server.
Registering the Service
In Program.cs:
builder.Services.AddHttpClient<AiAssistantService>(
client =>
{
client.BaseAddress =
new Uri("http://localhost:11434");
});
This configuration points to the local model endpoint.
Creating the API Controller
Now create a controller for handling requests.
[ApiController]
[Route("api/assistant")]
public class AssistantController : ControllerBase
{
private readonly AiAssistantService _service;
public AssistantController(
AiAssistantService service)
{
_service = service;
}
[HttpPost]
public async Task<IActionResult> Ask(
PromptRequest request)
{
var response =
await _service.GenerateResponseAsync(
request.Prompt);
return Ok(response);
}
}
The endpoint accepts prompts and returns AI-generated answers.
Testing the Assistant
Send a request:
POST /api/assistant
Request Body:
{
"prompt": "Generate a C# method to reverse a string"
}
Example Response:
public static string Reverse(string value)
{
return new string(
value.Reverse().ToArray());
}
The coding assistant can now answer programming questions.
Enhancing the Assistant
A basic implementation is useful, but real-world assistants require additional capabilities.
Code Explanation
Developers can submit existing code.
Example:
Explain this LINQ query.
The assistant analyzes and describes the logic.
Bug Detection
Example:
Find issues in this ASP.NET Core service.
The model identifies potential problems.
Documentation Generation
Example:
Generate XML documentation comments.
The assistant automatically creates documentation.
Unit Test Generation
Example:
Create xUnit tests for this service.
The model generates test cases.
Adding Conversation Context
Maintaining conversation history improves response quality.
Without context:
User:
Explain dependency injection.
Follow-up:
User:
Give me an example.
The model may not understand what "it" refers to.
With context:
User:
Explain dependency injection.
Assistant:
Explanation...
User:
Give me an example.
The model can generate more accurate responses.
Conversation history can be stored:
In memory
Redis
SQL Server
PostgreSQL
Security Considerations
When working with source code, security is critical.
Validate User Input
Always sanitize incoming requests.
if(string.IsNullOrWhiteSpace(request.Prompt))
{
return BadRequest();
}
Limit Prompt Size
Prevent abuse by restricting input length.
if(request.Prompt.Length > 5000)
{
return BadRequest("Prompt too large.");
}
Authenticate Users
Protect endpoints using:
JWT Authentication
OAuth
OpenID Connect
Protect Sensitive Code
Avoid exposing proprietary information to unauthorized users.
Best Practices
Use Focused System Prompts
Define the assistant's role clearly.
Example:
You are an expert ASP.NET Core developer.
Provide accurate and secure coding guidance.
Log Requests
Track:
Prompt volume
Errors
Response times
Usage trends
Cache Common Responses
Reduce model workload for frequently asked questions.
Implement Rate Limiting
Protect the application from excessive usage.
ASP.NET Core provides built-in support for rate limiting.
Monitor Performance
Measure:
Response latency
Token usage
Memory consumption
CPU utilization
These metrics help maintain a responsive user experience.
Real-World Use Cases
Organizations use AI coding assistants for:
Developer Productivity
Accelerate development workflows.
Code Reviews
Identify bugs and code quality issues.
Onboarding
Help new developers understand existing codebases.
Documentation
Generate and maintain technical documentation.
Internal Knowledge Systems
Answer organization-specific development questions.
Conclusion
Building an AI coding assistant with ASP.NET Core and local LLMs allows organizations to take advantage of AI-powered development workflows while maintaining control over their data and infrastructure. By hosting models locally, teams can improve privacy, reduce dependency on external services, and customize the assistant to meet their specific requirements.
Using ASP.NET Core as the foundation makes it easy to expose APIs, manage authentication, implement monitoring, and integrate with enterprise systems. As your solution evolves, you can extend it with conversation memory, code analysis tools, vector databases, and agent-based workflows to create a more capable and intelligent development assistant.
Whether you're creating an internal productivity tool or a full-featured coding platform, ASP.NET Core and local LLMs provide a powerful combination for building secure and scalable AI-driven developer experiences.