Learn generative AI in .NET with proven enterprise patterns — RAG, Semantic Kernel, streaming, and resilience in C#. Start building smarter apps today.
Generative AI in .NET has moved from experiment to expectation. In 2026, enterprise teams across the USA, UK, Canada, Australia, and India are being asked to ship AI copilots, document summarizers, and intelligent search into existing C# applications — and to do it with the same reliability standards as any other production service. The problem? Most tutorials show you how to call an LLM API in ten lines, then leave you alone with the hard parts: architecture, grounding, resilience, cost control, and testing.
This guide covers the architecture patterns that actually matter when you integrate generative AI into .NET enterprise apps: the abstraction layer, Retrieval-Augmented Generation (RAG), streaming, tool calling, and the operational guardrails that keep an AI feature from becoming a production incident. Every example is runnable C# targeting .NET 8/9 with Microsoft.Extensions.AI and Semantic Kernel.
Why Generative AI in .NET Needs Real Architecture
Calling an LLM is easy. Building an enterprise feature on top of one is not, because LLMs break three assumptions your architecture probably relies on:
- Non-determinism: the same input can produce different outputs, which breaks naive caching and traditional unit testing.
- Latency and cost per call: a single completion can take seconds and cost real money — you cannot treat it like a database query.
- Untrusted output: model responses are generated text, not validated data. They can be wrong, malformed, or manipulated via prompt injection.
The patterns below exist to contain each of these risks. If you take one thing from this article, take this: treat the LLM as an unreliable external dependency, like a flaky third-party API — because architecturally, that is exactly what it is.
Pattern 1: Abstract the Model Behind Microsoft.Extensions.AI
The single most common mistake in .NET AI codebases is scattering vendor SDK calls (OpenAI, Azure OpenAI, Anthropic, Ollama) throughout business logic. Model choice will change — pricing shifts, better models ship quarterly, and enterprise procurement may force a provider swap. Microsoft's Microsoft.Extensions.AI package gives you a standard IChatClient abstraction, the AI equivalent of ILogger.
using Microsoft.Extensions.AI;
// Program.cs — register the client once, swap providers in one place
builder.Services.AddChatClient(services =>
new AzureOpenAIClient(
new Uri(builder.Configuration["AI:Endpoint"]!),
new DefaultAzureCredential())
.GetChatClient("gpt-4o")
.AsIChatClient())
.UseDistributedCache() // response caching
.UseLogging() // telemetry
.UseFunctionInvocation(); // tool calling support
// Business code depends only on the abstraction
public class SummaryService(IChatClient chat)
{
public async Task<string> SummarizeAsync(string document, CancellationToken ct)
{
var response = await chat.GetResponseAsync(
[
new ChatMessage(ChatRole.System,
"You summarize legal documents in plain English, max 150 words."),
new ChatMessage(ChatRole.User, document)
], cancellationToken: ct);
return response.Text;
}
}
Why this matters: the decorator-style pipeline (UseDistributedCache, UseLogging) means caching, telemetry, and rate limiting are cross-cutting middleware, not code you rewrite per feature. When you switch from Azure OpenAI to a local Ollama model for dev environments, only the registration changes — a huge win for testability and CI.
Pattern 2: RAG — Ground the Model in Your Own Data
Retrieval-Augmented Generation is the highest-value generative AI pattern for enterprise .NET apps, and the answer to the most common stakeholder question: "Can it answer questions about our data?" Instead of fine-tuning a model (expensive, slow, stale), RAG retrieves relevant chunks of your documents at query time and injects them into the prompt.
The RAG pipeline in C#
public class RagService(
IChatClient chat,
IEmbeddingGenerator<string, Embedding<float>> embedder,
IVectorStore vectorStore)
{
public async Task<string> AskAsync(string question, CancellationToken ct)
{
// 1. Embed the user's question
var queryVector = await embedder.GenerateVectorAsync(question, cancellationToken: ct);
// 2. Retrieve the top-matching chunks from the vector store
var collection = vectorStore.GetCollection<Guid, DocChunk>("kb-chunks");
var results = collection.SearchAsync(queryVector, top: 5, cancellationToken: ct);
var context = new StringBuilder();
await foreach (var match in results)
context.AppendLine($"[Source: {match.Record.SourceFile}]\n{match.Record.Text}\n");
// 3. Ask the model, grounded in retrieved context only
var response = await chat.GetResponseAsync(
[
new ChatMessage(ChatRole.System,
"""
Answer using ONLY the provided context. If the context does not
contain the answer, say "I don't have that information."
Cite the source file for every claim.
"""),
new ChatMessage(ChatRole.User, $"Context:\n{context}\n\nQuestion: {question}")
], cancellationToken: ct);
return response.Text;
}
}
Why RAG beats fine-tuning for most teams: your data changes daily; re-indexing a vector store takes minutes, while fine-tuning takes days and freezes knowledge at training time. RAG also gives you citations — essential for enterprise trust — and lets you enforce document-level security by filtering retrieval results to what the current user is authorized to see. That last point is critical and frequently missed: apply your permission model at retrieval time, or your chatbot will happily leak the CEO's compensation memo to anyone who asks nicely.
Chunking: the unglamorous part that decides quality
Poor answers usually trace back to poor chunking, not a poor model. Practical defaults: chunk by semantic boundaries (headings, paragraphs) rather than fixed character counts, target 200–500 tokens per chunk, and overlap chunks by 10–15% so answers spanning a boundary aren't lost.
Pattern 3: Tool Calling with Semantic Kernel
RAG makes the model knowledgeable; tool calling makes it useful. Instead of only generating text, the model can invoke your C# methods — check an order status, create a ticket, query a database — with the model deciding when a tool is needed. Semantic Kernel makes this declarative:
using Microsoft.SemanticKernel;
using System.ComponentModel;
public class OrderPlugin(IOrderRepository orders)
{
[KernelFunction, Description("Gets the current status of a customer order")]
public async Task<string> GetOrderStatus(
[Description("The order number, e.g. ORD-12345")] string orderNumber)
{
var order = await orders.FindAsync(orderNumber);
return order is null
? $"No order found with number {orderNumber}."
: $"Order {orderNumber}: {order.Status}, expected delivery {order.Eta:d}.";
}
}
// Wiring it up
var kernel = Kernel.CreateBuilder()
.AddAzureOpenAIChatCompletion("gpt-4o", endpoint, credential)
.Build();
kernel.Plugins.AddFromType<OrderPlugin>(serviceProvider: services);
var settings = new PromptExecutionSettings
{
FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
};
var answer = await kernel.InvokePromptAsync(
"Where is my order ORD-12345?", new(settings));
The critical best practice: tools are an attack surface. Treat every tool parameter as untrusted user input — because via prompt injection, it can be. Validate inside the function, enforce authorization with the current user's identity (never a service account with broad rights), and keep write-actions (refunds, deletions) behind explicit human confirmation. A good rule for enterprise apps: the model can read freely within the user's permissions, but state-changing tools require a confirm step in the UI.
Pattern 4: Stream Responses for Perceived Performance
A 6-second wait for a complete answer feels broken; the same answer streaming token-by-token feels instant. Streaming is a UX pattern, but in enterprise apps it is also a resilience pattern — users can cancel early, saving tokens and cost. With ASP.NET Core:
app.MapPost("/api/chat", async (ChatRequest req, IChatClient chat, HttpContext http) =>
{
http.Response.ContentType = "text/event-stream";
await foreach (var update in chat.GetStreamingResponseAsync(
req.Messages, cancellationToken: http.RequestAborted))
{
await http.Response.WriteAsync($"data: {JsonSerializer.Serialize(update.Text)}\n\n");
await http.Response.Body.FlushAsync();
}
});
Note the use of http.RequestAborted: when the user closes the tab, generation stops and you stop paying for tokens nobody will read. Propagating cancellation tokens through your AI call chain is one of the cheapest cost optimizations available.
Best Practices for Generative AI in .NET Production
1. Add resilience — LLM APIs fail more than you think
Rate limits (HTTP 429) and transient timeouts are normal operating conditions for LLM endpoints. Use Microsoft.Extensions.Resilience (Polly) with exponential backoff, and set aggressive timeouts with a graceful degradation path — "AI summary unavailable" beats a spinner that never resolves.
2. Control cost with caching and model tiering
Not every request needs your most capable model. Route simple classification or extraction to a small, cheap model and reserve the flagship model for complex reasoning — a tiering strategy that routinely cuts spend 60–80%. Cache aggressively: identical prompts (FAQ-style questions) should never hit the API twice, which the UseDistributedCache middleware from Pattern 1 handles for free.
3. Evaluate, don't just unit test
You cannot assert Assert.Equal(expected, actual) on generated text. Instead, build an evaluation set — 50–200 real questions with graded reference answers — and score outputs on relevance, groundedness, and safety with the Microsoft.Extensions.AI.Evaluation libraries. Run it in CI whenever you change a prompt, a model version, or chunking parameters. Prompts are code; regressions in them should fail the build like any other regression.
4. Log everything (within privacy limits)
Capture prompt, retrieved context, model, token counts, latency, and user feedback for every AI interaction (with PII redaction per your compliance requirements — GDPR in the UK/EU, CCPA in the USA). When a user reports "the AI said something wrong," this trace is the only way to diagnose whether the failure was retrieval, prompting, or the model itself.
Common Pitfalls to Avoid
- Sending the entire conversation history forever. Context windows are finite and tokens cost money. Summarize or truncate history beyond the last N turns.
- Trusting model output as structured data without validation. When you need JSON, use structured output features and still validate with
JsonSerializer+ schema checks before acting on it. - Skipping the "I don't know" instruction. Without an explicit instruction to admit uncertainty (see the RAG system prompt above), models fill gaps with confident fabrication.
- Ignoring prompt injection. Any user-supplied or retrieved text can contain adversarial instructions. Keep system prompts separate from user content, and never give the model more authority than the user driving it.
- Blocking async calls. LLM calls are long-running I/O.
.Resultor.Wait()on them will exhaust your thread pool under load faster than almost any other mistake.
Conclusion: Generative AI in .NET Is an Architecture Problem
The teams succeeding with generative AI in .NET are not the ones with the cleverest prompts — they are the ones who treated the LLM as an unreliable, expensive, untrusted dependency and architected accordingly. Key takeaways:
- Abstract the model behind
IChatClientso providers, caching, and telemetry are swappable middleware. - Use RAG to ground answers in your data, with permission-aware retrieval and citations.
- Tool calling turns chat into action — but validate every parameter and gate writes behind human confirmation.
- Stream responses and propagate cancellation to improve UX and cut token cost.
- Evaluate prompts in CI like the production code they are.
Start small: pick one workflow where users currently read documents to answer questions, ship a RAG-powered assistant behind a feature flag, measure it with a real evaluation set, and expand from there. The patterns in this guide will scale with you from that first prototype to a fleet of AI features — without the 2 a.m. incidents.
Your go-to resource for C#, .NET, and modern software development. Follow along for daily tutorials, tips, and real-world examples.
Comments
Post a Comment