Generative AI in .NET: Enterprise Architecture Patterns

Learn generative AI in .NET with proven enterprise patterns: RAG, function calling, IChatClient, resilience and cost control. Start building with C# today.

Generative AI in .NET has moved from experiment to production requirement. In 2026, most enterprise teams building on C# are being asked to add chat assistants, document summarization, or intelligent search to existing line-of-business apps. The hard part is not calling a model. The hard part is doing it in a way that survives a compliance review, a traffic spike, and a vendor price change. This guide covers the architecture patterns that work for generative AI in .NET enterprise applications, with runnable C# examples and the pitfalls that catch most teams on their first project.

Why Generative AI in .NET Needs Real Architecture

A prototype that calls an LLM from a controller action works on a laptop. It fails in production for predictable reasons:

• Vendor lock-in. Model providers change pricing, deprecate models, and release better ones every few months. Hard-coding one SDK into your services means every change ripples through your codebase.
• Non-determinism. The same prompt can produce different output. Traditional unit tests and validation strategies do not apply cleanly.
• Latency and cost. A single model call can take seconds and cost real money. Unbounded retries or missing caching can turn a feature into a budget problem.
• Data governance. Enterprise apps handle PII, contracts, and financial records. Sending that data to a model without controls is a compliance incident waiting to happen.

Each pattern below addresses one or more of these concerns. Together they form a layered architecture: an abstraction layer, a knowledge layer (RAG), an action layer (function calling), and a cross-cutting layer for resilience, observability, and safety.

Pattern 1: Abstract the Model with IChatClient

Microsoft ships Microsoft.Extensions.AI, a set of provider-neutral abstractions for .NET. The core interface is IChatClient. Your application code depends on the interface, and the concrete provider (Azure OpenAI, Anthropic, Ollama for local models, or others) is wired up in dependency injection. This is the same pattern you already use with ILogger or HttpClient.

// Program.cs
using Microsoft.Extensions.AI;

var builder = WebApplication.CreateBuilder(args);

// Provider-specific client is created once, here, and nowhere else.
IChatClient providerClient = CreateProviderClient(builder.Configuration);

builder.Services.AddChatClient(providerClient)
    .UseLogging()            // structured logs for every call
    .UseOpenTelemetry()      // traces and metrics
    .UseFunctionInvocation(); // automatic tool calling (Pattern 3)

builder.Services.AddScoped<ContractSummaryService>();

var app = builder.Build();
app.MapPost("/summaries", async (ContractSummaryService svc, string text) =>
    Results.Ok(await svc.SummarizeAsync(text)));
app.Run();

The service itself knows nothing about which vendor is behind the interface:

using Microsoft.Extensions.AI;

public sealed class ContractSummaryService(IChatClient chat)
{
    private const string SystemPrompt =
        "You are a legal operations assistant. Summarize contracts in plain English. " +
        "Return exactly three bullet points: parties, term, and termination conditions. " +
        "If information is missing, say 'Not specified'.";

    public async Task<string> SummarizeAsync(string contractText, CancellationToken ct = default)
    {
        var messages = new List<ChatMessage>
        {
            new(ChatRole.System, SystemPrompt),
            new(ChatRole.User, contractText)
        };

        var options = new ChatOptions
        {
            Temperature = 0.1f,   // low creativity for factual tasks
            MaxOutputTokens = 400
        };

        ChatResponse response = await chat.GetResponseAsync(messages, options, ct);
        return response.Text;
    }
}

Why this matters: when procurement negotiates a better rate with a different vendor, you change one factory method and a config value. Your services, tests, and prompts stay untouched. It also lets you run a small local model in development and CI, keeping test runs free and offline.

Keep Prompts as Versioned Assets

Prompts are code. Store them in source control, give them version identifiers, and log the version with every request. When output quality changes, you need to know whether the prompt, the model, or the input changed. A simple approach is a Prompts folder of .txt files embedded as resources, loaded through a typed IPromptCatalog service.

Pattern 2: Retrieval-Augmented Generation (RAG) in C#

Models do not know your internal data, and fine-tuning is expensive and slow. RAG solves this by retrieving relevant chunks of your own documents and placing them in the prompt. It is the most common enterprise generative AI pattern because it keeps data fresh, auditable, and permission-scoped.

A RAG pipeline has two halves. The ingestion half chunks documents and stores embeddings. The query half embeds the user question, finds similar chunks, and builds a grounded prompt.

using Microsoft.Extensions.AI;

public sealed record DocumentChunk(string Id, string Text, ReadOnlyMemory<float> Vector, string TenantId);

public interface IVectorStore
{
    Task UpsertAsync(DocumentChunk chunk, CancellationToken ct);
    Task<IReadOnlyList<DocumentChunk>> SearchAsync(
        ReadOnlyMemory<float> query, string tenantId, int topK, CancellationToken ct);
}

public sealed class RagAnswerService(
    IChatClient chat,
    IEmbeddingGenerator<string, Embedding<float>> embedder,
    IVectorStore store)
{
    public async Task<string> AskAsync(string question, string tenantId, CancellationToken ct)
    {
        // 1. Embed the question
        var queryEmbedding = await embedder.GenerateEmbeddingVectorAsync(question, cancellationToken: ct);

        // 2. Retrieve, scoped to the caller's tenant (security boundary!)
        var chunks = await store.SearchAsync(queryEmbedding, tenantId, topK: 5, ct);

        // 3. Build a grounded prompt
        var context = string.Join("\n---\n", chunks.Select(c => $"[{c.Id}] {c.Text}"));
        var messages = new List<ChatMessage>
        {
            new(ChatRole.System,
                "Answer using ONLY the provided context. Cite chunk IDs in square brackets. " +
                "If the context does not contain the answer, reply: 'I don't have that information.'"),
            new(ChatRole.User, $"Context:\n{context}\n\nQuestion: {question}")
        };

        var response = await chat.GetResponseAsync(messages, new ChatOptions { Temperature = 0 }, ct);
        return response.Text;
    }
}

RAG Best Practices

• Chunk by meaning, not by byte count. Split on headings and paragraphs, and keep chunks around 300 to 500 tokens with slight overlap. Chunks that cut a sentence in half produce poor retrieval.
• Enforce authorization at retrieval time. The tenant filter above is not optional. If the vector store returns chunks the user cannot see, the model will happily quote them. This is the number one security defect in enterprise RAG systems.
• Use hybrid search. Combine vector similarity with keyword search for product codes, invoice numbers, and other exact-match terms that embeddings handle poorly.
• Return citations. Users trust answers they can verify. Citations also make quality review possible.

Pattern 3: Function Calling for Safe Actions

Summaries and answers are read-only. Real value comes when the assistant can look up an order, open a ticket, or schedule a callback. Function calling (also called tool use) lets the model request that your code run a specific method with typed arguments. The model never executes anything itself. Your code decides what runs.

using System.ComponentModel;
using Microsoft.Extensions.AI;

public sealed class OrderTools(IOrderRepository orders, ICurrentUser user)
{
    [Description("Gets the status and estimated delivery date for an order the current customer owns.")]
    public async Task<string> GetOrderStatus(
        [Description("The order number, e.g. ORD-10422")] string orderNumber)
    {
        var order = await orders.FindAsync(orderNumber, user.CustomerId);
        return order is null
            ? "Order not found for this customer."
            : $"Status: {order.Status}. Estimated delivery: {order.EstimatedDelivery:d}.";
    }
}

public sealed class SupportAssistant(IChatClient chat, OrderTools tools)
{
    public async Task<string> ReplyAsync(IList<ChatMessage> history, CancellationToken ct)
    {
        var options = new ChatOptions
        {
            Tools = [AIFunctionFactory.Create(tools.GetOrderStatus)]
        };

        // UseFunctionInvocation() in DI handles the call/return loop automatically.
        var response = await chat.GetResponseAsync(history, options, ct);
        return response.Text;
    }
}

Why this design is safe: the tool method takes the customer ID from the authenticated user, not from the model. The model can ask for any order number, but it only receives data the caller is allowed to see. Follow the same rule for every tool: authorization comes from your identity system, never from model output.

For anything that changes state (refunds, cancellations, payments), require an explicit human confirmation step in the UI before the tool runs. Treat model-initiated writes like unverified user input.

Pattern 4: Resilience, Caching, and Cost Control

Model endpoints rate-limit, time out, and occasionally return errors. Because IChatClient supports a middleware pipeline, you can add resilience without touching business code.

using Microsoft.Extensions.AI;
using Microsoft.Extensions.Caching.Distributed;
using Polly;

builder.Services.AddChatClient(providerClient)
    .UseDistributedCache()   // identical prompt + options = cached response
    .Use((inner, services) => new ResilientChatClient(inner))
    .UseLogging()
    .UseOpenTelemetry();

public sealed class ResilientChatClient(IChatClient inner) : DelegatingChatClient(inner)
{
    private static readonly ResiliencePipeline Pipeline = new ResiliencePipelineBuilder()
        .AddRetry(new()
        {
            MaxRetryAttempts = 3,
            BackoffType = DelayBackoffType.Exponential,
            UseJitter = true,
            ShouldHandle = new PredicateBuilder().Handle<HttpRequestException>()
        })
        .AddTimeout(TimeSpan.FromSeconds(30))
        .Build();

    public override Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken ct = default)
        => Pipeline.ExecuteAsync(
            async token => await base.GetResponseAsync(messages, options, token), ct).AsTask();
}

Cost Controls That Actually Work

• Cap output tokens on every request. An unbounded response is both slow and expensive.
• Route by task complexity. Use a small, fast model for classification and extraction, and reserve the largest model for reasoning-heavy work. A simple IChatClient router keyed by task type makes this a configuration decision.
• Cache aggressively. FAQ-style questions and document summaries repeat constantly. The distributed cache middleware above turns repeat calls into free calls.
• Track tokens per tenant. Record response.Usage in your metrics pipeline and set budget alerts before the invoice surprises finance.

Pattern 5: Guardrails and Evaluation

Enterprise generative AI in .NET needs two kinds of guardrails: input and output. On input, redact or block PII before it leaves your network, and reject prompts that exceed size limits. On output, validate structure. When you need machine-readable results, ask the model for JSON and deserialize into a typed record. If parsing fails, retry once with the error message included, then fall back gracefully.

public sealed record InvoiceExtraction(string VendorName, decimal Total, DateOnly DueDate);

public async Task<InvoiceExtraction?> ExtractAsync(string invoiceText, CancellationToken ct)
{
    // Structured output: the abstraction layer requests JSON matching the type.
    var response = await chat.GetResponseAsync<InvoiceExtraction>(
        [new ChatMessage(ChatRole.User, $"Extract fields from this invoice:\n{invoiceText}")],
        cancellationToken: ct);

    return response.TryGetResult(out var result) && result.Total >= 0
        ? result
        : null; // caller decides: queue for human review
}

Evaluation is your regression suite. Build a golden dataset of 50 to 200 real inputs with expected outputs, and run it in CI whenever a prompt or model changes. Score with simple assertions where possible (did the JSON parse, is the citation present) and use a second model as a judge for subjective quality. Without this, every prompt tweak is a blind deploy.

Common Pitfalls in .NET Generative AI Projects

• Calling the SDK directly from controllers. This couples your API surface to a vendor and makes testing impossible. Always go through IChatClient.
• Trusting model output as authorization. Never let the model supply user IDs, tenant IDs, or role claims. Read them from your identity provider.
• Skipping tenant filters in vector search. Cross-tenant data leakage through RAG is a common finding in security audits.
• Blocking threads. Model calls take seconds. Use async end to end and pass cancellation tokens so abandoned requests stop billing you.
• No fallback path. When the model is down, your feature should degrade to a search page or a "try again later" message, not a 500 error.
• Logging raw prompts with PII. Redact before logging, or your observability stack becomes a compliance liability.

Conclusion: Key Takeaways for Generative AI in .NET

Building generative AI in .NET for the enterprise is an architecture problem more than a model problem. The patterns that hold up in production are familiar to any experienced C# developer: depend on abstractions, enforce authorization at the boundary, add resilience through middleware, and test with real data.

• Use IChatClient from Microsoft.Extensions.AI so vendors are a configuration detail.
• Ground answers with RAG and always filter retrieval by the caller's permissions.
• Expose actions through function calling, with identity coming from your auth system and human confirmation for writes.
• Add caching, retries, timeouts, and token caps as middleware, not scattered through services.
• Validate outputs with typed structured results and run an evaluation dataset in CI.

Start with one well-bounded use case, such as document summarization or internal knowledge search, apply these patterns from day one, and you will have a foundation that scales to the next ten AI features your business asks for.

Tags: #generative AI .NET #C# AI integration #RAG in C# #Semantic Kernel tutorial #Microsoft.Extensions.AI #LLM enterprise architecture #.NET AI best practices

About csharp-coder.com
Your go-to resource for C#, .NET, and modern software development. Follow along for daily tutorials, tips, and real-world examples.