ML.NET Image Classification in C#: Train & Deploy a Model

Learn ML.NET image classification in C#. Train a custom image classifier with transfer learning, evaluate it, and deploy it in ASP.NET Core. Start now.

ML.NET image classification lets you train a custom image classifier in pure C# without leaving the .NET ecosystem. You don’t need Python, you don’t need to hand-write a neural network, and you don’t need a PhD. In this tutorial you’ll build an image classification model in C# using transfer learning, evaluate it properly, save it, and deploy it behind an ASP.NET Core Web API. Along the way we’ll cover why each step matters, the best practices that separate a demo from a production model, and the pitfalls that catch most developers the first time.

What Is ML.NET Image Classification and Why Use It?

ML.NET is Microsoft’s open-source, cross-platform machine learning framework for .NET. Its image classification API wraps a TensorFlow-based training pipeline and exposes it through the same MLContext and IDataView abstractions used everywhere else in ML.NET.

The key idea is transfer learning. Instead of training a convolutional neural network from scratch (which would require millions of labeled images and days of GPU time), ML.NET reuses a pre-trained model such as ResNet V2 or Inception V3. That model already knows how to detect edges, textures, shapes, and objects. ML.NET freezes those layers and trains only a small final layer on your categories. The result:

• Small datasets work. A few hundred images per class is often enough.
• CPU training is feasible. A laptop can train a useful model in minutes.
• Deployment is plain .NET. The output is a .zip you load with MLContext.Model.Load.

Typical use cases include product defect detection, document type routing, content moderation, and sorting user uploads in a web app.

Prerequisites and Project Setup

You need .NET 8 or later (the code below also works on .NET 9 and 10) and a folder of labeled images. The convention ML.NET expects is one subfolder per class:

data/
  cats/
    cat_001.jpg
    cat_002.jpg
  dogs/
    dog_001.jpg
    dog_002.jpg
  birds/
    bird_001.jpg

Create a console project and add the required packages:

dotnet new console -n ImageClassifierTrainer
cd ImageClassifierTrainer
dotnet add package Microsoft.ML
dotnet add package Microsoft.ML.Vision
dotnet add package Microsoft.ML.ImageAnalytics
dotnet add package SciSharp.TensorFlow.Redist

Why these packages? Microsoft.ML.Vision contains the ImageClassification trainer. Microsoft.ML.ImageAnalytics provides LoadRawImageBytes. SciSharp.TensorFlow.Redist ships the native TensorFlow binaries; without it you’ll get a confusing DllNotFoundException at runtime. If you have a CUDA-capable GPU, swap it for SciSharp.TensorFlow.Redist-Windows-GPU for a large speedup.

Step 1: Define the Data Classes

ML.NET is strongly typed. You describe your input and output rows as plain C# classes.

using Microsoft.ML.Data;

public class ImageData
{
    public string ImagePath { get; set; } = string.Empty;
    public string Label { get; set; } = string.Empty;
}

public class ModelInput
{
    public byte[] Image { get; set; } = Array.Empty<byte>();
    public uint LabelAsKey { get; set; }
    public string ImagePath { get; set; } = string.Empty;
    public string Label { get; set; } = string.Empty;
}

public class ModelOutput
{
    public string ImagePath { get; set; } = string.Empty;
    public string Label { get; set; } = string.Empty;
    public string PredictedLabel { get; set; } = string.Empty;
    public float[] Score { get; set; } = Array.Empty<float>();
}

Score is important. The trainer emits one probability per class, and you’ll use it to reject low-confidence predictions in production rather than blindly trusting PredictedLabel.

Step 2: Load Images from Disk

Walk the folder tree and produce one ImageData per file, using the parent folder name as the label.

using Microsoft.ML;

static IEnumerable<ImageData> LoadImagesFromDirectory(string folder)
{
    var allowed = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
        { ".jpg", ".jpeg", ".png", ".bmp" };

    foreach (var file in Directory.EnumerateFiles(folder, "*", SearchOption.AllDirectories))
    {
        if (!allowed.Contains(Path.GetExtension(file))) continue;

        yield return new ImageData
        {
            ImagePath = Path.GetFullPath(file),
            Label = new DirectoryInfo(Path.GetDirectoryName(file)!).Name
        };
    }
}

var mlContext = new MLContext(seed: 1);
var dataFolder = Path.Combine(AppContext.BaseDirectory, "data");

var images = LoadImagesFromDirectory(dataFolder).ToList();
Console.WriteLine($"Loaded {images.Count} images across " +
                  $"{images.Select(i => i.Label).Distinct().Count()} classes.");

IDataView data = mlContext.Data.LoadFromEnumerable(images);
data = mlContext.Data.ShuffleRows(data, seed: 1);

Why shuffle? Files enumerate class by class. If you split unshuffled data into train and test sets, your test set could contain only the last class and your accuracy numbers would be meaningless. Setting a seed on MLContext and on ShuffleRows makes runs reproducible, which matters when you’re comparing experiments.

Step 3: Preprocess and Split the Data

The trainer needs the label as a numeric key and the image as raw bytes. Apply both transforms, then split.

var preprocessing = mlContext.Transforms.Conversion
    .MapValueToKey(outputColumnName: "LabelAsKey",
                   inputColumnName: "Label",
                   keyOrdinality: Microsoft.ML.Transforms.ValueToKeyMappingEstimator.KeyOrdinality.ByValue)
    .Append(mlContext.Transforms.LoadRawImageBytes(
        outputColumnName: "Image",
        imageFolder: null,          // ImagePath already holds absolute paths
        inputColumnName: "ImagePath"));

IDataView preprocessed = preprocessing.Fit(data).Transform(data);

var trainTestSplit = mlContext.Data.TrainTestSplit(preprocessed, testFraction: 0.2, seed: 1);
IDataView trainSet = trainTestSplit.TrainSet;
IDataView testSet  = trainTestSplit.TestSet;

var trainValidationSplit = mlContext.Data.TrainTestSplit(trainSet, testFraction: 0.1, seed: 1);
trainSet = trainValidationSplit.TrainSet;
IDataView validationSet = trainValidationSplit.TestSet;

You now have three sets. The training set updates the weights. The validation set is used by the trainer during training to monitor overfitting and pick the best epoch. The test set is never seen during training and gives you an honest accuracy estimate at the end. Skipping the separate test set is the single most common reason people report 99% accuracy in a demo and 70% in production.

Step 4: Train the ML.NET Image Classification Model

This is the core of ML.NET image classification. Configure the trainer, then call Fit.

using Microsoft.ML.Vision;

var options = new ImageClassificationTrainer.Options
{
    FeatureColumnName = "Image",
    LabelColumnName = "LabelAsKey",
    ValidationSet = validationSet,
    Arch = ImageClassificationTrainer.Architecture.ResnetV250,
    Epoch = 50,
    BatchSize = 16,
    LearningRate = 0.01f,
    MetricsCallback = metrics => Console.WriteLine(metrics),
    TestOnTrainSet = false,
    ReuseTrainSetBottleneckCachedValues = true,
    ReuseValidationSetBottleneckCachedValues = true,
    WorkspacePath = Path.Combine(AppContext.BaseDirectory, "workspace")
};

var trainingPipeline = mlContext.MulticlassClassification.Trainers
    .ImageClassification(options)
    .Append(mlContext.Transforms.Conversion.MapKeyToValue(
        outputColumnName: "PredictedLabel",
        inputColumnName: "PredictedLabel"));

Console.WriteLine("Training... this can take several minutes on CPU.");
ITransformer trainedModel = trainingPipeline.Fit(trainSet);
Console.WriteLine("Training complete.");

A few of these options deserve explanation:

• Arch: ResnetV250 is the best default balance of accuracy and speed. InceptionV3 can score slightly higher on fine-grained categories but is slower. MobilenetV2 is the choice if the model must run on constrained hardware.
• Epoch: The number of passes over the data. With a validation set, ML.NET reports validation accuracy each epoch so you can see when it plateaus. Start at 50 and reduce if it converges early.
• BatchSize: Larger batches train faster but use more memory. 16 is safe on most laptops; 64 or more if you have a GPU.
• Bottleneck caching: Transfer learning runs every image through the frozen base model once to produce a “bottleneck” feature vector, then trains only the top layer. Caching those vectors in WorkspacePath means the second and later runs skip the expensive part entirely. This is why re-training with different hyperparameters is fast.
• MapKeyToValue: The trainer outputs a numeric key. This transform converts it back to the human-readable label so ModelOutput.PredictedLabel reads “cats” instead of “1”.

Step 5: Evaluate the Model Honestly

Never ship a model based on training accuracy. Run it against the untouched test set.

IDataView predictions = trainedModel.Transform(testSet);

var metrics = mlContext.MulticlassClassification.Evaluate(
    predictions,
    labelColumnName: "LabelAsKey",
    predictedLabelColumnName: "PredictedLabel");

Console.WriteLine($"Macro accuracy:  {metrics.MacroAccuracy:P2}");
Console.WriteLine($"Micro accuracy:  {metrics.MicroAccuracy:P2}");
Console.WriteLine($"Log loss:        {metrics.LogLoss:F4}");
Console.WriteLine(metrics.ConfusionMatrix.GetFormattedConfusionTable());

Macro vs. micro accuracy: Micro accuracy is overall correct predictions divided by total. Macro accuracy averages the per-class accuracy, so every class counts equally. If you have 900 cat images and 100 bird images, a model that always says “cat” gets 90% micro accuracy and roughly 33% macro accuracy. Watch macro accuracy when your classes are imbalanced. The confusion matrix tells you which classes are being confused, which usually points straight at the data you need to collect more of.

Step 6: Save the Model

var modelPath = Path.Combine(AppContext.BaseDirectory, "imageClassifier.zip");
mlContext.Model.Save(trainedModel, preprocessed.Schema, modelPath);
Console.WriteLine($"Model saved to {modelPath}");

Note that you pass preprocessed.Schema, not the raw data schema. The saved model expects a row with an Image byte array and a LabelAsKey, because that is what the trainer consumed. You’ll reproduce those columns at inference time.

Step 7: Deploy the Model in ASP.NET Core

Deploying an ML.NET model in a web API is straightforward, but there is one rule you must follow: PredictionEngine is not thread-safe. Creating one per request is slow, and sharing one across requests corrupts results under load. The solution is PredictionEnginePool from the Microsoft.Extensions.ML package, which manages a pool of engines for you.

dotnet new webapi -n ImageClassifierApi
cd ImageClassifierApi
dotnet add package Microsoft.ML
dotnet add package Microsoft.ML.Vision
dotnet add package Microsoft.ML.ImageAnalytics
dotnet add package Microsoft.Extensions.ML
dotnet add package SciSharp.TensorFlow.Redist

Copy imageClassifier.zip into the API project and set it to copy to the output directory. Then register the pool and expose an endpoint:

using Microsoft.Extensions.ML;
using Microsoft.ML;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddPredictionEnginePool<ModelInput, ModelOutput>()
    .FromFile(modelName: "ImageClassifier",
              filePath: Path.Combine(AppContext.BaseDirectory, "imageClassifier.zip"),
              watchForChanges: true);

var app = builder.Build();

app.MapPost("/api/classify", async (
    IFormFile file,
    PredictionEnginePool<ModelInput, ModelOutput> pool,
    CancellationToken ct) =>
{
    if (file is null || file.Length == 0)
        return Results.BadRequest("Upload an image file.");

    if (file.Length > 10 * 1024 * 1024)
        return Results.BadRequest("Image must be under 10 MB.");

    using var ms = new MemoryStream();
    await file.CopyToAsync(ms, ct);

    var input = new ModelInput { Image = ms.ToArray() };
    ModelOutput prediction = pool.Predict(modelName: "ImageClassifier", example: input);

    float confidence = prediction.Score.Max();
    const float threshold = 0.70f;

    return Results.Ok(new
    {
        label = confidence >= threshold ? prediction.PredictedLabel : "unknown",
        confidence,
        scores = prediction.Score
    });
})
.DisableAntiforgery();

app.Run();

Three production decisions are baked into that endpoint:

• Confidence threshold. A classifier always picks some class, even for a photo of a spreadsheet. Returning “unknown” below a threshold prevents embarrassing results. Tune the threshold using your test set.
• Size limits. Reading an arbitrary upload fully into memory is a denial-of-service vector. Cap it.
• Hot reload. watchForChanges: true lets you drop a newly trained .zip onto the server and have the pool pick it up without a restart.

Test it with curl:

curl -X POST https://localhost:5001/api/classify -F "file=@test-cat.jpg"

Best Practices for ML.NET Image Classification

• Balance your classes. Aim for roughly equal image counts per category. If that is impossible, monitor macro accuracy and consider collecting more examples of the minority class rather than duplicating images.
• Collect images that match production. If users will upload phone photos under bad lighting, don’t train exclusively on studio product shots. The model learns the distribution you give it.
• Resize before storing. The base models internally work at 224×224 (ResNet) or 299×299 (Inception). Storing 4000×3000 originals bloats disk and slows loading with no accuracy benefit.
• Keep the workspace folder. Cached bottleneck values make retraining with new hyperparameters take seconds instead of minutes.
• Version your models. Name saved files with a date or git SHA, and log which model version produced each prediction so you can trace regressions.
• Warm up the pool. The first prediction loads TensorFlow and can take a couple of seconds. Fire a dummy prediction at startup so real users never see it.

Common Pitfalls and How to Fix Them

• DllNotFoundException for tensorflow.dll: You forgot SciSharp.TensorFlow.Redist, or you’re on an ARM machine where the redist isn’t available. On Apple Silicon, run under Rosetta or train on an x64 machine and deploy only the saved model.
• Accuracy stuck near 1/N classes: Labels aren’t being read correctly. Print a few ImageData rows to confirm the folder name is the label, and confirm MapValueToKey targets the right column.
• Training accuracy 100%, test accuracy poor: Classic overfitting. Add more varied images, reduce epochs, or check whether near-duplicate images leaked across the train/test split.
• Corrupted predictions under load: You shared a single PredictionEngine across threads. Use PredictionEnginePool.
• Out of memory during training: Lower BatchSize. Very large images also inflate memory; resize them first.
• Model loads but throws a schema mismatch: You saved with the wrong schema or your inference input class doesn’t have an Image byte array column. Match the column names exactly.

Conclusion: Key Takeaways on ML.NET Image Classification

ML.NET image classification gives C# developers a practical path from a folder of labeled images to a deployed prediction API without leaving .NET. The important lessons from this tutorial:

• Transfer learning with a pre-trained architecture like ResNet V2 makes small datasets and CPU training viable.
• Always shuffle, then split into train, validation, and test sets. Evaluate only on the test set.
• Watch macro accuracy and the confusion matrix, not just a single headline number.
• Save the model with the preprocessed schema and reproduce those columns at inference time.
• In ASP.NET Core, use PredictionEnginePool, enforce upload limits, and apply a confidence threshold.

From here, natural next steps are automating retraining as new labeled data arrives, exporting to ONNX for cross-platform inference, or extending the same pipeline to object detection. But the foundation you’ve built here, a custom image classification model in C# that trains, evaluates, and serves predictions reliably, is exactly what most real-world applications need.

This article is part of the AI in .NET guide (LLMs, agents, RAG, ML.NET and AI coding tools).