Google Cloud Pub/Sub with C#: Complete .NET Tutorial

Learn Google Cloud Pub/Sub in C# with this .NET tutorial. Publish and subscribe to messages on GCP with runnable code examples. Start building today!

If you're building distributed systems on Google Cloud Platform, Google Cloud Pub/Sub is the messaging backbone you'll reach for again and again. It's a fully managed, globally distributed publish/subscribe service that decouples the services that produce events from the services that process them — no brokers to patch, no clusters to scale, no partitions to rebalance. In this tutorial, you'll learn how to use Google Cloud Pub/Sub with C# and .NET: creating topics and subscriptions, publishing messages asynchronously, consuming them with the high-throughput streaming subscriber, and handling the real-world concerns — ordering, retries, dead-letter queues, and exactly-once processing — that separate a demo from a production system.

What Is Google Cloud Pub/Sub and Why Use It?

Pub/Sub implements the classic publish/subscribe pattern: a publisher sends a message to a topic, and Pub/Sub delivers a copy of that message to every subscription attached to the topic. Subscribers pull messages from their subscription, process them, and acknowledge them. Until a message is acknowledged, Pub/Sub keeps redelivering it — which is precisely what gives you at-least-once delivery guarantees.

Why does this matter for your architecture? Because direct HTTP calls between services create tight coupling. If your order service calls your email service synchronously and the email service is down, orders fail. With asynchronous messaging, the order service publishes an OrderPlaced event and moves on. The email service, the analytics pipeline, and the inventory system each consume that event independently, at their own pace, with their own retry behavior.

Compared to alternatives you might know:

• vs. RabbitMQ: Pub/Sub is fully managed and scales automatically to millions of messages per second. You trade fine-grained routing (exchanges, routing keys) for zero operations.
• vs. Apache Kafka: No partitions to plan, no consumer group rebalancing to debug. Kafka wins for log replay over long retention windows; Pub/Sub wins on operational simplicity (though it does support message retention and replay via seek).
• vs. Azure Service Bus / AWS SQS+SNS: Functionally similar; the right choice usually follows the cloud you're already on. Pub/Sub combines the fan-out of SNS and the queueing of SQS in a single service.

Setting Up Google Cloud Pub/Sub in a C# Project

You'll need a GCP project with the Pub/Sub API enabled and the gcloud CLI installed. For local development, authenticate with Application Default Credentials — the Google client libraries pick these up automatically, so you never hard-code keys:

// Terminal, not C# — run once on your dev machine:
// gcloud auth application-default login
// gcloud config set project your-project-id

Then add the NuGet package to your .NET project:

// dotnet add package Google.Cloud.PubSub.V1

The Google.Cloud.PubSub.V1 package contains two layers: low-level generated clients (PublisherServiceApiClient, SubscriberServiceApiClient) for administrative operations, and high-level wrappers (PublisherClient, SubscriberClient) that handle batching, flow control, and connection management for you. Use the high-level clients for message traffic — they exist because doing this efficiently by hand is genuinely hard.

Creating a Topic and Subscription in C#

using Google.Cloud.PubSub.V1;
using Grpc.Core;

const string projectId = "your-project-id";

var publisherApi = await PublisherServiceApiClient.CreateAsync();
var subscriberApi = await SubscriberServiceApiClient.CreateAsync();

var topicName = TopicName.FromProjectTopic(projectId, "orders");
var subscriptionName = SubscriptionName.FromProjectSubscription(projectId, "orders-email-service");

try
{
    await publisherApi.CreateTopicAsync(topicName);
    Console.WriteLine($"Created topic: {topicName}");
}
catch (RpcException ex) when (ex.StatusCode == StatusCode.AlreadyExists)
{
    Console.WriteLine("Topic already exists — that's fine.");
}

try
{
    await subscriberApi.CreateSubscriptionAsync(new Subscription
    {
        SubscriptionName = subscriptionName,
        TopicAsTopicName = topicName,
        AckDeadlineSeconds = 60
    });
}
catch (RpcException ex) when (ex.StatusCode == StatusCode.AlreadyExists)
{
    Console.WriteLine("Subscription already exists.");
}

A note on AckDeadlineSeconds: this is how long Pub/Sub waits for an acknowledgment before redelivering the message. Set it slightly above your worst-case processing time. The high-level SubscriberClient automatically extends the deadline while your handler runs, but a sensible base value reduces spurious redeliveries during restarts.

Publishing Messages Asynchronously with PublisherClient

Here's the idiomatic publisher. Create it once and reuse it — it maintains gRPC channels and batches messages behind the scenes:

using System.Text.Json;
using Google.Cloud.PubSub.V1;
using Google.Protobuf;

public record OrderPlaced(string OrderId, string CustomerEmail, decimal Total);

public class OrderEventPublisher : IAsyncDisposable
{
    private readonly PublisherClient _publisher;

    private OrderEventPublisher(PublisherClient publisher) => _publisher = publisher;

    public static async Task<OrderEventPublisher> CreateAsync(string projectId)
    {
        var topicName = TopicName.FromProjectTopic(projectId, "orders");
        var publisher = await PublisherClient.CreateAsync(topicName);
        return new OrderEventPublisher(publisher);
    }

    public async Task<string> PublishAsync(OrderPlaced order)
    {
        var message = new PubsubMessage
        {
            Data = ByteString.CopyFromUtf8(JsonSerializer.Serialize(order)),
            Attributes =
            {
                { "eventType", "OrderPlaced" },
                { "schemaVersion", "1" }
            }
        };

        // Returns the server-assigned message ID once the batch is sent.
        string messageId = await _publisher.PublishAsync(message);
        return messageId;
    }

    public async ValueTask DisposeAsync()
    {
        // Flushes any locally batched messages before shutdown.
        await _publisher.ShutdownAsync(TimeSpan.FromSeconds(15));
    }
}

Two details here matter more than they look:

• Attributes are metadata, not payload. Putting eventType and schemaVersion in attributes lets subscribers filter messages server-side (subscription filters like attributes.eventType = "OrderPlaced") without deserializing the body. This is how you evolve schemas without breaking consumers.
• ShutdownAsync is not optional. The client batches messages in memory for efficiency. If your process exits without calling ShutdownAsync, messages sitting in the local batch are silently lost. Wire this into your host's graceful shutdown (in ASP.NET Core, register it with IHostApplicationLifetime or as a singleton implementing IAsyncDisposable).

Consuming Messages: The Streaming SubscriberClient

The SubscriberClient opens streaming pull connections and invokes your handler concurrently as messages arrive. Your handler returns a verdict: Ack (done, delete it) or Nack (failed, redeliver it):

using System.Text.Json;
using Google.Cloud.PubSub.V1;

var subscriptionName = SubscriptionName.FromProjectSubscription(projectId, "orders-email-service");
var subscriber = await SubscriberClient.CreateAsync(subscriptionName);

var cts = new CancellationTokenSource();
Console.CancelKeyPress += (_, e) => { e.Cancel = true; cts.Cancel(); };

Task subscriberTask = subscriber.StartAsync(async (PubsubMessage message, CancellationToken ct) =>
{
    try
    {
        var order = JsonSerializer.Deserialize<OrderPlaced>(message.Data.ToStringUtf8());
        if (order is null)
            return SubscriberClient.Reply.Ack; // Poison message — don't retry forever.

        await SendConfirmationEmailAsync(order, ct);
        Console.WriteLine($"Processed order {order.OrderId} (message {message.MessageId})");
        return SubscriberClient.Reply.Ack;
    }
    catch (JsonException)
    {
        // Malformed payload will never succeed on retry. Ack it (or better:
        // configure a dead-letter topic and Nack so it lands there).
        return SubscriberClient.Reply.Ack;
    }
    catch (Exception)
    {
        // Transient failure — Nack so Pub/Sub redelivers with backoff.
        return SubscriberClient.Reply.Nack;
    }
});

cts.Token.Register(() => subscriber.StopAsync(TimeSpan.FromSeconds(15)));
await subscriberTask;

In a real ASP.NET Core or Worker Service app, host this inside a BackgroundService — call StartAsync in ExecuteAsync and StopAsync when the stoppingToken fires. StopAsync waits for in-flight handlers to finish, giving you clean graceful shutdown.

Why At-Least-Once Delivery Demands Idempotency

This is the single most important concept in this entire tutorial. Pub/Sub guarantees at-least-once delivery, which means your handler will occasionally receive the same message twice — after a crash, a missed ack deadline, or an internal redelivery. If your handler charges a credit card, sending the same message twice must not charge twice.

The standard defense is idempotent processing keyed on message.MessageId or, better, a business key like OrderId:

// Example: idempotency via a unique constraint in your database.
// INSERT the business key first; a duplicate key violation means
// this message was already processed — ack and skip.
try
{
    await db.ExecuteAsync(
        "INSERT INTO processed_events (order_id) VALUES (@id)",
        new { id = order.OrderId });
}
catch (PostgresException ex) when (ex.SqlState == "23505") // unique_violation
{
    return SubscriberClient.Reply.Ack; // Already handled — safe to skip.
}

Pub/Sub also offers exactly-once delivery as a subscription setting, which suppresses redelivery of acknowledged messages within a region. It's useful, but treat it as a mitigation, not a replacement for idempotent handlers — your own code can still fail after the side effect but before the ack.

Google Cloud Pub/Sub Best Practices for .NET Developers

• Reuse clients. PublisherClient and SubscriberClient are thread-safe and expensive to create. Register them as singletons in your DI container. Creating one per request destroys throughput and leaks connections.
• Configure a dead-letter topic. Set DeadLetterPolicy on the subscription (with a MaxDeliveryAttempts of 5–10). Messages that keep failing get routed to a dead-letter topic you can inspect, instead of cycling through your subscriber forever and burning CPU.
• Use ordering keys only when you need them. Enabling EnableMessageOrdering and setting an OrderingKey (say, per customer ID) guarantees in-order delivery per key — but serializes throughput per key and pauses the key on failure. Most event streams don't need it; don't pay the cost by default.
• Tune flow control. SubscriberClientBuilder.Settings.FlowControlSettings caps how many messages are outstanding in your process at once. If your handler hits a database, cap concurrency near your connection pool size, or you'll trade a messaging bottleneck for a database one.
• Test locally with the emulator. Run gcloud beta emulators pubsub start, set the PUBSUB_EMULATOR_HOST environment variable, and the client libraries route to it automatically. No cloud costs, no shared state between developers, and it works in CI containers.
• Monitor the right metric. The key health signal is subscription/oldest_unacked_message_age in Cloud Monitoring. A growing backlog means your subscribers can't keep up — alert on it before your users notice.

Common Pitfalls to Avoid

• Nacking poison messages forever. A message that fails deserialization will fail on every redelivery. Without a dead-letter policy, it loops indefinitely. Detect permanent failures and route them out of the hot path.
• Forgetting that subscriptions expire. By default, a subscription with no activity for 31 days is deleted — along with any undelivered messages. Set ExpirationPolicy to never-expire for production subscriptions.
• Assuming ordering without ordering keys. Plain Pub/Sub delivery order is not guaranteed. If your consumer assumes OrderPlaced always arrives before OrderShipped, it will eventually be wrong. Either use ordering keys or design consumers to tolerate out-of-order events.
• Blocking in the handler. The handler delegate is async — use await throughout. Calling .Result or .Wait() inside it starves the thread pool under load, and the symptoms (rising ack latency, redeliveries) look confusingly like a Pub/Sub problem.

Conclusion: Key Takeaways for Google Cloud Pub/Sub in C#

Google Cloud Pub/Sub gives .NET developers a zero-ops path to asynchronous messaging: publishers and subscribers scale independently, failures in one service stop cascading into others, and the Google.Cloud.PubSub.V1 library handles the hard parts — batching, streaming pull, and deadline management — so your code stays focused on business logic.

• Use the high-level PublisherClient and SubscriberClient, registered as singletons, and always call ShutdownAsync/StopAsync on graceful shutdown.
• Design every handler to be idempotent — at-least-once delivery means duplicates are a certainty, not an edge case.
• Configure dead-letter topics and non-expiring subscriptions before production traffic, not after your first incident.
• Reach for ordering keys and exactly-once delivery only when the business requirement demands them; both trade throughput for guarantees.
• Develop against the Pub/Sub emulator locally and alert on oldest-unacked-message age in production.

From here, natural next steps are wiring Pub/Sub push subscriptions into Cloud Run services, adding schema validation with Pub/Sub schemas (Avro or Protobuf), and exploring the outbox pattern for publishing events transactionally with your database writes. Master those, and you'll have a genuinely production-grade event-driven architecture on GCP — all in C#.

Tags: #google cloud pub/sub c# #gcp pub/sub tutorial #pub/sub .net #google pub/sub publisher subscriber #asynchronous messaging c# #google cloud messaging .net

About csharp-coder.com
Your go-to resource for C#, .NET, and modern software development. Follow along for daily tutorials, tips, and real-world examples.