Messaging Patterns - A Comprehensive Guide For Software Developers

Publish/Subscribe messaging pattern and Message Queueing pattern are two important messaging patterns used in distributed systems. Both patterns help solve the communication problem between different components in a distributed system.

A publisher can send a message to numerous subscribers using the publish/subscribe messaging pattern if they are all interested in receiving it. On the other hand, the message queuing pattern uses a queue to enable asynchronous communication between various components.

What is a Distributed System?

A network of various autonomous computers functioning as a single system is known as a distributed system. By sending messages to one another over a network, the computers coordinate their actions and communicate with one another. By spreading the workload across several machines, distributed systems can boost performance, scalability, and reliability by handling complex problems that are beyond the capabilities of a single computer.

Distributed systems offer several advantages, including fault tolerance, scalability, and improved performance. The system can keep running even if one machine malfunctions by dividing tasks among several machines. Additionally, workload distribution can enhance performance and decrease response times by enabling tasks to be completed in parallel. Finally, distributed systems are more adaptable to changing demands because they can be scaled easily by adding or removing machines.

Why do people need messaging patterns, and what are the benefits of using these patterns?

These patterns are needed when several parts of a distributed system must communicate with one another in a dependable and scalable way.

The benefits of using these patterns such as:

Loose Coupling

The components are decoupled, and they don’t have to be aware of each other’s existence, making the system more resilient to changes.

Scalability

These patterns enable the system to scale horizontally by adding more instances of the components, improving the overall system’s performance.

Reliability

The messages are stored in a queue, ensuring they are not lost if a component fails or is temporarily unavailable.

Asynchronous Communication

The decoupling of components through messaging patterns allows for asynchronous communication, which reduces latency and improves performance.

Some Real-Life Usages of These Patterns

Publish/Subscribe and Message Queueing patterns are widely used in various industries, including finance, healthcare, transportation, and e-commerce.

For instance, these patterns are used in healthcare to exchange patient data between systems and finance to broadcast stock prices to various trading applications. Also, these patterns are used in e-commerce to process orders and notify customers.

How do these patterns work?

Publish/Subscribe Messaging Pattern

Publishers send messages to a topic, and subscribers receive messages from the topic in a messaging pattern called publish/subscribe. This pattern involves publishers sending messages to a topic rather than directly to subscribers. Following that, users can subscribe to the topic and start receiving messages as soon as they are published.

The flow of the publish/subscribe messaging pattern is as follows:

A publisher sends a message to a topic.

The topic receives the message and forwards it to all the subscribers that 
have subscribed to the topic.

Each subscriber receives a copy of the message.

+---------+         +------+        +-----------+
|Publisher|-------->|Topic1|------->|Subscriber1|
+---------+         +------+        +-----------+
     |                                     ^
     |                                     |
     |                                     |
     |              +------+               |
     +------------->|Topic2|---------------+
                    +------+
                         |
                         |
                         v
                    +-----------+
                    |Subscriber2|
                    +-----------+

Message Queuing Pattern

The message queuing pattern is a messaging pattern in which messages are sent to a queue, and one or more consumers receive messages from the queue. This pattern stores messages in the queue until a consumer consumes them. Multiple consumers can consume messages from the same queue, and messages are delivered to them in the order in which they were received.

The flow of the message queuing pattern is as follows:

A producer sends a message to a queue.

The message is stored in the queue until a consumer retrieves it.

A consumer retrieves the message from the queue and processes it.

+----------+      +------+     +---------+
|Producer1 |----->|Queue1|<----|Consumer1|
+----------+      +------+     +---------+
     |                               |
     |                               |
     |                               |
     |            +------+           |
     +----------->|Queue2|<----------+
                  +------+
                     ^
                     |
                 +---------+
                 |Consumer2|
                 +---------+

In summary, the publish/subscribe messaging pattern is used when publishers need to broadcast messages to multiple subscribers. In contrast, the message queuing pattern is used when messages need to be stored and delivered to one or more consumers. Both patterns have benefits and can be used to build reliable and scalable messaging systems.

Differences Between These Patterns

Message distribution
The publish/subscribe pattern distributes messages to multiple subscribers. In contrast, the message queuing pattern distributes messages to one or more consumers.

Relationship between producers and consumers
In the publish/subscribe pattern, the producer does not know about the subscribers. The producer sends messages to a topic, and the subscribers independently consume messages from the topic. In contrast, in the message queuing pattern, the producer sends messages to a queue, and one or more consumers retrieve messages from the queue.

Order of messages
In the publish/subscribe pattern, messages are not guaranteed to be delivered in the same order they were sent, as different subscribers may receive messages at different times. In contrast, in the message queuing pattern, messages are typically delivered in the order they were sent, as they are stored in a queue until consumed.

Message persistence
In the publish/subscribe pattern, messages are not typically persisted, as they are broadcasted to subscribers in real time. In contrast, in the message queuing pattern, messages are typically persisted in a queue until a consumer consumes them.

Scalability
Both patterns are scalable but in different ways. The publish/subscribe pattern is scalable in the number of subscribers, as the topic can have multiple subscribers. The message queuing pattern is scalable in the number of consumers, as multiple consumers can consume messages from the same queue.

Messaging Pattern Service Providers

Here are the common service providers for these messaging patterns:

Apache Kafka (https://kafka.apache.org)
Amazon Simple Queue Service (SQS) (https://aws.amazon.com/sqs)
Azure Service Bus (https://azure.microsoft.com/en-us/products/service-bus)
Google Cloud Pub/Sub (https://cloud.google.com/pubsub)
RabbitMQ (https://www.rabbitmq.com)

In this article, we will continue with Azure Service Bus.

What is Azure Service Bus?

Azure Service Bus is a fully-managed messaging service offered by Microsoft Azure. It provides features for both Publish/Subscribe messaging and Message Queueing patterns. Azure Service Bus is a cloud-based service that enables reliable messaging between different components in a distributed system.

It enables easy scaling of messaging infrastructure and provides high throughput.
It ensures reliable message delivery, even during failures or network interruptions.
It provides secure communication with features such as access control and encryption.
It offers integration with other Azure services, such as Azure Functions and Logic Apps.

Here’s an example of how you can use the publisher and subscriber in a sample C# project:

Publisher Class

using Microsoft.Azure.ServiceBus;
using System.Text;
using System.Threading.Tasks;
public class Publisher {
    private readonly TopicClient _topicClient;
    public Publisher(string connectionString, string topicName) {
        _topicClient = new TopicClient(connectionString, topicName);
    }
    public async Task SendMessageAsync(string messageBody) {
        var message = new Message(Encoding.UTF8.GetBytes(messageBody));
        await _topicClient.SendAsync(message);
    }
    public async Task CloseAsync() {
        await _topicClient.CloseAsync();
    }
}

This class creates a TopicClient in constructor and provides two methods to send a message to the topic and close the client. The SendMessageAsync method takes a message body and creates a new Message object with the body. It then sends the message to the topic using the SendAsync method. The CloseAsync method simply closes the client using the CloseAsync method.

Subscriber Class

using Microsoft.Azure.ServiceBus;
using System;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
public class Subscriber {
    private readonly SubscriptionClient _subscriptionClient;
    public Subscriber(string connectionString, string topicName, string subscriptionName) {
        _subscriptionClient = new SubscriptionClient(connectionString, topicName, subscriptionName);
    }
    public async Task StartReceivingAsync() {
        _subscriptionClient.RegisterMessageHandler(async (message, cancellationToken) => {
                Console.WriteLine($ "Received message: {Encoding.UTF8.GetString(message.Body)}");
                await _subscriptionClient.CompleteAsync(message.SystemProperties.LockToken);
            },
            new MessageHandlerOptions(ExceptionReceivedHandler) {
                MaxConcurrentCalls = 1,
                    AutoComplete = false
            });
        Console.WriteLine("Press any key to exit");
        Console.ReadKey();
        await _subscriptionClient.CloseAsync();
    }
    private Task ExceptionReceivedHandler(ExceptionReceivedEventArgs exceptionReceivedEventArgs) {
        Console.WriteLine($ "Message handler encountered an exception: {exceptionReceivedEventArgs.Exception}");
        return Task.CompletedTask;
    }
}

This class creates a SubscriptionClient in the constructor and provides a method to start receiving messages from the subscription. The StartReceivingAsync method registers a message handler using the RegisterMessageHandler method, which processes the message and completes it using the CompleteAsync method. The method also waits for a key press to exit the application and closes the client using the CloseAsync method.

The ExceptionReceivedHandler method is used to handle any exceptions that occur during message processing.

To run the publisher and subscriber separately, you can create two separate console applications, one for each class. In the publisher application, you can create a new instance of the Publisher class, call the SendMessageAsync method with the message body, and then call the CloseAsync method to close the client. In the subscriber application, you can create a new instance of the Subscriber class, call the StartReceivingAsync method to start receiving messages and then wait for a key press to exit the application.

TL;DR

The Publish/Subscribe Messaging Pattern and Message Queuing Pattern are common messaging patterns used in modern software development. The Publish/Subscribe Messaging Pattern is used when messages need to be broadcast to multiple subscribers. In contrast, the Message Queuing Pattern is used to store messages until a consumer can process them. Both patterns have unique benefits and use cases, and it is important to choose the right pattern based on the specific requirements of the application.

Azure Service Bus is a popular cloud messaging service that supports both patterns, making it a flexible and scalable solution for many modern applications. Using a simple C# code example above, we can see how easy it is to implement the Publish/Subscribe Messaging Pattern using Azure Service Bus.

Finally, I would like to thank you for taking the time to read this article. I hope it has provided you with a better understanding of these messaging patterns and their practical applications in modern software development.

Please feel free to comment below if you have any questions or suggestions.

Good luck and happy coding.