Using the Microsoft Message Queue MSMQ and C# ASP .NET

Introduction

There are several different kinds of queues available in the C# ASP .NET framework with the most popular one being the standard Queue collection object. While these collections are easy to work with and fairly robust they are still stored in memory and therefore temporary. Often, in enterprise application design, more stringent means of managing and storing temporary data are required.

In the case of receiving timely data to be processed, using a queue in your software design makes sense. When the data needs to be guaranteed for delivery and protected from loss, the Microsoft Message Queue MSMQ provides a scalable easy solution.

At its core, Microsoft Message Queue MSMQ is a queue collection with a persistent storage backing. This means that data sent to the queue is automatically persisted within an internal Windows database. The MSMQ is also thread-safe, which means a multi-threaded C# ASP .NET application may safely send and receive messages with the queue and be gauranteed a thread-safe enviornment, and that messages will not be lost. As powerful and complex as the Microsoft Message Queue MSMQ sounds, it’s actually quite easy and convenient to use.

Overall Look at the Microsoft Message Queue MSMQ

Before jumping into the detailed code, using the Microsoft Message Queue with C# ASP .NET is very similar to working with a queue collection with the exception that you have even more properties and tools available for working with the data. Messages can be stored as plain text, XML, binary, or even a custom selected format, dictated by the Message Formatter property of the queue. Below is a psuedo-code summary of how to use a Microsoft Message Queue MSMQ.

// Initialize the queue

// Begin listening for new events on the queue

// Send a message to the queue

// Close the queue

You Do Have a Queue, Don’t You?

Since Microsoft Message Queue MSMQ is part of the Windows operating system, you need to first verify that the component is installed on your PC and that you have a queue available to work with. To check if MSMQ is installed, open the Computer Management window by clicking Start->Settings->Control Panel->Administrative Tools->Computer Management. Click the plus sign next to Services and Applications. You should see an item called Message Queuing. If you need to install MSMQ, you can do so by entering the Add Remove Programs window, click Add Remove Windows Components, and checkmark the option for Message Queuing.

Once Microsoft Message Queue MSMQ is available on your PC, you need to create a queue. Two basic types of queues exist: public and private. While there are slight security differences between using public and private queues, they are for the most part, the same in usage. In code, they only differ by the path you use to connect to the queue. Specifically, two paths are shown below to connect to a public and private queue. The text “Private$” precedes the path for private message queues.

Public Message Queue Path
.\MyQueue”

Private Message Queue Path

“.\Private$\MyQueue

To create a new queue, right-click on the “Public Queues” folder and select New->Public Queue. It is generally a good choice to checkmark the Transactional option so that you queue will support transactions. Enter a name for the queue to create a new MSMQ queue. With the queue created, you can view the messages in the queue by clicking the plus sign next to your queue name, select Queue messages, and hit F5 to refresh the pane. Initially, the queue is empty until we send a few messages its way. To manually empty the queue, right-click on “Queue messages”, select All Tasks, and select Purge.

Getting a Message into the Queue

Using a Microsoft Message Queue MSMQ is very straight-forward and involves opening your message queue, sending a message, and closing the queue. Since the message queue object implements the IDisposable interface, we can utilize the “using” statement to automatically close and dispose of the queue for us. Below is an example of opening the queue, sending a basic message, and closing. Note, this article demonstrates sending two types of messages to the queue, therefore an enum type is defined to dictate which type of message we’re sending.

public enum MessageType
{
    MESSAGE_TYPE_PLAIN_TEXT = 0,
    MESSAGE_TYPE_HELLO_WORLD = 1
};

static void Main(string[] args)
{
   // Open the queue.
   using (queue = new MessageQueue(".\\MyQueue"))
   {
        // Enable the AppSpecific field in the messages.
        queue.MessageReadPropertyFilter.AppSpecific = true;

        // Set the formatter to binary.
        queue.Formatter = new BinaryMessageFormatter();

        // Since we're using a transactional queue, make a transaction.
        using (MessageQueueTransaction mqt = new MessageQueueTransaction())
        {
            mqt.Begin();

            // Create a simple text message.
            Message myMessage = new Message("Hello World", new BinaryMessageFormatter());
            myMessage.Label = "First Message";
            myMessage.AppSpecific = (int)MessageType.MESSAGE_TYPE_PLAIN_TEXT;

            // Send the message.
            queue.Send(myMessage, mqt);
        }
   }
}

After running the above code, if you view your message queue in the Computer Management window and refresh the messages, you will see a new message available in the queue. If you view the contents of the message, you’ll notice it’s encoded in XML and formatted in binary. This is because we chose to use the BinaryMessageFormatter(), which allows us to serialize any type of object in the queue’s message body and deserialize it back to its original object type.

Receiving Messages from the Queue

Receiving messages generally happens asynchronously in Microsoft Message Queue MSMQ. You add a ReceiveCompleted event to the queue and begin listening for new messages to arrive. When a message arrives, your event is triggered, passed a message object, and you can then deserialize the body and process the contents. Your final call in the ReceiveCompleted function would be to start listening again for new messages.

// Setup an event to listen for new messages.
queue.ReceiveCompleted += new ReceiveCompletedEventHandler(queue_ReceiveCompleted);

// Start listening.
queue.BeginReceive();

static void queue_ReceiveCompleted(object sender, ReceiveCompletedEventArgs e)
{
   lock (lockObject)
   {
      // The message is plain text.
      string text = (string)e.Message.Body;
      Console.WriteLine("Message received: " + text);
   }

   // Listen for the next message.
   queue.BeginReceive();
}

All Messages are Objects

Before showing the complete example code to work with Microsoft Message Queue MSMQ, it’s important to understand that the message bodies stored in the queue are actually objects. This is a powerful choice by Microsoft because it allows you to store any type of message, class, structure, and even algorithms, within a message body for transmission. With this information, we begin by creating an example class to hold our message content.

[Serializable]
class HelloWorld
{
    private string text = "";

    public HelloWorld()
    {
    }

    public HelloWorld(string _text)
    {
        text = _text;
    }

    public string GetText()
    {
        return text;
    }
}

Notice that the class is serializable. This is required in order to serialize the class into a message’s body. In the example above of sending a message to the queue, we used a simple text message (a string). In the example below, we’ll send a HelloWorld object and a string. This example shows how to send two different types of messages and process them accordingly. Note, MSMQ is generally used in a service form of application, rather than a console application. However, in this article, a console-style application is used for simplicity.

public enum MessageType
{
    MESSAGE_TYPE_PLAIN_TEXT = 0,
    MESSAGE_TYPE_HELLO_WORLD = 1
};

class Program
{
    private static MessageQueue queue = null;
    private static object lockObject = new object();

    static void Main(string[] args)
    {
        // Open the queue.
        using (queue = new MessageQueue(".\\MyQueue"))
        {
            // Enable the AppSpecific field in the messages.
            queue.MessageReadPropertyFilter.AppSpecific = true;

            // Set the formatter to binary.
            queue.Formatter = new BinaryMessageFormatter();

            // Setup an event to listen for new messages.
            queue.ReceiveCompleted += new ReceiveCompletedEventHandler(queue_ReceiveCompleted);

            // Start listening.
            queue.BeginReceive();

            using (MessageQueueTransaction mqt = new MessageQueueTransaction())
            {
                mqt.Begin();

                // Create a simple text message.
                Message myMessage = new Message("Hello World", new BinaryMessageFormatter());
                myMessage.Label = "First Message";
                myMessage.AppSpecific = (int)MessageType.MESSAGE_TYPE_PLAIN_TEXT;

                // Send the first message.
                queue.Send(myMessage, mqt);

                // Create a message containing an object.
                HelloWorld helloWorld = new HelloWorld("Hello World from an object.");
                myMessage = new Message(helloWorld, new BinaryMessageFormatter());
                myMessage.Label = "Second Message";
                myMessage.AppSpecific = (int)MessageType.MESSAGE_TYPE_HELLO_WORLD;
                myMessage.Priority = MessagePriority.Normal;

                // Send the second message.
                queue.Send(myMessage, mqt);

                mqt.Commit();
            }

            while (Console.ReadKey().Key != ConsoleKey.Q)
            {
                // Press q to exit.
            }
        }
    }

    static void queue_ReceiveCompleted(object sender, ReceiveCompletedEventArgs e)
    {
        lock (lockObject)
        {
            if ((MessageType)e.Message.AppSpecific == MessageType.MESSAGE_TYPE_HELLO_WORLD)
            {
                // The message is our HelloWorld object.
                HelloWorld helloWorld = new HelloWorld();
                helloWorld = (HelloWorld)e.Message.Body;
                Console.WriteLine("Message received: " + helloWorld.GetText());
            }
            else
            {
                // The message is plain text.
                string text = (string)e.Message.Body;
                Console.WriteLine("Message received: " + text);
            }

            // Listen for the next message.
            queue.BeginReceive();
        }
    }
}

Notice how in the code that sends the messages, we utilize the AppSpecific property of the message to indicate the type of message body being stored. The AppSpecific property is an integer unused by Windows and is available to the developer. This property is often used by developers in the case of messages possibly failing and being re-tried, in which case it is used to count how many times the message has been sent. After the value reaches a certain threshold, the message may be posted to the Dead-letter messages queue located under the “System Queues” folder in Computer Management. Dead-letter messages are also termed “poisoned messages”. However, in the example above, the AppSpecific property is being used to identify the type of message body (since we are sending two different messages to the same queue). If the AppSpecific property identifies our HelloWorld type, we deserialize the message’s body as a HelloWorld object. Otherwise, we know it will be a simple string.

When the above example is ran, you will see two lines printed to the console almost instantly, as Microsoft Message Queue MSMQ operates quite fast. If you refresh your queue in the Computer Management window, you should continue seeing an empty list of messages, as the messages were immediately processed and removed from the queue by your code.

The Message Loop

The final line in the example above was “queue.BeginReceive()”. This is a critical line to a successful implementation of Microsoft Message Queue in that it provides the means for continuous listening on the message queue. Each time a message is received, the listening process stops. This helps provide a thread-safe environment. However, it also means it’s the developer’s responsibility to resume listening to the queue.

Poisoned Messages and Retry Attempts

As mentioned above, while Microsoft Message Queue MSMQ provides a persistent backing store and thread-safe system for managing queued messages, it’s still possible for processing of the message to fail. For example, consider the case where each time a message is received you transfer a file over the Internet. File transfers are bound to fail at some point, such as a server temporarily going down. In which case, your message will be considered to have failed. Since this is a temporary failure, you would want to re-queue the message and try again. There are different ways of dealing with failed messages, with the most obvious being to simply put the message back on the top of the queue to immediately try again. However, if the failure of the message was due to some invalid data within the body, which would cause to fail every single time (such as a bad FTP address or other data), then the message is a poisoned message - meaning that it can never be successfully processed. If the message is put back on the top of the queue every time, you end up wasting valuable processing time since each receive call on the queue obtains the same message, which is bound to fail. After so many retry attempts (counted by incrementing a property such as the AppSpecific field), you can cease re-queing the message and instead move it to a failed queue. However, there are better ways to handle a failed, and possibly poisoned, message.

The first suggestion for dealing with a failed message was to increment its AppSpecific count (or other field), and re-queue the message at the top. Alternately, you could re-queue the message at the bottom of the queue by setting its MessagePriority to “Lowest”. This gives all newer messages a chance to be processed first before getting back to the failed message.

Depending on your situation, there are even more ways, with increasing complexity, of handling failed and poisoned messages. The method you choose depends on your particular software design.

Conclusion

Microsoft Message Queue MSMQ provides a robust and scalable system for handling timely data in a queue. MSMQ provides a reliable persistent backing store via an internal Windows database, which persists data on the queue between C# ASP .NET web application sessions and system restarts. Message queues can be public or private and support transactional commands in a thread-safe environment. Message bodies can contain a variety of formatted data and is easily created using message formatters such as the XMLMessageFormatter and the BinaryMessageFormatter. Receiving messages from is handled by listening for the Receive event, deserializing the message body, and processing the contents. Messages may fail when processing and can be retried in a number of ways. When a message is doomed to fail every time, it is considered a poisoned message and should be handled in a speedy and reliable manner.

Using Microsoft Message Queue MSMQ in C# ASP .NET web applications allows us to easily process queued data and provide scalable enterprise architecture solutions.

About the Author

This article was written by Kory Becker, software developer and architect, skilled in a range of technologies, including web application development, machine learning, artificial intelligence, and data science.

Sponsor Me