.NET on AWS – Queuing with SQS (Part-3)

Introduction

In previous posts, we learned the basics of AWS Queue and how to publish messages to it from a .NET web-api application.

Today our focus will be the Consumer part of the application. Following is solution design we are working towards throughout this series.

To consume the messages produced from our web-api application, the .NET solution includes a simple worker service application which will continuously look for new messages being published and process them.

Source code is available from this git repository. 

Lets implement the consumer functionality in following sections in this post.

Connecting with SQS Queue

To provide our consumer application with access to AWS Queue, we will start by providing some settings to worker application via appsettings.json file.

This may seem very similar to the part we did for publisher application. Following are the settings, corresponding model class and DI wiring for worker application:

With this in place, our application is now equipped with reading the settings from json file and wiring those up via dependency injection system to be available to worker application.

Implementing the Queue Consumer

To read and process message from the queue, I’ve defined the following simple interface

Interfaces provides abstraction and easy way to introduce new implementations as needed.

Following is the SQSEventConsumer class implementing the interface

This consumer class is using objects from AWSSDK.SQS NuGet package, we installed to our application in previous posts.

Configuring AmazonSQSClient

To interact with SQS Queue, we need to configure instance of AmazonSQSClient and that is done in the constructor:

Above shown code is very boiler plate and it is setting up the AmazonSQSClient to be later used in Start() and Stop() methods.

Consuming the Queue (Start() method)

Start method contains the code to read and process messages from the queue:

following is the code for actual messages consumption

As you can see that we are initializing a new RecieveMessageRequest and using it with _sqsClient for polling messages.

Once messages are received, worker application can process those. Here for our demo purposes, it is just logging those out to console, but you can do any processing here as per your application requirements.

Stopping the EventConsumer (Stop() method)

Stop method is very simple and its just setting up a flag variable (running = false) which will make the processing stop inside Start() method:

Wiring up the EventConsumer in Worker Service

Now we’ve SQSEventConsumer, we can wire this up with Dependency Injection as shown below

and use it in Worker class as shown below

Here eventConsumer is injected via constructor’s injection and then its simply matter of calling the Start() and Stop() methods appropriately.

Testing the Event Consumer

Now we can simply run the worker application from visual studio (Set as startup project) and following is the output

These are the messages we pushed to queue in previous post and now our consumer application is successfully able to poll and process those.

Deleting Queue Messages

One thing clients need to do is to delete the messages once those are processed, otherwise they will become available again in the queue after Visibility timeout is elapsed.

Deletion can be done easily by sending a DeleteMessageRequest vis SQSClient as shown below

with these changes in place, if we now run the consumer application again, it will delete the messages from queue after processing.

For further testing you can publish more messages via web-api (publisher) to queue and see that those will be picked up by consumer, processed and afterwards deleted.

Solution Update

At this point, our demo solution for SQS functionality is completed. Following is the diagram for your reference:

Summary

In this post, we implemented the SQS Queue Consumer part of our solution. We’ve implemented this functionality inside a worker service type of .NET project.

Throughout this series we learned the various components involving Amazon SQS queue creation and setup, .NET web-api as a message publisher and .NET worker service as a message consumer.

We saw that how AWSSDK.SQS NuGet package helps us writing boiler plate code to connect, send message and receive message from SQS inside our .NET applications.

You can get the source-code from this git repository. 

Let me know if you have some comments or questions. Till next time, Happy coding.