AWS SQS-SNS.md

AWS SQS/SNS

Introduction to AWS SQS/SNS

• AWS SQS (Simple Queue Service) and SNS (Simple Notification Service) are managed messaging services provided by Amazon Web Services. They enable communication between distributed applications and microservices using message queuing and pub/sub patterns. SQS provides a highly scalable and reliable message queuing service, while SNS enables messaging through topics and subscriptions.

• AWS SQS and SNS have a variety of features and pricing options. SQS offers standard and FIFO (First-In-First-Out) queues with different pricing models based on the number of requests, data transfer, and additional features. SNS offers a pay-as-you-go pricing model based on the number of notifications, data transfer, and additional features. Detailed pricing information can be found on the AWS website.

• To start using AWS SQS/SNS, you need to create an AWS account and configure the AWS SDK for .NET. You'll also need to set up Identity and Access Management (IAM) policies to grant the necessary permissions for working with SQS and SNS resources.

AWS SQS

• AWS SQS is a managed queuing service that enables communication between distributed applications using message queues. It ensures that messages are reliably delivered, even if the receiving component is temporarily unavailable. SQS offers two types of queues: standard queues, which provide at-least-once delivery and best-effort ordering, and FIFO queues, which guarantee exactly-once processing and strict message order.

• To create an AWS SQS Queue, you can use the AWS Management Console or the AWS SDK for .NET. In the console, navigate to the SQS service and click "Create queue". Choose the queue type, set the desired properties, and create the queue. To create a queue using .NET code, you can use the AmazonSQSClient class and the CreateQueueAsync method.

• To send and receive messages from an AWS SQS Queue using .NET code, use the AmazonSQSClient class. SendMessageAsync and ReceiveMessageAsync methods can be used to send and receive messages, respectively. To delete a message after processing, use the DeleteMessageAsync method.

• SQS allows you to configure message retention, visibility timeout, and dead-letter queues. Message retention defines how long messages are stored in the queue before being deleted. The visibility timeout determines how long a message is hidden from other consumers after being received. Dead-letter queues store unprocessed messages after a specified number of receive attempts, helping you identify problematic messages and handle errors. These settings can be configured when creating the queue or updated later using the SetQueueAttributesAsync method.

AWS SNS

• AWS SNS is a managed pub/sub messaging service that allows you to send messages to multiple subscribers through topics. A topic is a communication channel for sending messages and allowing subscribers to receive those messages. SNS supports various subscription types, such as AWS SQS, AWS Lambda, and HTTP/HTTPS endpoints.

• To create an AWS SNS Topic, you can use the AWS Management Console or the AWS SDK for .NET. In the console, navigate to the SNS service and click "Create topic". Set the desired properties and create the topic. To create a topic using .NET code, you can use the AmazonSimpleNotificationServiceClient class and the CreateTopicAsync method.

• To create an AWS SNS Subscription, you can use the AWS Management Console or the AWS SDK for .NET. In the console, navigate to the desired SNS topic and click "Create subscription". Choose the subscription type, set the desired properties, and create the subscription. To create a subscription using .NET code, you can use the AmazonSimpleNotificationServiceClient class and the SubscribeAsync method.

• To publish messages to an AWS SNS Topic using .NET code, use the AmazonSimpleNotificationServiceClient class and the PublishAsync method. You can provide the topic ARN, message body, and optional message attributes as parameters.

Advanced AWS SQS/SNS Concepts

• Message Attributes and Message Filtering: AWS SQS/SNS supports message attributes, which are metadata added to messages. Message filtering allows subscribers to receive messages based on the message attributes, reducing the number of irrelevant messages received.

• FIFO Queues and Message Grouping: FIFO queues in SQS guarantee message order and exactly-once processing. Message grouping allows related messages to be processed in order, while unrelated messages can be processed concurrently.

• Message Versioning and Content-Based Deduplication: Message versioning enables processing of different message formats, while content-based deduplication in SNS FIFO topics helps reduce duplicate deliveries.

• Message Encryption and Access Policies: AWS SQS/SNS supports server-side encryption of messages to protect sensitive data. Access policies control who can perform actions on your resources.

AWS SQS/SNS Best Practices

• Error Handling and Retry Mechanisms: Implement error handling and retry mechanisms to handle temporary failures, such as network issues or throttling.

• Load Balancing and Scaling: Use multiple producers and consumers to distribute workload and enable horizontal scaling.

• Monitoring and Management: Monitor your messaging infrastructure using the AWS Console and AWS CloudWatch. Set up alarms for important metrics and analyze logs to troubleshoot issues.

Examples using the AWS SDK for .NET

1. Creating an AWS SQS Queue using .NET code:

using System;
using Amazon.SQS;
using Amazon.SQS.Model;

public class SQSExample
{
    public async Task CreateQueueAsync()
    {
        var sqsClient = new AmazonSQSClient();
        var createQueueRequest = new CreateQueueRequest
        {
            QueueName = "MyExampleQueue",
            Attributes =
            {
                { "MessageRetentionPeriod", "345600" }, // 4 days
                { "VisibilityTimeout", "60" }, // 60 seconds
            }
        };

        var createQueueResponse = await sqsClient.CreateQueueAsync(createQueueRequest);
        Console.WriteLine($"Queue URL: {createQueueResponse.QueueUrl}");
    }
}

2. Sending and receiving messages from an AWS SQS Queue using .NET code:

using Amazon.SQS;
using Amazon.SQS.Model;

public class SQSExample
{
    private readonly AmazonSQSClient _sqsClient = new AmazonSQSClient();

    public async Task SendMessageAsync(string queueUrl, string messageBody)
    {
        var sendMessageRequest = new SendMessageRequest
        {
            QueueUrl = queueUrl,
            MessageBody = messageBody
        };

        var sendMessageResponse = await _sqsClient.SendMessageAsync(sendMessageRequest);
        Console.WriteLine($"Message sent with MessageId: {sendMessageResponse.MessageId}");
    }

    public async Task ReceiveMessagesAsync(string queueUrl)
    {
        var receiveMessageRequest = new ReceiveMessageRequest
        {
            QueueUrl = queueUrl,
            MaxNumberOfMessages = 10,
            WaitTimeSeconds = 20
        };

        var receiveMessageResponse = await _sqsClient.ReceiveMessageAsync(receiveMessageRequest);

        foreach (var message in receiveMessageResponse.Messages)
        {
            Console.WriteLine($"Received message with MessageId: {message.MessageId} and content: {message.Body}");
            await _sqsClient.DeleteMessageAsync(queueUrl, message.ReceiptHandle); // Deleting the message after processing
        }
    }
}

3. Configuring message retention, visibility timeout, and dead-letter queues:

using System;
using Amazon.SQS;
using Amazon.SQS.Model;

public class SQSExample
{
    public async Task CreateQueueWithDeadLetterQueueAsync()
    {
        var sqsClient = new AmazonSQSClient();
        
        // Create dead-letter queue
        var deadLetterQueueRequest = new CreateQueueRequest
        {
            QueueName = "MyExampleDeadLetterQueue"
        };
        var deadLetterQueueResponse = await sqsClient.CreateQueueAsync(deadLetterQueueRequest);
        var deadLetterQueueUrl = deadLetterQueueResponse.QueueUrl;

        // Get the dead-letter queue ARN
        var deadLetterQueueAttributes = await sqsClient.GetQueueAttributesAsync(new GetQueueAttributesRequest
        {
            QueueUrl = deadLetterQueueUrl,
            AttributeNames = new List<string> { "QueueArn" }
        });
        var deadLetterQueueArn = deadLetterQueueAttributes.Attributes["QueueArn"];

        // Create main queue with dead-letter queue configured
        var createQueueRequest = new CreateQueueRequest
        {
            QueueName = "MyExampleQueue",
            Attributes =
            {
                { "MessageRetentionPeriod", "345600" }, // 4 days
                { "VisibilityTimeout", "60" }, // 60 seconds
                { "RedrivePolicy", $"{{\"deadLetterTargetArn\":\"{deadLetterQueueArn}\",\"maxReceiveCount\":\"5\"}}" } // Send messages to the dead-letter queue after 5 failed processing attempts
            }
        };

        var createQueueResponse = await sqsClient.CreateQueueAsync(createQueueRequest);
        Console.WriteLine($"Queue URL: {createQueueResponse.QueueUrl}");
    }
}

In this example, we create a dead-letter queue and a main queue. We set the RedrivePolicy on the main queue to send messages to the dead-letter queue after 5 failed processing attempts. The dead-letter queue ARN is retrieved and included in the main queue's RedrivePolicy configuration.

4. Create an AWS SNS Topic using .NET code:

using System;
using System.Threading.Tasks;
using Amazon.SimpleNotificationService;
using Amazon.SimpleNotificationService.Model;

namespace SnsCreateTopicExample
{
    class Program
    {
        static async Task Main(string[] args)
        {
            var snsClient = new AmazonSimpleNotificationServiceClient();
            var createTopicRequest = new CreateTopicRequest
            {
                Name = "MySampleTopic"
            };

            var createTopicResponse = await snsClient.CreateTopicAsync(createTopicRequest);
            Console.WriteLine($"Topic ARN: {createTopicResponse.TopicArn}");
        }
    }
}

5. Create an AWS SNS Subscription using .NET code:

using System;
using System.Threading.Tasks;
using Amazon.SimpleNotificationService;
using Amazon.SimpleNotificationService.Model;

namespace SnsCreateSubscriptionExample
{
    class Program
    {
        static async Task Main(string[] args)
        {
            var snsClient = new AmazonSimpleNotificationServiceClient();
            var topicArn = "arn:aws:sns:us-east-1:123456789012:MySampleTopic"; // Replace with your Topic ARN

            var subscribeRequest = new SubscribeRequest
            {
                TopicArn = topicArn,
                Protocol = "email",
                Endpoint = "you@example.com" // Replace with your email
            };

            var subscribeResponse = await snsClient.SubscribeAsync(subscribeRequest);
            Console.WriteLine($"Subscription ARN: {subscribeResponse.SubscriptionArn}");
        }
    }
}

6. Publish messages to an AWS SNS Topic using .NET code:

using System;
using System.Threading.Tasks;
using Amazon.SimpleNotificationService;
using Amazon.SimpleNotificationService.Model;

namespace SnsPublishMessageExample
{
    class Program
    {
        static async Task Main(string[] args)
        {
            var snsClient = new AmazonSimpleNotificationServiceClient();
            var topicArn = "arn:aws:sns:us-east-1:123456789012:MySampleTopic"; // Replace with your Topic ARN

            var publishRequest = new PublishRequest
            {
                TopicArn = topicArn,
                Message = "Hello from AWS SNS!",
                Subject = "My Test Message"
            };

            var publishResponse = await snsClient.PublishAsync(publishRequest);
            Console.WriteLine($"Message ID: {publishResponse.MessageId}");
        }
    }
}