💡 摘要
一个用于在解耦架构中与 AWS 简单队列服务 (SQS) 交互的综合指南和代码库。
🎯 适合人群
🤖 AI 吐槽: “这是一本写得不错的 SQS 手册,只是忘了穿上它的“AI 代理技能”外衣。”
该技能需要具有 SQS 权限的 AWS 凭证,存在秘密管理风险。代码示例会调用 AWS API 进行网络通信。缓解措施:使用临时凭证(例如 AWS IAM 角色),切勿在代理环境中硬编码秘密。
name: sqs description: AWS SQS message queue service for decoupled architectures. Use when creating queues, configuring dead-letter queues, managing visibility timeouts, implementing FIFO ordering, or integrating with Lambda. last_updated: "2026-01-07" doc_source: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/
AWS SQS
Amazon Simple Queue Service (SQS) is a fully managed message queuing service for decoupling and scaling microservices, distributed systems, and serverless applications.
Table of Contents
Core Concepts
Queue Types
| Type | Description | Use Case | |------|-------------|----------| | Standard | At-least-once, best-effort ordering | High throughput | | FIFO | Exactly-once, strict ordering | Order-sensitive processing |
Key Settings
| Setting | Description | Default | |---------|-------------|---------| | Visibility Timeout | Time message is hidden after receive | 30 seconds | | Message Retention | How long messages are kept | 4 days (max 14) | | Delay Seconds | Delay before message is available | 0 | | Max Message Size | Maximum message size | 256 KB |
Dead-Letter Queue (DLQ)
Queue for messages that failed processing after maxReceiveCount attempts.
Common Patterns
Create a Standard Queue
AWS CLI:
aws sqs create-queue \ --queue-name my-queue \ --attributes '{ "VisibilityTimeout": "60", "MessageRetentionPeriod": "604800", "ReceiveMessageWaitTimeSeconds": "20" }'
boto3:
import boto3 sqs = boto3.client('sqs') response = sqs.create_queue( QueueName='my-queue', Attributes={ 'VisibilityTimeout': '60', 'MessageRetentionPeriod': '604800', 'ReceiveMessageWaitTimeSeconds': '20' # Long polling } ) queue_url = response['QueueUrl']
Create FIFO Queue
aws sqs create-queue \ --queue-name my-queue.fifo \ --attributes '{ "FifoQueue": "true", "ContentBasedDeduplication": "true" }'
Configure Dead-Letter Queue
# Create DLQ aws sqs create-queue --queue-name my-queue-dlq # Get DLQ ARN DLQ_ARN=$(aws sqs get-queue-attributes \ --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue-dlq \ --attribute-names QueueArn \ --query 'Attributes.QueueArn' --output text) # Set redrive policy on main queue aws sqs set-queue-attributes \ --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue \ --attributes "{ \"RedrivePolicy\": \"{\\\"deadLetterTargetArn\\\":\\\"${DLQ_ARN}\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\" }"
Send Messages
import boto3 import json sqs = boto3.client('sqs') queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue' # Send single message sqs.send_message( QueueUrl=queue_url, MessageBody=json.dumps({'order_id': '12345', 'action': 'process'}), MessageAttributes={ 'MessageType': { 'DataType': 'String', 'StringValue': 'Order' } } ) # Send to FIFO queue sqs.send_message( QueueUrl='https://sqs.us-east-1.amazonaws.com/123456789012/my-queue.fifo', MessageBody=json.dumps({'order_id': '12345'}), MessageGroupId='order-12345', MessageDeduplicationId='unique-id-12345' ) # Batch send (up to 10 messages) sqs.send_message_batch( QueueUrl=queue_url, Entries=[ {'Id': '1', 'MessageBody': json.dumps({'id': 1})}, {'Id': '2', 'MessageBody': json.dumps({'id': 2})}, {'Id': '3', 'MessageBody': json.dumps({'id': 3})} ] )
Receive and Process Messages
import boto3 import json sqs = boto3.client('sqs') queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue' while True: # Long polling (wait up to 20 seconds) response = sqs.receive_message( QueueUrl=queue_url, MaxNumberOfMessages=10, WaitTimeSeconds=20, MessageAttributeNames=['All'], AttributeNames=['All'] ) messages = response.get('Messages', []) for message in messages: try: body = json.loads(message['Body']) print(f"Processing: {body}") # Process message... # Delete on success sqs.delete_message( QueueUrl=queue_url, ReceiptHandle=message['ReceiptHandle'] ) except Exception as e: print(f"Error processing message: {e}") # Message will become visible again after visibility timeout
Lambda Integration
# Create event source mapping aws lambda create-event-source-mapping \ --function-name my-function \ --event-source-arn arn:aws:sqs:us-east-1:123456789012:my-queue \ --batch-size 10 \ --maximum-batching-window-in-seconds 5
Lambda handler:
def handler(event, context): for record in event['Records']: body = json.loads(record['body']) message_id = record['messageId'] try: process_message(body) except Exception as e: # Raise to put message back in queue raise return {'batchItemFailures': []}
CLI Reference
Queue Management
| Command | Description |
|---------|-------------|
| aws sqs create-queue | Create queue |
| aws sqs delete-queue | Delete queue |
| aws sqs list-queues | List queues |
| aws sqs get-queue-url | Get queue URL by name |
| aws sqs get-queue-attributes | Get queue settings |
| aws sqs set-queue-attributes | Update queue settings |
Messaging
| Command | Description |
|---------|-------------|
| aws sqs send-message | Send single message |
| aws sqs send-message-batch | Send up to 10 messages |
| aws sqs receive-message | Receive messages |
| aws sqs delete-message | Delete message |
| aws sqs delete-message-batch | Delete up to 10 messages |
| aws sqs purge-queue | Delete all messages |
Visibility
| Command | Description |
|---------|-------------|
| aws sqs change-message-visibility | Change timeout |
| aws sqs change-message-visibility-batch | Batch change |
Best Practices
Message Processing
- Use long polling (WaitTimeSeconds=20) to reduce API calls
- Delete messages promptly after successful processing
- Configure appropriate visibility timeout (> processing time)
- Implement idempotent consumers for at-least-once delivery
Dead-Letter Queues
- Always configure DLQ for production queues
- Set appropriate maxReceiveCount (usually 3-5)
- Monitor DLQ depth with CloudWatch alarms
- Process DLQ messages manually or with automation
FIFO Queues
- Use message group IDs to partition ordering
- Enable content-based deduplication or provide dedup IDs
- Throughput: 300 msgs/sec without batching, 3000 with
Security
- Use queue policies to control access
- Enable encryption with SSE-SQS or SSE-KMS
- Use VPC endpoints for private access
Troubleshooting
Messages Not Being Received
Causes:
- Short polling returning empty
- All messages in flight (visibility timeout)
- Messages delayed (DelaySeconds)
Debug:
# Check queue attributes aws sqs get-queue-attributes \ --queue-url $QUEUE_URL \ --attribute-names All # Check approximate message counts aws sqs get-queue-attributes \ --queue-url $QUEUE_URL \ --attribute-names \ ApproximateNumberOfMessages,\ ApproximateNumberOfMessagesNotVisible,\ ApproximateNumberOfMessagesDelayed
Messages Going to DLQ
Causes:
- Processing errors
- Visibility timeout too short
- Consumer not deleting messages
Redrive from DLQ:
# Enable redrive allow policy on source queue aws sqs set-queue-attributes \ --queue-url $MAIN_QUEUE_URL \ --attributes '{"RedriveAllowPolicy": "{\"redrivePermission\":\"allowAll\"}"}' # Start redrive aws sqs start-message-move-task \ --source-arn arn:aws:sqs:us-east-1:123456789012:my-queue-dlq \ --destination-arn arn:aws:sqs:us-east-1:123456789012:my-queue
Duplicate Processing
Solutions:
- Use FIFO queues for exactly-once
- Implement idempotency in consumer
- Track processed message IDs in database
Lambda Not Processing
# Check event source mapping aws lambda list-event-source-mappings \ --function-name my-function # Check for errors aws lambda get-event-source-mapping \ --uuid <mapping-uuid>
References
优点
- 结构良好的文档,涵盖核心概念和模式。
- 提供实用的 CLI 和 Python (boto3) 代码示例。
- 包含针对实际使用的最佳实践和故障排除指南。
缺点
- 主要是一个文档/指南,而非具有独特逻辑的新颖“技能”。
- 实用性依赖于外部的 AWS 凭证和权限。
- 除了标准的 AWS SDK 使用外,没有提供抽象或自动化。
相关技能
免责声明:本内容来源于 GitHub 开源项目,仅供展示和评分分析使用。
版权归原作者所有 itsmostafa.
