A media company hosts its video processing workload on AWS.

This solution addresses the scalability needs of the workload while preventing failed processing attempts due to resource constraints.

Amazon S3 event notificationscan be used to trigger a message to an SQS queue whenever a new video is uploaded.

The existing Auto Scaling group of EC2 instances can poll the SQS queue, ensuring that the EC2 instances only process videos when there is a job in the queue.

SQS decouplesthe video upload and processing steps, allowing the system to handle sudden spikes in video uploads without overloading EC2 instances.

The use ofAuto Scalingensures that the EC2 instances can scale in or out based on the demand, maintaining cost efficiency while avoiding processing failures due to insufficient resources.

AWS References:

S3 Event Notificationsdetails how to configure notifications for S3 events.

Amazon SQSis a fully managed message queuing service that decouples components of the system.

Auto Scaling EC2explains how to manage automatic scaling of EC2 instances based on demand.

Why the other options are incorrect:

A. AWS Lambda functions: While Lambda can handle some workloads, video processing is often resource-intensive and long-running, making EC2 a more suitable solution.

B. Using SNS with Lambda: Similar to A, Lambda is not ideal for large-scale video processing due to its time and memory limitations.

D. AWS Step Functions: While a valid orchestration solution, this introduces more complexity and overhead compared to the simpler SQS-based solution.