Amazon Web Services SAP-C02 Question Answer
A retailer is deploying a customer service AI agent that uses multiple tools to look up orders, process refunds, and make address changes. The retailer needs a solution for its CI/CD pipeline that blocks promotion if a new release regresses on specific metrics. The specific metrics include task completion, tool selection, and final response quality. The retailer wants to test the metric evaluation against a set of golden interaction traces. After deployment, the solution must also continuously monitor for agent degradation and perform a periodic human audit of production results.
Which solution will meet these requirements with the LEAST operational overhead?

