Summer Sale Special Limited Time 65% Discount Offer - Ends in 0d 00h 00m 00s - Coupon code: ac4s65

A retailer is deploying a customer service AI agent that uses multiple tools to look...

A retailer is deploying a customer service AI agent that uses multiple tools to look up orders, process refunds, and make address changes. The retailer needs a solution for its CI/CD pipeline that blocks promotion if a new release regresses on specific metrics. The specific metrics include task completion, tool selection, and final response quality. The retailer wants to test the metric evaluation against a set of golden interaction traces. After deployment, the solution must also continuously monitor for agent degradation and perform a periodic human audit of production results.

Which solution will meet these requirements with the LEAST operational overhead?

A.

Build custom checks by using Amazon CloudWatch Logs and AWS Lambda to parse tool calls and response text. Use weekly manual reviews to decide whether the deployed AI agent meets performance standards.

B.

Use an Amazon Bedrock model evaluation job on the underlying foundation model (FM) by using prompt-response pairs. Promote the release if the model ' s helpfulness and correctness scores improve over the previous release.

C.

Configure agent trace logging. Add Amazon Bedrock AgentCore Evaluations to the pipeline for on-demand evaluation by using built-in and custom evaluators. Enable online evaluation after deployment. Periodically review a sampled subset of sessions.

D.

Use Amazon Bedrock AgentCore Evaluations in online mode after deployment by using built-in evaluators. Use post-deployment rollback if the evaluation detects regression.

SAP-C02 PDF/Engine
  • Printable Format
  • Value of Money
  • 100% Pass Assurance
  • Verified Answers
  • Researched by Industry Experts
  • Based on Real Exams Scenarios
  • 100% Real Questions
buy now SAP-C02 pdf
Get 65% Discount on All Products, Use Coupon: "ac4s65"