An administrator is monitoring the performance of a deployed Large Language Model within the Nutanix...

Comprehensive and Detailed Explanation From Exact Extract of Nutanix Artificial Intelligence documents knowledge without any external URL or links:

The symptoms described—high CPU/memory saturation and increasing latency during peaks—indicate that the current inference infrastructure is underprovisioned for the load it is receiving.

The correct remediation in Nutanix Enterprise AI is to scale out the endpoint. By updating the endpoint configuration to increase the number of instances (replicas) or by allocating more compute resources (CPU/Memory per pod) if the model is constrained by the container size, the system can distribute the inference requests across more capacity. This parallelization directly addresses the bottleneck, lowering latency and preventing timeouts. Restarting is a temporary fix that does not solve the capacity issue, and API keys do not impact performance resources.