Enhancing Amazon Bedrock Agent Performance with AgentCore Observability

As large language models (LLMs) and intelligent agents become integral to production systems, robust observability becomes crucial for their performance, reliability, and debugging. Amazon Bedrock’s AgentCore offers comprehensive observability features designed to provide deep insights into agent behavior in live environments. This article delves into the various facets of AgentCore Observability, covering metrics, logging, and distributed tracing, all integrated seamlessly with Amazon CloudWatch.

Understanding AgentCore Observability

AgentCore Observability is a powerful toolkit that enables developers to monitor, debug, and optimize the performance of agents deployed within Amazon Bedrock. It provides detailed visualizations of each step an agent takes during its workflow, allowing for thorough inspection of execution paths, auditing of intermediate outputs, and identification of performance bottlenecks or failures.

This observability solution offers real-time visibility into operational performance through CloudWatch-powered dashboards. Key metrics like session count, latency, duration, token usage (input, output, and total), and error rates are readily available. AgentCore also enriches its telemetry data with metadata tagging and filtering capabilities, simplifying the investigation and maintenance of agents at scale. Crucially, AgentCore emits telemetry in an OpenTelemetry (OTEL)-compatible format, ensuring easy integration with existing monitoring stacks.

Beyond built-in metrics for agents, gateway resources, and memory, developers can further instrument their agent code to generate custom span and trace data, along with tailored metrics and logs, for even finer-grained control.

Deep Dive into Model Invocation Metrics

Within the CloudWatch console, under the GenAI Observability section, users can access “Model Invocations” metrics. These dashboards provide critical insights into how the underlying models are performing:

• Invocation Count: Tracks the total number of times a model has been called.
• Invocation Latency: Measures the response time of model invocations.
• Invocation Throttles: Indicates instances where model invocations were limited due to rate limits.
• Invocation Error Count: Logs the number of failures during model interactions.
• Token Count by Model: Details the number of input, output, and total tokens processed by each model, offering insights into cost and model efficiency.
• Requests Grouped by Input Token: Provides a breakdown of requests based on the volume of input tokens.

These metrics are essential for understanding model utilization, identifying performance degradations, and managing operational costs.

Enabling Model Invocation Logging

To gain deeper insights into the specific interactions with models, model invocation logging can be enabled. This feature provides detailed records of each model call, including input prompts, tool usage, and the model’s output.

To activate this:
1. Navigate to the “Model Invocations” panel in CloudWatch’s GenAI Observability.
2. Select “enable model invocation logging.”
3. Configure the settings page by enabling logging, selecting desired data types (e.g., full input/output), and choosing CloudWatch Logs as the destination.
4. Specify an existing CloudWatch Log Group (e.g., agentcore-logging) and define a new IAM role (e.g., bedrock-role) to grant Bedrock the necessary permissions.
5. Save the configuration.

Once enabled and the agent is invoked, logs will flow into the specified CloudWatch Log Group. Each log entry can be inspected to reveal the model’s input, any tools used and their results, and the final model invocation output, providing an invaluable resource for debugging and understanding agent reasoning.

CloudWatch Transaction Search for Bedrock AgentCore

For a holistic view of agent execution, enabling CloudWatch Transaction Search integrates Bedrock AgentCore traces with AWS X-Ray. This feature allows for end-to-end tracing of requests as they flow through your agent and associated services.

To enable this functionality:
1. In the CloudWatch GenAI Observability console, locate the “Bedrock AgentCore” panel and press “Configure.”
2. This will direct you to the X-Ray traces area within the “Transaction Search” panel.
3. Click “Edit,” then check “Enable Transaction Search.”
4. Optionally, adjust the “X-Ray Trace indexing” rate (note that only a percentage of spans are indexed as trace summaries for free).
5. Save the changes. The “Ingest OpenTelemetry spans” status will update and eventually show as enabled.

Integrating AWS Distro for OpenTelemetry (ADOT) SDK

To populate CloudWatch GenAI Observability with rich trace data, the AWS Distro for OpenTelemetry (ADOT) SDK is essential. ADOT is an AWS-supported, production-ready distribution of the OpenTelemetry project, offering open-source APIs, libraries, and agents for collecting distributed traces and metrics. By instrumenting applications once with ADOT, developers can send correlated metrics and traces to various AWS monitoring solutions.

When deploying an agent using the Amazon Bedrock AgentCore Runtime starter toolkit, ADOT integration is largely automated. The generated Dockerfile typically includes:

• Installation of aws-opentelemetry-distro (e.g., pip install aws-opentelemetry-distro>=0.10.1).
• Instrumentation of the agent with opentelemetry-instrument (e.g., CMD ["opentelemetry-instrument", "python", "-m", "agentcore_runtime_demo"]).

This automatic integration ensures that the agent’s execution path is automatically captured and sent to CloudWatch for analysis.

Bedrock AgentCore: Agents, Sessions, and Traces View

After invoking the agent multiple times, the “Bedrock AgentCore” panel in CloudWatch GenAI Observability provides detailed views:

• Agents View: Offers an overview of an agent’s general metrics, including the number of sessions and traces, error rates, and throttle rates.
• Sessions View: Presents a list of all agent sessions, each representing a single conversation or interaction with the agent.
• Traces View (Linked from Sessions): Clicking on a session reveals its associated traces. Each trace captures the full lifecycle of a request within that session, broken down into granular spans.

Within the trace view, developers can explore:

• Spans: A hierarchical representation of individual operations performed by the agent. This might include invoking /invocations endpoints, authenticating with Cognito, interacting with the AgentCore Gateway, the agent’s internal event loop, and calls to specific tools or models (e.g., Amazon Nova Pro Model invoking a getOrdersByCreatedDates tool). Each span displays its latency, and clicking on it reveals the complete payload.
• Timeline View: A visual representation of spans arranged chronologically, showing the duration of each operation within the agent invocation chain. This is invaluable for identifying where time is spent.
• Trajectory View: Provides a high-level flow diagram of the agent invocation. This view is particularly useful for quickly pinpointing errors, as failed spans are highlighted in red, allowing for rapid identification of problem areas within the agent’s execution path.

Conclusion

AgentCore Observability in Amazon Bedrock is a comprehensive suite of tools designed to ensure the robust performance and reliability of AI agents. By integrating with CloudWatch for detailed metrics and logs, and leveraging AWS Distro for OpenTelemetry for end-to-end tracing with X-Ray, developers gain unparalleled visibility into agent behavior. These capabilities empower teams to trace, debug, and monitor agent performance effectively, leading to more resilient and efficient AI applications. The ability to track model invocations, examine detailed logs, and visualize complex agent workflows through spans, timelines, and trajectories makes AgentCore Observability an indispensable part of developing and managing production-ready Bedrock agents.