Amazon OpenSearch Service now brings software monitoring, native Amazon Managed Service for Prometheus integration, and AI agent tracing collectively in OpenSearch UI‘s observability workspace. You may question Prometheus metrics with PromQL alongside logs and traces saved in Amazon OpenSearch Service, hint an AI agent’s full reasoning chain right down to the failing device name, and drill from a service-level well being view to the precise span that precipitated a checkout failure, all with out leaving the interface.
On this publish, we stroll via two real-world eventualities utilizing the OpenTelemetry pattern app: a multi-agent journey planner dealing with gradual processing, and a checkout move quietly failing on one microservice. We chase each to its root trigger utilizing these new capabilities.
State of affairs 1: An underperforming AI agent
Your multi-agent journey planner is stay and customers begin reporting gradual responses. With the brand new AI agent tracing functionality in Amazon OpenSearch Service, you’ll be able to hint the agent’s full processing path to pinpoint precisely the place issues went mistaken.
In any observability workspace in OpenSearch UI, navigate to Software Map within the left navigation pane.
You may see the complete topology of your system together with the journey agent and the sub-agents it calls. The journey agent node exhibits elevated latency and occasional errors. Choose it, and the aspect panel confirms that latency is up however the latency chart exhibits intermittent spikes moderately than constant degradation.
The appliance map tells you one thing is mistaken, however understanding why an AI agent is underperforming requires seeing its reasoning chain. Choose Agent Traces within the left navigation pane, then filter by service title and time vary.
Choose one of many traces to see the hint tree. Not like a conventional span waterfall, this view organizes across the agent’s reasoning chain: the foundation agent span, the LLM calls it made, the instruments it invoked, and the way they nested every step color-coded by kind. The hint map offers a visible directed graph of the identical execution. You may see which mannequin was known as, what number of enter and output tokens had been consumed, and the precise messages despatched to and acquired from the mannequin.
A device name contained in the climate agent errored out. The agent then spent extra time reasoning concerning the failure earlier than returning a partial response explaining the intermittent latency spikes and occasional faults.
Why this issues for AI brokers
Brokers make autonomous selections primarily based on LLM responses, device outcomes, and chained reasoning. Not like conventional microservices with deterministic code paths, agent habits varies throughout executions. With out semantic tracing that captures these AI-specific indicators, root-cause evaluation is guesswork. The hint tree surfaced the mannequin title, token counts, and failing device name as a result of the journey planner was instrumented with OpenTelemetry’s generative AI semantic conventions. The subsequent part describes how.
Instrumenting AI brokers
OpenTelemetry auto-instrumentation enriches spans with well-known attributes for HTTP, database, and gRPC calls. AI brokers want a unique set of attributes akin to which LLM was known as, what tokens had been consumed, which instruments had been invoked, that commonplace instrumentation doesn’t cowl.
The OpenTelemetry gen_ai semantic conventions outline commonplace attributes for these indicators, together with gen_ai.operation.title, gen_ai.utilization.input_tokens, gen_ai.request.mannequin, and gen_ai.device.title. When Amazon OpenSearch Service receives spans with these attributes, it categorizes them by operation kind (agent, LLM, device, embeddings, retrieval) and renders the agent hint tree and hint map views.
The Python SDK offers one technique to generate these spans. To ship traces to Amazon OpenSearch Ingestion, configure the SDK with AWS Signature Model 4 (SigV4) authentication. The AWSSigV4OTLPExporter cryptographically indicators every HTTP request to assist forestall unauthorized knowledge ingestion. The calling id wants an IAM coverage that grants osis:Ingest in your pipeline’s ARN. Credentials are resolved via the usual AWS credential supplier chain.
from opensearch_genai_observability_sdk_py import register, AWSSigV4OTLPExporter
exporter = AWSSigV4OTLPExporter(
endpoint="https://pipeline.us-east-1.osis.amazonaws.com/v1/traces",
service="osis",
area="us-east-1",
)
register(service_name="my-agent", exporter=exporter)
Use the @observe decorator to hint agent capabilities and enrich() so as to add mannequin metadata:
@observe(op=Op.EXECUTE_TOOL)
def get_weather(metropolis: str) -> dict:
return {"metropolis": metropolis, "temp": 22, "situation": "sunny"}
@observe(op=Op.INVOKE_AGENT)
def assistant(question: str) -> str:
enrich(mannequin="gpt-4o", supplier="openai")
knowledge = get_weather("Paris")
return f"{knowledge['condition']}, {knowledge['temp']}C"
outcome = assistant("What is the climate?")
The SDK additionally helps auto-instrumentation for OpenAI, Anthropic, Amazon Bedrock, LangChain, LlamaIndex, and others. As a result of the instrumentation is constructed on OpenTelemetry requirements, any agent framework that emits spans with gen_ai.* attributes is appropriate with OpenSearch UI.
State of affairs 2: Investigating a microservice difficulty
AI brokers are just one a part of most manufacturing environments. The identical interface surfaces telemetry from typical microservices, the place the troubleshooting workflow follows a extra acquainted path.
Your ecommerce checkout begins paging throughout a busy visitors window. From OpenSearch UI, navigate to APM Providers within the left navigation pane. Each instrumented service is listed alongside its well being indicators. The checkout service exhibits an elevated error charge.
Choose the affected service. The element view exhibits Request, Error, and Length (RED) metrics: request charge is climbing, fault charge has spiked within the final quarter-hour, and p99 length has doubled. You may see precisely when the degradation began.
Drill into the correlated spans for the affected time window. The span checklist exhibits a number of failed requests, all hitting the identical endpoint. Choose one to see the complete hint waterfall. The checkout service known as prepareOrder, which failed attempting to retrieve a product from the catalog. The error message within the span particulars tells you precisely what went mistaken, that’s your root trigger.
Checking the infrastructure with PromQL
In each eventualities, the pure subsequent query is whether or not the issue originates within the software or within the infrastructure beneath it. With the brand new Amazon Managed Service for Prometheus integration, you’ll be able to reply that query with out leaving OpenSearch UI.
Prometheus metrics at the moment are queryable instantly from the identical workspace utilizing native PromQL syntax, alongside the logs and traces you’ve already been navigating.
For the database timeout in State of affairs 2, run a PromQL question to examine the database occasion’s learn/write throughput for a similar time window. For the agent latency difficulty in State of affairs 1, examine the LLM endpoint’s response time metrics to see if the slowness originates from the mannequin supplier.
This can be a key architectural resolution: metrics proceed to stay in Amazon Managed Service for Prometheus, logs and traces proceed to stay in Amazon OpenSearch Service, and neither sign is copied or warehoused right into a second retailer. Every backend stays the only retailer for the info kind it’s purpose-built to deal with, whereas OpenSearch UI federates queries throughout each at runtime. The fee, retention, and operational mannequin of every retailer keep intact whereas the troubleshooting workflow collapses right into a single interface.
To configure the OpenTelemetry Collector and OpenSearch Ingestion pipelines that route metrics into Amazon Managed Service for Prometheus, see Ingesting software telemetry.
The way it’s wired collectively
The next diagram exhibits the end-to-end structure. Purposes instrumented with OpenTelemetry ship traces, logs, and metrics over OTLP to Amazon OpenSearch Ingestion. OpenSearch Ingestion routes every sign to the suitable retailer: traces and logs land in Amazon OpenSearch Service, whereas metrics move into Amazon Managed Service for Prometheus. OpenSearch UI then queries each shops to render the Software Map, Providers catalog, Agent Traces, and Metrics views.
Your entire expertise rests on open-source foundations, Prometheus for metrics, OpenSearch for logs and traces, and OpenTelemetry for instrumentation, so groups already operating an OpenTelemetry collector can undertake it by updating the collector’s export configuration to level at Amazon OpenSearch Ingestion, with no proprietary brokers or rewritten instrumentation required.
Getting began
To allow these capabilities, log in to OpenSearch UI’s observability workspace, choose the Gear icon within the backside left nook to open Settings and setup, and confirm that the Observability:apmEnabled toggle is on beneath the Observability part. OpenSearch UI is accessible at no extra cost for Amazon OpenSearch Service clients.
Discover domestically first. The OpenSearch Observability Stack offers you a totally configured surroundings together with software monitoring, agent tracing, and Prometheus integration, operating in your machine with a single set up command. It ships with pattern instrumented companies, together with a multi-agent journey planner, so you’ll be able to discover the complete workflow with actual telemetry knowledge out of the field.
For AI agent improvement. Agent Well being is an open-source, evaluation-driven observability device designed for native improvement. It offers you execution move graphs, token monitoring, and power invocation visibility proper in your improvement loop, earlier than you push to manufacturing.
For manufacturing. The Python SDK offers one-line setup and decorator-based tracing with gen_ai semantic conventions, with auto-instrumentation assist for OpenAI, Anthropic, Amazon Bedrock, LangChain, LlamaIndex, and others. See the Amazon OpenSearch Service documentation and the Amazon Managed Service for Prometheus integration information for the complete managed expertise.
In regards to the authors
