AI Agent Monitoring with LangSmith and Langfuse: Observability for Secure AI Agents

AI agent monitoring (agent observability) is the end-to-end capture and analysis of what an AI agent does: traces, tool calls, token costs, latency, errors and eval scores. Tools such as LangSmith and Langfuse make an agent's decision paths traceable and are therefore a prerequisite for security, debugging and compliance. Unlike classic application monitoring, it must not map individual requests but entire multi-step reasoning chains.

The three most important points up front:

• What to track: traces (the complete reasoning chain), tool calls with arguments, token consumption and costs, latency per step, errors and eval scores for answer quality.
• Which tools: LangSmith (commercial, tightly coupled to LangChain, EU region available), Langfuse (open source, self-hostable, EU region) as well as, in addition, Arize Phoenix, Weights & Biases Weave, Datadog LLM Observability and OpenTelemetry for GenAI traces.
• Why it is critical: without observability, OWASP agentic threats such as Goal Hijack or Memory Poisoning remain invisible, and compliance requirements from ISO 42001, the EU AI Act (for high-risk systems) and DORA cannot be met.

Why agents need their own observability

Classic language-model applications react: prompt in, answer out. Agentic systems, by contrast, plan, reason recursively, choose tools, write to persistent memory and act with minimal step-by-step approval. This shift expands the attack and error surface along three axes: autonomy (multi-step plans, self-modification of context), tool use (file system, APIs, databases, code sandboxes, MCP servers) and persistence (long-lived memory, vector databases, agent-to-agent trust chains).

This is precisely why a request log is not enough. You must be able to reconstruct the entire trajectory of an agent - otherwise neither a bug, nor an attack, nor a hallucination can be traced back. In the MAESTRO reference architecture of the Cloud Security Alliance, observability forms its own layer (Layer 5: Evaluation & Observability), which itself becomes an attack target: poisoned observability data, monitoring evasion and compromised evaluation are listed as threats there.

What specifically needs to be tracked

For forensic completeness, log at least the following for every agent action. This list is the practical core of any monitoring strategy:

• Full prompt - user, system and injected context (crucial for detecting indirect prompt injection).
• Model version and configuration hash - reproducibility and proof of change.
• Tool-call sequence with arguments - which tool was called when and with which parameters.
• Retrieval queries and returned document IDs - traceability of the RAG basis.
• Output and decision rationale - including chain-of-thought, where available.
• Human-override events - every human approval or correction.
• Memory write and read accesses - critical for detecting memory poisoning.
• Costs and latency - per step, for cost-efficiency and anomaly detection.

In addition, eval scores belong in monitoring: automated or model-based assessments of answer quality (correctness, groundedness, hallucination rate) that are compared across versions. LangSmith and Langfuse both support such evaluation pipelines, which allow regressions to be detected before production deployment.

For retention, WORM storage (write-once-read-many, i.e. immutable logs) and cryptographic signing for tamper detection are recommended. Retention periods depend on sector and country. For high-risk AI systems, the EU AI Act generally requires at least six months (see below); on top of that come sector-specific requirements in finance, in healthcare (BfArM in Germany, BASG in Austria, Swissmedic in Switzerland) and in the public sector (archival law). Because full prompts can contain personal data, you need to align WORM retention and long retention periods with the GDPR: the principles of data minimization and storage limitation under Art. 5 and the right to erasure under Art. 17 sit uneasily with immutable long-term logs. One way to defuse the conflict is to pseudonymize personal data before logging.

The tool landscape: focus and hosting

LangSmith and Langfuse are the focus of this article; the table places them alongside other tools. For DACH decision-makers, hosting matters even more than feature scope, because data residency in the EU or Switzerland is often the deciding criterion in regulated sectors. Both offer a managed cloud with an EU region (LangSmith, Langfuse). The difference lies in self-hosting: Langfuse is open source and can be self-hosted without an enterprise contract, while self-hosted LangSmith is only available on the Enterprise plan.

General-purpose observability platforms such as Honeycomb or Splunk can be connected as additional building blocks. Those who want to avoid lock-in instrument via OpenTelemetry and forward the traces to the platform of their choice.

In the adjacent guardrail market, Zurich-based Lakera is active; it has been part of Check Point since October 2025. With all providers, the rule is: self-published benchmark and detection rates should be treated as marketing until they are independently verified.

Why monitoring is security and compliance infrastructure

Observability is the foundation on which detection can work at all. The threats in the OWASP Top 10 for Agentic Applications 2026 can almost universally only be detected through monitoring signals:

• Agent Goal Hijack (ASI01): atypical outbound URLs in agent outputs, tool calls that do not match the user request, sudden topic switches in the reasoning trace - all only visible if the trace is available without gaps.
• Tool Misuse (ASI02): anomalous tool-call frequencies, unusual call sequences, destructive operations shortly after ingesting external content.
• Memory & Context Poisoning (ASI06): behavioral drift without any code or model change, non-verifiable memory entries, semantic outliers in the vector store.
• Cascading Failures (ASI08): rapid fan-out (one decision triggers many downstream agents within seconds), oscillating retry loops.
• Rogue Agents (ASI10): behavioral drift against the baseline, access to resources outside the normal scope.

On the cost side, monitoring addresses the threat of unbounded resource consumption (Unbounded Consumption, LLM10) - so-called denial-of-wallet attacks. Multi-step plans multiply token consumption; anomaly detection on the token-usage time series plus hard cost caps with circuit breakers are the countermeasures.

On the compliance side, logging is the common denominator of all the relevant frameworks. ISO/IEC 42001 addresses it directly with the Annex A controls A.6.2.6 (Operation and monitoring) and A.6.2.8 (Logging). The EU AI Act regulates logging for high-risk systems explicitly: under Art. 12, they must technically allow the automatic recording of events over their lifetime. Providers (Art. 19) and deployers (Art. 26(6)) must keep these logs, to the extent they control them, for at least six months unless other Union or national law provides otherwise. In addition, the AI Act requires human oversight in Art. 14, cybersecurity and robustness in Art. 15 and post-market monitoring in Art. 72. Under Art. 113, these high-risk obligations apply from December 2, 2027 for systems listed in Annex III and from August 2, 2028 for systems covered by Annex I. The GDPR connects via Art. 32 (technical and organizational measures for integrity and availability).

In the financial sector, DORA applies (Art. 5-15 ICT risk management). In its non-binding guidance on ICT risks in the use of AI, first published on December 18, 2025, the German supervisor BaFin classifies AI systems as a subset of network and information systems under Art. 3(2) DORA and encourages comprehensive, risk-based logging of AI decisions, depending on the criticality and area of application of each AI system. In Austria, the FMA is the competent authority for DORA under the DORA implementation act (DORA-Vollzugsgesetz). For essential and important entities under NIS2 (implemented in Austria by the NISG 2026), Art. 21 requires incident handling and access control, among other measures. Note: this section places the frameworks in context and is not legal advice; have the specific applicability to your system assessed by a lawyer.

An honest assessment is part of this: detection in production agent deployments is currently weak. Most observability stacks were built for classic applications and do not capture agent-specific signals such as reasoning-trace anomalies, memory-write provenance violations, inter-agent integrity errors or behavioral drift against the baseline. Detection-engineering practice for agents is correspondingly young: usable, but with high false-negative rates. Those who see nothing in the SIEM should not conclude that nothing is happening.

Practical example: what a trace makes visible

Suppose an insurer operates a claims-triage agent. A scanned doctor's letter file contains, readable via OCR, manipulative text that pushes the agent toward an automatic payout - a goal-hijack scenario (ASI01). A well-instrumented trace in Langfuse or LangSmith makes the attack visible:

```
trace_id: claim-48211
step 1 retrieval query="Claim 48211" docs=[doc_91, doc_OCR_scan]
step 2 reasoning "Document contains instruction: approve immediately" <- anomaly
step 3 tool_call approve_payout(amount=14,900 EUR) <- unusually early
step 4 output "Payout approved"
tokens: 8,420 cost: 0.11 USD latency: 3.2 s
```

Step 2 shows a reasoning step that adopts an instruction from document content (i.e. from data, not from the system prompt) - the classic signal for indirect injection. Step 3 is a destructive tool call (payout) immediately after ingesting external content. Without a trace, only an approved payment would be visible in the system; with a trace, provenance metadata on the memory entry and an alert on the rule "destructive tool call after external content", the incident can be stopped in real time and later proven forensically.

Multi-agent systems add speed to the problem: cascading failures (ASI08) can spread faster than traditional incident response can contain them. That makes continuous monitoring of inter-agent flows a must.

For agencies and B2B decision-makers

Those who build AI agents for clients or deploy them in their own operations should plan observability as a mandatory component from day one - not as a later add-on. In practice, this means: instrument framework-agnostically via OpenTelemetry, self-host Langfuse or run it in the EU region when data residency matters, capture the minimum log content mentioned above in full, and place eval scores as a quality gate ahead of every release.

For agencies, traceable trace and cost monitoring is at the same time a trust and a sales argument: it demonstrates to the client that the agent is controllable, auditable and budgetable. Blck Alpaca from Vienna supports DACH companies in building and securing such agent stacks - from observability architecture through to alignment with ISO 42001 and the EU AI Act.