> ## Documentation Index
> Fetch the complete documentation index at: https://langchain-5e9cc07a-preview-update-1770920925-e15dbde.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Alerts in LangSmith

<Note>
  **Self-hosted Version Requirement**

  Access to alerts requires Helm chart version **0.10.3** or later.
</Note>

## Overview

Effective observability in LLM applications requires proactive detection of failures, performance degradations, and regressions. LangSmith's alerts feature helps identify critical issues such as:

* API rate limit violations from model providers
* Latency increases for your application
* Application changes that affect feedback scores reflecting end-user experience

Alerts in LangSmith are project-scoped, requiring separate configuration for each monitored project.

## Configuring an alert

### Step 1: Navigate to create alert

First navigate to the Tracing project that you would like to configure alerts for. Click the Alerts icon on the top right hand corner of the page to view existing alerts for that project and set up a new alert.

### Step 2: Select metric type

<br />

<div style={{ textAlign: 'center' }}>
  <img src="https://mintcdn.com/langchain-5e9cc07a-preview-update-1770920925-e15dbde/4x_4Eu33iESkLFLy/langsmith/images/alert-metric.png?fit=max&auto=format&n=4x_4Eu33iESkLFLy&q=85&s=5bc3d0ec30599504a5ffa1b5269ce4fb" alt="Alert metrics" width="597" height="134" data-path="langsmith/images/alert-metric.png" />
</div>

LangSmith offers threshold-based alerting on three core metrics:

| Metric Type        | Description                         | Use Case                                                                                                                                                             |
| ------------------ | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Errored Runs**   | Track runs with an error status     | Monitors for failures in an application.                                                                                                                             |
| **Feedback Score** | Measures the average feedback score | Track [feedback from end users](/langsmith/attach-user-feedback) or [online evaluation results](/langsmith/online-evaluations-llm-as-judge) to alert on regressions. |
| **Latency**        | Measures average run execution time | Tracks the latency of your application to alert on spikes and performance bottlenecks.                                                                               |

Additionally, for **Errored Runs** and **Run Latency**, you can define filters to narrow down the runs that trigger alerts. For example, you might create an error alert filter for all `llm` runs tagged with `support_agent` that encounter a `RateLimitExceeded` error.

<div style={{ textAlign: 'center' }}>
  <img src="https://mintcdn.com/langchain-5e9cc07a-preview-update-1770920925-e15dbde/4x_4Eu33iESkLFLy/langsmith/images/alerts-filter.png?fit=max&auto=format&n=4x_4Eu33iESkLFLy&q=85&s=0d9a406bccbc2ec58261ebfd49ae3bb5" alt="Alert Metrics" width="407" height="273" data-path="langsmith/images/alerts-filter.png" />
</div>

### Step 2: Define alert conditions

Alert conditions consist of several components:

* **Aggregation Method**: Average, Percentage, or Count
* **Comparison Operator**: `>=`, `<=`, or exceeds threshold
* **Threshold Value**: Numerical value triggering the alert
* **Aggregation Window**: Time period for metric calculation (currently choose between 5 or 15 minutes)
* **Feedback Key** (Feedback Score alerts only): Specific feedback metric to monitor

<br />

<div style={{ textAlign: 'center' }}>
  <img src="https://mintcdn.com/langchain-5e9cc07a-preview-update-1770920925-e15dbde/mxFjzB3ASP1J2dsF/langsmith/images/define-conditions.png?fit=max&auto=format&n=mxFjzB3ASP1J2dsF&q=85&s=b9b031b8789c8237070f48f7de9a0d76" alt="Alert Condition Configuration" width="597" height="112" data-path="langsmith/images/define-conditions.png" />
</div>

**Example:** The configuration shown above would generate an alert when more than 5% of runs within the past 5 minutes result in errors.

You can preview alert behavior over a historical time window to understand how many datapoints—and which ones—would have triggered an alert at a chosen threshold (indicated in red). For example, setting an average latency threshold of 60 seconds for a project lets you visualize potential alerts, as shown in the image below.

<div style={{ textAlign: 'center' }}>
  <img src="https://mintcdn.com/langchain-5e9cc07a-preview-update-1770920925-e15dbde/4x_4Eu33iESkLFLy/langsmith/images/alert-preview.png?fit=max&auto=format&n=4x_4Eu33iESkLFLy&q=85&s=cb1cfa7774a0e40773d943731457ac0f" alt="Alert Metrics" width="863" height="545" data-path="langsmith/images/alert-preview.png" />
</div>

### Step 3: Configure notification channel

LangSmith supports the following notification channels:

1. [PagerDuty integration](/langsmith/alerts-pagerduty)
2. [Webhook notifications](/langsmith/alerts-webhook)

Select the appropriate channel to ensure notifications reach the responsible team members.

## Best practices

* Adjust sensitivity based on application criticality
* Start with broader thresholds and refine based on observed patterns
* Ensure alert routing reaches appropriate on-call personnel

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/langsmith/alerts.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
</Callout>

<Tip icon="terminal" iconType="regular">
  [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
</Tip>