# `ConduitMcp.PromEx`
[🔗](https://github.com/nyo16/conduit_mcp/blob/v0.11.0/lib/conduit_mcp/prom_ex.ex#L2)

A PromEx plugin for monitoring ConduitMCP operations.

This plugin captures metrics from the ConduitMCP library's telemetry events:
- Request metrics (all MCP method calls)
- Tool execution metrics
- Resource read metrics
- Prompt retrieval metrics
- HTTP and message rate limit metrics
- Authentication metrics
- Cancellation metrics
- Janitor cleanup metrics

Every event in `ConduitMcp.Telemetry.events/0` feeds at least one metric.

## Installation

Add `:prom_ex` to your dependencies:

    def deps do
      [
        {:conduit_mcp, "~> 0.9.0"},
        {:prom_ex, "~> 1.11"}
      ]
    end

## Usage

Add this plugin to your PromEx module's plugins list:

    defmodule MyApp.PromEx do
      use PromEx, otp_app: :my_app

      @impl true
      def plugins do
        [
          PromEx.Plugins.Application,
          PromEx.Plugins.Beam,
          {ConduitMcp.PromEx, otp_app: :my_app}
        ]
      end

      @impl true
      def dashboard_assigns do
        [
          datasource_id: "prometheus",
          default_selected_interval: "30s"
        ]
      end
    end

Then add your PromEx module to your supervision tree:

    def start(_type, _args) do
      children = [
        MyApp.PromEx,
        # ... other children ...
      ]

      Supervisor.start_link(children, strategy: :one_for_one)
    end

## Configuration Options

- `:otp_app` (required) - Your application name
- `:duration_unit` (optional) - Unit for duration metrics (default: `:millisecond`)

## Metrics Exposed

All metrics are prefixed with `{otp_app}_conduit_mcp_`.

### Request Metrics

- `{prefix}_request_total` - Counter of MCP requests
  - Tags: `method` (e.g., "tools/list"), `status` (:ok | :error)
- `{prefix}_request_duration_milliseconds` - Distribution of request durations
  - Tags: `method`, `status`
  - Buckets: [10, 50, 100, 250, 500, 1_000, 2_500, 5_000, 10_000]

### Tool Metrics

- `{prefix}_tool_execution_total` - Counter of tool executions
  - Tags: `tool_name`, `status`
- `{prefix}_tool_duration_milliseconds` - Distribution of tool execution durations
  - Tags: `tool_name`, `status`
  - Buckets: [10, 50, 100, 500, 1_000, 5_000, 10_000, 30_000]

### Resource Metrics

- `{prefix}_resource_read_total` - Counter of resource reads
  - Tags: `status`
- `{prefix}_resource_read_duration_milliseconds` - Distribution of resource read durations
  - Tags: `status`
  - Buckets: [10, 50, 100, 500, 1_000, 5_000]

### Prompt Metrics

- `{prefix}_prompt_get_total` - Counter of prompt retrievals
  - Tags: `prompt_name`, `status`
- `{prefix}_prompt_get_duration_milliseconds` - Distribution of prompt retrieval durations
  - Tags: `prompt_name`, `status`
  - Buckets: [10, 50, 100, 500, 1_000]

### Rate Limit Metrics

- `{prefix}_rate_limit_check_total` - Counter of HTTP rate limit checks
  - Tags: `status` (:allow | :deny)
- `{prefix}_rate_limit_check_duration_milliseconds` - Distribution of check durations
  - Tags: `status`
  - Buckets: [1, 5, 10, 25, 50, 100, 250]

### Message Rate Limit Metrics

- `{prefix}_message_rate_limit_check_total` - Counter of message rate limit checks
  - Tags: `status` (:allow | :deny), `method` (MCP method name)
- `{prefix}_message_rate_limit_check_duration_milliseconds` - Distribution of check durations
  - Tags: `status`, `method`
  - Buckets: [1, 5, 10, 25, 50, 100, 250]

### Authentication Metrics

- `{prefix}_auth_verify_total` - Counter of authentication attempts
  - Tags: `strategy` (:bearer_token | :api_key | :function), `status`
- `{prefix}_auth_verify_duration_milliseconds` - Distribution of auth verification durations
  - Tags: `strategy`, `status`
  - Buckets: [1, 5, 10, 25, 50, 100, 250]

### Cancellation Metrics

- `{prefix}_request_cancelled_total` - Counter of recorded
  `notifications/cancelled` (no tags: request ids, scopes and reasons
  are unbounded or client-supplied)
- `{prefix}_cancellation_cleanup_removed_total` - Sum of expired
  cancellation rows removed
- `{prefix}_cancellation_cleanup_in_flight_removed_total` - Sum of
  in-flight rows of dead request processes removed
- `{prefix}_cancellation_janitor_runs_total` - Counter of cancellation
  janitor runs (its `removed` duplicates the cleanup sum, so it is
  counted, not summed)

### Cleanup Metrics

- `{prefix}_session_cleanup_removed_total` - Sum of expired sessions evicted
  - Tags: `store`
- `{prefix}_tasks_cleanup_removed_total` - Sum of terminal tasks evicted
- `{prefix}_ext_tasks_cleanup_removed_total` - Sum of expired
  tasks-extension tasks evicted
  - Tags: `store`

### MRTR Metrics

- `{prefix}_mrtr_input_required_total` - Counter of `InputRequiredResult`s
  sent
  - Tags: `method` (`tools/call`, `resources/read`, `prompts/get`)
- `{prefix}_mrtr_input_requests_total` - Sum of input requests carried by
  them
  - Tags: `method`

### Client Metrics

From `ConduitMcp.Client`. Tool names and notification methods come from
the server, and client names may be per-tenant, so none of them is a tag.

- `{prefix}_client_request_start_total`,
  `{prefix}_client_request_exception_total` - Counters of requests sent
  and of requests whose caller raised
  - Tags: `method`
- `{prefix}_client_request_total` - Counter of finished requests
  - Tags: `method`, `status`, `error_kind`
- `{prefix}_client_request_duration_milliseconds` - Distribution of
  request durations
  - Tags: `method`, `status`
- `{prefix}_client_tool_call_start_total`,
  `{prefix}_client_tool_call_exception_total` - Counters of tool calls
- `{prefix}_client_tool_call_total` - Counter of finished tool calls
  - Tags: `status`, `error_kind`, `is_error`
- `{prefix}_client_tool_call_duration_milliseconds` - Distribution of tool
  call durations
  - Tags: `status`
- `{prefix}_client_connection_transition_total` - Counter of connection
  state changes
  - Tags: `to`, `reason`
- `{prefix}_client_request_cancel_total` - Counter of requests the client
  cancelled
  - Tags: `reason`
- `{prefix}_client_response_dropped_total` - Counter of late responses
  dropped
- `{prefix}_client_notification_total` - Counter of server notifications
  received
- `{prefix}_client_cache_refresh_start_total` - Counter of cache
  refreshes started
  - Tags: `kind`
- `{prefix}_client_cache_refresh_total` - Counter of finished cache
  refreshes
  - Tags: `kind`, `status`, `error_kind`
- `{prefix}_client_cache_refresh_duration_milliseconds` - Distribution of
  cache refresh durations
  - Tags: `kind`, `status`
- `{prefix}_client_tool_rejected_total` - Counter of listed tools
  dropped at intake
  - Tags: `reason`
- `{prefix}_client_tool_definition_changed_total` - Counter of known
  tools whose definition changed
- `{prefix}_client_policy_decision_total` - Counter of policy
  decisions on governed calls, reads and gets
  - Tags: `target`, `decision`, `source`, `reason`, `approval`
- `{prefix}_client_hook_total` - Counter of hook runs that returned
  within their contract
  - Tags: `kind`, `status`
- `{prefix}_client_hook_duration_milliseconds` - Distribution of hook
  durations
  - Tags: `kind`
- `{prefix}_client_hook_exception_total` - Counter of failed hook runs
  - Tags: `kind`, `reason`
- `{prefix}_client_hub_server_failed_total` - Counter of hub server
  clients that exited
  - Tags: `reason`
- `{prefix}_client_hub_server_restarted_total` - Counter of hub servers
  started again after a failure
- `{prefix}_client_hub_collision_total` - Counter of exposed tool names
  left out because several tools mangled to them
- `{prefix}_client_adapter_strict_fallback_total` - Counter of OpenAI
  tools sent non-strict because their schema has no strict form
  - Tags: `api`, `reason`
- `{prefix}_client_oauth_{op}_start_total`, `{prefix}_client_oauth_{op}_total`,
  `{prefix}_client_oauth_{op}_exception_total` - Counters of the
  client's OAuth operations, `op` one of `discovery`, `registration`,
  `token_exchange`, `refresh`, `step_up`
  - Tags (`_total` only): `status`, `reason`
- `{prefix}_client_oauth_authorization_required_total` - Counter of
  times the user had to authorize
  - Tags: `reason`

### Subscription Metrics

- `{prefix}_subscription_start_total` - Counter of `subscriptions/listen`
  streams opened
- `{prefix}_subscription_stop_total` - Counter of streams ended
  - Tags: `reason` (`client_closed`, `max_lifetime`, `token_expired`,
    `idle_timeout`)
- `{prefix}_subscription_duration_milliseconds` - Distribution of how long
  streams stayed open
  - Tags: `reason`
  - Buckets: [1 000, 10 000, 60 000, 300 000, 900 000, 1 800 000, 3 600 000]
- `{prefix}_subscription_drop_total` - Counter of notifications skipped
  for a stream whose mailbox reached `ConduitMcp.Subscriptions.max_queue/0`
  - Tags: `topic` (`tools`, `prompts`, `resources`, `resource`, `task`)

## PromQL Examples

### Request Rate by Method

    rate({otp_app}_conduit_mcp_request_total[5m])

### Error Rate Percentage

    100 * (
      rate({otp_app}_conduit_mcp_request_total{status="error"}[5m])
      /
      rate({otp_app}_conduit_mcp_request_total[5m])
    )

### P95 Request Duration

    histogram_quantile(0.95,
      rate({otp_app}_conduit_mcp_request_duration_milliseconds_bucket[5m])
    )

### Slow Tool Executions (>5s)

    histogram_quantile(0.95,
      rate({otp_app}_conduit_mcp_tool_duration_milliseconds_bucket[5m])
    ) > 5000

### Authentication Success Rate

    100 * (
      rate({otp_app}_conduit_mcp_auth_verify_total{status="ok"}[5m])
      /
      rate({otp_app}_conduit_mcp_auth_verify_total[5m])
    )

## Alert Examples

### High Error Rate

    - alert: ConduitMcpHighErrorRate
      expr: |
        100 * (
          rate(myapp_conduit_mcp_request_total{status="error"}[5m])
          /
          rate(myapp_conduit_mcp_request_total[5m])
        ) > 5
      for: 5m
      annotations:
        summary: "High error rate in ConduitMCP ({{ $value }}%)"

### Slow Tool Executions

    - alert: ConduitMcpSlowTools
      expr: |
        histogram_quantile(0.95,
          rate(myapp_conduit_mcp_tool_duration_milliseconds_bucket[5m])
        ) > 5000
      for: 10m
      annotations:
        summary: "Tool executions are slow (p95: {{ $value }}ms)"

### Authentication Failures

    - alert: ConduitMcpAuthFailures
      expr: |
        rate(myapp_conduit_mcp_auth_verify_total{status="error"}[5m]) > 0.1
      for: 5m
      annotations:
        summary: "Authentication failures detected"

## Cardinality Considerations

This plugin is designed to minimize metric cardinality:

- ✅ LOW: `method` (limited set of MCP methods)
- ✅ LOW: `status` (only :ok or :error)
- ✅ LOW: `strategy` (only 3 auth strategies)
- ✅ LOW: `tool_name` (user-defined but typically limited)
- ✅ LOW: `prompt_name` (user-defined but typically limited)
- ❌ HIGH: `uri` is NOT included (unbounded)
- ❌ HIGH: `server_module` is NOT included (not useful)
- ❌ HIGH: the rate limit `key`, cancellation `request_id`, `scope` and
  `reason`, and subscription `filters`, are NOT included (unbounded,
  client-supplied, or a list)

All string values are normalized to prevent cardinality explosion.

---

*Consult [api-reference.md](api-reference.md) for complete listing*