Argosvix
Docs menuGuides

Guides

How-to guides for the main features, organized by use case. For individual API signatures, see the REST API Reference or MCP Server pages.

Sections:

  1. Alerts & webhooks
  2. Prompt management
  3. Eval criteria & runs
  4. Annotations
  5. LLM budget management
  6. Autonomous AI ops
  7. Framework-based instrumentation

1. Alerts & webhooks

Setup flow

  1. Open dashboard.argosvix.com/alerts.
  2. Click "+ New alert". Pick a preset from the gallery (11 templates: monthly cost, error rate, anomaly detection, etc.) or build one from scratch.
  3. Pick one or more notification channels (webhook / Slack / Discord / Teams / PagerDuty / Email).

Test webhook delivery

Before registering a webhook URL in a production alert, probe it (Pro+):

curl -X POST \
  -H "Authorization: Bearer $ARGOSVIX_API_KEY" \
  -H "Content-Type: application/json" \
  https://ingest.argosvix.com/v1/alerts/test-webhook \
  -d '{"url":"https://example.com/hook","alertName":"setup test"}'

From Claude Desktop / Cursor via MCP, the test_webhook tool gives the same.

⚠️ Webhook URLs must be HTTPS. Private / loopback / cloud metadata IPs are rejected (SSRF defense). Rate-limited to 5 / min per account.

Verify signatures on the receiver side

When the alert has a secret, the webhook delivery carries HMAC-SHA256 signature headers:

X-Argosvix-Signature: sha256=<hex>
X-Argosvix-Timestamp: 2026-06-03T11:22:33.456Z

Receiver HMAC-SHA256s ${timestamp}.${rawBody} with the secret and compares with the header. We recommend rejecting timestamps more than 5 min off server time for replay defense.

Silence and acknowledge

  • silence_alert / unsilence_alert: Mute an alert for a duration (default 24h).
  • acknowledge_alert: Mark a specific firing event as handled. Orthogonal to silence.

2. Prompt management

Manage prompts by name + version (Pro+). The same name can coexist across multiple versions.

Register a new version

curl -X POST \
  -H "Authorization: Bearer $ARGOSVIX_API_KEY" \
  -H "Content-Type: application/json" \
  https://ingest.argosvix.com/v1/prompts \
  -d '{
    "name": "customer_support",
    "version": "v1",
    "template": "You are a polite customer support agent. Question: {{question}}",
    "variables": { "question": "sample" },
    "labels": ["production"],
    "description": "Customer support, v1"
  }'

name + version must be unique within your account; duplicates return 409.

Partial update & rename

  • PATCH /v1/prompts/:id — Updates template / variables / labels / description. name and version are immutable.
  • POST /v1/prompts/:id/rename — Changes name and version. Useful for typo fixes.

Logical sunset

Deleting an old version detaches it from past eval runs, so those runs lose their link to the prompt (provenance is lost). Add sunset to labels instead:

curl -X PATCH \
  -H "Authorization: Bearer $ARGOSVIX_API_KEY" \
  -H "Content-Type: application/json" \
  https://ingest.argosvix.com/v1/prompts/7 \
  -d '{"labels":["sunset"]}'

Operate from chat

Via MCP: create_prompt / update_prompt / rename_prompt / delete_prompt.

You:    Add "be polite" to the customer support prompt.
Claude: (uses update_prompt to patch the template)
Claude: Added. The production label is unchanged.

3. Eval criteria & runs

LLM-as-judge: have one LLM score another's responses.

Default & custom criteria

GET /v1/eval-criteria returns global defaults (helpfulness / safety / conciseness etc.) and your account's custom criteria. Custom criteria are Pro+.

Create custom:

curl -X POST \
  -H "Authorization: Bearer $ARGOSVIX_API_KEY" \
  -H "Content-Type: application/json" \
  https://ingest.argosvix.com/v1/eval-criteria \
  -d '{
    "name": "domain_specificity",
    "rubric": "Score 1-5 based on how well the answer uses domain-specific terminology correctly.",
    "scaleMin": 1,
    "scaleMax": 5
  }'

⚠️ Deleting a custom criterion also removes all scores recorded against it, breaking historical comparison. For rename, prefer PATCH (full replace).

Run an eval

curl -X POST \
  -H "Authorization: Bearer $ARGOSVIX_API_KEY" \
  -H "Content-Type: application/json" \
  https://ingest.argosvix.com/v1/eval-runs \
  -d '{
    "name": "weekly review",
    "recentCount": 50,
    "label": "production",
    "promptRegistryId": 7,
    "idempotencyKey": "weekly-2026-w23"
  }'

idempotencyKey dedupes within 60 min (defends double-clicks and retries).

Read results

GET /v1/eval-runs for the list, GET /v1/eval-runs/:id for details. Per-call scores and reasoning are returned under scores; trajectory scores (see below) under trajectoryScores.

Trajectory (multi-turn) evaluation

Create a criterion with "scope": "trajectory" to score a whole multi-turn agent run as one unit instead of per call. At run time, every call sharing a traceId (set automatically by withTrace, or passed explicitly) is folded — in timestamp order, with any tool/retrieval/sub-agent observations — into a single transcript and judged once. Good for "did the agent achieve the goal?", "was tool use efficient?", "did it stay on task across turns?". Trajectory criteria are llm_judge only and returned under trajectoryScores (each with traceId, score, turnCount, reasoning).

Operate from chat

You:    Has our chatbot's quality dropped this past week?
Claude: (uses list_eval_runs to scan the last 7 days)
Claude: Out of 5 evaluations, only Tuesday's came in low (helpfulness mean 2.4).
        The rest stayed around 4.

4. Annotations

Attach labels, quality scores, and free-text comments to call records.

Use cases

  • Flag issues like "badly-summarized" manually
  • Build a golden dataset for evaluation
  • Manual quality review of customer support responses

Annotate a single call

curl -X POST \
  -H "Authorization: Bearer $ARGOSVIX_API_KEY" \
  -H "Content-Type: application/json" \
  https://ingest.argosvix.com/v1/annotations \
  -d '{
    "callId": "call-xyz-123",
    "annotationText": "Answer covers only half of the question",
    "label": "incomplete",
    "qualityScore": 2
  }'

At least one of annotationText / label / qualityScore is required.

Cross-call search by label

GET /v1/annotations?label=incomplete lists all calls flagged with that label. The dashboard annotation view also supports filtering.

Operate from chat

You:    Mark any of the last 10 responses where the answer only covers half the question.
Claude: (reads via query_calls, then uses create_annotation with the
        "incomplete" label on the matching ones)
Claude: Marked 3 (two from Customer A's queries and one from Customer B).

5. LLM budget management

The safety classifier, PII secondary audit, and eval runs call OpenAI (and others) internally. These monthly LLM costs are capped per account and manageable from the API.

Read current budget state

curl -H "Authorization: Bearer $ARGOSVIX_API_KEY" \
  https://ingest.argosvix.com/v1/account/llm-feature-budget

Response:

{
  "budgetUsd": 5,
  "spentUsd": 0.42,
  "remainingUsd": 4.58,
  "periodStart": "2026-06",
  "defaultBudgetUsd": 5,
  "minBudgetUsd": 5,
  "maxBudgetUsd": 500
}

periodStart is YYYY-MM; auto-resets across calendar months.

Raise or lower budget (Pro+)

curl -X PATCH \
  -H "Authorization: Bearer $ARGOSVIX_API_KEY" \
  -H "Content-Type: application/json" \
  https://ingest.argosvix.com/v1/account/llm-feature-budget \
  -d '{"budgetUsd":30}'

Range $5–$500 in $0.01 increments. A new value lower than current spentUsd is accepted but remainingUsd becomes 0 (next month resets from 0).

Monitor budget from chat

You:    What's left of this month's LLM budget?
Claude: (uses get_llm_budget)
Claude: $4.58 left of $5 ($0.42 spent, 91.6% remaining).

You:    Bump the cap to $30 — I want to run more checks this month.
Claude: (uses raise_llm_budget to set 30)
Claude: Monthly budget updated to $30.

6. Autonomous AI ops

Beyond observation, into action. From Claude / Cursor / Codex CLI you can close the loop in a single conversation — infra health → anomaly detection → alert design → batch classification → eval criteria proposal (Pro+ only).

The Claude responses below are illustrative examples, not real production output. Actual numbers depend on your environment and traffic.

One-question infra health check

You:    How is my LLM infra doing right now?
Claude: (calls get_account_health for the 24h window)
Claude: Past 24h: 1,247 calls, $4.12 cost, 2.1% error rate, p95 latency 1,820 ms, budget 18% used. Verdict: "ok".

Internally fans out to 4 endpoints (aggregate / percentiles / llm-budget / audit-log) in parallel; per-endpoint failures land in partialFailures.

One-prompt anomaly scan

You:    Anything unusual going on?
Claude: (calls detect_anomaly with 24h window + normal threshold)
Claude: 2 anomalies: (1) cost is 2.4× baseline = major, (2) error rate at 8% is 4× baseline = critical.

Threshold: sensitive (1.5×) / normal (2×, default) / conservative (3×). error_rate is evaluated in percent (0-100).

Have the AI design your alerts

You:    Design a sensible alert set from my traffic pattern.
Claude: (calls propose_alert_rules with 14-day lookback)
Claude: 4 alerts proposed:
  - cost_daily threshold $0.50 (1.8× baseline mean $0.28)
  - latency_p95 threshold 3000ms (1.6× recent p95 of 1,820ms)
  - error_rate threshold 5% (2.4× baseline 2.1%)
  - anomaly_cost detection enabled
Apply them?

Proposal-only (zero side effects). The user adopts via create_alert separately.

Bulk classify unclassified calls right now

You:    Classify all calls from last week.
Claude: (calls classify_calls_batch with maxRecords=100)
Claude: 87 scanned, 85 assessed, 3 flagged, 0 failed, 2 skipped.

Complements the automatic background scan (every 15 minutes) with an immediate "scan now" trigger. The results are recorded as on-demand runs (distinct from scheduled scans). Rate limit: 30 req / 60s fixed window (account-wide).

Have the AI propose evaluation criteria

You:    Propose criteria to measure my prompt's quality. Use case: customer support bot.
Claude: (calls propose_eval_criteria with useCaseHint + 3 sample calls; gets 5 proposals)
Claude: 5 criteria proposed:
  1. tone_appropriateness — Does the tone feel polite and professional?
  2. resolution_clarity — Does the response convey the resolution clearly?
  ... (and so on)
Adopt them?

Proposal-only — nothing is created automatically (you apply it via create_eval_criterion separately). When you pass sampleCallIds, Argosvix decrypts those calls and sends excerpts to OpenAI gpt-4o-mini for proposal generation; please be aware of this data flow. Rate limit: 30 req / 60s.

⚠️ All endpoints in this section are Pro+ only. They enforce budget limits, write audit-log entries, and apply rate limits. See MCP Server and REST API Reference for detailed specs.

7. Framework-based instrumentation

wrap() instruments the client instance you create. When a framework creates the provider client internally, wrap() can't reach it. Here's the per-case approach:

Using the provider SDK directly (most reliable)

If you create the client yourself (new OpenAI() etc.), just wrap that spot (Quickstart). Centralize client creation in one module and export the wrapped client, and it's effectively one place.

Vercel AI SDK (TypeScript)

The Vercel AI SDK has built-in OpenTelemetry. Pass experimental_telemetry: { isEnabled: true } to your generation calls and point an OTLP/HTTP JSON exporter at Argosvix.

  • exporter: @opentelemetry/exporter-trace-otlp-http (JSON over HTTP)
  • endpoint: https://ingest.argosvix.com/v1/traces
  • header: Authorization: Bearer argk_... (optionally X-Project-Id: <project>)
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";

const exporter = new OTLPTraceExporter({
  url: "https://ingest.argosvix.com/v1/traces",
  headers: { Authorization: `Bearer ${process.env.ARGOSVIX_API_KEY}` },
});
// Register this exporter with your OTel SDK / @vercel/otel, and pass
// experimental_telemetry: { isEnabled: true } to generateText/streamText.

Argosvix /v1/traces accepts both JSON and protobuf OTLP/HTTP encodings. Exporters work with their default settings.

LangChain / LlamaIndex (Python)

Two options:

  1. Wrap the inner client: if you create the provider client yourself before handing it to the framework, wrap() it first — most reliable.
  2. OpenTelemetry auto-instrumentation (e.g. OpenLLMetry): point the OTLP exporter at https://ingest.argosvix.com/v1/traces with a Bearer header. Python's standard OTLP HTTP exporter (protobuf by default) works as is.

Argosvix OTLP endpoint (shared spec)

  • POST https://ingest.argosvix.com/v1/traces
  • OTLP/HTTP accepts both JSON and protobuf (Content-Type: application/json or application/x-protobuf)
  • Auth: Authorization: Bearer argk_...; optional X-Project-Id to route records
  • Ingested spans are normalized into call records (tagged source=otlp)

Let your AI do it

Ask Claude / Cursor:

"Instrument this project's LLM calls for Argosvix. If it uses the provider SDK directly, use @argosvix/sdk wrap(). If it uses the Vercel AI SDK, enable experimental_telemetry and point an OTLP/HTTP JSON exporter at https://ingest.argosvix.com/v1/traces."

Support

hello@argosvix.com

Last updated: 2026-07-09