> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ninjachat.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Observability
> Connect request IDs, routing traces, usage, balance, health, and signed webhooks into one operating loop.
Use `request_id` to connect your application logs with NinjaChat usage and request traces.
## What to record
| Signal | Source | Keep it for |
| ----------------------- | ----------------------------- | ----------------------------------------------- |
| `request_id` | Every response and error | Support, incident correlation, and trace lookup |
| `model` and `provider` | Response or request trace | Route attribution and provider regressions |
| `cost_usd` | Successful generated response | Per-feature and per-customer spend |
| latency and TTFB | `GET /requests/{id}` | Streaming and generation SLOs |
| status and `error_code` | Error plus request trace | Alert grouping and retry analysis |
| balance | `GET /balance` | Low-credit protection |
Do not log prompts, response bodies, API keys, or webhook signing secrets by default.
## Trace one request
Store the response `request_id`, then retrieve the normalized trace when you need routing detail.
```typescript TypeScript theme={null}
const response = await client.responses.create({
model: "ninja/auto",
input: "Summarize this incident report.",
});
const trace = await client.requests.get(response.request_id);
logger.info("ai.request", {
requestId: trace.request_id,
model: trace.model,
provider: trace.provider,
latencyMs: trace.latency_ms,
ttfbMs: trace.ttfb_ms,
retries: trace.retries,
costCents: trace.cost_cents,
statusCode: trace.status_code,
});
```
```python Python theme={null}
response = client.responses.create(
model="ninja/auto",
input="Summarize this incident report.",
)
trace = client.requests.get(response["request_id"])
logger.info("ai.request", extra={
"request_id": trace["request_id"],
"model": trace["model"],
"provider": trace["provider"],
"latency_ms": trace["latency_ms"],
"ttfb_ms": trace["ttfb_ms"],
"retries": trace["retries"],
"cost_cents": trace["cost_cents"],
"status_code": trace["status_code"],
})
```
The trace also contains the requested candidates, routing policy, excluded rails, each attempt, and the final `served_by` route. Use it for debugging; use aggregated usage for dashboards.
## Build the operating loop
Add `request_id` to the same structured log or trace span as your user action or background job.
Query `client.usage("1d" | "7d" | "30d")` for requests, spend, per-model latency, and endpoint volume.
Query `client.balance()` and subscribe to `balance.low` so a prepaid balance does not surprise production traffic.
Register signed webhooks for video and pipeline completion, failed jobs, project-budget thresholds, and low balance.
```typescript TypeScript theme={null}
const [usage, balance, health] = await Promise.all([
client.usage("7d"),
client.balance(),
client.health(),
]);
console.log(usage.total_requests, usage.total_cost, usage.by_model);
console.log(balance.balance, balance.currency);
console.log(health);
```
## Health and alerting
`client.health()` reads the public gateway-health endpoint. It is appropriate for an external availability check, but it does not replace a real request canary.
Use two checks:
* Poll `client.health()` (or `GET /api/v1/health` without an SDK) for gateway reachability.
* Run a small authenticated generation against your normal model policy on a slower cadence to verify auth, balance, routing, and provider execution together.
Alert on customer impact: elevated `5xx`, repeated `429`, latency or TTFB regression, fallback exhaustion, low balance, and webhook delivery failures. Do not page on one provider rail if fallbacks are still serving traffic successfully.
## Webhook events and delivery
| Event | Fires when |
| -------------------- | ----------------------------------------------------------- |
| `video.completed` | An async video job finished and its `video_url` is ready |
| `video.failed` | A video job failed (the charge is refunded) |
| `pipeline.completed` | A `/pipelines` job finished |
| `pipeline.failed` | A pipeline failed |
| `budget.alert` | A project spend limit crossed 50%, 80%, or 100% |
| `balance.low` | Your prepaid balance dropped below \$5, and again below \$1 |
Each account can register up to **5** endpoints. URLs must be `https` on a public host. Omit `events` when creating an endpoint to subscribe to all six. Register in [Developers → Webhooks](https://www.ninjachat.ai/developers/webhooks) or via `POST /webhooks`.
Every delivery is a `POST` with `Content-Type: application/json` and this body:
```json theme={null}
{
"id": "6f1c…",
"event": "video.completed",
"created": 1787529600,
"data": { "request_id": "req_…", "...": "event-specific fields" }
}
```
| Header | Value |
| ------------------- | ---------------------------------------------------------------------------------- |
| `X-Ninja-Signature` | Hex HMAC-SHA256 of `"${timestamp}.${rawBody}"` using the endpoint's signing secret |
| `X-Ninja-Timestamp` | Unix seconds when the delivery was signed |
| `X-Ninja-Event` | The event name |
Delivery is at-least-once. A non-2xx response or a 10-second timeout is retried with backoff of 1 minute, 5 minutes, 15 minutes, 1 hour, then 6 hours — **6 attempts** in total — after which the delivery is marked `dead`. `GET /webhooks/deliveries` lists the last 30 days with each row's `status` (`pending`, `retrying`, `delivered`, `dead`), `attempts`, `last_error`, and `request_id`.
## Verify signed webhooks
Verify the raw request body before parsing JSON. The SDK rejects invalid signatures and timestamps older than five minutes by default.
```typescript TypeScript theme={null}
import { verifyWebhookSignature } from "@ninjachat/sdk";
const rawBody = await request.text();
const valid = await verifyWebhookSignature(
rawBody,
request.headers.get("x-ninja-signature") ?? "",
request.headers.get("x-ninja-timestamp") ?? "",
process.env.NINJACHAT_WEBHOOK_SECRET!,
);
if (!valid) return new Response("Invalid signature", { status: 401 });
const event = JSON.parse(rawBody);
```
```python Python theme={null}
from ninjachat import verify_webhook_signature
raw_body = request.get_data()
valid = verify_webhook_signature(
raw_body,
request.headers.get("X-Ninja-Signature", ""),
request.headers.get("X-Ninja-Timestamp", ""),
os.environ["NINJACHAT_WEBHOOK_SECRET"],
)
if not valid:
abort(401)
event = request.get_json()
```
After creating an endpoint, call `client.webhooks.test(endpoint.id)` and inspect `client.webhooks.listDeliveries({ endpointId: endpoint.id })` before relying on live events. In Python, use `client.webhooks.list_deliveries(endpoint_id=endpoint["id"])`.
## Useful links
Full trace response and routing-attempt schema.
Periods, model aggregation, latency, and cost fields.
Endpoint creation, event types, deliveries, and test sends.
Typed errors, retry behavior, and support correlation.