Skip to main content
A process can be “running” and still broken. Postgres is down, Redis dropped the connection, the worker pool deadlocked — the agent is still binding port :3773 and answering TCP, but the next real request fails. So /health in Bindu does more than return 200 OK. It walks the runtime — storage, scheduler, task manager — and returns 503 Service Unavailable when any of them is missing or not running. /metrics exposes Prometheus text-format counters, gauges, histograms, and summaries for request traffic, latency, task state, errors, and per-agent task completion. You do not configure either. Both endpoints are registered on BinduApplication startup, both are on the auth allowlist, and both are exempt from the startup gate so Kubernetes can probe the pod while storage and the scheduler are still coming up. Point your load balancer at /health and your scrapers at /metrics — you have visibility from request one.

Liveness, readiness, metrics

Bindu does not split liveness and readiness across multiple paths. There is one health endpoint and it makes the strict choice for you: 200 means every dependency is ready, 503 means at least one is not.
Strict readiness means storage + scheduler + task manager are all initialized and the task manager’s run loop is active. Anything less and /health returns HTTP 503 with "status": "degraded", "ready": false, "health": "degraded".

Readable

/health returns a single JSON document with runtime, application, and system blocks.

Scrapeable

/metrics is Prometheus text format version=0.0.4 — scrape it with anything OpenMetrics-compatible.

Probe-safe

Both paths are exempt from the startup gate and the auth middleware allowlist, so probes never get a 500 or 401.

Request flow

The metrics middleware runs on every request except /metrics itself (skipped to avoid feedback loops), sanitizes path parameters (UUIDs and numeric IDs are rewritten to :id to keep cardinality bounded), and feeds the global PrometheusMetrics singleton.

/health

1

Probe it

2

Read the fields

3

Wire it to your orchestrator

Kubernetes treats any 2xx as healthy; 503 fails the probe. That maps directly to strict_ready, so a single config covers both liveness and readiness:
During boot — before storage and scheduler initialize inside the ASGI lifespan — /health returns 503 without raising. The startup gate in BinduApplication.__call__ whitelists /health, /healthz, and /metrics, so probes get a proper response code instead of a 500.
Only /health is registered as a route. The /healthz path is whitelisted in the startup gate (so it cannot 500 mid-boot) but no handler exists for it — a request to /healthz returns 404 once the app is running. Use /health everywhere.

/metrics

Returns text/plain; version=0.0.4; charset=utf-8 with Cache-Control: no-cache. Before generating output, the endpoint refreshes the per-agent active-task gauge by counting tasks in submitted, working, and input-required states from storage.

Metric reference

Total HTTP requests handled by MetricsMiddleware, labelled by method, sanitized endpoint, and status code.Path parameters are normalized: UUIDs match [0-9a-f]{8}-… and runs of digits both collapse to /:id. That keeps cardinality bounded even with high-traffic task endpoints like /tasks/<uuid>.
Request latency in seconds. Buckets: 0.1, 0.5, 1.0, +Inf. Exposes _bucket, _sum, _count.The histogram is global, not per-endpoint — there are no method or path labels on the buckets. If you need per-route p95, build it from request count buckets in your aggregator, or alert on the global p95.
Tasks currently in submitted, working, or input-required state for the agent. Updated lazily — only when /metrics is scraped, by querying storage.count_tasks(status=…) for each active state. Only emitted when at least one agent has reported.
Cumulative finished tasks per (agent_id, status). Status values come from the task manager and are typically success, failed, or canceled. Only emitted when at least one task has completed.
End-to-end task execution time, labelled by agent_id and status. Buckets: 1, 5, 10, 30, 60, +Inf seconds. Exposes _bucket, _sum, _count. Only emitted when at least one task has finished.
Errors raised during task execution, labelled by agent_id and error_type (e.g. timeout, validation, execution). Only emitted when at least one error has been recorded.
Aggregate byte totals and counts for request and response bodies. Exposed as Prometheus summaries with _sum and _count only (no quantiles). Sizes are read from the Content-Length header — chunked responses without a length register as zero.
Number of concurrent in-flight HTTP requests. Incremented in the middleware before call_next, decremented in the finally block, so it always reflects current concurrency even on exceptions.Always emitted, even at zero — useful for “agent is alive but idle” dashboards.

Prometheus scrape config

The metrics middleware skips collection for /metrics itself, so scrape calls do not inflate http_requests_total. Your dashboards measure user traffic, not Prometheus polling.

Use cases

Use /health for both probes. The endpoint returns 503 while storage or scheduler is initializing, so kubelet does not route traffic to a half-booted pod, and it returns 200 only once task_manager.is_running is true.
Point your LB at GET /health. Anything other than 200 should drain the instance. status: "degraded" is your signal that the process is up but should not receive traffic.
Scrape /metrics every 15s. Build dashboards on rate(http_requests_total), histogram_quantile(0.95, …) for p95 latency, agent_tasks_active for concurrency, and agent_errors_total for fault rates.
If an agent appears live but rejects work, hit /health and inspect the runtime block. A null storage_backend or scheduler_backend means initialization never completed; task_manager_running: false means the loop exited or never started.
Bindu’s Sentry integration filters /health, /healthz, and /metrics out of transaction traces by default (SentrySettings.filter_transactions), so probe and scrape traffic does not eat your Sentry quota or pollute performance dashboards.

Best practices

Probe readiness, not just the port

A TCP check on :3773 will pass while storage is still connecting. Use HTTP /health and treat any non-200 as not-ready.

Scrape every 15s

The default Prometheus interval is enough for agent_tasks_active to track real concurrency. Below 10s mostly buys noise.

Alert on degraded, not down

By the time /health is unreachable you are already on fire. Alert on status="degraded" or health="degraded" returns for an early warning.

Keep both paths public

Both /health and /metrics are in the auth allowlist by default. Do not put them behind your OAuth gateway — probes and scrapers do not carry tokens.

Sunflower LogoWith Bindu, agent health is like a field of sunflowers — each one standing on its own, yet easy to observe and trust.