Give each check one question

A liveness check asks whether the process can continue operating. A readiness check asks whether this instance should receive new work. Inventory event-loop responsiveness, configuration loading, local resources, and required dependencies separately. Avoid placing every external service inside every process liveness check.

Define a maximum duration and explicit outcome for each check. Preserve diagnostic categories internally while exposing a generic public response. A health endpoint must not reveal connection strings, credentials, private hostnames, or internal routing. Review who can access detailed diagnostics before enabling them.

Budget end-to-end verification separately

Routine probes should not trigger a real model generation every time. Use initialized application state for basic readiness, then schedule any necessary end-to-end request as a separate task with a budget, bounded frequency, and a clear stop control. Account for the number of instances multiplying the probe frequency.

Use synthetic, non-sensitive inputs when an external capability must be checked. Record the scope and time, and verify current model support, pricing, and account permissions. A passing sample is evidence about that observation, not a guarantee for every request or workload.

Rehearse withdrawal and recovery

Mark a test instance unready and confirm that new work stops arriving while existing jobs follow the chosen completion policy. Inspect interruption behavior for streaming connections and preserve an understandable task state. Do not automatically replay generation because a connection ended.

On recovery, finish initialization before admitting new traffic. Record readiness transitions alongside business failures and investigate repeated flapping. Single-instance replacement can still cause interruption; health checks alone do not create a deployment without downtime.

Sources & further reading

An owned publication maintained by NexoToken. Product references reflect the publisher’s commercial interest and are not independent certification.