X

Health Checks and Readiness Signals for Web Applications

A web server can return a response while the application behind it is still starting, stuck, overloaded, or unable to reach a dependency it needs. A health check turns a small signal into an action, such as holding traffic back or restarting a process. The important part is choosing a signal that matches the action.

Use readiness to decide whether an instance should receive new requests, liveness to decide whether a restart could recover a stuck process, and startup to give slow initialization time to finish. These signals are related, but they are not interchangeable. The platform interpreting them determines what happens after a check fails.

What each health signal should mean

Signal Question Typical response to failure
Startup Has initialization finished enough for normal checks to begin? Keep startup in progress or restart after the configured startup limit.
Liveness Can this process still make progress, or is it stuck in a state a restart could fix? Restart the process or container after the failure threshold.
Readiness Should this instance receive new user traffic right now? Stop routing new traffic to it, then check again.

These are platform actions, not universal HTTP standards. In Kubernetes, a failed liveness or startup probe can restart the container, while a failed readiness probe marks it unready so Services stop sending it normal traffic. Cloud Run has its own probe configuration and behavior. Read the documentation for the platform that will act on your endpoint.

Dependency checks are inputs to these signals, not necessarily a fourth kind of probe. If the application cannot serve useful requests without its database, database availability may affect readiness. It usually should not affect liveness: restarting every application instance will not repair an unavailable database.

Keep the endpoints small and safe

Use a lightweight route, such as /health/live or /health/ready, rather than treating the home page as a health check. A home page may depend on a template, cache, analytics service, or third-party API that is unrelated to whether the process can handle requests.

For example, the endpoints might return:

GET /health/live   -> 200 {"status":"ok"}
GET /health/ready  -> 503 {"status":"not_ready"}

The response body is for simple status, not a diagnostic dump. Do not expose connection strings, credentials, internal hostnames, stack traces, or detailed dependency errors from a publicly reachable endpoint. Keep diagnostic detail in access-controlled logs and monitoring instead.

Make every check cheap and bounded. A readiness request should not perform an unbounded query, call several third-party APIs, or wait indefinitely for a connection. If a dependency is essential, use a small check with a short timeout and return not-ready when it exceeds that budget. Avoid allowing every instance to create extra load during a dependency outage by repeatedly running expensive checks.

An application that can still serve some requests while a nonessential service is down may remain ready. Decide based on user-facing behavior: if the dependency is optional or a fallback is available, a blanket not-ready result can remove capacity unnecessarily. If most requests would fail without it, readiness may be the right place to represent that failure.

Separate shallow checks from dependency checks

A shallow check asks whether the local process is responsive. It is appropriate for liveness: the endpoint should return quickly if the server can handle a small request. It should not fail simply because a remote database or payment service is temporarily unavailable.

A dependency-aware check asks whether the instance can serve the requests that matter. Use it for readiness only when the dependency is required for those requests. Do not make every dependency equally critical. A temporary outage in email delivery, for example, may not mean that a storefront or dashboard should stop serving all traffic.

Kubernetes describes this distinction directly: liveness can report that the application itself is healthy, while readiness can also check required back-end services. Its documentation warns that poorly designed liveness checks can trigger cascading failures by restarting containers under load. For more on keeping monitoring signals useful, see our guide to the observability debt problem.

Configure startup, liveness, and readiness in Kubernetes

For a containerized web app with a slow initialization phase, a Kubernetes configuration can use separate endpoints:

startupProbe:
  httpGet:
    path: /health/startup
    port: 8080
  periodSeconds: 5
  timeoutSeconds: 2
  failureThreshold: 24

livenessProbe:
  httpGet:
    path: /health/live
    port: 8080
  periodSeconds: 10
  timeoutSeconds: 2
  failureThreshold: 3

readinessProbe:
  httpGet:
    path: /health/ready
    port: 8080
  periodSeconds: 5
  timeoutSeconds: 2
  failureThreshold: 3

This is an example, not a universal tuning recipe. With a startup period of five seconds and a failure threshold of 24, the startup check allows roughly two minutes of repeated failures before Kubernetes treats startup as failed. Measure the slowest expected initialization in your environment, then leave enough margin for normal variation.

When a Kubernetes startup probe is configured, liveness and readiness probes do not run until startup succeeds. That lets you give startup its own allowance without making liveness so forgiving that a real deadlock takes too long to recover from. A readiness failure keeps the container running while Kubernetes continues checking it.

Account for load-balancer and hosting behavior

An application-level probe and an edge load balancer's target check may look similar, but they can have different consequences. An AWS Application Load Balancer health check, for example, normally routes requests to healthy targets. AWS also documents a fail-open behavior: if every registered target is unhealthy, the load balancer can route requests to all of them anyway. A health check is therefore not a substitute for application-level error handling or a tested recovery plan.

Managed platforms can add their own readiness rules. For Cloud Run, a successful startup probe is treated as ready to receive traffic, even before the first readiness probe completes. Its documentation recommends making startup success mean that the container can safely accept traffic immediately. Check platform-specific details before assuming that a readiness endpoint alone controls the first request.

Also confirm how the platform reaches the endpoint. Cloud Run documents HTTP health-check endpoints as externally accessible, so return only a minimal status there and ensure that application middleware does not block the platform's request. Other platforms may use a private network path, a TCP connection, or an executable command instead of HTTP.

For more deployment context, see our guide to common hurdles in Kubernetes deployments.

Choose timeouts and thresholds to avoid restart loops

Health checks are sampled signals. A timeout that is shorter than normal response variation, or a failure threshold that reacts to one transient delay, can mark a healthy instance as failed. A long timeout can delay detection and tie up resources when the app is genuinely stuck.

Set the endpoint's own dependency timeout shorter than the platform's probe timeout. Then choose the platform interval and consecutive-failure threshold based on how quickly you need to detect a real problem and how much transient variation is normal. Do not copy another project's values without checking its startup time, request latency, traffic pattern, and shutdown behavior.

Use startup checks for slow initialization instead of making every liveness check wait a long time. Keep liveness focused on failures a restart can plausibly fix. When an instance is temporarily overloaded or waiting on a required dependency, readiness can stop new traffic while leaving it alive to recover.

Observe the result, not just the endpoint

A green check is not proof that a complete user journey works. Monitor probe failures and latency alongside application errors, resource use, and the number of instances or targets still receiving traffic. Keep enough context in logs to identify which check failed and why, but avoid logging sensitive response details or creating noisy records for every successful probe.

Test the failure cases in a staging environment before relying on the configuration:

  1. Delay startup and confirm the startup allowance is long enough.
  2. Make a required dependency unavailable and confirm readiness changes without triggering a restart storm.
  3. Simulate a process that cannot make progress and confirm the liveness policy eventually recovers it.
  4. Check that the load balancer or hosting platform can reach the endpoint and interprets its status as expected.
  5. Restore the dependency or process and confirm the instance becomes ready again.

Health checks help a platform make a narrow decision; they do not prove that data can be recovered. Keep operational checks alongside a documented database backup and restore plan.

A practical default

Start with a cheap liveness endpoint and a readiness endpoint that reflects only the dependencies required to serve useful requests. Add a startup probe when initialization time makes liveness unreliable. Configure the platform to act on each signal deliberately, then observe and test both failure and recovery.

The right health check is not the one that reports the most detail. It is the smallest reliable signal that triggers the right action without turning a temporary dependency problem into an outage.

Official sources

Categories: Development
Related Post