Workload health KPIs
Infrastructure metrics tell you what the machine is doing. Workload health KPIs tell you whether the service is doing its job. A service can be at 20% CPU and failing.
The standard. A small set of metrics per component that show whether the workload is healthy, with thresholds, owners and runbooks that make them actionable.
Three components
Definition
A small set of metrics per component, tied to what the component must do for the service to work. Each has an owner, a baseline and a target.
Collection
CloudWatch, X-Ray, structured application logs, and third-party telemetry where the customer already has it.
Action
Thresholds that raise an alert, an owner to receive it, and a runbook that says what to do.
Selecting KPIs
A KPI is well formed when it satisfies all of these:
- It maps to a user or business outcome — not to a resource.
- It has a baseline measured before the engagement's changes.
- It has a target agreed with the customer.
- It is measurable continuously, from telemetry rather than surveys.
- Someone owns it and can act when it moves.
- It is stable — not so noisy that it triggers on normal variation.
Common failure modes of KPI selection
| Anti-pattern | Why it fails |
|---|---|
| Dashboards with 60 metrics and no owner for any | Nobody knows which one matters |
| Alerting on every metric | Alert fatigue; the real signal is ignored |
| KPIs that measure activity, not outcome | "Number of deployments" rises while reliability falls |
| Targets with no baseline | No way to know whether anything improved |
| Customer-facing KPIs owned only by engineering | The business does not see or trust them |
KPI catalogue by component
Web / edge
| KPI | Definition | Target pattern |
|---|---|---|
| Availability | Successful responses / total requests | ≥ 99.9% |
| p95 response time | 95th percentile end-to-end latency | Per page or endpoint |
| Error rate | 5xx / total requests | < 0.5% |
| Cache hit rate | Cache hits / requests | Per workload, usually > 80% |
Application
| KPI | Definition | Target pattern |
|---|---|---|
| Request success rate | Successful business transactions / attempts | ≥ 99.5% |
| p99 latency | Tail latency on the critical path | Per endpoint |
| Saturation | Queue depth, thread pool, connection pool utilisation | Below a tested ceiling |
| Throughput | Requests or jobs per minute at peak | Meets forecast |
Data
| KPI | Definition | Target pattern |
|---|---|---|
| Query latency | p95 for the dominant query pattern | Per query class |
| Connection saturation | Active / max connections | < 80% |
| Replica lag | Seconds behind primary | < 5s for read paths |
| Failed transactions | Rollbacks, deadlocks, timeouts | Near zero |
Async / background
| KPI | Definition | Target pattern |
|---|---|---|
| Queue age | Age of the oldest message | < processing SLA |
| DLQ depth | Messages that failed all retries | 0 |
| Consumer lag | Records behind head | < one processing window |
| Job success rate | Completed / attempted | ≥ 99% |
Business
Business KPIs are what makes the technical ones meaningful to the customer:
- Orders completed per hour, compared with the same period last week.
- Payment success rate by provider.
- Sign-ups completed versus started.
- Exam submissions accepted versus attempted.
- Report generation completed within the expected window.
These are the metrics that belong on the leadership dashboard. They also catch failures that infrastructure metrics miss — a dependency returning 200 with wrong data will not raise an error rate.
Logging for detection
Structured logs make KPIs computable and incidents diagnosable.
{
"timestamp": "2026-11-06T10:14:22.481Z",
"level": "error",
"service": "checkout",
"environment": "prod",
"requestId": "b1c2d3e4",
"traceId": "1-6789abcd-0123456789abcdef",
"message": "payment capture failed",
"provider": "gateway-a",
"errorCode": "TIMEOUT",
"durationMs": 3021,
"attempt": 2
}
| Requirement | Reason |
|---|---|
| Structured JSON | Queryable without regex |
| Correlation id | Ties a request to its log lines |
| Trace id | Ties a log line to a trace |
| Level | Filter for errors without scanning everything |
| Domain fields | Makes the log useful for business KPIs too |
| No secrets or personal data | Compliance and cost |
Thresholds
| Method | Use when |
|---|---|
| Static threshold | The target is contractual or well understood |
| Baseline-relative | The workload is seasonal and a static number would be wrong |
| Anomaly detection | The metric has a stable pattern and sudden deviation is the signal |
| Composite | Several indicators together indicate a real incident |
Thresholds are reviewed after every incident: was the alarm early enough, and did it point at the right thing?
The KPI review cadence
| Cadence | Participants | Output |
|---|---|---|
| Daily | On-call | Alert review; anything unexplained becomes a ticket |
| Weekly | Delivery team | KPI trend against target; capacity and waste items |
| Monthly | Customer and delivery lead | KPI report, incidents, improvement actions |
| Quarterly | Sponsor and delivery lead | Target review; are these still the right KPIs? |
Evidence
- KPI definition document per component, with owner and target.
- Dashboard showing each KPI against its baseline.
- Alarm configuration with thresholds and routing.
- Log schema and a sample query used in an investigation.
- A worked example of an operational event detected from a KPI and resolved via its runbook.