Build and test
A build is only useful if it is reproducible and a test is only useful if it fails when it should. This standard defines both — the automation chain, the test layers, and the gates that stop a change.
The standard. A reproducible build and test chain that turns a commit into a deployment-ready artefact, with gates that stop a change rather than merely reporting on it.
Why automate
Builds. Consistency (the same inputs produce the same artefact), speed, early integration feedback, removal of manual error, and scaling without a proportional increase in effort.
Tests. Early defect detection, enforced quality, confidence that only tested code reaches production, unattended execution, and coverage across configurations that manual testing cannot reach.
The automation chain
source → build → unit test → static analysis → package → artefact registry
│
integration test ◄─────┘
│
deploy to environment
| Step | Runs on | Produces |
|---|---|---|
| Source checkout | Every push and PR | — |
| Build | Every push and PR | Compilation output |
| Unit tests | Every push and PR | Test report, coverage |
| Static analysis | Every push and PR | Quality gate result |
| Security scan | Every push and PR | Dependency and secret findings |
| Package | Merge to a protected branch | Versioned artefact |
| Integration test | Merge to dev or stage | Environment-verified result |
| Publish | Merge to main | Immutable artefact in registry |
Reproducibility
A build is reproducible when the same commit produces the same artefact.
- Pin dependencies. Lock files are committed. Base images are referenced by digest, not tag.
- Pin tool versions. The CI image, compiler and build tool versions are declared, not inherited from whatever the runner has today.
- No build-time network surprises. Package registries are pinned or proxied; the build does not depend on an invisible upstream.
- Build once, promote. The artefact tested in staging is the artefact deployed to production.
- Record provenance. The build records the commit, the toolchain and the dependency manifest alongside the artefact.
Test layers
| Layer | Scope | Runs | Target |
|---|---|---|---|
| Unit | A single function or class | Every commit | Fast; high coverage of logic |
| Component | A service with its own dependencies mocked | Every commit | Fast; verifies service behaviour |
| Contract | Interaction between consumer and provider | Every PR | Catches interface drift |
| Integration | Service with real dependencies in a container | Merge to dev | Catches wiring and configuration errors |
| End-to-end | The deployed system through its interface | Merge to stage | Catches cross-service failure |
| Performance | Behaviour under expected and peak load | Before production release | Verifies against SLO |
| Security | SAST, DAST, dependency and secret scanning | Every commit; DAST before release | Blocks high and critical findings |
What to test
Test behaviour, not implementation. A test that breaks when a function is renamed but not when the logic changes has negative value — it adds maintenance cost and provides no protection.
Prioritise:
- Business rules that are expensive to get wrong.
- Failure paths: timeouts, retries, partial failures, duplicate delivery.
- Boundaries: empty input, maximum size, unicode, time zones, currency rounding.
- Security-sensitive paths: authentication, authorisation, input validation.
Quality gates
A gate blocks the pipeline. A warning does not, and a warning that is routinely ignored should be removed or promoted.
| Gate | Threshold | Action on failure |
|---|---|---|
| Unit test pass rate | 100% | Block |
| Coverage on changed code | ≥ 80% | Block |
| Coverage overall | ≥ 60% | Warn |
| Static analysis | No new critical or high issues | Block |
| Dependency vulnerabilities | No critical or high with a fix available | Block |
| Secret scan | Zero findings | Block |
| Container image scan | No critical findings | Block |
| Licence compliance | Allow-list only | Block |
| Integration tests | 100% pass | Block |
| DAST (pre-release) | No high or critical | Block |
| Performance | p95 within target | Block |
Server provisioning
The build and test process also covers the infrastructure the tests run against:
- Ephemeral environments created by Terraform for integration tests, destroyed after the run.
- Preview environments per pull request where the customer's cost model supports it.
- Self-hosted runners where the workload requires network adjacency or specific hardware, managed with the same patch discipline as production.
Teaching the process
For each customer engagement we run a session covering:
- How to structure a build so it is reproducible.
- Which tests to write first, and how to make them fail meaningfully.
- How to set a coverage or quality threshold that is achievable and useful.
- How to read a failed pipeline and distinguish a flaky test from a real defect.
- How to change the pipeline without disabling the gates.