You inherited a codebase with no reliable tests, and an architecture change is already on the release calendar. The tempting response is to demand broad test coverage before anything moves. I would not do that: coverage built against poorly understood behavior can certify the wrong system. QA’s first job is to identify which existing behaviors the change could break, then make those behaviors observable before approving it.
Teams mistake missing tests for missing evidence
When engineers say “there is no baseline,” they often mean there is no automated test suite. That distinction matters because the running product may still provide evidence: HTTP responses, database records, audit entries, queue messages, support reports, and client retry patterns. Ignoring those sources costs time, since QA then has to invent expected behavior while the architecture work is already under way.
I would push back on Scope an Architecture Upgrade Without Rewriting the Product with a stricter QA condition: a proposed boundary is not ready for approval until someone can name the behavior visible on either side of it. If an order endpoint moves behind a new service, “the API remains compatible” is too vague. Does a duplicate submission return the same HTTP status? Does the stored order retain the same identifier? Does an audit event appear once, even after a retry?
Start with a small evidence ledger. For each changed path, record the request or trigger, the observed response, the durable side effect, and the person who can confirm whether the behavior is intentional. Mark uncertain observations as uncertain rather than turning them into specifications. RFC 9110 defines HTTP status semantics, and an OpenAPI 3.1 document can describe an intended contract, but neither proves that clients do not depend on an undocumented field or retry response.
The cost of skipping this ledger is particularly high for inherited code: a test written from a design document may pass while a client breaks, because the client has been using the deployed behavior rather than the document. Git history and incident tickets can help explain a surprising behavior, but production observations still need privacy review before they become fixtures.
Broad coverage is the wrong first purchase
A team under pressure may propose hundreds of unit tests before touching architecture. I would not set a coverage percentage as the entry ticket, because line coverage counts executed code without showing whether an important boundary was asserted. The resulting cost is a delayed release and a suite that can stay green while serialization, retries, or persistence change.
Instead, ask which failure would escape the current checks and be expensive to repair. Put tests around those paths first. An OpenAPI schema check can catch a missing response field; a Pact contract test can catch an incompatible consumer expectation; a pytest test can pin a calculation that will move between modules. None substitutes for the others, because each observes a different boundary.
For a read-only JSON endpoint, a comparison probe can establish an initial signal before the team has a test harness. With OLD_URL and NEW_URL set to staging hosts that contain the same fixture order, this Bash script compares their responses. Change the path and ignored fields only after checking that those fields are genuinely noncontractual:
#!/usr/bin/env bash
set -euo pipefail
: "${OLD_URL:?set OLD_URL}"
: "${NEW_URL:?set NEW_URL}"
old=$(mktemp)
new=$(mktemp)
trap 'rm -f "$old" "$new"' EXIT
curl -fsS "$OLD_URL/orders/42" | jq -S 'del(.request_id, .generated_at)' > "$old"
curl -fsS "$NEW_URL/orders/42" | jq -S 'del(.request_id, .generated_at)' > "$new"
diff -u "$old" "$new"
The curl -fsS flags make an HTTP error fail the command while retaining an error message; jq -S prevents object key order from creating a false difference. The probe still cannot prove equivalence, because it checks one fixture and does not observe stored state. Its value is narrower and more honest: it gives reviewers a reproducible mismatch to investigate before they debate coverage targets.
Response parity can conceal a costly state change
A response comparison becomes dangerous when teams treat it as a release decision. Two implementations can both return “accepted” while one writes twice, loses a queue message, or records a different timestamp. That mistake costs more than a visible API break because the defect may persist in data after a rollback.
I would also challenge Scope Architecture Changes Without Rewriting the Codebase on what counts as a change worth testing: moving a write across a boundary changes timing and failure modes even if the source-level business rule stays put. For each write path, QA should ask what happens when the caller times out after the server commits, when a worker retries, and when the old and new components briefly run together. Those questions identify the cases a happy-path response test cannot reach.
There is an explicit choice between contract probes and read-only traffic replay. Contract probes win when the team has a few critical requests and controlled fixtures; they cost less to maintain but miss unfamiliar input combinations. Read-only replay wins when request shapes are poorly documented and realistic variation matters; it costs more in capture, data sanitization, and result triage. Neither should replay writes into a live destination, because duplicate side effects would make the experiment itself a source of defects.
Where PostgreSQL 16 is involved, compare durable rows and constraints as well as responses. Inspect query behavior with pg_stat_statements when a new boundary changes database access, because a functionally correct endpoint can impose a damaging query load. If the rollout includes an index, PostgreSQL’s CREATE INDEX CONCURRENTLY avoids the usual write-blocking index build but cannot run inside a transaction block; treating it as an ordinary migration step can therefore derail deployment. A rollback plan also needs to say what happens to data written in the new format, since reverting code does not automatically revert records.
A green canary cannot defend a metric nobody defined
Teams often say they will “watch the canary” without deciding which signal would stop it. That costs detection time: a dashboard can look calm while the affected request class is too small to move a service-wide average. QA should require the changed path, outcome, and comparison group to appear in the observation plan before rollout begins.
For example, a team might measure 6 divergent responses in 120 matched staging requests; that is an observed result for that rehearsal, not an acceptable production rate. A proposed 0.5% mismatch limit is a threshold to tune against the path’s risk and sample size, not a universal standard. Prometheus documents a default global scrape_interval of 1 minute; that published default may be too coarse for a short canary, so the team should verify its actual configuration before promising rapid detection.
Use OpenTelemetry traces and W3C Trace Context’s traceparent header to connect a request to downstream calls when the boundary moves. In Grafana, separate errors and latency for the affected route rather than relying on a service-wide panel. A Prometheus latency histogram can support a p95 comparison if its buckets suit the route, because an average can hide a slow tail. Logs should carry stable request identifiers but should not copy sensitive request bodies merely to make comparison easier.
The canary decision also needs a stop rule that a person can execute. A 10-minute rollback target is a value to rehearse, not a promise to put in a slide: it is credible only if the deployment can be reversed and the resulting data remains readable. If observation cannot distinguish old from new traffic, the canary has limited evidential value, because a healthy aggregate can mask failures confined to the new path.
QA sign-off should reject unknown ownership, not unknown code
An inherited codebase will contain unknowns at release time. Blocking until every function is understood costs effort without necessarily reducing the risk of this particular change. The more useful objection is that no one owns an observable failure: a mismatch appears, but engineering calls it expected, operations calls it an application issue, and QA lacks authority to stop the rollout.
Require the change owner to name who classifies response differences, who inspects durable side effects, and who orders rollback. Record the acceptable differences before deployment. If a timestamp may shift because processing becomes asynchronous, specify the allowed relationship between the timestamp and the event that produced it; “timestamps may differ” would permit a regression no one intended. Keep a separate list of unanswered questions so a passing test does not silently convert them into approved behavior.
A 48-hour observation window might be appropriate for a path exercised daily, but it is a schedule to tune to actual usage, because a rarely used operation may see no meaningful traffic in that period. Likewise, a clean CI run says the selected checks passed; it says little about an unobserved migration path. QA can sign off on bounded evidence and a named stop rule without pretending that an untested inheritance has become fully tested.
First, pick one changed request path and write down its current response, persistent side effect, retry behavior, and rollback owner. Capture one safe staging example, then ask the change team to identify what the new boundary could alter in each entry. If they cannot answer, postpone that path’s approval—not to demand a rewrite, but to prevent an unmeasured change from becoming QA’s evidence after release.