What strategies do you use for API performance testing?
Testing APIs: three different questions
Three suites, three questions: behaviour, compatibility, capacity. The contract test is the one only an API needs.
For this question · chapter 3 of 3
Four load shapes, four questions
- Two suites have now proven behaviour and compatibility. Capacity is a different question. You ask it with the shape of the traffic. A load test keeps the traffic at the level you actually expect: 800 requests per second for 30 minutes. The p95 latency (the time within which 95% of requests finish) stays at 180 ms, under a 300 ms budget. So this run tells you about a normal day and nothing else.
- A stress test gives two numbers, and the second one matters more. The first is the ceiling, 3,000 requests per second, where latency goes over the budget as the traffic keeps rising. Latency then stays over the budget and reaches 4,900 ms at the top of the ramp. The second number is the first thing to break, the database pool at 200 connections.
- A spike test measures recovery, not the peak. Traffic rises from 800 to 5,000 requests per second in ten seconds and stays there for thirty seconds. Under that load, the p95 reaches 4,100 ms and 11% of requests fail. But the measurement is the 90 seconds the service takes to get back under the budget after the spike ends.
- This endurance run finds a leak, and no ten minute run can show a leak's shape. The test sends the same ordinary traffic for eight hours. But p95 latency still rises from 180 ms to 940 ms, and the heap grows in a straight line from 410 MB to 3.1 GB. People skip this test, but it explains why a service needs a restart at 3am.
- A test that checks only the status code proves almost nothing. Every signal says the load run passed: 1.4 million responses, all of them 200, and p95 at 180 ms. Only the schema assertion failed, because every response body had an empty items list and no total field. So assert an error rate, the body's shape, and a percentile instead of an average, because an average hides every slow response.
Two suites have now proven behaviour and compatibility. Capacity is a different question. You ask it with the shape of the traffic. A load test keeps the traffic at the level you actually expect: 800 requests per second for 30 minutes. The p95 latency (the time within which 95% of requests finish) stays at 180 ms, under a 300 ms budget. So this run tells you about a normal day and nothing else.
Say which question each suite answers. Functional tests prove behaviour. Contract tests prove compatibility. Performance tests prove capacity. Then name the contract test as the API-specific one. The reason: only the contract test fails your build on a change your own suite has already approved.
© LearnThatStack - diagrams may not be republished without permission.
Want a quick review of the fundamentals? See the API Design cheatsheet.
74 of 92 API Design answers are in Pro.
Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.
- Full answers + code
- AI explanations, simpler or deeper
- 1,000 AI credits / month
- Cancel anytime