Blog
Understand API Load Testing vs. API Performance Monitoring
With over a billion more mobile devices projected to become connected over the next year, it’s never been more important to ensure SLA performance and functional reliability for business-critical APIs. While QA, development, and SRE teams have traditionally relied on API load testing and uptime or ping monitors to mitigate performance risk, too many companies have felt the pain of APIs, mobile apps and web platforms failing under stress or unexpected traffic patterns. For instance, a ticket sales and live events company might expect its API management platform to throttle traffic during expected spikes due to holidays, sudden changes in sporting or concert event times, etc. But ensuring API reliability and SLA performance through all scenarios has become far more challenging due to the sheer scale of integrations involved in even a single API flow. SRE and performance engineering teams looking to eliminate performance bottlenecks should understand the important differences between traditional API load testing vs. true API performance monitoring.
Learn why today’s performance monitors must do more than check for 200 OK pings by hitting endpoints with stress tests. Also learn why API management platforms and siloed load testing tools are unable to provide the unified visibility that SREs now need to properly manage risk for business-critical APIs.
What Is API Performance Testing?
API performance testing is any type of test that evaluates how well APIs perform under a given set of conditions.
For example, an API performance test could assess:
Whether an API returns the expected output in response to given input. This is known as functional API testing. Functional tests often evaluate how API responses change along with variables like the geographic origin of an API request or the client device’s browser or operating system.
How quickly the API responds to requests. This is called API latency or speed testing.
How API behavior and performance change along with the request rate, or load, that the API experiences. This is known as API load testing. As we’ll see below, API load testing can be broken down into several sub-categories.
Given these varying test types, API performance testing/monitoring is a pretty broad category. If you want to run performance tests on your APIs, you can’t just run a single test and call it done. Instead, you should divide your performance testing strategy into different components and run different tests to address each one.
Types of API Load Testing
API load testing is arguably the most complex category under the API performance testing umbrella. That’s because, unlike other types of API performance tests, API load testing includes several different types of tests:
Standard load testing: A standard API load test assesses your API’s performance when it receives requests at the standard, expected rate that it will encounter in production.
Stress testing: API stress tests evaluate whether your API continues to respond normally when it receives unusually high levels of requests.
Spike testing: In spike testing, you intersperse moderate request levels with peaks in order to assess how the API behaves when request rates spike suddenly.
Soak testing: API soak testing involves exposing your API to high request rates for a prolonged period (such as several hours) to detect issues that may not emerge during shorter periods of high load.
Why would you run multiple types of API load tests instead of just sticking with standard load tests? The answer is that testing your API under normal load conditions won’t necessarily reveal issues that arise due to inconsistent or unexpected traffic patterns. For example, consider a banking app that suddenly experiences a spike in traffic due to an unannounced, “viral” ticket sales event for a popular concert. Despite having passed its load tests with flying colors, the app may become non-responsive during the sales event because it sees request rates that are ten or twenty times higher than the standard load.
As a variation on that example, imagine that the app is initially able to handle the spike in requests by using page files to work around the demand placed on its memory. But as the surge in requests continues for hours, the page files fill up, at which point the application fails entirely. In this case, even if you had stress-tested or spike-tested your API, you may not have detected the potential for failure. Only a soak test that evaluated API performance in response to a prolonged increase in load would have revealed the issue.
Combining API Functional Tests and Performance Tests
If having to run so many types of API load tests on top of your other API performance tests feels daunting, there’s an approach that could significantly simplify your overall API testing strategy.
It’s known as API functional load testing, and it involves reusing functional tests as load tests. Under this method, you write tests that evaluate API functionality, then run them under different load conditions.
The advantages of this technique are twofold. First, it saves time on test development and execution because you can extend your functional tests into functional load tests, instead of having to write and run functional and load tests separately.
Second and most important, API functional load testing allows you to evaluate not just whether the API can handle a given load pattern or not, but how well it handles it. Conventional load tests only reveal whether the API fails or not in response to a given load condition. But with functional load testing, you can track functionality – a more nuanced and granular insight – under varying load conditions.
Conclusion
API performance testing is one of those things that seems simple enough on the surface, but is actually pretty complex when you unpack it. This is probably part of the reason why the OWASP API Security Top 10 includes a failure by API publishers to rate-limit their APIs, which can open the door to DDoS attacks, presumably because they fail to test their APIs under heavy loads. To run API performance tests well, you’ll need to write multiple types of tests.
The good news is that you can simplify this process by extending your API functional tests into API functional load tests, which provide the most nuanced level of insight into your API’s overall performance. Ultimately, SRE teams should be able to reuse API performance tests as API performance monitors that can integrate the positive and negative test scenarios of functional load testing in order to significantly increase SRE confidence in production and live environments. This approach also expands visibility across the entire SDLC, and when abstracted across a company’s entire API infrastructure, gives management the global insights and historic quality and performance trends to drive more efficient and effective continuous improvement.
