My previous blog in this series was about the role of functional testing in the cloud environment. In this blog, let’s see how performance testing plays a major role in cloud migrations.
Performance testing in the cloud is fundamentally similar to how it is done in a non-cloud environment. However, there are differences while planning and running performance tests on the cloud – as certain category of performance tests may not be applicable in a cloud environment (depending on the type of cloud adoption – public, private or hybrid cloud). In a public cloud, the performance tests that can impact other customers are not performed (e.g. stress test). Care should be taken to baseline and measure the response times, as the response time on a cloud environment may not be similar to that for a non-cloud based infrastructure. Elasticity testing is applicable in the case of cloud-based performance testing. Typical tests applicable in the context of cloud are:
- Load test: Start with getting information from the cloud provider on the load statistics of other customers sharing a public cloud platform. This will give us insights in what kind of response time to expect (as load on other applications hosted on the public cloud can impact our application). Run the tests when the load is expected to be high (i.e. real time load situations). Importantly, run these tests regularly to get a range of response times in order to establish “minimum response time”, “maximum response time” and “average response time”.
- Stress test: As stress test in a public cloud can impact other customers, extreme care should be taken when planning and executing stress tests in a public cloud situation, as compared to a private cloud context where the cloud is in our exclusive ownership and control. However, using the boundary value analysis technique, stress just beyond the agreed range can be applied to verify whether the situation is gracefully handled.
- Endurance or volume test: Using memory monitoring tools, endurance or volume test is conducted. In the case of IaaS, the test team can monitor the memory by installing memory monitoring tools in the virtual environment itself.
- Testing elasticity and scalability: The objective of elasticity testing is to determine whether the performance meets the defined SLAs and does scalability happen at the determined point. In a pay-per-use model, elasticity testing is necessary to validate the generation of financial invoice based on the amount of infrastructure really consumed. The approach consists of gradually increasing the load to cross the threshold of the current limit, thereafter check whether the system scales up as the load increases and scales down as the load reduces. In this case, the boundary value analysis testing technique may be useful to determine the edge of different limits.
What has been your experience? How did you go about addressing some of the challenges in doing performance testing in a cloud environment? In my next blog I plan to share my experiences with security and interface testing in a cloud environment.