Test strategies for different cloud models – An overview

In my previous blog, I wrote about the important factors impacting cloud testing, with a particular emphasis on the iStock_000056644844_Mediumeffect of various software development models on the testing approach. In this blog, I’ve elaborated the different aspects of testing that needs to be considered while migrating to the cloud. However, I would like to point out that in a way what is mentioned below is a generic test strategy and does not cover every possible test dimension. Depending on situations and the environment, this strategy should be modified.

  • Infrastructure as a Service (IaaS): From the testing perspective, if the sole purpose of cloud migration is to utilize IaaS capabilities, then focus should be more on testing application performance (availability, scalability, elasticity and similar features). This is similar to a web based testing team having a good quality test environment that is always available and can be virtualized with ease.
  • Platform as a Service (PaaS): Test strategy or approach will not change significantly between IaaS and PaaS. Since PaaS makes available a development environment (platform for application development on the cloud along with the ability to virtualize the environment and the supported hardware), the only difference between IaaS and PaaS is the need for interface testing which is more pronounced in PaaS compared to IaaS. Since applications using PaaS inherit cloud characters, tests to validate scalability, high-availability, and multitenancy should be run.
  • Software as a Service (SaaS): SaaS offers ready to use applications as a service. From an end user perspective, in most cases, no download of software is needed to access SaaS applications (sometimes users, however, may need to download a plugin). Access to the software is generally governed through an authentication and authorization mechanism. “Pay-as-you-use” model is the default standard for customers utilizing SaaS applications. From a testing perspective, SaaS environment is where the full spectrum of testing is needed. This includes, testing for upgrades, installation, configurability, multitenancy, backward compatibility, metered usage (testing different subscription models), security, compliance, legacy integration, reliability, data integrity, privacy, scalability, elasticity, SLA’s and user experience. It should be noted that, irrespective of SaaS being a delivery model, customers will not compromise the functional accuracy because of SaaS. In most cases, traditional testing tools will be sufficient while testing SaaS based application.

Functionally, there will be not much of a change in the test approach between a web based and cloud environment. However, the test teams should focus on certain aspects like performance, security, environment availability, integration with interfaces, customer experience and adherence to SLAs while testing in a cloud based environment.

Depending on the service models and implementation models selected, the test strategy will undergo changes. The choice of tools and techniques will be similar as in the case of web based applications, however during tool selection one should consider the fact that these are available on the cloud and support the pay-as-you-use model.

Implementation models

  • Private cloud: Since the infrastructure is owned by one customer, concerns on shared resources resulting in security violations and performance impact due to excessive load on shared resources are eliminated. From a testing perspective, performance, scalability and security tests provide better results (reliable test results) in a private cloud environment compared to other models. Access to the private cloud is generally restricted.
  • Community cloud: This is more like a shared cloud infrastructure for community or closed group of individuals. Access is restricted to specific groups or individuals who are members of the community. From a testing perspective, the focus would be on testing authentication and authorization.
  • Public cloud: Cloud infrastructure is publicly accessible but controlled through authentication and authorization. In this model, the focus of Testing would be on performance, scalability, elasticity, security, broad network access, and user experience among others.
  • Hybrid cloud: This consists of two or more independently operating clouds (private, community or public) that are integrated. Depending on the combination, and considering the service model, there may be changes to the focus of testing. However, integration and interface testing, performance, scalability, security, user experience and broad network access tests are common in this environment.

Based on the choice of cloud platform services and the implementation model, organizations need to design their test strategy, taking into consideration the possible test environments as specified above.

Also, look out for my next blog highlighting a structured process that organizations can use as a checklist while doing cloud based testing.

Venkata Ramana Lanka

Director - QA, Virtusa. Venkata Ramana Lanka (LRV) is an accomplished software test management professional with an extensive background in software testing, test automation framework design, building domain specific testing solution accelerators, leading large software testing teams and supporting presales initiatives. LRV is an hands-on manager with proven ability to direct and improve quality initiatives, reduce defects and improve overall efficiency and productivity. He has in-depth and proven expertise in developing and implementing test strategies and operational procedures. LRV has extensive experience in working with multiple commercial and open source test tools. As a QA thought leader, LRV has written multiple White papers and articles some of which have been published in various media. Further, he has spoken and presented papers at various software testing conferences.

More Posts - Website