Testing in the Cloud and of the Cloud.

The Cloud computing paradigm offers many opportunities for testers to add great value to their testing at little, or no, additional costs. This is because the Cloud’s scalability is precisely the property that testers have been needing for at least 10 year.

Traditionally, testers have needed additional IT support services to create a range of suitable test environments that testers can work in. Often these environments are dedicated to a single development project and often incur considerable capital ans support costs, in addition to the project test resources.

Today, the Cloud enables testers to grow and shrink their test infrastructure needs on a constantly changing basis, dependant on the testing needs of the project. Adding an other browser to the compatibility matrix? No problem! Just add another test system to the list and kick of all the tests at the same time. This eliminates one of the most frustrating tradeoff testers have to deal with. Do you one only one platform to do all the testing on and have an increasingly lengthing test cycle as more tests are added to the plan? Or, do buy more test kit so that the tests can be run in parallel but now you have yet again increased capital costs and reduced total utilisation by having even more systems sitting idle when they are not needed?

This is where the Cloud excels. You only have to pay for what you use. If you want a peak of 20 test platforms for 2 hours each, that is what you get. In a traditional environment you would have to have 20 platforms available 24/7 just to meet the capacity needs to get the testing done in 2 hour. Alternatively, you have one test platform running all the tests and it takes 40 hours to complete 1 cycle of testing. Now testing is the bottleneck in your SDLC.

Developing for the Cloud is not only a way to control costs for production systems, it also solved some very real dilemmas that testers have to face every day.

While Cloud deployment has many benefits, it should also be kept in mind that Cloud architecture is still developing and performance characteristics of Cloud applications cannot be guaranteed to scale limitlessly. Additionally, the applications rely on the security provided by your Cloud infrastructure supplier.

In conclusion, Cloud computing offers many interesting solutions for testers, but it also requires them to address the unique attributes of Cloud platforms and plan to test the performance and security of their Cloud platform.

Cloud Computing: Auto-Scaling pros and cons

There is some debate in the world of cloud computing about the value of auto-scaling. Auto-scaling is an option provided by cloud infrastructure providers to allocate additional resources from the cloud when an unexpected increase in operational load occurs.

One of the main selling points of cloud infrastructure is that users only pay for the resources that they actually use. There should be no need to pay for capacity that is not being used. This is precisely why many recent startups ups love the concept of cloud computing. In this current economic climate, the traditional route of setting up an internal IT infrastructure capable of meeting possible exponential growth is prohibitively expensive. Also, it pulls your investment capital in two directions at once. Firstly, explosive growth usually requires major investment in marketing. Secondly, there is a lag time for the procurement and implementation of IT infrastructure, so you have to hope that the marketing budget is being spend wisely and that the expected user growth will follow. If the marketing budget fails to deliver, you now have two problems to deal with. You need a new marketing strategy and you are also paying for a grossly underutilized IT infrastructure.  This is a nightmare scenario for any recent IT startup.

What to do? Marketing is, to some extent, a black art with no guarantee of success. However, cloud computing can solve the IT infrastructure procurement problem. At the core of the cloud computing offering is a proposition where the cost can be closely aligned to the revenue derived from the provision of the infrastructure. However, there are scenarios where this is not true.

A common concern is a distributed denial of service attack (DDoS) upon the application from a hostile third party. Some argue that the application provider should cope with the attack and continue to provide normal users with their services. Auto-scale is an obvious solution to this problem, but is the cost justified? This question has to be left to the service provider to answer. Where the case for auto-scaling is dubious, is when the application exceeds its known capacity. In order to know the capacity of the system, it needs to be tested to the maximum expected load. If no load testing has been done, how do you know what capacity you can actually scale to?

Many cloud advocates give scalability as the biggest benefit delivered by cloud infrastructure, but this misses an important point. All software architectures have their limits and you can only truly know that limit when you actually see it occur. This is the reason why you should load test the architecture before deploying to your customers. Not only will there be natural bottlenecks in any given software system, there are also likely to be defects in the construction that are only triggered under operational loads. Simple functional testing will not uncover these problems. They are performance related and only performance testing will uncover them. If you let your cloud service provider auto-scale your application and it hits a performance related bug, you have effectively given your cloud infrastructure provider a blank cheque to spend your money on feeding buggy software, rather than providing the services that you thought the software was delivering.

Auto-scaling has its place in the cloud, but it needs to be used wisely. It is not an alternative to capacity planning and it does not eliminate performance bottlenecks and performance related bugs. If you want to auto-scale your cloud infrastructure, you must know its limits, and that can only be achieved with a rigorous phase of load testing before the system is declared operational.