Creating a Cloud Load Testing Strategy for your Enterprise

Cloud Load Testing continues to be an increasingly discussed and important topic. Nearly everywhere I go, questions arise as to: what, why, how, and when these capabilities are important. Perhaps you are looking for a quick read to understand how to start defining a Cloud Load Testing strategy for your enterprise. That is what I am setting out to do in this blog and provide to you.

1. What are you delivering to your end users?

As an end user, I always wonder, why is it that organizations want me to find their defects? Not sure if you have had this same experience, but surely you have, and I am sure your experience is equally as frustrating…if not more.

So as an organization, we need to think about ‘What are we delivering to our end users?’ because if we don’t, we can be sure our competitors are. Most successful organizations with longevity have a ‘best intent’ to deliver valuable products / services to their end users at a fair price.

Thinking of and recognizing the results they are delivering against this ‘best intent’ is often not done. This limits the continuous assessment and feedback of the market and performance of their products / services in the hands of their end users.

So, what are you delivering, how do you get feedback, and how are you continuously improving your end users experience? The reasons I ask this question is simple; what if you had a way to capture these results from production, then replay them across your pre-production environments, getting results you could use to optimize your applications for maximum value to your end users? Now won’t that be really good for the business!

2. Why do you spend cycles proactively ensuring it works?

Consider your position in the market, what the customers are saying, and how your competitors are constantly improving. Do you even stand a chance if your strategy is status quo? I think not, and now is the time to figure out how to pivot and proactively make a difference…or else your book will end at Chapter 11.

With the Olympics and Holiday Shopping as two large upcoming events, many organizations see these as make or break, and are seeing this as a reason to spend cycles proactively ensuring their systems are working. Of course, with these events (and others like them) volumes are going to be extraordinarily high; which if your business can hold up to the increased demand, there is much to be gained in a number of ways. Do you measure any of these ways today?

Perhaps you have ways to correlate flows, transactions speeds, downtime, user sentiment, employee satisfaction and the like; to that of revenue, brand value, competitive advantage, customer (acquisition and retention), employee satisfaction, and user sentiment. In today’s organizations, this is typically not the case, and an area lacking which can result in exponential returns.

Of course, perhaps you are thinking “Yeah right Todd, easier said than done.” Would not be the first time I have heard that by the way. My suggestion, let’s start with one flow; perhaps measuring the correlation between ‘transaction response time’ of a specific page on your site or app, and that of ‘revenue’ related to that system. I believe when you start to look at the trend, and find quick ways to improve that ‘transaction response time’, you will be shocked to see the change in ‘revenue’…even if only 1%, what does that mean annualized for the organization and you? [that answer is often measured in millions]

3. How are you maximizing value, efficiency, cost, and time?

Everyone opens bananas differently. And, is there really a right or wrong way? The objective is to get to the edible fruit and nurture your body with vitamins and other goodness. Whether you start with the stem or the bananus, the shared goal is the same; there is an easier and better way, how are you doing it?

In this metaphorical reference to proactively working to maximize value, efficiency, cost and time is a very strong relationship. Again, most if not all organizations with paying customers are thinking about these factors, and working proactively to bring the best end user experience.

Along this path, whether primate or human; we are learning by doing, and observing the results, then repeating. Organizations and individuals are evaluated on how well they maximize the value of investments, along with efficiencies being built-in to lightweight processes, reducing the costs to an appropriate level, then spending only the amount of time needed to maximize the overall equation.

In a recent poll, 58% of individuals opened the banana by the bananus. This is the same behavior observed by primates. Not saying there is a right or wrong, just a majority. Where do you count yourself, and why? Is it because of how you are doing something…which could be done better another way?

4. When do you focus on these capabilities? 

“Always, at the beginning and throughout”…that’s what I wish I heard. Sadly, when the topic of working proactively to ensure a good performing experience for your end users arises, it is often something raised after a massive production incident of some sort.

Another common topic that is risen has to do with ‘highly skilled resources’ and/or ‘scheduling to have a scenario run’ as excuses as to why this has been deprioritized and not done. Seems this is another huge limiting factor to when people can run these types of tests and get the results they need, so as to improve the end user experience proactively and earlier.

I don’t know what your personal experience has been…but, when “it’s hit the fan” most CIO’s and Business Leaders are NOT saying ”Glad we did not waste time and effort thinking about performance for our end users.” It is often “How will you prevent this from happening again!” When this happens, it is a great opportunity to have a prepared response, so you can appropriately address the immediate concern, and the future for the organization and your end users.

In short, this consideration is something that should be built-in from the beginning with accountability and responsibility across the technology and business teams to the end users. Making this a reality is a bit of a cultural adaption, but having visibility on the impact this has on your organization, enables you to leverage this tipping point for the moment for the beginning of a transformation.

In the four (4) above points we can see why the capabilities of Cloud Load Testing would be important to ensure that the end user experience is optimum. There are a number of reasons why organizations find Cloud Load Testing as an easy adoption and high value. I have outlined five (5) of these below.

1. Burst: Leveraging Cloud Load Testing as a way to supplement your existing load and/or performance testing capabilities. For example, perhaps you typically run a 1000 virtual user test, but for an upcoming event, you want to ensure you can handle and tune for a 5000 virtual users’ scenario. Rather than purchasing an additional 4000 virtual users, simply leverage Cloud Load Testing to drive additional traffic to your application under test only for the required period.
2. No Infrastructure to maintain: By using the capabilities of Cloud Load Testing, you are leveraging ‘The Cloud’ to host your load generators and engines, so maintaining a separate infrastructure for this capability is no longer required. Build and maintenance of these core infrastructure components is the responsibility of the vendor you are using, as they are responsible to deliver this as a service to you. In some examples this is a significant savings of capital and operational expense.
3. Integrations: As this capability has been further adopted broadly over the years, so has the demand across the varied stakeholders. Resulting from this has been the flurry of integrations, which now makes it very simple to schedule, execute and report on Cloud Load Testing results. This is because now it can be triggered from any number of locations including: IDE’s, Build Systems, Quality Reporting Systems, Load and Performance Tools, and more.
4. Run concurrently with existing load test: Scheduling and shared resource models have increased the complexity and reduced the availability of load and performance testing resources (people and tools) in the past. Now with Cloud Load Testing, this capability is available exponentially, and can be triggered by anyone provided with access to do so. This capability along with Lifecycle Virtualization has enabled teams and organizations to introduce continuous testing scenarios with an extremely reduced costs model.
5. Adding on supplemental testing types: When driving fast and adding value, often others will turn to you for considerations on how to help solve for other challenges. One of these I have often run into is, “If you can do this with performance, can you also add: Functional, security, usability, etc.?” Well, the answer is yes. And, quite frankly, I wish more people would think this way. It makes you ask some hard questions about what your production environment is like, and how your end users will experience life with your capabilities. Just imagine, what it would mean for your organization and business if you could add performance, functional, security and usability as all capabilities under a single test with a single result which runs continuously. The time is now!

*Other resources:
Blog: How Cloud Load Testing helps you deliver amazing software
Trial: StormRunner Load (trial)
eBook: 2015-16 State of Performance Engineering
Book: Effective Performance Engineering Book

About the Author:Todd DeCapua is the Chief Technology Evangelist with Hewlett Packard Enterprise and cofounder of TechBeacon.com thought leadership site for IT Heros. DeCapua is a seasoned software professional with 20+ years of experience in IT applications development, IT operations, technology integrations, channels operations, and business development in several domains, including Mobile, Agile, Cloud, and Performance. Over the years Todd has transformed three organizations to Agile/DevOps, consulted with 100+ organizations worldwide, and amassed a variety of perspectives and practical experiences. He has earned an MBA in Finance and a BS; has been recognized with several industry certifications and awards; and is an industry-renowned leader, speaker, and author.