You are listening to QA talks, a podcast for quality assurance executives implementing digital transformation in their organizations. In this show, we focus on the unique pitfalls inherent in quality assurance and quality engineering and how these executives are navigating them to position their organization for the future. Let’s get into the show. 

Logan: Welcome back to QA Talks. I’m Logan Lyles, your host for today’s episode from Sweet Fish Media. We’ve got with us today two very experienced speakers on the podcast. I’m joined by Mark Bland, Senior Test Manager over at Premier Farnell. Mark, how is it going sir? 

Mark: Hey it’s good. Nice to meet you, Logan. 

Logan: Great to have you on the show, Mark. I’m also joined by Nanda Padmaraju. He is the SVP at Cigniti Technology. Nanda, welcome back. Great to have you on the show for this episode. 

Nanda: Hey Logan. Thank you so much for having me back on this podcast. Pleasure! 

Logan: Absolutely. Gentlemen, we are going to have a great conversation today about agile transformation, test automation, and some other hot topics for listeners today. Before we do that, I want to provide a little bit of background on both of our speakers for some context. As we mentioned, Mark Bland is senior test manager over at Premier Farnell with 15 years of experience. During that time, he’s worked on everything from large–scale platform replacements to small feature delivery on websites and mobile apps. He’s managed onshore and offshore resources both directly and outsourced in multiple locations and in functional automation and non-functional disciplines, implementing a number of tools and frameworks in automation and non-functional testing along the way. He says he accidentally fell into software testing after 10 years of working as an electronic and electro–mechanical engineer but found that he immediately loved the industry. And now a little bit more about our other guest today, Nanda.  

Nanda Padmaraju is senior vice president at Cigniti Technologies. He is our co–speaker for this edition of the podcast. Nanda heads up the UK–EU business unit for Cigniti. He brings in 22 years of experience in consulting, advising, and helping global enterprises across domains in their quality assurance and quality engineering transformation.  

All right. Now that we’ve set the stage a little bit, Mark, I want to turn to you with our first question for today. You know, agile and DevOps really transform the way enterprise functions today. I think all of our listeners really understand that. I’d love to kick it off with a question for you about, you know, really, what would you say are the primary characteristics required in an agile transformation from leadership? 

Mark: Yes. Well, I think first and foremost, you’ve got to be open–minded and willing to embrace change. I think I’ll give you an opportunity to work together as a team and really deliver something with a common goal rather than traditional methods, which would have been some way that teams working against each other at times. So you’ve got the ability to listen to other people’s opinions. They’re willing to have your opinion challenged. I found that that is probably one of the one of the bigger ones. They just haven’t been willing to have your opinion challenged and accept that, you know, whilst your way might not be a particularly wrong, it might not be the best way. You’ve got a collaborative mindset. You’ve got to be one willing to work with other people and and contribute to work as a team. And one of the things I think I find really helped me along the way is learning to step back and allow my team to self-organize and not only make but learn from their own mistakes without jumping in and micromanaging them. I think as a manager, Agile really is about taking a step back and letting the teams do what we hired the teams to do and get to round it. Oh, I would summarize that. You know, it takes a lot of courage and trust. I think as a manager, I need to trust my team and I need to have the courage to like stand back and let them deliver using the skills that I hired them for, but also pass that message back up to my management and go the other way with that. You know, and coach my management to trust that we will deliver what’s been asked of us. 

Logan: Mark, I love the way that you rounded that up there. You know, for listeners, I think you hit on four key elements there – humility and approach of collaboration, leadership that does not micromanage, trust – fostering that within your team, and also kind of managing up and looking to build trust and instill trust in in your leadership as well. You know, Mark, some people talk about this difference between doing agile and, you know, kind of dipping your toe in the water and being agile. Do you see a big difference between those two mindsets in your experience with your teams? 

Mark: Oh, absolutely yes. Something, simply put, you can follow all the steps and run an agile process and stick to all the rules and guidelines set out for best practices. We’re running agile and you can do all that while still underneath I’ll believe it’s just a fad or a trend and just pay the whole thing lip service. So, you’ll manage, you’ll get well delivered, and you’ll be doing agile. I think what tends to happen – I love teams – I’ve seen this in transition as far as you start out and that will be some members of your team being in leadership, the people within the team will start out with that, doing agile following the rules, mindset. But then what you hope happens is that over time all those people will start to realize the benefit of delivering in this way and will start to buy into the principal and the culture of it. I think that’s when you stop being agile, when everything that you do is approached in the agile, collaborative way. So, any challenge you face, you don’t go back to the drawing board and go back to old methods of dealing with these problems. You come together with a new solution, not problem. And here you do that in a way that best serves the scrum team. And if you’re doing scrum out there in the delivery team, whichever message you use in best delivers the best result for your end user and your customer. 

Logan: I love that, Mark. So, talking about the end result for the end user and customers, you know, customers today are really expecting a world–class digital experience. They expect it to be seamless, frictionless – all of those buzzwords that we hear, you know, and I know that you’re involved in making that digital experience better for your customers at Farnell. Can you share some specific ways that you’ve started to orchestrate and implement some of those changes for customer–facing experience? 

Mark: Yeah, I’m by no means an expert on this. I have my views and things that work for me. But I think implementing testing practices as a whole that enable a faster delivery and the test code in a really short timescale is the key. Contirbuting more to the team, for me, getting more involved with the solution and the sculpting side of the applications, the new things that we were delivering and making sure we are catching any failure at all and risks, but also making sure that we can feed back at that point the experiences we’ve had with customers and the negativity we’ve seen with things that haven’t worked in the past. I think one of the things that moved us forward a lot in terms of providing a better solution for our customers was introducing the likes of security test into our basic automation test, smoke tests, and principles. So covering another risk that customers base is very aware of in terms of their security and financial data security, those kind of things, but also looking at how you can then expand on that and look at introducing early and testing for accessibility in those kinds of usability challenges that you’ve started to come to the forefront really in terms of creative design and and application design. All the things we tried to focus on were expanding our automation coverage to cover more variations, of user scenario, a more compatibility type way. So different browsers, different types of setup within the actual application, including mobile variation and tablet variations, all those kinds of things mean that you are trying to deliver a much more rounded quality experience to you customer. Beyond that, I think it all comes back to the plumbing, really. 

Logan: Yeah, that makes a lot of sense, Mark. So, you talked about, you know, mobile variations and browser compatibility. Can you go a little bit further there, Mark? You’ve mentioned testing and how that helps you go faster. Can you speak a little bit about, you know, automation testing and in some of the specific increases in effectiveness, efficiency, and coverage of the software testing? In addition to helping you just increase that release velocity, which you mentioned, you know, is a great benefit to the customer experience when you’re able to deliver solutions faster, are there other things that you’ve seen in the benefits of testing automation in the projects that you’ve been involved with?  

Mark: So, I think automation, if you improve it right, if you implement it correctly, it doesn’t just add another layer of testing. What, in theory, it should do is move obstacles out of your current test team’s way to enable them to do much more complex and interesting testing. So, you get your teams more engaged because they’re doing the more interesting testing themselves manually. But then you give yourself this capability to expand the breadth of your automation coverage, and your test coverage to cover multiple web browsers, cover multiple combinations of compatibility issues to cover security, to cover a lots more functionality. For ourselves, one of the big challenges we had was that we actually have 48 websites. Across those 48 Web sites, we cover 28 languages. So, there’s a lot of variation in that as a manual test. If you were to test all of that and make sure it was right in all those scenarios, you’ll be there for weeks. The automation gives us the ability to write a single set of tests that we can roll out with just environment variables to cover those variations. I think, in theory, it gives you that capability to test wider, reduce regression times, and expand the amount of tests you are doing, all in a much smaller timeframe, which is invaluable because, as we all know, time–to–market is the biggest demand now. I’m a firm believer that there’s a shift in our market from quality assurance to delivery assurance where people are more concerned about getting things fast and about getting things right, you know. So, we’ve really got to cater to our customers with that and be able to deliver at that pace while keeping that risk low. 

Logan: Well, Mark, that is a really good connection because anytime that we approach things with the end in mind and what is the customer’s viewpoint, you make a really great connection there between the expectation of speed of delivery, even more so sometimes over quality and thinking about those shifts and how they impact how do we go to market.  

Nanda, we’ve introduced a lot of your experience at the top of the episode, and I haven’t given you a chance to to chime in here. Since we’re jumping in on testing a good bit here, as Mark’s been talking about automation, you know, many customers have benefited from Cigniti’s test advisory and transformation service. Can can you elaborate on some of the benefits customers have seen from this service while we’re on the subject of testing here. 

Nanda: Sure, Logan. I’ll take a real example. We work with a large Airline conglomerate. They have five brands. Most of them came into their fore through M&A route and hence have their own code base, their own teams, their own vendors, frameworks, and tools. 

When we landed up with the client’s place, we gave them a thirty–thousand feet view of why they need to kick start a common approach that is then drawn down by all the brands. Aspects such as scalability, reusability, maintainability can drive a lot of efficiency gains that translate into multi-million dollar savings per year for large enterprises. Tool rationalization, focused approach to increasing release velocity, and moving from waterfall agile – agile is a topic that Mark touched upon earlier, and very rightly so – and now to DevOps, which includes all aspects, including the culture, people, tools, and technologies, are all initiatives that are an outcome of the advisory and transformation service that Cigniti offers. We have a group of top–notch consultants who are the best of the breed come with loads of relevant experience and they normally operate at a CIO–level Logan. 

Logan: That is a great rundown there, Nanda. That sounds like the team has been doing some really top–notch work. Can you speak a little more broadly as well Nanda on just the importance of having an enterprise test strategy and maybe a little bit on how Cigniti can help build one for customers, so one two punch there and in the response here.  

Nanda: Sure, Logan. At Cigniti, we believe in having a signed off enterprise test strategy is fundamental to ensuring quality in everything that goes into production. The test strategy will act as a Bible and not just for the testing function, but also for the upstream Dev and downstream Ops. It will guide the business analyst to document not only the functional requirements, but also the non-functional requirements upfront. It will analyze how we can drive the ambiguity out of the requirements early on using the shift–left practices. It will clearly state that quality gate criteria. It will encourage the adoption of agile and DevOps as their enterprise test strategy needs to be aligned with the overall objective of IT and that of the business. Leveraging the latest tool sets and focusing on being an independent voide of quality in the SDLC is better achieved when you have an agreed enterprise test strategy. Cigniti leverages our collective wisdom of serving hundreds of customers and bring in the best practices while understanding the context and constraints of each customer in putting together the most relevant, practical enterprise test strategy that works and is aligned to the overall vision of the CIO office. 

Individual projects will then draw down and prepare project level test strategies or test plans to ensure they stay aligned and are combined with the enterprise test strategy. Back to you, Logan. 

Logan: Thank you for that, Nanda. That collective wisdom, I loved hearing that line that you just shared. You know, Mark, Nanda was speaking a little bit there about the toolset. You know, automation testing is highly tool–dependent. Can you talk about the challenges that you’ve had to address to identify the right automation testing tool for your application landscape? 

Nanda: Absolutely, yeah. So we like all companies and organizations that think about our fair share of challenges with this. When I earlier joined the team that I’m in now, I really didn’t have much experience in testing. And I was given QTP – an old version of QTP. The record inside of that worked great. And it added some benefits. I really didn’t understand the principles back then. I guess it’s how you should be looking to build out a test framework to act as an augmented resource for your activities. So I went through a number of journeys I think with test tools – trying different tools, trying different approaches. The first really successful framework we manufactured here was put together was on an old scripting language called Ruby, a test framework called Walter, web application tested in Ruby, really simple language to code with a test framework. We built a framework that worked very much in a kind of way, you could call it, from a simple web browser. And that were all open–source. And I think this is one of the challenges I find with companies. A lot of the the time, people want automation, but they don’t want to spend a lot of money on tool for automation upfront. They want to know that it can work. To anybody out there who’s looking to implement automation, I strongly recommend you pull up proof of concept together first and prove its worth before trying to get buy-in for funding. It just makes life so much easier. That is one of the biggest challenge I had to be getting for it to implement. But when we got the idea, then implement a framework. And like I said, the first Ruby framework actually worked fantastically and was a great tool until Firefox started doing weekly updates to their code. So once Firefox started doing agile delivery and start rolling out new versions of Firefox every couple of days, the plugins were needed to enable the framework to work and we weren’t being there quick enough. We quickly fell behind in terms of compatibility and had to have to take a step back. So, we took a look at the industry again and a lot of people seem to be using Selenium. And so, we implemented the selenium framework – again a very basic one at first. But then gradually you learn from your mistakes. You learn from trying to overcomplicate an automation framework and you learn what was actually important to the framework and gradually get to this point through those exercises where you understand what the key things are that you need to put in place and the tool becomes agnostic, if I’m honest. At this point, once you understand what is it that you’re trying to achieve with the automation tool, it really doesn’t matter which tool you use. That said, there’s a lot of new tools on the market which are introducing a whole new dimension to automation for me that I’d love to start exploring, but that’s probably a story for another time. 

Logan: I love it, Mark. Some really good context there. Nanda, I’d love to hear your thoughts here on picking the right test automation tool and also where you’ve seen customers leverage Cigniti’s test automation framework. You know, Mark spoke to the importance of developing your framework there. So, another kind of two–part question coming back at you, Nanda. 

Nanda: Sure, Logan. And just going back to what Mark said, I 100% support that whole thought process of laying the right emphasis on the right test automation tool is half the battle won. So Cigniti has an automation center of excellence which comprises of automation architects that build and maintain best automation frameworks to cater to various underlying platforms, applications, and technologies. Cigniti also has partnership with all the leading tool vendors, which entitles us to have a preview of the upcoming version of the test tools. With the skill and capability backed by the tool partnership and deep technical expertise, Cigniti offers tool feasibility study as one of its offerings. So, we go to a technology landscape and understand the technology lay of the land, the future technology roadmap, the current and future application roadmaps as well. And put in a mix of open-source and commercial tools to then build and run a few automation scripts and compare them objectively. So we have an objective rating criteria across 30 parameters and then aggregate that and do a total score which will help the customer make the right choice. There are several parameters, not just technical, commercial, but ease of use and maintainability, scalability, robustness, and future proofness. So, all of these together give you an objective score that has significantly helped the customer make the right choice. And once they make the right choice, Cigniti, having served hundreds of customers, has the right frameworks independent of the underlying technology landscape to be able to aggregate all the classes, frameworks, and reusable tools to be able to bring to bear the jumpstart kit, which usually saves about 20 to 30 percent of the effort for all new customers whoare embarking on this automation journey. Logan? 

Mark: If I may.. 

Logan: Go ahead, Mark. 

Mark: To add to that, I think companies like Cigniti, you’re starting from nothing with automation, you just begin to ream into implementing automation. Then for a lot of cases, I think partnering with a company like Cigniti, that’s actually a godsend because they bring all that learning wisdom, they bring experience with multiple applications and all the challenges that you will face if you try and develop your own frameworks have already been covered. And all the other challenges that find people how with automation, if trying to automate too much too soon. If you just automate every test that you do, what you actually create is a massive maintenance task, not helpful automation framework. And I think companies like Cigniti bring a wealth of experience that really helped you start that journey. And you know, like Nanda said it gets you the way through right from the start of understanding what you need. I think that insight can be absolutely invaluable if you come into this blind. 

Logan: I love that context, Mark. You know, talking about speed and the benefit there from earlier in the conversation, like you said, getting kind of a 30 percent jumpstart can be so invaluable. You know, I think both of you gentlemen have alluded to this a little bit, but I’d like to round out the conversation a little bit here with with you, Mark to share some best practices on how to get the right mix of automation and manual testing. We’ve spent a good bit of time talking about the value and the applications of automation in testing. But I imagine that doesn’t lead you to believe that automation should just be applied blindly everywhere, right?  

Mark: You know, I’m currently in the process of potentially changing my opinion on this. I’ve always been a strong advocate that you require both a manual test phase and automation test phase. But for purely pragmatic reasons, up until very recently, automation tests have only been able to deliver simple functions that are repeatable tasks. But with the advent of machine learning and AI, we’re getting to a point now where automation frameworks can, in theory, make decisions about what the right route to take in effect, what the right data to use. And, you know, these are things that up until now, automation hasn’t been able to do. It hasn’t been able to make those intelligent decisions about for this particular kind of test. With this feedback I’m getting from the application, I now need to make it simple, whereas AI and ML give you a whole new world of possibilities that will only improve as you implement it and as it grows. Right now, personally, I think we’re not quite there yet. So, I would always argue that you will not get to 100 percent automation test coverage and companies have said that they have. In my experience, when you really break it down, what they mean is they’ve got to 100 percent coverage of automatable all tasks. I think that’s the real question here is not how much can you ultimate, but how much of your application is automatable, how much of it is complex and requires too many inputs and that is challenging. I think I and I know a really going to blow the doors wide open on that. And we may actually get to a point now where you can 100 percent automate things. I imagine it’ll be a good 10 years before that mainstream fall for every other company out there. It will be led by Googles and Amazons and people of that nature. But I have genuinely changed my opinion in the last couple of years due to things I’ve seen and worked with. 

Logan: Well Mark, that’s a really good context. I appreciate the insights into how you’re thinking about this now and also, you know, seeing the trends. And I you know, I just jot it down is as you were talking that question that I think listeners should be thinking of as you change the context of how much have we automated and what is automable and what’s potentially going to be automable in the next few years and kind of just making that subtle shift in where you look at applying your test automation. Nanda, I’ll kick it back over to you as we round out the conversation. When give you a chance to to chime in on Mark’s context that he just shared there.  

Nanda: Sure. Thanks, Logan. I’ll go back to one of the earlier comments that Mark made, and rightfully so. When he touched upon mercury toolsets, I got nostalgic. I believe about 16 to 20 years ago when I personally made my hands dirty with WinRunner at that time, which then went on to become the QTPs of the world and the UFTs of the world, and now the LeanFTs. So, tools change, technology changes that I think we’ve improved. There are two main outcomes old concepts. A one a lot of people believe that life is too short to focus on your weaknesses. Focus on your strengths and lean on the experts to blunt the weaknesses. So that’s fair. I think Cigniti comes into play. Why reinvent the whole wheel in terms of automation when we have learnt the hard way and we can bring in the best of practices? Again supported by Mark’s views on this subject. From there, we at Cigniti over the last five years have significantly seen the resource profile within Cigniti to move from a traditional testing mindset to now a completely SDET mindset. So, everybody comes with an automation–first approach. So, about five years ago, everybody was talking about mobile first approach in everything that they do in the development. Now, if a company is still not embarked on the automation journey and is not talking about the automation–first approach, I think we are left far behind on the curve with the Googles of the world, with the Netflixs of the world, Amazons of the world, Facebooks of the world. They are benchmarking to what level test automation can be taken and leveraged upon. And while not everybody has to be there at that mark. But they just proved that the technology is out there for you to leverage and harness your potential for the best things that you could do outside, which are manual repetitive tasks, right. So I fully agree with Mark’s views. The trend is slowly getting to a position where you leverage AI, ML to do things that you previously thought would never be possible through test automation. The tools are out there. The technology is out there, and they are maturing by an order of magnitude as we speak. The pace at which the technology is changing is opening up a plethora of opportunities for everybody in the testing community to become smarter, to add disproportionate and geometric-proportionate value to our customers as, again leaning back to the thought that Mark said, release velocity, time–to–market are so critical today and the technology as well supports that. So if you go wrong, you fail fast, you roll back and you keep moving, right. So, with that mindset coming from a field of pure testing fraternity, for us, quality is paramount. But at the same time, we have to meet the demands of the time to market that are out there. So balancing these two, leveraging on test automation and harnessing it to the fullest potential is the only way out for all the enterprises out there. 

Logan: Nanda, thank you so much for that. It was really great context and I love the parallel that you drew there – the connection between the shift that we all needed to make to mobile first in development and seeing a similar shift to automation first in testing. I think those are wise words and I can just tell the passion that both of you have in sharing with the community on the trends, the challenges, and the opportunities that everyone listening to this is really facing. So I want to give you guys just a quick opportunity to let listeners know how they could reach out or stay connected with both of you, providing, you know, so much wisdom and context from your experience today on the podcast. If they have follow-up questions or anything like that and would like to reach out. Mark, I’ll start with you. What would be the best way to reach out or stay connected with you, sir?  

Mark: Yeah, I think LinkedIn is probably the best way to stay connected with me. I checked that quite regularly. I keep in touch with a lot of the testing groups on their networking sites. You can reach me on Twitter. That’s probably about it on social media for me. I’m afraid I’m a bit of a dinosaur. 

Logan: You make it nice and easy for folks. That’s A-okay with us, Mark. Nanda, how about you? LinkedIn the best spot to go for folks, too, that want to follow up and reach out to you? 

Nanda: Yes, Logan. LinkedIn is the best and most preferred option. Well, I’m also reachable on my email ID nanda@cigniti.com. And on the lighter note, just to finish off. By touching the topic of mercury toolsets, I think both Mark and I, inadvertently gave away our age, how old we are within this community. So be it. We are passionate people within testing. It’s a small world out there. We love what we are doing, and we love to share what we’ve learned. We also learn a love to learn from others as well. So, keep the community feeling going, going on. I think it will benefit the larger audience. 

Mark: Yeah, yeah. I think what we’ve also done that nowadays we prove that you don’t have to be young to take advantage of the new technologies and methodologies until you are passionate about the change in our industry.  

Nanda: Absolutely, Mark. 

Logan: I love the passion that you guys are bring in and. You know, whether you’d inadvertently, you know, shared your ages or not. I think that what listeners can can say is what I mentioned earlier is that there is a lot of collective wisdom, even just right here and as we’ve mentioned on the Cigniti team and the teams that you guys represent. So, I hope that listeners will take next steps, reach out to you guys. This has been a great conversation, I think, looking at some of the challenges and some of the opportunities in the years to come. Gentlemen, thank you so much for joining me on the podcast today. 

Mark: Thank you very much for the opportunity Logan and Nanda. It’s been a pleasure 

Nanda: Thanks, Logan. Thank you for having me. 

Quality assurance is vital to the success of an organization’s digital transformation. Lack of control can quickly derail a company’s technological presence, costing thousands. At Cigniti, our resolution is to build a better world with better quality software. Renowned for the global quality thought leadership in the industry, we draw expertise from over a decade of test engineering experience across verticals. To learn how we do it, visit cigniti.com.  

You have been listening to QA talks. To ensure that you never miss ay episode, subscribe to the show in your favorite podcast player. Thank you so much for listening, until next time.