How automated quality assurance (QA) works at 3Pillar

One of the core principles of The Product Mindset is minimizing time to value. Our strong belief is that by building and releasing product in slices in order to quickly collect customer feedback, we can develop digital products that drive outcomes our clients seek.

To work at this pace, all of our disciplines—product management; user experience; and engineering—must work in alignment, and be aware of the entire software development life cycle (SDLC). At 3Pillar, we send our quality assurance (QA) teams in early on projects. They listen from the outset, and hear directly about the customer’s needs and requirements. As part of our agile process, we define what ‘done’ means, which helps alleviate confusion. Then we plan our QA strategy and process based on the needs, scale, and timelines of the project. Our experience has shown us that the quality of the process leads to higher quality products. We then take those requirements and look at our decision matrix to help us determine the right tools to use.

Some of the key points to determine the right tools include:

1. Assessing organizational maturity and identification of opportunities for test tool support.
2. Assessing appropriate objectives for test tool support.
3. Analyzing tool information against objectives and project constraints.
4. Estimating the cost-benefit ratio based on a solid business case.
5. Identifying compatibility of the tool with the system under test (SUT) components.

While QA can be done manually (to some extent), there are benefits and advantages to automating the process when it comes to regression testing, load testing, performance testing. This is especially true if there are Continuous Integration pipelines implemented. Our QA teams, which include delivery leads, architects, and others, look at the technologies that are going to be used in actually building the project. Adi Balogh, a 3Pillar Global QA lead in Romania says, “For example, if I know that I’m going to have really dynamic web pages that are changing constantly, based on my previous experience I know that Selenium is not a very good tool for handling situations like that. Working with it in the past, I had to find out all sorts of workarounds and hacks and they were not reliable because the tool itself is not reliable in that specific use. While It’s good for web automation, it lacks in other areas”.

Like any experienced craftsperson, finding the right tools is a crucial aspect to complete the task.

Adi says “there’s no perfect tool, but if I have a tool or a framework that will give me speed into writing or covering tests, or solving problems, I will definitely push for a new tool even though that means challenging the status quo, or having people to invest in changing what their knowledge is, or expanding it.”

To find the right tool, Adi sets up a decision matrix and starts digging into the characteristics of each tool. He sets up a small proof of concept (POC) to see if each tool he’s considering meets his needs, and then grades them based on the requirements. Adi says that “this is good for boiling down to things that really matter for the client, and also for your team. If I’m going to pick a tool that solves all my problems, but it takes me three weeks to do one simple task, it is not going to be effective, and we might not meet desired quality standards by our deadline .”

Once a candidate for a well-suited tool is chosen, Adi communicates with the product architect and the rest of the team, to drill down into any further requirements. Adi says “and after that basically, we just start implementing testing in our cases, building the framework, and having tests running against the app or whatever we are developing. And that’s the journey and how we get there.”

An effective QA test strategy, with the QA team involved from the beginning of the process, all disciplines involved and communicating, and using our product mindset principles, are the keys to finding the best test automation solution that’s aligned to the requirements, and drive the best products.