AGILE TESTING: WHO'S ASKING THE RIGHT QUESTIONS?

Declan Whelan, an Agile coach who contributes to the Agile Journal, looks at the crucial role of the Agile tester in posing questions that no one else thinks of.

Whelan asserts that good testers don’t just question the final product; they question the product before work has even started and while it is being developed. And they challenge the team itself. While this a hallmark of good traditional testers, Whelan maintains that  this line of questioning is more prevalent on Agile teams, likely because a “whole team” mentality is encouraged and quality is a team responsibility.

An Agile testing matrix, developed by Brian Marik and expounded on by Whelan, help frame the meta questions an Agile tester should ask and suggests activities for each quadrant.


Whelan notes that the questions in the right quadrants are equally valid on traditional and Agile projects, but these activities run throughout the product life-cycle rather than being phased as post-development work. To illustrate the intent, some of the questions pertinent to each section are listed below.

Questions for the Left Quadrant

Are we building the right thing?
Programmers are often inwardly focused asking the question “How can I make it work?,” while testers balance this view with a more customer-centric perspective by asking outward facing questions to the product owner during iterative planning, such as:

• What benefit would users get from this story?
• How would we know the story is working?
• What are alternate ways this story could work?
• In what ways could this story fail?

Are we building it right?
The intent of questions here is to help instill a quality mindset in the team by finding and preventing defects while work is underway. The focus is on helping the team to get stories "done, done" meaning that that the story is fully complete and ready to deploy. The questions here promote interactions with the entire team.

• Can we try it together before we check it in?
• Does the happy path work?
• Is this a problem?
• Can we fix this now? (usually in the best interest of the team to fix newly found defects asap, the result of this question should be either:
• An immediate pairing session with the developer to fix the defect
• A defect card if it can’t be fixed now but still can be within the sprint OR
• a defect story is added to product backlog
• Is this story done?

Whelan believes that Agile testing in the left-most quadrants requires strong team collaboration. This adds significant value to help the team get to “done, done,” preventing defects and finding them earlier. Working in the left-most quadrants the tester mindset needs to be “Are we done?” rather than “Is it shippable?”

Questions for the Right Quadrants

Traditional testing techniques will work very well on agile teams when operating in the right-most quadrants. The main shift for testers will be to pragmatically automate some of this work to support continuous testing and to radiate information back to the team on a regular basis.

Did we build the right thing?
Whelan finds exploratory testing focuses on asking questions that trigger discovery.  Context driven testing principles and session-based test methods provide an effective framework for asking exploratory questions and sharing the discoveries and Whelan says, more importantly, session-based exploratory testing is more effective at finding defects. Outputs from exploratory sessions provide the team with a deeper awareness of the system strengths and weaknesses.

For a starting point on a more complete list of testing heuristics talk, check out the Test Heuristics Cheat Sheet by Elisabeth Hendrickson, James Lyndsay and Dale Emery. Alternatively, talk to a skilled exploratory tester.

• How might a different user use the system?
• How could the system be used in different scenarios?
• How does the system behave at boundary conditions?

Did we build it right?
This is, per Whalen, a "traditional" testing question applied to para-functional aspects of the system like performance, reliability and so on. Again, the questions below are a small subset of what should be asked.

• Is the response time of the system acceptable?
• What are the bottlenecks in the system?
• What are the largest loads that the system can handle?
• How well does the system recover when over-stressed?

Contact Software Consortium or call 1-877-850-9393 if you would like to discuss how to leverage our top-level talent to empower your business.

 

Baltimore Office
Local Phone: 410.583.9393
sales@softwareconsortium.com