How to reduce the scope of testing during testing

It is common to reduce the scope of testing before the test. In my test charters I use things like feature to be tested or planned time. For me this is a way to prioritise my tests. Here are some stories about scoping testing in full testing mode.

Reported as usual

This story happened in real life. In order to share it with you, the reader, I used the VIP cinema appointed of the real app. The new feature of the app was that the clients could make tailor made purchases. Some custohhhmers like free parking or discounts for snacks. So, customers could make a package with the discount set to 50 percent.

Not in my backyard

One day I had to test a feature for a package. My steps were to make a package, add components, and use the package. I focused on the normal situation avoiding making typing errors and entering invalid values.

The start was quite promising: I made a new package and gave it a name. Then I added a decent number of components to the fresh package. When it was time to use the package, I noticed something strange. It was not usable. The package did not give the promised benefits to the user.

It took me a few seconds to realise that I found a bug. Now it was time to make a bug report. It should contain all steps that led to this strange situation. In my notes I found all the information I needed.

Not in my front yard

I made a new package, but after adding some components, something went wrong again. I carefully looked at the new situation and noticed a new bug. Now I needed to repeat all steps again for a new bug report.

So, the whole series of steps had to be repeated for a decent bug report. After a few times I could show the bug every time, so I added all the steps to the bug report.

Not in any of my yards

I tried to do the next few steps. Again, I noticed something wrong, so I had to make another bug report. Once again, the whole stuff repeated, I had to repeat all the steps until I could reproduce the bug and then write another bug report.

And this pattern repeated until I got to the 15 steps for initial bug. It was very difficult, because every 3 steps I got an error for which I had to make another bug report showing in the steps I took. I only wanted to make a package with some components.

It has my attention

A few weeks later a programmer told me: “You made those many bug reports?” I remembered the bug reports and was proud about my findings. Then he continued: “All the bug reports had the same issue.”
My spirit dropped after his remark.

I had reported a lot, but actually one single bug report was more than enough. Were the bug reports a useful way to report the bug to the  programmer? Sure, but it costed a lot of time. I spent at least one morning to write bug reports.

What I remember is that it took so long to get some feedback on my bug reports. If I just could get feedback after the first one, it would just save me so much time.

Interrupted as usual

During one test session in another company, I saw something strange. I tried to make eye contact with the programmer. But he was fully engaged in his programming, so I called his name and he looked up.

“I saw something strange. Would you have a look?”, I said. He came to my desk and looked how I reproduced all the steps.

Sometimes he asked questions to clarify the situation: “What did you do?” or “Why did you do this?”. Then he went silent, stood up, and went back to his laptop without saying a word.

Wait a minute

I just waited.

After a long silence I got my answer: “It is a bug.” Now I knew that everything in this place I had to avoid during testing. I needed to reduce my scope of testing that very moment. Everything using this specific function could have a strange side effect because of the found bug.

This way of reporting a bug was on the request of the programmer. He knew that I could find things he had not expected.

He also knew that if I would continue testing, that it was very hard for him to reconstruct what went wrong.