Category Archives: AgiLe

How to reduce the scope of testing during testing

It is common to reduce the scope of testing before the test. In my test charters I use things like feature to be tested or planned time. For me this is a way to prioritise my tests. Here are some stories about scoping testing in full testing mode.

Reported as usual

This story happened in real life. In order to share it with you, the reader, I used the VIP cinema appointed of the real app. The new feature of the app was that the clients could make tailor made purchases. Some custohhhmers like free parking or discounts for snacks. So, customers could make a package with the discount set to 50 percent.

Not in my backyard

One day I had to test a feature for a package. My steps were to make a package, add components, and use the package. I focused on the normal situation avoiding making typing errors and entering invalid values.

The start was quite promising: I made a new package and gave it a name. Then I added a decent number of components to the fresh package. When it was time to use the package, I noticed something strange. It was not usable. The package did not give the promised benefits to the user.

It took me a few seconds to realise that I found a bug. Now it was time to make a bug report. It should contain all steps that led to this strange situation. In my notes I found all the information I needed.

Not in my front yard

I made a new package, but after adding some components, something went wrong again. I carefully looked at the new situation and noticed a new bug. Now I needed to repeat all steps again for a new bug report.

So, the whole series of steps had to be repeated for a decent bug report. After a few times I could show the bug every time, so I added all the steps to the bug report.

Not in any of my yards

I tried to do the next few steps. Again, I noticed something wrong, so I had to make another bug report. Once again, the whole stuff repeated, I had to repeat all the steps until I could reproduce the bug and then write another bug report.

And this pattern repeated until I got to the 15 steps for initial bug. It was very difficult, because every 3 steps I got an error for which I had to make another bug report showing in the steps I took. I only wanted to make a package with some components.

It has my attention

A few weeks later a programmer told me: “You made those many bug reports?” I remembered the bug reports and was proud about my findings. Then he continued: “All the bug reports had the same issue.”
My spirit dropped after his remark.

I had reported a lot, but actually one single bug report was more than enough. Were the bug reports a useful way to report the bug to the  programmer? Sure, but it costed a lot of time. I spent at least one morning to write bug reports.

What I remember is that it took so long to get some feedback on my bug reports. If I just could get feedback after the first one, it would just save me so much time.

Interrupted as usual

During one test session in another company, I saw something strange. I tried to make eye contact with the programmer. But he was fully engaged in his programming, so I called his name and he looked up.

“I saw something strange. Would you have a look?”, I said. He came to my desk and looked how I reproduced all the steps.

Sometimes he asked questions to clarify the situation: “What did you do?” or “Why did you do this?”. Then he went silent, stood up, and went back to his laptop without saying a word.

Wait a minute

I just waited.

After a long silence I got my answer: “It is a bug.” Now I knew that everything in this place I had to avoid during testing. I needed to reduce my scope of testing that very moment. Everything using this specific function could have a strange side effect because of the found bug.

This way of reporting a bug was on the request of the programmer. He knew that I could find things he had not expected.

He also knew that if I would continue testing, that it was very hard for him to reconstruct what went wrong.

Accessibility Poker – This is my view

Accessibility Poker – This is my view.

This is a friendly reminder, that all stories with George are fiction.

The story so far

By assigning Accessibility Points to tickets with obstacles for blind people, it is easier to prioritise work. A user story cannot contain more than 1 action with more than 5 Accessibility Points. This way users can use the web site in an accessible way. Also, the sum of Accessibility Points for a single task should be limited.

Some problem

“Hi George.”
“Hi Polly.
How are things going?”
“During the A/B testing something noticeable happened: sales dropped enormously.”
“What happened, Polly?“
“They changed something to improve accessibility, George.

There was a new page and they thought that they did something pretty smart. Before, it had only a picture and some texts and they replaced it by a movie with a sound track and captions.

If someone was blind, then she or he could hear the movie. Deaf people could read the captions in the movie. We thought that we did a decent job but the A/B testing proved otherwise. We cannot find a solution. George. How do I solve this problem?”

“Polly, your team thought that people were either blind or deaf. If someone is both blind and deaf, what would be the use of sound and captions of the movie?“
“They cannot hear the movie and they cannot see the movie, so they don’t get any information. George. That is really insightful.”

Some thing

“There is something different.
In May sales also dropped. We did the promotion for haptic gloves. Something went wrong though. The sales went up, after the promotion article on the homepage disappeared.“

“Could you tell me more about this article, Polly?“
“Of course, George. It was for a new haptic glove for the fashion industry. It also included a movie.”
“Can you still play this movie, Polly?”
“Sure, that is easy.”
“Let’s do a simple experiment: you play the movie without showing it to me. At the same time, you tell what is being shown for 10 seconds.”

“George, I found the movie. Let me start:
A user with haptic glove … fabric … screen … appreciation …. Order… happy.
That were 10 tough seconds. It was too hard to tell what happened on the screen. ”

“What would usera with a visual impairment experience?”
“I think that they have a lot of problems to recognise the pictures, George.”
“Indeed, for me this story would be really difficult to follow.”

“You said: “me”.”
“I mean a user with a visual impairment.”
“George, you have a visual impairment. That’s why you got a headache, when I was quickly browsing a website a few calls ago.”
“You are smart, Polly.”
“Your secret is safe with me, George.”
“Thank you, Polly.”

Some personas

“George, something with the Accessibility Points will go wrong.”
“What do you mean?”
“At the moment we have personas for a blind person and a deaf person. Now we need personas for a blind and deaf person and a person with a visual impairment.”
“You know, that there are different visual impairments, Polly?”
“then we need to use more personas. This is my view.

I do not like to add multiple numbers of Accessibility Points to a ticket. It is too much administration.”
“What is the minimal number of Accessibility Points you need?”
“At least 1.”
“If there are 7 numbers of Accessibility Points for different personas. Which one would you take?”
“I would take the highest. This sounds lean to me.”

Some offer

 “It’s almost 9 o’clock. This was our last call. So, it’s time to say goodbye.”
“There is no other meeting lined up for me, Polly.”
“In that case I have a special offer for you George. You can work for our company full time get all usual benefits like health insurance stock options. A lot of colleagues are thrilled to work with you.”

“Polly, half hour a day is more than enough work for me.”
“George, this is hard to sell to management though.“
“I’m sorry, Polly.”

“Thank you for all the support. My team and I really appreciated it. Take care. Goodbye, George.
“Good luck. Goodbye, Polly.”

Some disappointment

“Good morning, Polly.”
“Good morning, John.”
“Did George take our offer?”
“I’m afraid not.”
“Did we not show that we have the right mentality and use technology in the right way?”
“It was something different.”

The end.

Accessibility Poker – Watch your step

This is a friendly reminder, that all stories with George are fiction.

The story so far

By assigning Accessibility Points to tickets with obstacles for blind people, it is easier to prioritise work. A user story cannot contain more than 1 action with more than 5 Accessibility Points. This way users can use the web site in an accessible way.

Supplier chain

“Hi George.”
“Hi Polly.”

“That is strange. You did not wave back, George.”
“I blocked your camera. The last time I got too many impressions, when you browsed a web site.”
“If I promise not to share my screen, are you willing to look at me?”
“That is fine.”

“Hi Polly.”
“Hi George, It is nice, if people wave back.”

“Polly, how are things going?”
“Since our last call my team made a list of all the suppliers of components for our web site. There were a lot. I never thought that there would be that many. Any way, we focused on accessibility testing on these components.

Most of them were accessible or they could be made accessible in an easy way. But we had some troubles with the cookie banner.”
“How did you solve this, Polly?”

“We called the customer service for help. It was the first time that they heard of this problem. The customer service agent requested us to tell about the problems. We had about 12 problems.”

Supplier strain

“The same day an account manager called me for some clarification. So I showed some bug reports. He was genuinely shocked and wanted to help us.”
“This sound good to me.”

“Then the talk changed. He noticed the Accessibility Points of the bug reports.
So I told him that these points represented the amount of energy used by disabled people. Furthermore actions with more than 3 Accessibility Points should be avoided. Then he started to talk about velocity.”

“Is this supplier agile?”
“Maybe. So I told him that it would be great, if his company would deliver the solutions within a few weeks. According to him velocity was not about the speed of solving the tickets, but about the velocity of the user.”

“Was he expecting that the user would solve his problems?”
“I was having the same thoughts and asked him for a specific example.”
“He talked about the state of the toggle button, which could not be detected by a screen reader. If this was possible, then the number of Accessibility Points would drop to 1. This way a blind user would go faster. The velocity would be higher.”

“I told him that I could follow his explanation, but it was confusing to use the word velocity for a user.”

Supplier drain

“The first demo went right. It was simple to navigate with a screen reader while the screen was turned off. The account manager started again over velocity, which lead to a lot of confusion.
Soon another demo was announced. This time a blind person would use the improved cookie banner.

This demo was cancelled at the last minute. The programmers still needed to do some tweaking. In the meantime we already used a cookie banner from another provider. After a few weeks of promises and delayed demos we definitely switched from supplier.”

Supplier vain

“It looks like everything is solved, Polly.
What kind of help do you need from me?”

“The cookie banner problem is solved. There is something different.
People in my company started to use velocity with relation to Accessibility Points.
I did a bit of research. There is something wrong with the word velocity. I even found this tweet of Ron Jeffries about velocity. I put it in the chat.”

“Polly, can you tell me more about the first demo?”
“The screens were quite compact. It only contained a short understandable explanation and a single question. The tester in our team could easily navigate these screens using the screen reader.”

“Polly, I have another story to explain this to you.”
“I am all ears, George.”

“During one of our calls you told me that your husband and you share running as a hobby.
Now imagine that you are trained well enough to run a whole marathon. Then there is an announcement in the newspaper that there is a marathon in your neighbourhood. You both buy tickets.

On the day of the race both of you want to use the car, but it does not start. So, you put on your running gear and walk to the bus stop nearby. You missed the bus by minutes. This is no problem, because a train will be leaving soon. So, you run to the railway station.

Again you have bad luck and miss the train. So, you decide to run to the start of the marathon. After one and half hour of running you arrive.”

Supplier stain

“George, I would not run the marathon.”
“You have trained well and you have bought tickets.”
“It is just too long.”
“You made it to the start of the run.”

“Yes, I made it. The velocity was good. The number of steps to the start and the number of steps of the marathon are too many. It will cost me too much energy.”

“How would you summarize it?”
“Watch your step.”
“This can still be confusing with velocity, Polly”
“Maybe I should say: watch the total number of steps.”

“Let me do a search on Twitter, Polly.
In the chat I put a link to a tweet of Ron Jeffries that story points should be added.”

“George, I do not understand, what you mean with the running story.”
“You told me about the small screens in the first demo. What do you think what happened with the biggest screen?”
“It had about 30 toggle buttons. I would expect a lot of small screens. I would stop after 20 screens. This is really unpleasant.

Wait. You compare a screen with a step.
This could be a good explanation for the delayed cookie banner demos. Even a person without sight problems would leave the website in frustration.
Thank you for the insight.”

“Polly, it’s 9. I have to go to the next meeting.
Goodbye Polly.”
“Goodbye George.”

Supplier pain

“Hi Polly.”
“Hi John,”
“Did you discover what might have gone wrong with the cookie banner supplier? Did he use the wrong tools?”
“It was something different.”

To be continued.