Q & A Bits of performance testing

“Why are you writing this Q & A?”
“Because people have questions, which are unanswered about ‘Bits of performance testing’ . ”
“Like?”
“The use of drawings and perfornance test it self.”
“So I can send in questions?””
“Sure on this page. Why not?”

“Hey, I am the one who is supposed to ask questions.”

Plan
“You used a lot of pictures. Is this not a waste of time?”
“In my blog post I drew a picture about the customer journey. This led to the conclusion that the Wifi network should be split in a private and public ones. This is not a waste of time.”
“I agree. But there were other pictures, which did not have that big impact.”
“That is right. But the drawing of picture takes minutes. Implementing the wrong performance scripts takes days.”

“You used SFDIPOT instead of another heuristic., FIBLOTS”
I am aware that this heuristic exists. I also know that a performance tester made it. And I just forgot it.
The reason I chose to pick SFDIPOT is to use this heuristic in another context. I learn by taking small detours. What if I do it in another way? My main reason was that I wanted to write about performance test in another way.
This kept my spirit high and extended my Circle of Comfort. That is a comfortable thought.”

“Your story about the performance test is sometimes difficult to follow. Why did you write a nonlinear story?”
“Testing is an activity which is unpredictable. I can find bugs on the strangest moments. This can trigger other ideas.
In this blog post I tried to describe what is going on in my head.”
“How can you sell this to your boss?”
“Just ask her or him, whether business cases are also written in a linear way?”

“I miss disk storage That is an important resource to watch.”
“You are right. I missed that one.”
“How do you know that this is a good load profile?”
“Some data have to be gathered. Think about log files and analytic stuff. The challenge is not to confuse frequency with resource usage.”
“Would you please explain that?”
“If 1000 users look to a simple web page, then almost no resources are used. If 1 user asks all articles in stock, then a lot will happen. The database is bring queried and much data is moved over the network. So look for resource usage.”

“So I only have to focus on user actions with heavy resource usage?”
“That is only possible with a very simple web site. Sometimes web sites or programs on the backend do not remove their garbage.”
“What do you mean with garbage?”

“Suppose you have a cinema web site. The purchasing department wants to know what kind of drinks and snacks are ordered in advance. Suppose all the results of the queries are stored. It might be interesting for their department and it should be stored on one of their systems. But not in a module of the web site.

Another thing is to simulate the customer. Use the customer journey. A customer does not only buys cards, snacks and drinks. She or he will also collect them. This lead to click paths, how does the user maneuver through the system: which screens will be visited and which options are used?”

To be extended

Agile Manifesto during test

The appointed ticket was about a business rule. I had done all the preparation stuff and I only had to analyse the data. In most cases a front end or a good looking application is enough to determine the quality of the work. In this case it would miss a lot of points. A front end shows a summary of the data in the database. And a summary was not enough for this case. I mean database.

What I write down, is a sanitised version of the story: all confidential stuff has been changed.

Individuals and interactions over processes and tools
I opened the database client. With this program I can easily browse through the tables. I got a connection error. That was not a good start. I retried to establish a connection and got the same connection error. I tried another server: same connection problem.
At this point I could just stop. Hey, I am a …. professional tester.

I had to start to analyse my test environment. I am not someone who runs to the helpdesk on the first encounter with an error. So I tried to make a connection with a terminal. This connection was possible which I did not expect. But thanks anyways.

Now I had a connection with the right server, I could do more things. Yes, the test environment was right back in my mind.
I remembered that it was possible to use a primitive database tool. I could use a database prompt, which let me query the database. I entered a command in the terminal: unknown command.
I switched to a browser and just entered the wrong command in the search engine, which suggested the right spelling. Using this command I saw a database prompt. The internet has lots of knowledge. Now I could use my queries from the day before.

I called a fellow devops and explained the situation. I reported about my failed attempts with the database client and he agreed that the database prompt was feasible for testing.

He told me how to connect to the right database. Thanks.
And off I tested.

Working software over comprehensive documentation
While I was still figuring out my test environment I realised, that I had forgotten to log. I opened a template of a test charter. It contained useful information like date, name, and application. Mine was based on some templates. Thanks Jean-Paul.

During my exploration I wrote some quick notes in my charter. They were mental notes to me. At the end of the day I would put a comprehensible summary in the ticket.

I copied the query from a file to the database prompt and pressed the Enter key. The database tool gave feedback, that the queries could not be used. Did I mention 1 query by the way?

My query contained multiple lines which were interpreted as multiple queries. There was some logic in it. E.g.
“Show the dates
of all Saturdays in 2017.”
The tool processed “Show the dates.” as follows:
“I have dates in my databases, but I am not sure which table: the birthdays, the working days or the school Holiday days.”
The second line “of all Saturdays in 2017.” would be interpreted like
“What would you like to know about the Saturdays in 2017? The times of the sunset or the number of Saturdays?”

My fellow devops had mentioned the option to execute the database prompt with the parameter /i. This stands for input file. This is a time saver.
So I left the database prompt and saw the system prompt again.

I executed the database tool with /i query file.
I got a response that /i was an invalid parameter.

Time to get help from the database tool:
I execute the tool with /help.
I got a list of all valid parameters and their usage. Then I settled for /f, which is short for input file.

During the next attempt I executed the database tool with /f query file. And I got a decent feedback from the database.

Both my hands went in the air.!!

Responding to change over following a plan
My plan was to use a nice looking database client and I ended up with a rough and tough database tool. Okay, but I could deliver value.

I looked to the output of the file and it was hard to read. All data was shown in ASCII or text. No table for easy scrolling and reading. No fancy stuff. The table was shown in multiple line: the header was scattered over several lines and the same was true for every record. So it was hard for me to determine which attribute was linked to which value.

I put the data in a text editor. It was still hard to read. I had to remove line breaks. No way.
But I did not need all the attributes. Oops, I was hoarding data again.
So I reduced the number of attributes to a minimal set. I accidentally maximised the window of the terminal and all data was shown in more readable table.

Now a new pattern emerged:

  • Adjust the file with query in an editor.
  • Go to the system prompt.
  • Execute the database tool with the latest version of the file.
  • Look at the output.
  • Correct the query.
  • And start again.

All that opening and closing of the input file was quite cumbersome.  I felt like a file jockey. So I opened a second terminal. Now I had one screen with the query in an editor and another screen for the execution of the database tool and analysis of the output.

Time for a tweet.

Customer collaboration over contract negotiation
After reading the Agile Manifesto I realised that I had not really listened to the customer or her/ his proxy.

The next working day I went to the business to talk about the business rule. Did I really interpret this well? Instead of conditions and codes I heard a story, what the story was about. About people who needed the right service.

PS Task 4

Technical bits of performance testing

The story so far: 3 blog posts ago I introduced a web site which offered the new option to buy drinks and snacks in advance for a cinema visit. This is an imaginary web site. Why should you need an extra drink? For the desert scene in Monsters Unlimited. Also imaginary stuff.

1 blog post ago an anxious project leader dropped by for a performance test for the next day. Somehow it ended up as a cliff hanger for … [Some dots] SFDIPOT in performance test.
Now I can continue to write about my standard talk for project leaders about performance tests. Let me add some flavour. Stirred, not shaken.

SFDIPOT is basically used to determine new angles.  Up till now I have not used it for performance test. I could make a rough picture about the system. Why not? Visual testing anyone?

This is Structure. What are the building blocks?

basic structure

The customer uses a web site to buy tickets. There is an extra module to handle orders of drinks and snacks. So an extra flow is added.
extended structure
Decades ago I would be happy with this picture. But I know now that I would miss important information.

Now I have a look to the Function or what the system is expected to do.

“Hey, what’s that?”
“The marketing department is keeping a close eye on revenue.”
“I understand that they want to increase it, but this can impact the performance of the system.”
“You are right, so analysis takes place outside this system.”
“Okay.”

“But the counters are inside the system.”
“Sure enough.”

Another way to look at functions is to visualise it.

customer journey

”Okay I do understand the pictures, but what is that rectangle with the heart?”
”It is the toilet.”
”But it has nothing to do with web site.”
“But it can impact the cinema visit. (: P)
This picture offers another perspective. You might call a visual user journey. ”

Data is all the information which exchanged in the system.
“You probably have a link to the purchasing department.”
”Yep and this one is also handled outside the website.”

data

”If I zoom in, I see tickets, drinks and snacks going this way. It’s first just raw data.
I miss some money. ”
”That is no problem. A supplier takes care of the payments. ”
”Ok.
What about all those snacks and drinks? ”
”Those are bar codes and numbers.”

Interfaces are ways to interact with the system. So …
”I am curious whether the system can handle streaming film trailers.”
”Sure no problem. The web site has two servers: one is completely dedicated for streaming. ”

”Do you have free WiFi?”
”Sure, but …”
”What happens, if a lot of visitors start to watch a movie trailer on their mobile?”
”Wait. You are telling me that my internal network could be slower. I did not think about this option. We need to take measures like a separate network for visitors. ”
”Yes.”

”How did you figure this out?”
”I was thinking very fast.
Like . ”
”Could you please slow down?”
“# $ €
|:| |:|”
”Man, you think fast. Let me see: # stands for numbers, €$ for money and |:| |:| for … ”
”For movie. ”
”Okay. I know a lot of symbols, but this one is new. ”
”I just figured that one out.
In the past films were on celluloid rolls. There was picture and some holes to transport the roll. : are the holes. You know like this. ”
celluloid

”I still not get it.
How did this image popped up? ”
”I looked to my user journey.
So my customer has picked up a ticket, drink, and snack? What can she or he do?

This.”
[Hand moves to a pocket and takes out a smartphone.]
“Wow.”
“That’s why it is so important to use multiple senses to plan a proper test.
This movement [hand collecting phone] is muscle memory. ”

Now we have a look at the platforms or where is the system used?

“No problem. The web site and the connected systems work on one platform. It is used frequently. Known technology.”
“But the browsers can be used on smartphones and tablets with different browsers and different operating systems.”
“This is also known technology.”

platforms

Operations is about how the system is or will be used. This could lead to the following plausible talk.

“After our last talk I talked with the marketers. They noticed the following patterns:

  • Most tickets bought in advance are sold at least one day in advance.
  • Most tickets bought in advance are sold for the weekend.
  • The week patterns are almost the most same in the month except for the first week. “

expected ticket sales

“First week. Can you explain it?”
“Salary is paid in the last week of the month. People pay more for entertainment, if there is more money on their bank accounts. ”

“Of course there are exceptions on the months. December and the summer months are way higher because of the holidays. Watching a  movie is a nice way to spend holiday hours.
There are of course still unexpected and expected mega ticket selling movies.”

“Do you also take discounted movie tickets in account?”
“Yes, this lead to some increases. About 5 to 10 percent.”

“I would suggest to use the December holiday as the standard load profile. This way the system can handle peaks. To be sure a percentage of 10 % is added.

“Is your customer also interested, when the system breaks down?”
“We already took precautions: the modules for the tickets, drinks, and snacks are placed in the cloud and their resources are automatically scaled.”

“So everything is covered.”
“It looks good. It is important to know which factors have a major impact on the performance like CPU or chip, RAM or memory and use of network.”

resources subsystems

“Of course  you are interested in the change of use of the resources.

“You go to the ticket counter. You get a ticket for the movie and vouchers for snacks and drinks.

If I look to the user journey I miss a step between the ticket counter and the shop counter?”

“Wait the lines are wrong, it should be like this:”
different flows
“The ticket counter should have access to the drinks and snacks ordered.

“Okay I update the pic later.”

“If I go with 5 excited kids, I end up with 6 movie tickets, 6 snack vouchers and 6 drink vouchers. Double the number of drinks for Monsters Unlimited with the desert scene.”

That is a lot of printing.
So I have 24 cards. This is bigger than the set of business cards I end with after a conference. And the set of followers on Social Media. Todo cursive.

So the network can handle the extra information for the printers.”
“Yes, it is insignificant number of bytes.”

How much time does it take to print ’em all?”
“0.5 sec a ticket.”
“So I have to wait 12 seconds.
Wait 180 people go to a movie. That makes 4 x 180 pieces of paper in worst case (1 movie ticket, 2 drinks, and 1snack person). Only the printing will take 4 x 180 x 0.5, which makes 360 seconds. That is exactly 1 hour.
Of course people will not arrive after each other, but like this. If I add the number of voucher for drinks and snacks, I get …”

“Could you please stop?”
“Did I do some miscalculation?”
“No. I remembered something else. It took us months to determine the right number of ticket counters. My customer had to increase the number of counters. In this case it would be by 4. That is not pretty.

This is out of my scope. I am only responsible for IT.”
“Printers are also IT.  The system is not a single web site. Look at the pictures.

You might consider to reduce the number of vouchers. There is enough room on the paper for other things.”

The following item of interest is about the relationship between the system and Time. Yes finally.

A few months ago there was a question about ambiguous requirements. As a performance test coordinator I often heard: “It is fast enough.”

My solution was to perform a performance test on the old and the new system and compare the results.

Sometimes I was lucky and I had some real acceptance criteria. Like 95 % of the response time of function X is under 20 seconds. And a completely filled web site. The same content might give the same response. Or the other way around: almost no content is almost no delay.

“Now let me look at the response times. In the current system these are the response time for the tickets page, the payment module, and the ticket counter.
That payment module is swift and it is outsourced.
For the drink and snack page and the shop counter the times are added. Notice, that there is night between the day and the cinema visit.
response time actions

I can add the numbers for one movie. Then I get these numbers Fir two movies on an evening there is reassuring low response time.

“Let me guess your system is fast enough to handle a blockbuster evening.”
“You bet.”
“I don’t take bets I can lose.

But what about this?
Snacks and drinks ordered in advance are collected by the cinema people.
There is probably an acceptance criterion that snacks and drinks can be collected by the customer within 2 minutes after collecting the ticket.
So if I come with 5 kids collecting 6 snacks and 12 drinks, someone has to be quite fast. But there might be more people needed who collect their drinks and snacks.”

“That will be a problem. But I am only responsible for IT.”
“What about the people who have to work it. And your customer who depends on order drinks and snacks in advance page?”

“One apple juice please.”
“Huh. Oh yeah, you won the bet.”
“No, I am thirsty after all that talking.”
“You ain’t seen the desert scene of Monsters Unlimited.”

[Small update: there is a Q&A a few blog posts later about this very blog post.  You can even join the party.]

Non Violent performance testing

A colleague once told about a strange request. Now let me imagine the whole situation.

“Could you perform a performance test tomorrow?”
“If this is an unknown web site it takes 4 weeks: 1 week for test planning, 1 week for scripting, 1 week for execution of the scripts, and 1 week for test rest reporting.
If there are a few minor changes, it takes 2 weeks. If the test scripts and test data do not have to be changed, it will be one week.”

Now imagine the face of the project leader. Poor fellow.

Now I change the request of the project leader.
“I have a web site which must be tested on performance. I am exasperated.
I have a need for participation.
Would you please perform a performance test on my web site tomorrow?”

I used all the elements of the model of Non Violent Communication as described in my previous blog post.

I could add a little more detail.
“The web host provider needs a test report of the web site, that the performance is good. Within 3 days the web site must go live.”

This is a decent request and it is almost impossible to say: No.
Really? I have to fit at least a week of testing in 1 day. It is fair to say that this is not possible. It is not a reasonable request, so it is better to decline.

Then the dance can start. Let me first focus on the need of the project leader.
“I notice that you do not like my answer. Am I right?”
“Yes, you are right.”
“And you have now a need for clarity?”
[after a slow nod]
“I have a need for cooperation.
I am willing to cooperate, but a test plan takes one week.”

Now the dance is focused on the performance tester. It is her or his turn to explain. Almost.

“You extended a cinema web site with an option to order drinks and snacks in advance. It is just one page.”
“Can you not have the computer just click on the page to see what happens? That’s what computers are good at.”

“Would you please let me explain?”
“Yes. Sure.”
“If I would go with 2 friends I would buy three drinks and 3 snacks. Unless it is Monsters Unlimited with the desert scene.
It takes at most 15 clicks for 1 visit.
But there more than 3 visitors for the movie.

Now many people can look to the movie?”
“180. “
“What is the average number of tickets in 1 order?”
“I do not know.”
“Let’s assume it is 2. Then 90 different people will go to the drinks and snack page. So I have 90 sessions for ordering.“

“I can check with marketing and we can go.”
“But those 90 orders are not placed on the same moment.”

At that moment the project leader would look for another performance tester willing to help him.

My colleague had lost a customer. In this case there is no problem.

What happened?
A lot of talking.
“Would you please be more specific?”
“Sure.”

There were needs and feelings mentioned. The performance test coordinator focused on the project leader. And he explained a lot. A lot of talking. But no project though.

It is also possible to focus first on the test coordinator instead of the project leader.

Ready? Set. Speak.

“How can I help you?”
“I talked to my customer. She agreed to postpone the go live date of drink and snack page in the cinema web site.”
“The last time I had not enough time to explain why a performance test takes this long.
I was disgruntled.
I have a need to be understood.
Would you please let me explain, why the performance test takes time?”

More than 5 years ago in my real life I talked with my boss about a performance test. I had calculated that the investment in a performance would be earned back in 3 months. According to some financial standards this is quite good. But budget was too tight. My boss just asked, whether I had signed anything. This was not the case, so he advised to stop.

Other things that happened
A week ago a fellow TestBash speaker wrote a personal story about her needs which were not fulfilled. Poor fellow. There was a lot of emotion in the post. So she quit. And found joy back in little things.

Now you might doubt whether people would really read this story. But there were. Reactions came in. People who were touched by her story. Feelings and needs can be a big + or PLUS in communication.

For the people who are really struggling:

  • Describe the situation in an objective way.
  • Go to the feelings inventory.
    And determine your feeling (watch the flags for other languages available)
  • Go to the needs inventory.
    And determine your need. ([Other languages …] Chinese any one?)
  • Now formulate your request and make it reasonable.

There are other ways to handle difficult situations. One of my most read blog posts last year was about being fired. I unconsciously used stoicism. My need was clear: a need for support my family. But I had no bad feelings on that moment. I neglected the negative ones.

No violence intended

A few weeks ago I talked about a locker problem with a woman of my sport school.
“I put my stuff in locker 8, but the door came loose.”
She smiled:
“Thank you for reporting. We’ll pick this and ..”

“then I moved all my stuff to locker 6.”, I continued at the same pace.
The woman halted. Wrinkles of concern appeared on her face.
“I could not close the door, because there was no current. Then I moved all my stuff to locker 4.”

I waited for a moment.
The woman could not wait any longer:
“What went wrong this time?”
“This one was all right.”
The apologising smile was back on full strength.
I got another set of apologies and a promise to fix.

Incoming heading

What about the rest?
It is not my purpose to leave a trail of bug reports behind. I just noticed something and I shared this with someone who was really interested.

I did not use any violence to break the door off. The hinges had been clicked to the locker. They just unclicked and the woman rememberedthis within milliseconds. I was also Non Violent. I used an element of Non Violent Communication.

What I did, was to tell my observation in an objective way. I did not use any upsetting adjectives like stupid or dumb. It was just a door.

I did not have to use the other parts of model, the feeling the need, and the request. The woman had enough information to fix.

In my previous blog post I already mentioned Non Violent Communication. I realised that I did not write enough about the non violent part. So this is my rebound blog post.

Feeling
After the observation I could tell about my feeling. “I felt annoyed. ”
Wait a sec. Annoyed reads very offensive. It is a feeling that I had at that moment. I am writing for myself. It is not targeted at a person. I just had an experience and I was annoyed about it.

I know there are people who would consider this as an attack. This should not be the case. That is the responsibility of the person who hears my story.

Need

That’s a proper heading.

I had a need for perfection. As a tester I am aware that this is not always possible. But the opening and closing of the lockers can easily be arranged. Common technology I would write.

My need is personal. Some people might whine about it or like it. That is their responsibility. I am the boss of my own feelings and needs. Of course you can help me to determine them, but I can tell which are appropriate. To me.

Request

My request would be like:
“Would you please repair lockers 8 and 6?”

Please notice “Please”. This is a request. It is also used in other languages: bitte, s’il vous plaît, or alstublieft. It actually means: if it pleases you. So it is completely fair to disregard this request.

I just stress it again: it is no order. I am a customer and not a boss. The action would help me, who asks for this action. My bad feeling will go away and my need be fulfilled. That is rather pleasing. For me.

My sport school could do nothing with my request. Depending on previous requests from my side I could stop my subscription.

So I am heading to …

The closing section
Once I read a tweet of some one. I interpreted that this person had enough of a situation. I tweeted:
” I would determine my need and make a reasonable request.”

In my next blog post I will write about being Non Violent in the testing field.

House rules for bug reports

Some people liked the TV serie about a doctor with a walking stick. This blog is about rules, which are applicable in a company or an office

“There is 1 rule: There are no rules.”
3rd movie about Mad Max

“This is not logical.”
Leonard Nimoy

Let me finish
The most dreaded part of the game: she or he will win for sure. Or worse we have to hang on for another hour. Of mental mockery. This is a sign of unbalanced game. A game you only play once. I am not writing about life here. This is not serious.

Some smart people developed some house rules to balance the game or to accelerate the game. Preferably both.

There is also some serious stuff to do.

One of the basic skills of a tester is to write decent bug reports. There are complete courses for them. I want to share some of my thoughts about them. Particular how house rules are applied to bug reports.

The trick for a good tester is to find the house rules for bug reports. Sometimes they are familiar. Let’s examine an imaginary situation.

  • Select movie Monsters Unlimited.
  • Select two for tickets.
  • Press Next.
  • Add Cola.
  • Set Cola to 1.
  • Add Lemonade.
  • Set Lemonade to 2

Actual situation:
The following message is shown: “You ordered too many drinks.”

Expected result:
It must be possible to order more than 1 drink per ticket. A warning should be shown.

That looks like a pretty example, how to describe a problem.

Let me determine some house rules:

  • Every button press is described.
  • It is completely repeatable.
  • The actions are described in an objective way. No hard feelings. Devoid of emotions.

This is logical.
Leonard Nimoy.

  • It describes what I would expect as a tester.

There are some serious drawbacks:

  • It is hard to read for a dev.
    One dev once gave up after reading my reports.
  • It took me time to write it well.
    You might have notice the tickets and drinks are set in different ways. And I am quite experienced.
  • A standard reaction is: “No user would ever do that.”
    Bug reports are not about proving the odds, but about telling the plausible.

So the trick is to modify or add house rules for bug reports. As a tester I have to make reports. As a dev someone else has to solve them.

It is time for Finding Marlin
“Finding Marlin you wrote?”
“Yep and I …”
“Don’t write any more.” [I just did.]
“It is about three fishes. Now the dad has been caught. Now his son and his blue thin friend whose name I always forget, go on a heroic journey to get dad back.”
What is your favourite sea dweller?”
“Dolphin..”
“So the dolphins are going to help.
And…”

“Thanks for your help.”
“Huh, I was just brainstorming about a movie spoiler. ”
“You just described a realistic situation, what people do in a particular situation.”
“Talking about fish?”
“Yes and showing how people can act in a natural way. ”
“I just happen to like movies.”
“And that’s what makes this talk completely plausible.”

Actually, Marlin stands for ‘Make a real life impression now’.

If I can tell a realistic user story, then devs are compelling to solve the problem. It is not something like this:
“As a cinema visitor I want to order my snacks and drinks before the visit, so I can save time.”
This is an abstraction.

It is about thoughts and needs. Let’s read a mind.
“In the movie Monster Unlimited there are too many monsters, so Mike and his blue furry friend have to move to the desert. That thought makes me thirsty. I need 2 lemonades for sure ”

Is this plausible? I think so.
Let’s ask the PO or Product Owner. Fine with you? So what are we waiting for?

Let’s tweak again
I do a small rewrite of my three drink bug report:

  • Order two tickets for Monsters Unlimited.
    [Hey. That is great. I can order my drinks in advance. Let me see]
  • Order 1 Cola.
    [Looking at a desert makes me thirsty, so I take an extra drink.]
  • Order 2 lemonades.

Actual result:
A message is shown, that at most 2 drinks can be ordered. [I need that drink. This is ridiculous.]

Expected result:

It must be possible to order more than 1 drink per ticket. A warning should be shown. [I take the consequences].

Let me extract some modified and new house rules:

  • The actions are written in a more natural way.
    This speeds up my reporting and it is more readable. I am not programming the programmer. I wouldn’t dare to.
  • Thoughts have been added, so people can identify themselves with the user.
  • Emotion has been added.
    This is a dangerous one. It is extremely helpful in the steps and can become harmful in the actual result. My thumb of ruie is, that it is allowed for a customer and in a few cases for a tester. A customer is not always a tester.

Are you in for a little experience? Have another read.

  • Order two tickets for Monsters Unlimited.
  • Order 1 Cola.
  • Order 2 lemonades.

Actual result:
A message is shown, that at most 2 drinks can be ordered. [This is ridiculous.]

Expected result:

It must be possible to order more than 1 drink per ticket. A warning should be shown.

“This is ridiculous,” sounds a lot harder than in the first rewrite, when only 1 thought has been added.

I will come back to the emotional part later and a few decimeters lower.

A more important question is: Why did I stop the presses? I mean the description of presses on a button or mouse.

This might be a lack of detail for some people.
“You ordered 1 cola and 2 lemonades. Right?”
“Yep.”
“So how did you do it?
Add 1 cola, add 1 lemonade and add 1 lemonade.
Or was it: add 2 colas, drop 1 cola, add 2 lemonades.
Or: add 1 cola, add 1 ginger ale. Drop 1 ginger ale, add 2 lemonades
I could even add some steps to go back, select Finding Marlin, order 2 drinks, change movie to Monsters unlimited, add third drink. But you wrote, that you did not do this. … Maybe.”

Sometimes programmers are great debriefers for testing.
What did you actually do? Why do you think this was a useful test? Did you also have a look at the other features? Did you also check the interaction with the other modules?

“They scare, because they care?”
That is not the appropriate way to write about devs. I do not have to fear, if I can answer their questions. And yeah I make mistakes like devs. Then I have to admit, that I was wrong.

Back to the example:

  • Order 1 Cola.
    [Looking at a desert makes me thirsty, so I take an extra drink.]
  • Order 2 lemonades.

Now I can make another house rule: take the shortest route.
Of course the devs have to agree. Then you have a new rule. In da house.

Can this lead to a discussion?
You bet.

“Did you test other combinations?”
“No, I was just exploring the web site.”
“Did you know you cannot order the lemonade in the winter?”
“No”
“So you better test it. It took me some time to program that.”
“Thanks for the information. I wonder how I can change the system time. Maybe you can help?”
“Well that is easy. You …”
I just leave these 2 techies alone. I have another interesting section coming up. In the meantime ..

is a scenic tour also plausible:

  • Order 3 donkeys.
  • Order 2 dragons.
  • Reduce the number of donkeys to 1.
  • Reduce the number of dragons to 1.

This is still the shortest route. Every step was needed.
You might call it Advanced Donkeys and Dragons. So long the troll stays away.

Lectori Ludum – Game for the reader
The game shown in the picture is the pocket version of the Hive. It is a 3 dimensional game and it has house rules.  And it is about bugs.
I like it’s complexity. In case you want to buy it, you could use this link.
End of commercial break.

Let me get emotional
I promised you to explain, why emotions are important. I already gave a small hint.

This is something I experienced.
I was in the hospital. The nurse behind the desk tried to formulate excuses to me:
“I did not give [your case] a high priority. You sounded completely in control. ”

This is a big disadvantage of being a tester. I had been programmed – I know: bad word choice. – to give an objective report, which put me in the middle of the line.

A bug report is a thing I not only make for work, but also in private life. If I don’t like something, then I want to get things fixed. So if I order a book and the book is damaged on arrival, I file a bug report. If the book seller does not provide a good service, I get angry. So why can I not show anger to get things fixed?

Do users of software have needs and feelings?
I think they do.
What about persona?

This is a description of customer with all her or his interesting characteristics. A good name can help a lot.

I do not have any experience with it. Personally I think it can be useful. No pun intended. I am a personal ally of the User experience or UX designer. That’s an intended pun.

Let’s start a thought experiment.
So we have the excited Kid, the bragging teenager, the unlimited cinema visitor, ….

Of course the excited kid will not handle the payment on a cinema web site, but she or he will definitely want to choose her or his own drink and snacks. “Dad, how can I order a Cola? It has already been chosen.”

A small side step.
For marketing people this is cool: can I sell packages to the kids?
For the regular movie visitor a limited collectible cup for the drink can be quite tempting. Gotta have them all.

Back to the Excited kid.
“Dad, did you really choose Finding Marlin?”
“Yes, I did. Just pick your drink and snack, dear.”
“What is a kid package?”
“I think it is a drink, a snack and maybe a toy in a box. Does the web site show some description?”
[A little silence]
“I can use a link. Yes you are right, dad. ”

[Another little silence]
“How can I get back on the page?”
“Push the back button.”
“It does not work.”
“Just close the tab.”
“It works.

Where is the page for the drinks and snacks? I cannot find it dad.”
“Let me have a look dear. That is strange I cannot find the tickets.
We have to start all over again.
O dear, all tickets have been sold out.”
This will start an outburst of emotions. In turn these form a good starting point for Non Violent Communication.

Last week while I was still assembling this blog post, I saw a tweet of Santosh asking about NVC. He had read a conversation between Jari and Lucian. And he couldn’t resist himself to “barge” in. I sent him a link to some basic  resources.

NVC stands for Non Violent Communication. This model uses 4 elements: observation, feeling, need, request. Luckily there is a cinema example available for use.

  • Observation
    The kid tried to figure out, what a kid package was. This led to a situation, that tickets were lost.
  • Feeling
    The kid is sad and the dad angry.
  • Need
    The kid has a need to share: to tell about the movie at school. The dad has a need for independence, that his kid can order her or his own drink and snack.
  • Request
    Would you please provide a way to show information about the kid package without losing the ordered tickets?

Browsing through these four elements makes the request reasonable. I am tempted enough to call it logical.

[Answer on, why his parents married]
“That was logical.”
Leonard Nimoy [TV serie]

By the way it took me a while to determine the need of the angry dad on https://www.cnvc.org/Training/needs-inventory. A need is personal. The basic thing is to describe the point of the view of the user. As an observer I have to be very careful to fill in the need of the user. It is not possible to read one’s mnd, but it is possible to ask about someone’s need.

Now I am quite close to the heart of the bug report. If someone asks to solve a bug report, then it is easy to program the expected result. If I test something, it is easy to check only for the expected result. And of course I can test all the other impacted functions and data, but does it really solve the problem?

Let’s have another look at the kid package problem.
A programmer could program the interaction as follows:

  • if the mouse is on the kid package, a small message will be shown containing the description of the package.
  • The message will disappear, if the mouse is moved.

As a tester I can easily determine, that the impact on the data and features is minimal. So it looks easy to test.

Well. There is a huge problem lurking there.

Let’s take a closer look
“So I have 2 tickets for Finding Marlin, 1 cola, and 1 carrot.”
“A carrot?”
“What’s up, doc?

So I go to the ticket counter and get tickets, the package, the drink and the veggie.”
“No, you only get the tickets. The rest you have to collect in the shop.”
“Why?”
“There is no place for all the snacks and drinks.”
“Sounds fair to me.”

“If I enter the shop I pick up my order and continue to the movie.”
“You actually have to collect all your ordered stuff yourself.”
“Why?”
“Just imagine a cola standing there for a few hours. It might have less bubbles and is warmer. Now think about ice creams. Or your carrot’s waiting days for a nibble.”
“You’ve got a point.”

“But where does it state that I have to collect all the stuff?”
“It is on the voucher.”
“Which voucher?”
“The one you get at the ticket counter.”
“Okay. Can you please show me 1?”
“Sure. Here’s one.”
“Hm that is small font.
That means I still have to be 10 minutes earlier to collect my order.”
“Uhuh.”

“So I collected.my order, I show my voucher and see the movie.”
“Yes, you just go to the counters to claim the voucher.”
“Wait. You said counters. But I do not have to pay again.”
“The counters have a scanner for the vouchers. There are so many orders, that there is no standard voucher.”
“Okay, let me summarise again.”
“Collect order, go to the counters, show voucher and see movie.”
“You’ve got it.”

“So I go with my voucher to the voucher queue.”
“There is no voucher queue.”
“Wait a minute. I have to get in the same queue like the other people.”
“Yes.”

“But there are no real benefits to the service. I only pay in advance.”
“Yes, but I like the idea of a special voucher counter.”

“The user story was:
‘As a cinema visitor I want to order my snacks and drinks before the visit, so I can save time.’
If I look for a need, i would say the need for ease. Within minutes I should get everything: just show the tickets and collect everything.

What about this?
How much time does it take to collect the order for a customer?”
“3 minutes.”
“So if I couple the GPS location of the customer to the order, then it is easy to collect the order. What about that?
Faster service and happier customers. Just make them smile.

And I could continue writing about what to do with the money in case if the customer does not show up. Or the customer changes his mind over the snack. Every solution should be focused on the need for ease.

Let me speed up the reporting
Navigation can be compressed using arrows. E.g. Menu => New.
If too many arrows are used, then the user experience should be improved.

It is possible to save time by adding bug descriptions in the comments instead of linking new bug reports to the report. If the devs can keep up with your pace of reporting, this saves lots of time. In one project I had a supplier, who had no overview and was slow. Then separate bug reports were extremely handy.

During my first year in an agile team
“I found a bug.
Do I have to report it?”
“You can talk about it?”
“So I do not have to write it.”, I wondered.
“We are talking.”, my scrum master answered with a smile.

Let me think
Post Ludum [After the game]
A continuing thought from me: are there other house rules to break? For better quality of life and work.

Why am I now thinking about retrospectives? It must be a flash of insight. I just wrote one : )

 

 

 

 

 

 

Let me write
I thank you for that.

Let me talk
I wanted to write a small blog post about bug reports and it almost turned into a talk. Too many stories in my head. So …

my proposals for talks for test conferences will be sent in the following months.

Let me thank you.
Thanks for reading. Real thanks again.

daD Talk

One of the things I wanted to develop is critical thinking. Not only by myself, but also by my kids themselves. The led to a rather unpleasant start of one of those dad kid conversations.

There was no way back: a subject I tried to delay for a few years:
computer security.

The complaint about a program was packaged as a request:
“I want to have a computer, which can execute [dangerous module] programs without using [dangerous module].”

I exhaled. My kid had absorbed the information and realised that the use could have a severe consequence for the computer. No more computer time. On the other hand the disadvantages were too big to forget about it.

I tried to find a solution, but I could not find one. If a program can change things on a computer, then it can do bad things.

While blogging I realised I was wrong. There was a work around.
There are programs, which can do same things like the original program, but they are built differently. They are called emulators. Some gamers like to play low resolution games on emulators of very old operating systems.
Wow, that’s my kid.

It’s hammer time
“If you have a hammer, then you can use it to break a window. But that’s not right.”
My kid nodded.
“So I program the hammer, so it cannot be used for a window glass. Then I can go to a door and use it to break a lock. I can program it not to break a lock. Then I can use it for a window frame.”

It would be easier to tell the hammer it could only be used on wood. This looks brown and it has grains. But it could be changed, so that everything looks like wood.”
I made a wide gesture with my arm pointing to different objects in the room.

“But I could change the picture. All objects would look like wood. That is not a good idea, so I store the picture in a book. But the picture in the book can still be changed.

Then I could place a lock on it. But the lock could be picked. I could place a better lock on it, but then the whole book could be replaced by another book.

And that’s why it is so difficult to secure things.”

Another unpleasant guest
My kid had seen a cool app. And it should be installed absolutely. So I did my dad thing:  looking at the permissions, which I would grant to the app. It could handle my files. It was just a game and why should game have a peek at my files? Time for the bad news.

So I told my kid, that the app would access files on the phone. The reply was to buy a phone just for games. Then I told that after a while the phone would be also used for other purposes like making pictures. “You don’t want your pictures in someone else’s hands.” There was a lack of nod.

I needed another way to tell the warning. A visual one.
“Suppose someone comes in. He looks television for the whole evening. And he eats the whole fridge empty.

If you protest, he will say:
“You said I could come in.”

The next evening he comes back. He takes the table and the sofa out of the house.

If you protest, he will say:
“You said I could come in.”

Delegate report about CITCON 2017 Amsterdam

“We did it already 14 times.”
Jeff paused for a moment,  “and this one was amazing.”
He was quite modest.

Gotta attend this 1
A few months ago a speaker told about his experiences at a conference. He had difficulties to make contact. One of the responses intrigued me. Mohinder Khosla mentioned CITCON in Amsterdam.

For a Dutchman this was interesting. The price was also reasonable 0 Euro with a modest request to cover the costs. But how could I determine whether the price was right? Among the participants I noticed Gojko and Cirilo. Wow. Then I read a recommendation of @testobsessed and I was sold. I mean the ticket.

Not bad for a conference with a marketing budget of 0 Euro.

Evening 1
Before I want to describe, what happened, I have a small warning. This post will focus on the process and not on the content. The reason I chose to is simple: people came there to share information and difficult situations. So reader beware.
Now it is my task to draw you in the atmosphere of CITCON 2017 Amsterdam.

The first day I used public traffic to go to the venue. It was hosted by Xebia. This office had all the elements to affect people’s mood. There was a standard meeting room with glass walls, a comfy corner including sofas and game computer. Did I mention the TV? There were rooms where I could see concrete shining through.

Evening 1 started awkward. I only knew one participant and he was not present, so I talked with new people. The talks were friendly with a formal undertone. Really polite. Just probing around.

Jeff and P.J. started the conference in a room with more than 100 seated people. They casually introduced the format. Asked for feedback (“5 stars is good.”). And let all participants introduce themselves and telling about things they were excited about or personal struggles. Those small stories let me connect to the people telling them.

The subject proposal session was described. At the moment everyone felt at ease P.J. and Jeff told, that they would take a step back. They would only help in case of problems. It was “your conference”. This lead to an extension of the Q&A. After the last question the organizers withdrew from the flip boards. It was up to us, the delegates.

After the warming up the real stuff started people were requested to pick a subject and clarify it to the audience. A queue formed in the room. People with posts its interested in solutions or volunteering to share information.

I watched the process for a half hour and I noticed the time. Time to send text message to my family, that I would come home later.
I really liked the way, how the subjects were presented. The question part was really clarifying. Participants tried to understand the content. The atmosphere had changed: it was warm and people were supportive.

Let me give an example. I proposed a session to share information about blogging. For me it is a way to clarify my thoughts and reflect. I was questioned about it:
“Why do you call it ‘Blogging as a service’”?
“I needed a title … and it is a service to the Community and outside the Community.”
BTW there was also a session proposed “What’s in it for me?” It was listed In the final program.

After the subject proposals I voted and went home. Afterwards I heard from a fellow Dutchman, that he was requested to compose the program. He kindly declined. He had already arranged the venue. This was a reasonable excuse.

Morning 1
The next morning I entered the office, where I had a small chat with Pati about conferences and IT. I skipped my second breakfast and collected a good coffee. There was a friendly buzz in the air.

I went to the program and browsed through the stickies on the flip boards. My session about blogging was accepted. Yay. Then the bad news sank in: there were too many interesting sessions at the same time. That happened to me years ago.

I arrived early for the first session. I remembered that several stickies or subjects were combined. The room was quite big for the small group. I moved my chair to the middle of the room for better interaction.

The first session was about testing and I tried to participate as good as possible. I had to watch my politeness, but I had no time. The subject was too interesting. I bent slightly forward and started contributing. I was in business mode: polite and helpful.
The day before the code of conduct was explicitly mentioned. Proper use of language was also checked :
“Can I use the F word here?”

For the second session I had to find the stairs. Glass walls wrre exceptionally handy in this particular case.

The previous session was still in full swing. I put me in the background, while observing the room. Sofas. These are great: It is difficult to maintain a neutral position. So I had an extra indicator for inclusiveness.

So I was in my session about blogging. And someone else. Okay time to start the Q & A. There were good questions. I got other questions than expected. Those made me think and reflect on my actions.

After a while other people joined. They had used the Law of Two Feet. If delegates were not interested in their session, they were allowed and encouraged to change the session. For my session it was welcome.

Lunch 1
During the lunch people were still having sessions. One session was about strange effects of particular Unicode on programs. A bit too tech for me.

You might have noticed that I changed the category to ‘Techies conferring’. My first thought was, that CITCON was about testing, until I read the description. It was about integration and testing. Other technical people would also be present.

During the lunch I joined a conversation about 3,000 pipelines. This was another world for me. So I had to drop my idea of an extremely small set of pipelines. There can be more than 1.

In a later session “mister G” had a good suggestion to improve builds. Suppose you have suppliers who are only focused on their own software. “Just let them provide tests to each other they can use in their build. If the build breaks they have something to talk.”
G thanks.

Writing about the afternoon let’s switch sections.

Afternoon 1
After the section switch I try to stay of content, but that is difficult.
One participant told a story about switching lines in code in order to cause bad unit test results. If the unit test did give an OK, there was likely a bug in the unit test.

I also saw some really awful Gherkin to describe actions instead of abstractions. Gherkin is good for testing of flows. For tabular stuff FIT of Fitnesse are the tools to use. And Concordia is another good option for the remaining option. Thanks Gojko.

For one of the final parallel sessions people were requested to bring real life testing problems. I joined this session. The facilitator did a good job to clarify the problem. A lot of whys and whats. I thought I could handle a lot of testing problems, but these were really hard to solve.

O yeah. Back to the process. I went to a big meeting table and met someone else. After some small talk we decided to start. About …
I volunteered to take a picture of the stickies. My hand went to my pocket and stopped. The location had been changed. I had been warned. So I warned the other and up we joined the session in progress about Gherkin. You might have read about it. Some centimeters higher.

After the session everyone was gathered in a room to share their AHA moments. There were a lot. I saw people who showed emotions. I saw people ready to take action when they would be back in the office.

People referenced to the ‘Gentle punch in your face’ session with Jeff. Other people had some great conversations. Outside in the sun. Hopefully a dev snd a tester. The truce is out there.

Evening 2
With more than an hour travel time and past my dinner time I had to find some place to eat. “It always works out.” Jeff reassured the people. Indeed a big group assembled on a terrace enjoying snacks, French fries, and burgers. They even had veggie ones.

I talked with other testers, who knew about context driven testing and cynefin. I don’t meet these testers very often.

There was a friendly guy with a nice sweater with a known software supplier on it. That made me curious:
“How did you get that sweater?”
“I work there.”
“I use your software.”
“Me too.” another techie joined in.

I ended in a group of Finnish guys. Talking about things. Things different from IT and work.

Thanks P.J. and Jeff for the conferring.

Pizza dough though

One of my kids read the recipe.
“Now we need salt. So I take the Baker’s Salt.”
And I had to switch gears: Baker’s Salt?!

For a starter …
On my days at home my wife hands all the cooking stuff over to me and I have to figure how to cook, grill, or bake. In my family weekend starts on Wednesday. My wife and I puzzle about the meals.
One of the no brainer recipes is pizza. I thought.

My wife was still collecting recipes. It basically meant, that I had to sift all kinds of papers. I already used the pizza dough recipe a few times, so I could tell the basic characteristics: white paper with a eightsome steps.

I browsed through the recipe folder: nothing.
I opened the Baking Bible: niente.
My kid was thinking all along the way.
“Let me have a look at the folder.”
Again: nope.
Then I remembered a book written by a national known baker. My kid had enough information and retrieved the book with half centimeter stack of recipes: bingo.

The next course is …
Next step to the dough was machine assembly. I put the machine on a comfortable place and picked the K-beater. Yep, that thing in the picture.  I still remember my first impression of this part. there is a K in it. When my wife mentioned the K-beater …

All parts were fixed. Now I needed the ingredients. My kid helped me measure the water and the flour:
“We need to use the patent flour”.
I agreed.
I was proud to remember to put the water first into the bowl. My wife had stressed that several times.

Then Baker’s Salt had to be weighted. A special spoon with a built in scale was used for this purpose.
“That is little.” I heard.
Before I could help: “It was 0.3 gram.”
I heard nothing more, so the accuracy problem was handled.

Then I wanted to add the Baker’s Salt to the water and the flour. But my kid was determined: it had to be added later on. OK. But we still needed the yeast.

My kid picked a small plastic cap out of the drawer.
“The salt and the yeast must not be mixed.”
The cap was placed in the spoon and the scale was reset.. Then the yeast in the cap was measured and added. Oil was also added.

Then it was time for me to start mixing. After a minute the salt was added. The K-beater had a heavy time, so I switched it after a small consult with my kid. Imagine a dancing kitchen machine. Not funny. So I used a new part. It looked like the left hand (?) of Captain Hook.

After 18 minutes I got the dough out of the bowl. My kid made a circle in the air with two hands:
“Mom always does this.”
“Can you show it again?”
The movement was a few times repeated. So I gently moved my thumbs over the ball of dough. My kid was not content. The ball was pcked out of my hands and placed on a kitchen cupboard. Then the ball was firmly cupped with two hands. The shape looked familiar to me. Nice touch.

I then let the dough swell.

For dessert …
“The pizza bottom is crispy.”, my wife remarked.
I bit and agreed.

Some people might have noticed that I did not use any computer related word. Except the bolded one. Let me continue.

My point is of course related to my profession. It was not about looking beyond the instructions neither about using knowledge in practice. This story is all about cooperation.

DUMB heuristic

During the Rapid Software Testing course James Bach advised to name things. If I cannot tell, what I am doing, my boss would think: “What the < beep > is he doing?”
So after the course I came up with the DUMB heuristic. This heuristic I use frequently during my testing.

Suppose my boss asks me how the testing goes. My answer could be: “I do uh my best.” A busy tester is always good, but if I am too busy things might go wrong. All my energy and brain power are focused on the work at hand. I forget to think. That’s DUMB.

When I figured out my heuristic, I had to find some smart explanation for this abbreviation. It became Do Uh My Best. Most heuristics are acronyms or lists of words. In turn every word is explained in detail. You know: turned upside down and shaken. But I just stick to the abbreviation.

Some readers have a good reason to ask. So hold on. The emphasis of the heuristic is on Uh. This is an alarm bell. Hesitation is a sign, that too much is going on in my head. I might forget to pay the deserved attention to the real problem.

This following discussion I had several times with my scrum master:
“I am writing test cases.”
“Why do you make test cases?
How often will the tests be executed? Who will maintain the test cases?”
So I was doing my best. And my scrum master got an Uh out of me.
He let me reflect on my work. What was I doing? And why?

Many test heuristics I know are acronyms. Some I do use. The S in SFDIPOT stands for Structure, how is the application under test built? So which components do I have to test? Etcetera.

If I apply this to DUMB then I would get things like: Do is the activity I am involved at that moment. It can be taking notes in my test charter or just thinking. Etc. Etc. The only thing I need on that moment is a short reminder to reflect on my work. DUMB. One syllable. Keep it sweet and short.

When do I use this heuristic? If there is a lot of work to do. When I have hours of straight testing ahead, I can go in my little cocoon and do something testy and something nice comes out. Tadah.

What about the focus and defocus stuff? Most of the time I use this when I got stuck. What can I do now? Let’s take a step back. Now I see the big picture. I figure out at a high level. Let me zoom in.

The point is, that doing my best is equal to doing stuff which is in front of my nose. I just pick it up and start running. Maybe I get it done today or this morning. Completely forgetting to ask for the reason. And that is DUMB. NOK?

When I was writing this blog, something hit me in the face. That was my imaginary hand palm. This so called heuristic is maybe already known under another name or in another way. Like a proverb.

And yes Madam within a few minutes the following proverbs came in my mind:
Eile mit Weile
Dépechez lentement
Easy does it

A day later followed by
Haastige spoed is zelden goed.

So why am I not using proverbs to keep my testing right? It is too long. Maybe a short heuristic could be useful.

“My scrum master would say something like Does it help?

This is really STUPID which stands for Some Thing …. I Do. UP are two words, so ….

Do I have to figure this out? WHY?”
“We Hear Yes.”

 

 

 

 

 

 

 

 

 

 

 

 

 

“GIRLS”
“Good In Real Life Software”

 

 

“GUYS”
“Graphic Userinterface You See”

 

 

“PLEASE”
“People Lik E A Software Errooor”

“CHEERS”
“Co Herent Entertaining Erroneous Random Synthesis”

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

So you are a mindful reader. Congrats. Here is the bonus.

Why did I not throw this blog post in the waste bin? People already thought about it and distilled their experiences to proverbs.

It is about my thought process. I do not want to stop thinking. Once in a while I discover that other people already have figured it out. Sometimes I find something new. But apart of discovery of new thingies it is about thinking thingies through. Getting better to verbalise my thoughts and explain them to you the reader.

That was what I forget a while ago. Thanks for the attention.

Optional reading Also Known As scientific stuff for nerds like me
When I Do Uh My Best, System 1 is operating. More information about System 1 can be found in this blog post.

WYSIATE stands for What You See Is All There Is. If I am too busy, I forget to look for other relevant information as mentioned in Thinking Fast and Slow.

The Lindy effect is about using knowledge especially from books. Thinking Fast and Slow is quoted for years by some testers. If a book is relevant for 5 years, it will probably remain relevant for another 5 years.