May 2018 Testing

For the interested British reader this is not about politics. It is about testing software so that it complies with the General Data Privacy Regulation or GDPR in May 2018.

For the people who are only concerned about money. It can cost your company 4% of the global annual income of your company or 20 million Euros. That is seriously a lot of money.
Thanks for your attention.


I am not a legal expert. So please have a look at my used sources. Or contact a GDPR expert.

I am just a tester finding test ideas about GDPR.
Thanks for joining in advance.

Just show it to me

Suppose you have a cinema and a special web site. You can order tickets, drinks, and snacks in advance. This is a unique selling point.

A marketeer has a nice idea:
“Let’s make some profiles. We’ve got lot of sales numbers, so boost those numbers.”
“What do you have a mind?”
“We just tag customers: B movie, Friday night, first week, ..”
“First week?”
“Like ‘I want to see the movie in the first week after release.'”

If I would go to  this specific cinema, all my actions are recorded.
Big Buyer is being watched too. This sounds creepy. This is my alarm bell as a tester.

My simple question is:
Is profiling allowed?
More accurately, is profiling of European citizens allowed for this cinema web site according to General Data Protection Regulation?

What makes someone a European citizen?

sketchnote with cradle, parents passport and database

Obvious candidates are:  parents, place of birth, passports.  I just stick to Citizenship Administration. I found this one while doodling in my head.

Let me give you a royal example. The Dutch queen has the Dutch nationality, but had Argentine parents and was born in Argentina.

Let me show some graphs:

  • European Union
  • People with no nationality
  • People with 1 nationality
  • People with 2 nationalities

I could make these 2D graphs:

One chart of part of Europe and three coloured graphs about number of nationalities

I could try to stack them and squeeze them afterwards:

One more try:

3D graph made of a chart of a piece of Europe and pieces of sticky notes depicting the number of nationalities

So the best way to define an European citizen is that she or he is registered as an EU citizen in a Citizenship Administration in the EU. Now comes the difficult part: as a web site owner I have no access to this administration. Well. That is a good one.

How can I determine whether an European citizen is in my database?
In most cases I don’t. Because nationality or EU citizenship is not always registered.

“Is an address not sufficient?”
“What about An American in Paris?”
“Okay, email address.”
“What about or”
“The nationality is registered.”
“Good. What about EU citizens with two nationalities? ”

Looking at the context: if no nationality or EU citizenship has been registered, then I would suggest to look at GDPR. Otherwise definitely use it.

But this is a premature advice. This is a warning. Please read on.

Finding GDPR

If there is one thing I hate about learning, it is memorising information for the sole purpose of memory. I like to have some fun in a good sense of humour.

Here’s where deliberate practice comes in.
Determine a strange situation and look it up.

On my search to the official GDPR document I quickly determined that my target was:
Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)
Yes, it takes some time to read it.
And a natural person is human being. Like you and me.

I am well aware that English is not everyone’s native language. Now the EU has this little nice webpage with links to GDPR in your favourite language. Hopefully.
No Chinese, but maybe French?

Profiling and data subjects

Profiling can take place after informing the data subject, who has agreed to these terms for data processing. [GDPR 32, 42]
That is a lot of info.

Let’s go a step back to nationality. I warned you for this.
I am not familiar with the American laws. Remember I am not a legal expert.
Suppose profiling of natural persons is legal according to the American law. For example ‘s sake.

Take a case of an American woman who starts buying action movie tickets. My guess is that her new boyfriend is lucky. Piece of case.

It is very easy to make a profile of her boyfriend. Now this lucky guy happens to be British. And has some royal blood. It rhymes on What?!.

There is still no problem, because it cannot be traced back to some palace. Unless I would couple the data with the email address of a fortunate American actress. Oops intended.

Chain of Gift

The American woman is a data subject. All kind of data is collected, but there is an unpleasant side effect: her boyfriend or fiancee also ended up as a data subject. I doubt whether he would have given any permission. No thank you.

Actrice gives something to a prince.

The Chain of Gift leads to interesting doodles. In orange is the American woman and in blue an European Citizen wearing something called a crown.

Quick explanation for the colours: the European flag is a blue flag with yellow stars. So the EU citizen is blue. For the American woman the colours white and red remain. Somehow these are not appropriate. So I chose orange.

So there is a difference between buyer and user. A man can buy movie tickets and give them to his children. ‘Finding Marlin’ and ‘Monsters Unlimited’ seem quite innocent pieces of data to share.

Dad gives movie tickets to children.

Is it possible to determine the birthdays of the children just based on his cinema visits?
Not based on the movie titles. There is a better chance looking at the number of bought children tickets.

“When are we going to the cinema with my friends?”
“What do you think?”
“On my birthday?”
“Good girl.”
[Big smile]

Birhday party

Another interesting case: a man who buys gifts for his grand grandchildren. Depending on the gifts I could guess gender, age, and hobbies. If those grand grandchildren live in the EU, you might have a major problem.

Man gives gift to daughter, who gives it to her children.

With a low number of children per family it is relatively easy to make a family tree.
I can guess that princess cookies are for 5 year old grand granddaughter and that superhero suit with XS size is for …
You get my points.

My best guess is to make a GDPR compliant approach for my whole customer base. There is no way to determine which European people you are profiling.

Permission granted
Scenario 1
Suppose I am in the living room and one of my kids tries to sneak out of the room. I look in the right direction and get eye contact. The door is opened and closed.

A few days later a man is at my front door with a box of 20 tablets. You know those fancy computer things.
The name of my kid is on the box. O oo O.

Scenario 2
Suppose I am in the living room and one of my kids tries to sneak out of the room. I ask:
“What are you up to?”
“I gonna hack. You don’t mind?”
“Yes, but”
The door is opened and closed.

A few days later a man is at my front door with a box of 20 tablets. The name of my kid is on the box.
“Where can I place the other 500 boxes in my truck?”

Scenario 3
Suppose I am in the living room and one of my kids tries to sneak out of the room. I ask:
“What are you up to?”
“Just read this legal document and you will be just fine.”
“It has more than 10 pages.”
“Can I go now?”
The door is opened and closed.

A few days later a man is at my front door with a box of tablets. The name of my kid is on the box.
“There are three extra trucks coming with tablets. Where can we unload the four trucks?”

Let me finish the three scenarios at the same time.

A box, one truck and a group of 4 trucks on the way to a finish

“Excuse me, I have to call someone.
Would you please wait outside?”
I close the door and the mobile phone is in my left hand instantly. My kid picks up the phone right away.
“A package arrived for you.”
“The tablet arrived?”
“You can better say: ‘Tablets.’”
“Huh, those are the most expensive tablets on the world. They cost a fortune.”
“That’s why I am calling you. How can you afford these things?”

“You know dad, I needed some purpose in a life.”
“So I learned to hack.”
“O no.”
“It’s worse.”
“Legal hackers don’t get paid much. I had my eyes on this tablet. So I said: ‘You pay me in Those Tablets.’
If I got one extra, I could always give it to my Best Friend.”
You’ve got a friend in me.

Websites sometimes are like kids. Scenario 1 would look like:
A window where no permission is asked but just taken
No permission is asked, just taken.

Scenario 2 would lead to the following picture:
A window with a default permission for profiling
A very fast designer filled in a preference.

Scenario 3:
A window with unreadable text with a request to accept these conditions
O yeah. The legal stuff one.
At least the checkbox for the conditions has not been filled. But I cannot install the program, unless I agree with them. Hmm.

GDPR forbids all these three options. They lack the support for the user who wants to protect her or his privacy. Website 1 must use transparency, website 2 a default for no profiling. And finally website 3 must use concise and plain language. [GDPR 32]

Thanks for jumping in

For the interested British reader this is not about politics. It is about testing software so that it complies with the General Data Privacy Regulation or GDPR in May 2018. Déjà vu.

There might be readers in my audience who had another association with May 2018. I know that Harry is a major export product for the UK. And I am not writing about the scarred man who has been featured in a lot of books, movies and a theme park.

Some people are more interested in an upcoming royal wedding of Harry. That might have some impact on your online Harry product web shop. For the people interested in performance tests here are some nice blog posts about performance test and Q&A. From yours Mindfully.

Some research notes

A lot of you who are reading this can still follow me. What you actually missed, is my nonlinear search. For the answer on my question: Is profiling of an EU citizen allowed according to GDPR?

The first thing I did was to download all relevant legislation. With a search engine a legal document could easily be found. Then my inner critic voiced his concerns: where are you basing this blog post on?

What I needed, were traceable sources for my research. The more EU the better. Again I am not writing about politics.
I found some links to some non EU websites. But my main target was the GDPR on an official EU website. This took me some browsing. At last I downloaded the wanted document and saw no differences with the other document on first sight.

I took no risk and started to use the official document as main source for this blog post. There was one big but. BUT the document was a pdf. This format is widely supported by all kinds of apps, but not search friendly. A search takes a while on my smartphone.

I converted the document to epub. Now I had a significant win in time. There was no more interruption in my flow of thoughts.

Let them flow.
[On the melody of Let it go.]

So I sought on the word child and hit my next obstacle: the word article. Now are articles quite common in laws, but to my dismay I had not encountered this word before.

I did another search: article. My references to this document were obviously wrong. So I was referring to numbers between parentheses. I switched back to the pdf document to find exact starting point of the first article. It was roughly at the same spot: 38.6 % of the document. Apparently I was referring to some notes in the introduction. And that is not a problem. I think.

Kids, definitions and laws

Of course there are some exceptions. And exceptions on exceptions. This is a great playground for testers. For sure. For ever.
Because people tend to change their minds. This is my most political statement BTW.

Writing about kids reminds me about the definitions debates which pop up every now and then.
“Children have special protection.”
“What do you mean?”
“You need the permission or consent of the people who take care of the child.” [GDPR 38, article 8]

“And the exceptions are…”
“services for prevention and counseling. In these cases you need consent of the child after asking it in a way easily understandable for child. It is not about child proof but about child friendly.”
“What is a child according to GDPR?”
“A person who is not older than 16 years.” [GDPR Article 8]
“No exception?”
“Of course. Glad you asked. Some national laws can set the limit on 13 years.” [GDPR Article 8]

The first time I read about laws. I thought about stacking them like this.

national privacy law stacked on GDPR

A few weeks later I came up with this.

A pyramid with the following layers from the bottom up; Human rights, GDPR, National privacy law, Region law, and Place law

Yes, another test pyramid.
Why? Because the lower the law, the bigger the impact of the law.
And this model is dead wrong.
Small reminder: it is my model, which is wrong.
Next is my proof.

Let me focus on two layers of this pyramid: GDPR and a national privacy law. If I am a judge judging about a privacy case in Belgium, this is my route: GDPR, Belgian privacy law.
Sign with GDPR pointing pointing to sign with Belgian Privcay Law

Time to add some complexity. You know exception on exception. I have to judge a person with two nationalities.

Sign with GDPR pointing to signs with Belgian Privcay Law and Spanish privacy lawas pointing in the same direction

This is my route: GDPR, Belgian privacy law. and Spanish Privacy law.
I am really lucky. Both laws lead to the same judgement.
Now people will say:
“Hey. I can still use the pyramid?”
“I can make it a camel case”
[Pun intended]
GDPR block with two smalls blocks on top: Belgian privacy law and Spanish Privacy Law

“What about this?”
Sign with GDPR pointing to signs with Belgian Privcay Law and Spanish privacy lawas pointing in the different directions

Summarised: the test pyramid uses impact instead of direction, which is rather complicating things.

Finders fixers

The one, who finds a problem, solves it. This is common practice in my DevOps team. I made a model for testing purposes and found a fault in it, so I have to correct it. Fair enough.

When I was looking for the best law to apply, I thought about the strongest law. Something with the most articles and most severe penalties.

I looked on the internet and found a page in Wikipedia about Conflict of laws. My children are quite sceptical about Wikipedia. “My teacher told me that you cannot trust Wikipedia, because everyone can edit the page.”

A flag, a house, and an arrow pointing to a big dot

Anyways, the following laws seem proper candidates: the law of the country where you live or the law of one of the nationalities or the proper law.
So my mental picture of the signs is the right one. Sign intended.

Writing about signs. I could make a model like this:

A sign which points to 2 signs, which in turn point to 2 signgs
But this model is also too simple. The Benelux, a union of 3 countries, is more complex than this model. The Netherlands is part of the Benelux and has 12 regions. It is difficult to show this in a 2D figure.

A few sticky notes, which hold smaller sticky notes, which in turn hold smaller sticky notes.

But frankly this is even for me confusing. So I rebuilt this 3D by using sticky notes with blue lines:

Sticky notes with 3 blue vertical lines on them

Then I put a sticky note with curly red lines to one sticky:

Sticky notes with verticla blue lines and one has a ticky note with red curly lines.

An then I connect some very small sticky notes with a single orange lines to the last attaches sticky note:

Sticky notes with verticla blue lines and one has a ticky note with red curly lines, which have sticky notes on it with orange line

This model gives me a more appropiate way to handle the laws.

Also on Wikipedia there is a page which described how to determine the right law.  There is basically a set of rules which a judge must follow.

And yes, I do mind the warnings of my kids and their teachers. Kids are like websites: sometimes I cannot ignore them.

If your company is GDPR compliant, then there is no time to rest. You still have to browse through the national laws. [GDPR 8]

This might sound complicated. Let’s take a huge example: the United States of America. If you live in Florida, you have to stick to the laws which are used for all states and the Florida State Law.

What now?

So have a chat about GDPR with the people from the legal department. They can become your best friends in the coming months. And beyond.

To boldly go where no techie has gone before.

January Testing

Somehow I ended up with this test term or test type. Actually is a subset of boundary value analysis. But I got your attention.

That’s my right

It was the second day of the year 2018. I was about to place a new post on my web site. I just knew something was wrong.

I went to the web site lay out. It took me a few clicks to open the footer. Then I changed the text to
“2014 – 2018. Mindful Tester. All rights reserved.”

Now I could add my post.

It is my right
for which I fight

That’s my audit input

The same week.
For the audit I ran a query in the defect registration system. The number of items on the list was startling low: 0. My query was wrong. That bugged me. Last year it gave the right results. Actually a few weeks earlier.

I had a look to the query and noticed:
I don’t know all the commands, but I could make a good guess.
This year started at January 1st 2018. I was one year off.
The report was about 2017 and not about 2018.

A few hours later I had to go to my boss. He still used the same old query. It was easily explained.

It is not the query
I marry

A test idea approach

Let me generate some test ideas:

  • Is there a checklist for things to be updated in the new year?
  • If yes, so when is it updated?
  • Are queries based on fixed dates instead of relative dates?
  • Are there changes in laws which I have to pay attention to?

Still wondering about the pic with door?

Janua is the Latin word for door.

What about May 2018 testing? Excuse me GDPR testing.
Do you know what the effects of the General Data Protection Regulation are? An European customer has the right to be forgotten. But what about payments?


Hide or Seek Outside the Comfort Zone

You can only listen
Aaltje Vincent about Twitter

The day after Christmas I saw a question from Kim about test types on Twitter. She could not find the desired information. I remembered a long serie of Jean-Paul about test types.

I did a quick search on the internet and found some useful definitions. On my reply a most grateful reply returned. Twitter can be quite helpful. In deed.

The truth is out there.
Special Agent Fox William Mulder

In order to prepare my tests I use Gherkin to describe my test ideas. Okay. I slow down a bit.
In the defect registration system I can add comments. I use Gherkin, because it is a high level language. It saves me the pain of dealing with all kinds of technical details. Which can change of course. Which is out of my control.

The devops in the team use Gherkin for their unit tests, so they can use my test ideas in case of jumping in. One devops tends to browse my scripts with test ideas. So it actually saves me time and I can do more exploratory testing. The unit tests are quite thorough.

One day I was looking for test ideas for removing items. It was simple and therefore deceiving. I did my run with SFDIPOT.
O stands for Operations, how will it be really used?

I imagined the users looking at the same item. What would happen, if two users would delete the same item?

I stood up and saw the devops who had the best overview of the system. I walked a few meters and started a chat about the simultaneous removal actions. He listened and agreed that this should be taken care of.

My scrum master joined in with a minimum of words: “Race conditions”.
Back on my desk I searched for race conditions.

I felt comfortable when I asked my question. The 2 words of my scrum master increased the safety. For me and the others

Ask yourself, “What would my cheerleader say to me right now?”
Denise Jacobs

One of the best things about the testing community is 30 days of testing. On twitter you can find a lot using the search term #30daysoftesting. The hash tag is a label which can be used to find relevant information or questions. E.g. #testing, #softwaretesting.

One of my best Twitter months was July 2016. There were small and big challenges in 30 days of testing and participants were encouraged to share their progress on the internet. Testers over the whole world participated.

I liked the atmosphere. Trying to create strange tweets which still fulfilled the challenges posed. Some of my tweets got likes and I became more comfortable with Twitter. It was possible to share my thoughts about testing. With people I never met.

I think #30daysoftesting is great. Thanks for making it happen
Bret Pettichord

“Just name two people who are quite influential in the test community about complexity.”
“Let me guess.”
“Hey I was first. To let you guess.”
“Dave Snowden and Nassim Taleb”

“Why those two?”
Snowden made Cynefin. For me it is a model how to handle in certain situations. Nassim Taleb made INCERTO. He focuses on becoming anti fragile.”

What was I struggling with?
Is there any relationship between Cynefin and INCERTO?

So I used Google and another search engine. I could not find something on the web which addressed my question. With a bit of luck this is the first one with a first step.

The same day I answered Kim’s question I looked to the tweets from Dave Snowden. He was busy with the principles of antro-complexity. He welcomed questions. For three days I had been polishing my question.
So I tweeted my question how these principles did fit in the genealogy of INCERTO.

I got a reaction.

Without doubt, you have to leave the comfort zone of base camp and confront an entirely new and unknown wilderness.
Stephen Covey

Skin in the game of diversity

So I had my own piece of constructive feedback for tech conferences a few weeks ago.
Okay you may call it a rant. It was close.
Actually it was. For a good cause. Diversity.

As a blogger I could just lean back.
Now it was time for my action. Skin in the game.

Target One

This year I was on a test conference. The number of female speakers was low. Some male speakers might say:
“Let’s double the number of female speakers. Fine with us.”
“Thanks guys.”

There was one female co speaker. That makes two female co speakers.
Wait. Now let’s look at the incredible number of female keynote speakers of 0.
2 times 0 makes 0. If this would be quadrupled or octodupled, it would remain a disappointing 0.

In my experience there are more great female testers than 2 in the Netherlands. Let’s give them a place on stage. Main stage please. Thanks in advance.

In the past no correspondence was possible about the proposal selection process of this conference. Discussion takes time especially with Dutchmen. So a bit of transparency might help.

“You’ve got a question. What is your question?”
“I think that there are great stories of women out there. We don’t have the time to coach them.”
“Just go to”

“But this is a Dutch conference.”
“Indeed. But English is no problem for the attendees.”
“There are no Dutch coaches.”
“That also worries me a lot. A testing country without Dutch speaking coaches is strange.”

My first tweet to the conference was a wish for 2018: more female speakers than in 2017. No reaction at all. That was my target practicing. Next.

The next tweet was aimed to the chairman. I remembered two announcements of keynote speakers:

  • “When I was in London for BCS, I met [white male speaker]. I was quite impressed with his talk, so [ ….]”
  • “The other keynote speaker is [white male speaker] . I met him at TestBash Netherlands. “

Now let me stress that I am impressed with the test experience and knowledge of these keynote speakers. Now imagine two female keynote speakers and my reaction would be double wow.

Ready, aim, tweet.
My friendly request to the chairman was to have a look at a list of female testers in case of keynotes.
The answer was considerate: some of the ladies had already spoken as keynote speakers, but he would try to get other ladies on stage.

My tweet had an unexpected side effect, which I had not anticipated. A case of collateral praise. One female speaker reacted with “cool and Very proud” to be on this list.

Target zero
A few weeks ago Rosie Sherry pointed to a whole discussion about #PayToSpeak. As a speaker it is the norm to pay your travelling and accommodation costs. Both Rosie and I don’t like this.

I reacted with a Balanced Conference Card.
Just answer enough questions with Yes and you have a balanced conference.

“Why was TestBash target 0?”
“It was not. TestBash is one of the conferences focused on balance.”
“Why did you call this paragraph ‘Target 0’?”
“It is an inside joke.
Programmers always count from 0.”

Anyways my blog post ended up in the newsletter of Ministry of Testing. Thanks.

A week ago there was another discussion about female speakers. I pointed to my blog post. Somehow I ended up as an ally.

Target Two Too
Next tweet target was a huge European test conference.
You know: this size fits only one in Europe.
I actually was hesitant to send a tweet, but a disappointed tweet of a famous tester about another male lineup at a conf pushed me out of my comfort zone into my action zone.

Now comes the scary part.
The program chair had no Twitter account. A few years ago this was already mentionable according to his track chair. So I picked the conf itself.

I thought it was good to praise the conference organisers for actions taken for a diverse lineup this year. The aforementioned famous tester was also pleased with the steps taken in the past. So I retweeted her reaction with the request to keep up the good work.

This left an unaddressed program committee.
I tweeted a female member of the program committee to repeat the success of diverse lineup of this year. She did not react.

The conf did. Gender bias was taken care of by making the proposals anonymous. Reads good to me.

Target Two Too
By now you know the drill:

So if you want to have more diversity, you have my permission to use these actions.
Excuse me for scaring you.

Some tech confs have questions about diversity.  I respect their point of view. I heard really good stories about first time speakers supported by Speak easy
Oops let me add this to the list above.[v]

Conferences want praise from their attendees. So give New Voices a stage like Agile Testing Days. You know what: those first time speakers have a lot of keynote potential in themselves.

Little thought experiment
Bio 1: I am a white male speaker. My dad sent me to the best schools and universities, because he could afford it. I now work at a Fortune 500 company.

Bio 2: I am an African American woman. I grew up in the Bronx. We were really poor. The only times we saw a computer it was on the television.

One day I was invited by Per Scholas. They thought I was bright. I laughed. They suggested something with computers. Me and computers, I could not stop laughing for 1 minute.

Anyway I got some really good teachers. And I could touch a computer without some comment like:
“Hey, what you’re doing?”
It was not that difficult to pick up things. It was logical.

For me the time with Per Scholas was like a Christmas story. After school it would be back in the shop selling veggies. You know what. I got a good job offer. Ain’t that great?

“Wait you are writing everything down.”
“But that is not a good bio.”
“I assure it adds so much flavour.”

Of course I am biased, if I see those bios.

  • A very influential tester once told me, he would like to follow the Per Scholas course. And he is already good.
  • These days I have to learn a lot just to keep up. What could this woman teach me about struggling and keeping faith?
  • I want to see the underdog.

Okay recruiters, don’t send me mails now.
I have a disappointing message for you. Both bios are fake.
Because this is a thought experience: you know like imaginary or hypothetical or dreamed up.

Target Me
Now what is all that stuff about skin in the game?
Good question. Which needs a good answer.

If I get accepted by a tech conf and there is big unbalance, then I will bail out.
I won’t speak.
This talk will be dearly missed on my CV and my Conference Speaker Bingo Card,  but that is my skin in the game.

Side notes

  • I am going to submit to 2 #PayToSpeak confs in the next few weeks. Although I don’t like this.
  • One conference is part of yearly package deal membership under 100 Euro. This club offers a lot to the testing community. So I give something back.
  • One conference is so nearby I lose a small amount of money for travelling. I can sleep at home.
  • And I want break my streak of speaking every other year. In the meantime I have this, a blog to practice my speaking skills.

A Balanced Conference Card

This year a woman thanked for joining a group of attendees at a tech conference. I forgot it, until social media showed a lot of interest in bad behaviour.

I planned to post this post weeks later, but the stories became more unbelievable as time progressed. One female speaker blogged about an invitation to share a room with the event organiser. Another female speaker did a remote talk after receiving death threats.

Last week a whole discussion on Twitter started about paying speakers. This basically means that not all relevant voices are heard because of the costs. According to me that is bad. I’ll write later on why.

We live in an unbalanced world. We live in a time, that we need role models, who look like us. We live in a world, where talent still can be found.

These are some questions for conferences to provide better balance.

1. Conference Announcement

  • Are there any pictures of attendees and speakers?
  • Are there differences in gender and colour on these pictures?
  • Would you be comfortable with them?
  • Were any incidents in previous conferences handled well?

2. Call for papers

  • Has the program committee women and men?
  • Are there is a good mix of consultants and perople who are not consultants in the committee?
  • Are pictures shown of speakers from previous speakers?
  • Is the proposal clear?
  • Is help provided with the proposal?
  • Is it clear, what kind of talks are asked?
  • Is it clear, which costs of the speakers is being covered?
  • Does the conference cover all  costs of the speakers?
  • Are special slots for first time speakers?
  • Is help provided for speakers?

Awesome example:

3. Program announcement
Looking at the selected speakers and talks, is there a good balance in

  • Gender
  • Race
  • Experience in topic
  • Experience in speaking
  • Hands on talks / workshops versus high level talks
  • Soft skills vs hard skills
  • Are keynote speakers female and/or people of colour?

4. Communication
For attendee

  • Is it clear, what you need for a session? Laptop or Java knowledge.
  • Is the level or the talk clear?
  • Is clear speaker info provided?
  • Is speaker information only sent to the speakers?

For speaker

  • Do you have all information to give your talk before you enter the conference centre?

5. Conference day(s)
For speaker

  • Is it clear, who the track chair is for the presentation?
  • Does the track chair support the speaker before, during, and after the talk?
  • Are the technical facilities taken care of?
  • Is the Code of Conduct used?
    Great example is Agile Testing Days.

For attendee

  • Is there room for questions?
  • Is it possible to talk to the speaker after her or his talk?
  • Is the Code of Conduct used?

6. The days after

  • Is there an evaluation?
  • Are relevant actions taken to provide a more balanced program?

How do we know as conference organisers, whether the conference is balanced?

There is healthy number of yes.

Bottom line

  • Is it safe to organise?
  • Is it safe to speak?
  • Is it safe to attend?

Elaborate notes
Note about hands on:
If you want a dev conference, I understand live coding sessions and an occasional manager with useful tips for devs. For a C level conf – only people with Chief in their job title – I expect only high level speeches. A coder with the right level of abstraction might fit in. The target audience is something to think about.

Note about choices:
There are conferences which focus on the quality of the talk. This is dangerous: attendees get great stories from the same people or the same companies for years. Their company is not my company. I need different contexts to make a good choice what to apply in my firm. As my scrum master once summarised: “It was a consultant selling his own product.”

Note about #paytospeak:
Why is there so many fuss about #paytospeak?
Let’s suppose I get accepted by conference in Italy. They offer me a free ticket to the three day conference. That sounds quite generous.
The following costs I will make for

  • Traveling to the airport
  • Plane ticket
  • Cabs or public traffic in Italy
  • Four nights in a hotel  with breakfast
  • Lunches and dinners for 4 days

I would not be surprised to spend 1000 Euro just for speaking. This sum might increase, because conferences often take place in touristic places.
So I could go to my employer, but my budget is limited. I once paid all my travelling costs and I was really lucky that accommodation and food was included.

If I would be accepted for 2 talks at tech conferences abroad, I have to use money on my saving account. So it is pretty costly to talk abroad, if these are #paytospeak conferences. I have to pay to speak. I don’t even break even and I will lose money.
And I don’t bill all the hundreds of hours I spent to get a decent talk.

Now I gonna write something weird. Suppose I have a new revolutionary way to test software and I would speak at all #paytospeak conferences abroad, then my bank would really notice this within a year. Even if I am a keynote speaker.

Note about diversity:
Do you know that story about the white male project leader, the yellow man and the African European woman?
No. Okay. The white male project leader, the yellow man and the African European woman entered the conference centre. In the keynote the white male project leader gave his extended pitch about the context. Then I, the yellow man, blew the minds of the audience telling about a multi dimensional test model. The African European woman remained seated and she deserved a place on stage.

Months earlier In my second week on the project there was something harsh in the air. For months the team had struggled with this software test model. The progress was low in the previous weeks. The male project members were starting to grind their teeth, when the African European woman politely requested to test two models. There were still grunts, but the other project members eventually gave in after friendly words from her.

So the project team split in two sub teams using two different models. At the end the outcomes were compared. The result was almost the same. Her model was the most easy one to use. The team spirit was back in town.

In that same week I got my Eureka moment. I could make some nice extensions to the model. Think about third and fourth dimensions. I only needed a push and she provided that.

One project day I met her. She was all smiling. She told me about a presentation of the model to her team mates. They were very experienced testers and they could not believe that the model was feasible for their context. She just answered all the questions all alone. And I had missed all the fun.

In the weeks before the keynote I asked her to speak several times. She replied with:
“That is not a place for me. You just talk.”
My project leader raised his shoulders:
“She is too modest.”

This is the story about an African European woman who used her kindness to save a project. This story is also about cultures. This is something I want to share with you the reader.

Note about proximity:
This year Marcel Gehlen explained the zone of proximity to me. If someone is in my zone of proximity, then I can easily tell how things can be done. An example, if you are an experienced tester and familiar with mind maps, I can point you to Test Insane.
An example how things could go wrong: one of my kids sometimes sighs very hard:
“Could you please explain this in children’s language?”

Most conferences tend to choose speakers with years of experience: a decade minus or plus 5 years in the subject. If I am unfamiliar with the subject, then there is chance that I don’t understand the speaker. I would call it a case of different proximity zone.

One evening I was browsing through a blog post. The same kid showed attention, so I showed a Visual First Person Customer Journey. Sorry for the click bait. I could not resist it.

I asked: “What is this about?”
“It is about a cinema visit.”
“How do you know?”
“It is the popcorn.”
[Pointing to the door with the heart]
“What does this mean?”

“It is the toilet.”
“How do you know that?”
“They use it in comics.””
I just had entered the kid’s Zone of Proximity.

Let me change the situation. I have to tell about my struggle with Test Driven Development in the first years. The first year I can really recall all the pitfalls I fell in. In the second year things become natural for me and the impact on TDD starters is becoming less and less. Basically my story has an expiry date. It will be lost unless I blog about it.

Experience reports can be useful:
If I want to learn advanced HTML, I need JavaScript and CSS. If I want to learn CSS, I need HTML. If I want to learn JavaScript I need HTML. You want to know how I tried to solve this puzzle?

Experience does not always count. Warning: some absurd situation ahead.
“How many times did you do this?”
“About four times”
“And you think we should support you. Listen to your story?”
“Look I brought you a potato and tobacco. People will love them.”
“We had something else in mind: spices. Do you understand, Mister Christopher Columbus?”

Note about female speakers:
At the beginning of this month Cory Foy had almost no female candidates for CTO. He just tweeted and got 27 awesome candidates.  And counting.

What really stuck, was this statement:
“I don’t want to ever here an excuse for not having amazing females for senior positions or at conferences again. ”
Focus on conferences again? Good, keep this in mind.

This year a list of top test automation people was published on the web. This list was strange: there were almost no women on the list. A spontaneous brainstorm session started. A lot of names of women were mentioned. This lead to the following two lists of female testers:

In case a conference organiser still cannot find good female testing speakers, I honestly won’t understand it.

Note about compensation;
This year an upcoming female speaker asked a #paytospeak conference about, how costs could be compensated. The answer was none except for keynote speakers. She was in the right league, so she asked for specifics. The conference answered that there was a limit. This could be called reasonable.

Unfortunately the limit amount could not cover the trip from New Zealand to Europe. This lady is one of the organisers of a conference in New Zealand which covered travel and accommodation costs for all speakers. Including one from the UK and one from the USA. This year.

Also this year. A female speaker had a question about a compensation for a co speaker. She wanted to talk about pairing in software development. The conference answered that only one speaker would be compensated. Now you can argue, that pairing cannot be demonstrated by a single woman or man. This lady happened to organise a conference which compensate the costs of all speakers, even the co speakers. BTW on Twitter I already saw one relieved female speaker of Euro Testing Conference. The plane tickets were compensated two months before the start of the conference.

TestBash has also great compensation for speakers. It was updated and republished after the #paytospeak discussion.

Note about harassment:
It was late in the evening, but the disco was packed. People were dancing on the floor having a good time. Some young women were approached by men. There was no contact, so another man tried, retried, and retried again. I knew these women. As a man I had to do something.

I just stood there looking up. I was like uncle Vernon looking at Hagrid angry that some family history had not been shared with Harry. The music started to fade for me as we locked eyes. I put all my indignation in my stare and did not back off. He put all his irritation in his stare …
and backed off.

Slowly the volume of the music went up for me. I noticed that people were dancing.

The following day one of the women thanked me. Apparently I had done something right.

The burden of a script

I was speechless. I felt devastated. After 5 days I still could not use internet on my smartphone. I had not prepared myself for failure.

Day 1
With a lot of reassuring mails in my mailbox I assumed that switching mobile provider would be a piece of cake. My new SIM card seemed to work well except for the Internet. After several failed attempts I looked on the website. I could not find information to solve my problem.

I called the service desk. The agent was friendly and suggested several options.
“Cross test? Never heard of it.”
I had to use my SIM card in another phone. Good test idea though.

My request to set the access point was denied.
A lot of resets followed.
Another agent took over. Another set of instructions.
Another serie of failed internet connections.
My request to set the access point was denied again.

A new SIM card would arrive within two working days. Too bad it was Friday.

Day 5

My family reported that no SIM card was delivered on the second working day. So I did a courtesy call to the help desk.

My math was 1 day off. I try to explain my reasoning: you could have sent it on Monday, so I would receive on Tuesday. Helpdesk math is unbeatable.

Back to the internet. At least that was really wanted. My request to set an access point was denied again. I got a bit annoyed. But the agent was willing to browse through all settings. I kindly denied. Already two of his colleagues had failed.

Day 6

On this day my SIM card was in my mailbox. Great.

I switched the SIM cards. No internet. Moved the new SIM card out and in. No internet. Tried to turn the card around. No SIM card detected and no internet.

Then I lost track of all my attempts. I did a cross test with the new SIM card on the phone of my wife. Internet!

Switched back the SIM card to my phone. No internet. I had no words.

Day 8
My wife had noticed my gloominess. She let me go to a computer shop to fix my internet connection problem. In the shop I rattled off all tests I had done like a professional helpdesk agent. Of course some tests were repeated. My phone was still capable to connect with the internet, if I could use the SIM card of the computer shop guy forever. Bad idea.

Now comes the big surprise: he did configure an access point for me. While making remarks about the illogical structure of the web site of my mobile provider.
Alas still no internet.

I was advised to grab my car to go to the nearby shop of the mobile provider. Because I had already cycled halfway, I cycled a bit longer.

In the mobile provider shop I had a longer list of attempts to recite. The man listened to my story and repeated a few attempts. Just to be sure.

“My colleague is resetting the internet.”
“Is this not dangerous?” I asked.

Believe it or not: this man configured the right access point for internet. I was relieved and grateful.

Day 9
What is the lesson learned?

 Provide a good web site. This saves time and shipping costs. And this blog post, which might go viral.
 Test the SIM card before migration on all kinds of phones.

No notes

I had no notes
No music came into my mind. Silence.
It was my turn.

I had no notes.
No melody, no bass line, and even no chords came into my mind.
The people in the room expected me to do something.

I had no notes. After I had looked in my subdirectories: no database scripts. No relevant test charters. Actually I was supposed to test, not to make music. Mind you.

Getting back on track
It was time for my first flashback. A week earlier I had to test the same application. The import function had not been implemented yet. So I used some code of the unit tests.

I told myself to write a short note in the knowledge management system. This hunt for the code should not be repeated every test session.

The first step was to open my IDE or Integrated Development Environment. This tool helps me to program, build, and use version control among other things.

It was simple to find the right repository: it was still in view as I left it.

Another flashback came in. I could not use the code and one of the devops replied that I did not use the latest version.

So after the flashback I did a pull request and got the last version of the code.

My purpose was to find code to fill the database. I went to the unit test. A unit test has several phases. First I focused on the setup and breakdown. I could easily copy the code to make a table and the code to throw the table away. There were more commands for the breakdown than expected, so I had a small chat with one of the devops.

The second step was to find a way to fill the table. No other database commands could be found in the file. I saw a method to put a record into a table, clicked on it and saw the code of the wanted database stuff.

Then I reformatted the code. Now I could make, fill, and destroy the table at my own convenience.

During the test session I opened a test charter for notetaking.

End of the track
Did I put my steps in the knowledge management system?

Last flashback. I promise.
I was talking to a team member. He explained that he never bothered to make notes for these cases. Things changed continuously. His Best Friend was the IDE.

Tweaking My Website Security

WordPress is frequently used for websites and therefore attractive to some unfriendly people. So I reconfigured my WordPress security plugin.
And the mails of failed logins started coming in. It was not me, so someone else wanted to use this web site.

A short history about my tooling
For me web site security is something to review on a regular basis. It all started with an article in a magazine.  I put some elementary stuff in place: limited number of log ins and removed the login from the web site.

Over the months I added extra stuff like SSL. It encrypts the traffic between the browser and my web site. In other words my user name and password are unreadable for interested bad guys
Troy Hunt mentioned SSL in his free web course with the haunting name: Hack Yourself First.  Cheers mate.
In case you missed it SSL can be obtained for free at Let’s Encrypt.

At a regular basis I updated the software for my web site. I thought I was quite good until I changed the settings.

A short note about security
Some people might complain about the default security settings of their web site settings. Believe me things can be improved. If you do not set the WordPress settings right, then the user name is shown instead of your writer’s name on the blog post. Luckily there are plugins. (As a Dutchman I could not ignore the free ones.)

I thought about the default security and try to explain to you. If I buy a house, it has standard locks. If I want to keep the baddies out, I have to use the keys.
There are no special keys and locks involved. In case I need them I have to change them.
My new house has no vault or armed guards. If I need them, then I have to change something.

Shorten my list of security mails
So I had changed something and security mails came into my mail box. I noticed that there were mails with wrong user names and passwords. Not good.

After a few days I expected them to stop. You know: “Oops wrong web site. Sorry for that.” But the flow of failed login attempts did not stop. So I had to change something. Again.

I remembered a firewall in one of my WordPress plugins, so I had my first taste of a firewall. Dry, not shaken.
I had IP addresses of the sources of attack. Courtesy service of one of my WordPress iplugins.
An IP address consists of 4 numbers separated by a dot (.) like the invalid 345 345.345 345.

So I put the most offending IP addresses on the black list.

Three strikes and you are out.

The brute force attacks continued. The following combinations were used:

table with failed login attempts

The  user name is in the heading and the password is  shown in the first column. More details about this teaser will be added in the appendix.

My action did not change the flow. I used the asterix. 345 345.345.*. All people coming from IP addresses starting with 345.345.345 got blocked.

Wrong zone. Offsite. Stop the game.

It looked like I had put oil on fire. My normal mails were somewhere between the security mails.

I also noticed that black listed IP addresses still passed through. So there were apperently some smart guys pick locking the door of my web site. I’ll add some words to this assumption  at the end.

It was time for harsh measures. I was so focused on the mails, that I skipped my notetaking. In my logs other URLs were mentioned.  I clicked on one containing wp-admin and noticed that I saw my login page.
I changed a name somewhere and the security mails did not come in any more. Phew.

Brief briefing about red teaming
My list of WordPress plugins would be quite interesting for the people who really want to block out the intruders. The main reason I do not list them is red teaming. This military term is like give my plan to the red team, who will misuse this knowledge to my full disadvantage. Did you notice that “full” sounds like “fool”?

My steps for red teaming of my web site:

  1. Install the web site with all plugins.
  2. Configure the web site and the plugins.
  3. Look at for any bugs.
  4. Misuse the listed CVE or Common Vulnearbilites  and Exposures.
  5. Go to the subdirectories and look for strange files.
  6. Look whether those files are accessible from the outside.

This reads like the plot of a bad B movie. But it works.

A short note about security
Some people might complain about their default website settings. Believe me things can be improved. If you do not set the WordPress settings right, then the user name is shown instead of your writer’s name on the blog post. Luckily there are plugin.

I thought about the default security and try to explain it to you. If I buy a house, it has standard locks. If I want to keep the baddies out, I have to use the keys.
There are no special keys and locks involved. In case I need them I have to change them.
My new house has no vault or armed guards. If I need them, then I have to change something.

Had a short glance
The days after the intentional reduction of my mail I had another look to my log files. My login page was requested several thousands times in a month. And I can assure you that I was not blogging so much.

There were other pages or URLs which led to my login page. So a check on the hits on my login page would give me the wrong impression of safety. There are people who do not like to use numbers or metrics. Some numbers can be really useful when pondered upon.

Somehow I had not paid attention. Too much focus on blogging. Obviously.

An article of Santosh Tuppad was quite helpful to increase the security. Thanks mate.

I even noticed that wp-content was open. So any pictures of draft blog posts could be viewed before publication. I even discovered a CSS file of a WordPress security plugin, which I could access without logging in. It was like finding a business card of a security team at the doorstep.

Wait a moment.

Let’s turn this into a multiple choice question.
What is the reaction of thieves on the business card?
A. Let’s skip this house.
B. I know how these guys operate. Piece of cake.
C. Look at the big bird and the shield of armor. That is pretty neat. We need 500 of those cards.

Definitely something for an action movie.

Some tips:

  • Read the reviews of the WordPress plugins.
  • Install WordPress plugins from the official site.
  • Write down, what works.
    Some plugins do not mix. This might be the cause of the strange behaviour of my firewall.
  • Make an offline copy of the website before tweaking.
  • Tweak the website security several times a year.
  • Go to your web site on a regular basis and install the updates.
  • Keep on an eye on Social Media.
    Troy and Santosh are great sources.
  • Basically, explore your web site security.

Appendix A bit of data crunching
For my first real life forensic investigation I wanted to use the gathered data. As in Data the Gathering. In order to process my e-mails I used baregrep, vim, Javascript, CSS, HTML.

People had attempted to break in my web site. I expected a concentrated set of failed attempts like
expected heat map

When I looked to the patterns I noticed this:
observed heat map
This is an example of a Blink Test. Lots of info processed in milliseconds and still getting useful info.


  • Combinations were entered once.
  • Combinations where user name was the same as the password were frequently used.
  • The same for combination with user name equal to admin


  • There is a high chance that a group tried to break in. There is a moderate chance that there were more groups which used different lists.
  • A popular user name is admin. See the first column.
  • Single words are favourite, followed by words and numbers.
  • Some user names and passwords were linked to my blog.
  • My blog posts are read.

Q & A Bits of performance testing

“Why are you writing this Q & A?”
“Because people have questions, which are unanswered about ‘Bits of performance testing’ . ”
“The use of drawings and perfornance test it self.”
“So I can send in questions?””
“Sure on this page. Why not?”

“Hey, I am the one who is supposed to ask questions.”

“You used a lot of pictures. Is this not a waste of time?”
“In my blog post I drew a picture about the customer journey. This led to the conclusion that the Wifi network should be split in a private and public ones. This is not a waste of time.”
“I agree. But there were other pictures, which did not have that big impact.”
“That is right. But the drawing of picture takes minutes. Implementing the wrong performance scripts takes days.”

“You used SFDIPOT instead of another heuristic., FIBLOTS”
I am aware that this heuristic exists. I also know that a performance tester made it. And I just forgot it.
The reason I chose to pick SFDIPOT is to use this heuristic in another context. I learn by taking small detours. What if I do it in another way? My main reason was that I wanted to write about performance test in another way.
This kept my spirit high and extended my Circle of Comfort. That is a comfortable thought.”

“Your story about the performance test is sometimes difficult to follow. Why did you write a nonlinear story?”
“Testing is an activity which is unpredictable. I can find bugs on the strangest moments. This can trigger other ideas.
In this blog post I tried to describe what is going on in my head.”
“How can you sell this to your boss?”
“Just ask her or him, whether business cases are also written in a linear way?”

“I miss disk storage That is an important resource to watch.”
“You are right. I missed that one.”
“How do you know that this is a good load profile?”
“Some data have to be gathered. Think about log files and analytic stuff. The challenge is not to confuse frequency with resource usage.”
“Would you please explain that?”
“If 1000 users look to a simple web page, then almost no resources are used. If 1 user asks all articles in stock, then a lot will happen. The database is bring queried and much data is moved over the network. So look for resource usage.”

“So I only have to focus on user actions with heavy resource usage?”
“That is only possible with a very simple web site. Sometimes web sites or programs on the backend do not remove their garbage.”
“What do you mean with garbage?”

“Suppose you have a cinema web site. The purchasing department wants to know what kind of drinks and snacks are ordered in advance. Suppose all the results of the queries are stored. It might be interesting for their department and it should be stored on one of their systems. But not in a module of the web site.

Another thing is to simulate the customer. Use the customer journey. A customer does not only buys cards, snacks and drinks. She or he will also collect them. This lead to click paths, how does the user maneuver through the system: which screens will be visited and which options are used?”

“So if you have a lot of data, what do you do?”
“The challenge is to find patterns. Joe Common is more like this:”
customer journey Joe Common
“Why did you not draw the payment page?”
“It is important. I left it out because of the page size. My plan was to sketch out a rough journey. Just wait a few pics.”

“I suppose you left out snacks out for the same reason. Smart ”

“Why does he go one screen back?”
“That is hard to say. You can make some assumptions or hypotheses: the price was too high. Or the wrong movie was selected. You can always ask some questions to validate the assumptions.”

customer journey no deal

“Hey there is no deal. What’s wrong?”
“Most of the yearly cinema visitor is looking around or scouting. Were there any changes in the web site? Which movies can I see? ”

child birthday party
“What is going on here?”
“A children birthday party.”
“That looks like a lot of fun.”

noted bar graph

“What are you showing me?”
“This is the number of screens and locations which were visited, and their corresponding numbers.”
“The bars in the graph are getting lower. Is this not frustating for marketing? ”
“Actually this is normal. This is a sales funnel.”

extended sales funnel

*It is like commercials: not everyone buys what they see on the television. They do not buy everything they see in the shop. You actually want people to do things which bring profit. The Call To Action or CTA for this web site is buy tickets, drinks, and snacks.”

“Okay Mr. Monologue. I’ve got a few questions.
How did you make that funnel?*
*Like this.*

bar graph to funnel

*Why do you call this an extended sales funnel? Your CTA is buying tickets, drinks and snacks.*
*But there are two moments to buy drinks and snacks: in advance and on location.*

extended sales funnels with CTAs

“If I had put the CTA in the first funnel picture, it would be cut off right under. By delaying this action I got a better view on the situation.

The second CTA is successful, if the first good. John Common is not willing to buy extra drinks and snacks after a slow performance of the web site.”

*Are you writing that models can confuse people?*
*Yes, they do. Let me give another example.*
*Be my writer.*
*if you go twice to a cinema, you go twice through the sales funnel.

two funnels in a row
*So you should take repeat customers in account. *

For some businesses good customer contact is vital. In ‘Delivering Happiness: A Path to Profits, Passion, and Purpose’ it is called creating a moment of Wow.”

“And if things go wrong?”
“Just watch social media. Several companies have web care teams who actively look on the internet and engage with grumpy customers.”

“I would be interested in the numbers. So how would it look like?”

draft breakdown numbers

“Wait. This concept looks better than the draft one.”

concept break down numbers

“Are these estimated numbers for all visitors?”
“That is a good question. It is only for the website visitors, who also ordered drinks or snacks in advance. So I have to add a another set of numbers for the website visitors, who did not order drinks or snacks in advance and the visitors who pass the web site.”

“What is no showup?”
“That is a situation, when a customer does not get, what she or he paid for. He or she is ill. She or he found another group of friends and skipped the cinema.”

“This is strange: there are people who go to the shop without the ticket.”
“This is a common group mistake. A group of people at the cinema asking each other: “Who bought the tickets?””

“Why did you show the draft version?”
“I do not jump to solutions. I want to share my way of thinking. My thoughts and my pictures.”

“But those reversed trees with numbers is not a good starting point for scripting, I assume.”

“You are completely right. All that number crunching can take the attention away. If we have 3525 visitors coming in, then this must be translated in scripts somehow. We have to look at the chances.”
“But this is a cinema web site. It is not a casino.”

“Just have a look.”

click paths

“What does 70 % at Movie mean?”
“This means, that there is a chance of 70 % that a visitor goes from the Select Movie Window to Select Ticket Movie. You can call these click paths.”

“You are basically stating that the software has to roll a dice to determine the next step.”
“Yes. There is no single ideal path through the system. I already showed pictures of customer journeys.”

“Wait a sec.”
“The web site can easily be tested using a standard performance test, but a shop with real people is difficult. Do we need to test them? ”
“What is the system under test?”
“The web site and the shop.”
“Can you explain it to me?”
“The software supporting the web site and the shop are connected.”

“What does hold you back?”
“How can you test it?”
“I have not much experience with this particular situation. My suggestions would be: make a special interface for the shop just for performance test, mock the users in the shop, or use actual human beings. The latter option I call hybrid testing.”

“”Do you have experience with hybrid testing?”
“Yes, I have. For a web site we had to do a performance test. The visitors were simulated by the software. The web masters were real people including your interviewee. The actions of the web masters were too complicated to script, so we did testing manually. For more info there is a Dutch presentation, A performance test with a tail.”

“What is a good way to determine whether the performance is good enough?”
“A lot of people want a fixed number like a certain response time. Like the application will respond within 6 seconds. I’ve got an example.
In a performance test percentile is used: at least 90 % of the users have a response time of 6 seconds or less, if they do a specific action. This reduces discussions like:
“40 % has a response time of 5.3 seconds. That is quite good.”
“50 percentile has a response time of 6 seconds and the limit is 80 percentile.

Another thing to consider is to determine the worst realistic conditions for the system under test.”


“Can automated tests be used with performance tests?”
“I assume you are talking about automated functional tests.”
“I once got this request and it does not solve the problem. Is your manager writing a business case, a roadmap, and items for the back log for the same release at the same moment? I guess not. Every artefact has his own purpose.
But I wrote a blog post about the combination of automated tests and performance test.”

“When can the scripting start?”
“It is good to have a testable version before the final one.”
“But performance issues might still be in there. Also the interface can still change.”
“Change is always a companion of a tester. What I mean is that a tester should focus on things which will not change on long term. The technical basic components will probably not change. Test data is always needed. Data about the usage of the system is quite stable. Etc. ”

“There are always discussions about the test environment:
“We just put a part of the database in the test environment. “”
“Yes, a copy of the production environment is very expensive.
And yes, a performance test on a reduced environment can give a false impression.
I have a blog post about a huge test environment: Do you really need it?”

Execution or performance
“What would you advise during a performance test?”
“Have a hot line. If things go out of control, make sure everyone knows who to call or mail. Or your favourite communication style.”

“So what should a test report contain?”
“In the report it should be described whether the performance criteria have been met. Percentile graphs are really cool. It is all about a special point in the graph:”

special point

“I would like to see some real life graphs.”
“Sure be my guest. They are on slides 14 and 15 of  the  presentation, A performance test with a tail .”

“What about the resources?”
“That is a good question. This should be covered. A nice pic would be great, but that is something for another time.”

“Thanks for answering.”
“Thanks for questioning. And thanks for reading.”

Agile Manifesto during test

The appointed ticket was about a business rule. I had done all the preparation stuff and I only had to analyse the data. In most cases a front end or a good looking application is enough to determine the quality of the work. In this case it would miss a lot of points. A front end shows a summary of the data in the database. And a summary was not enough for this case. I mean database.

What I write down, is a sanitised version of the story: all confidential stuff has been changed.

Individuals and interactions over processes and tools
I opened the database client. With this program I can easily browse through the tables. I got a connection error. That was not a good start. I retried to establish a connection and got the same connection error. I tried another server: same connection problem.
At this point I could just stop. Hey, I am a …. professional tester.

I had to start to analyse my test environment. I am not someone who runs to the helpdesk on the first encounter with an error. So I tried to make a connection with a terminal. This connection was possible which I did not expect. But thanks anyways.

Now I had a connection with the right server, I could do more things. Yes, the test environment was right back in my mind.
I remembered that it was possible to use a primitive database tool. I could use a database prompt, which let me query the database. I entered a command in the terminal: unknown command.
I switched to a browser and just entered the wrong command in the search engine, which suggested the right spelling. Using this command I saw a database prompt. The internet has lots of knowledge. Now I could use my queries from the day before.

I called a fellow devops and explained the situation. I reported about my failed attempts with the database client and he agreed that the database prompt was feasible for testing.

He told me how to connect to the right database. Thanks.
And off I tested.

Working software over comprehensive documentation
While I was still figuring out my test environment I realised, that I had forgotten to log. I opened a template of a test charter. It contained useful information like date, name, and application. Mine was based on some templates. Thanks Jean-Paul.

During my exploration I wrote some quick notes in my charter. They were mental notes to me. At the end of the day I would put a comprehensible summary in the ticket.

I copied the query from a file to the database prompt and pressed the Enter key. The database tool gave feedback, that the queries could not be used. Did I mention 1 query by the way?

My query contained multiple lines which were interpreted as multiple queries. There was some logic in it. E.g.
“Show the dates
of all Saturdays in 2017.”
The tool processed “Show the dates.” as follows:
“I have dates in my databases, but I am not sure which table: the birthdays, the working days or the school Holiday days.”
The second line “of all Saturdays in 2017.” would be interpreted like
“What would you like to know about the Saturdays in 2017? The times of the sunset or the number of Saturdays?”

My fellow devops had mentioned the option to execute the database prompt with the parameter /i. This stands for input file. This is a time saver.
So I left the database prompt and saw the system prompt again.

I executed the database tool with /i query file.
I got a response that /i was an invalid parameter.

Time to get help from the database tool:
I execute the tool with /help.
I got a list of all valid parameters and their usage. Then I settled for /f, which is short for input file.

During the next attempt I executed the database tool with /f query file. And I got a decent feedback from the database.

Both my hands went in the air.!!

Responding to change over following a plan
My plan was to use a nice looking database client and I ended up with a rough and tough database tool. Okay, but I could deliver value.

I looked to the output of the file and it was hard to read. All data was shown in ASCII or text. No table for easy scrolling and reading. No fancy stuff. The table was shown in multiple line: the header was scattered over several lines and the same was true for every record. So it was hard for me to determine which attribute was linked to which value.

I put the data in a text editor. It was still hard to read. I had to remove line breaks. No way.
But I did not need all the attributes. Oops, I was hoarding data again.
So I reduced the number of attributes to a minimal set. I accidentally maximised the window of the terminal and all data was shown in more readable table.

Now a new pattern emerged:

  • Adjust the file with query in an editor.
  • Go to the system prompt.
  • Execute the database tool with the latest version of the file.
  • Look at the output.
  • Correct the query.
  • And start again.

All that opening and closing of the input file was quite cumbersome.  I felt like a file jockey. So I opened a second terminal. Now I had one screen with the query in an editor and another screen for the execution of the database tool and analysis of the output.

Time for a tweet.

Customer collaboration over contract negotiation
After reading the Agile Manifesto I realised that I had not really listened to the customer or her/ his proxy.

The next working day I went to the business to talk about the business rule. Did I really interpret this well? Instead of conditions and codes I heard a story, what the story was about. About people who needed the right service.

PS Task 4