GDPR – The forgotten tests – Test 3

Showing the status code 451 instead of a website is not enough to avoid GDPR penalties in particular cases.

Management report

Showing an error message instead of the website to users with a laptop or PC in EU is not enough. Tracking EU citizens without consent is still possible and therefore not GDPR compliant.

The section ‘Advice’ provides a more detailed description.

Disclaimer

I am not a legal expert. So please have a look at my used sources. Or contact a GDPR expert.

I am just a tester finding test ideas about GDPR. Thanks for joining in advance.

Experience report

This is my way to reflect on my research in GDPR of the last months. It took me lots of hours.

If I missed a legal or W3C link, you can always contact me. I am happy to update this blog post.

 

This spring I prepared a workshop about blogging. I tweeted about the use of sketch notes to find fieldstones. It got attention from @ConstanceHermit and Mike Rohde.

Mike had a familiar name. I bought his book about sketch noting.
He asked me for a sketch note for testing. OK. Wow. WOW.
Sure no problem.

I only had to wait for a good opportunity to put his request in practice. After a few months I saw a tweet about code on a web page:
“451: the website cannot be shown because of legal reasons.”

I visualised some scenarios and found some problems in the chosen solution. In case of impatience you can skip to the end of the article for the sketch notes. Be my guest.

Numbers are fast to communicate. If people want a pizza and call numbers, then I can go to the website and just enter the called numbers.

A pizza menu was used to abbreviate the pizza names: 16 is pizza Salami, etc. This way a protocol was set up.

The internet Hypertext Transfer Protocol is used for web sites. Status codes like 451 provide information to the user.

The problem with being a tester is to make an understandable message. This is quite hard. It is like telling how a car works without using names of car parts. I wanted to put 451 in the sketch note, but that was intimidating. I also skipped flow diagrams.

I also wanted to show off with test techniques. This was again: Not done. This is only nice for testers, but this is no good for people unfamiliar with testing. I can guarantee you that their number is way bigger than the number of testers.

Several drafts later.
One sketch note became 2 sketch notes. First I drew with a dark marker, then I used other markers for more details.

Then I set a new deadline for myself. I would use the sketch notes in a presentation. If a speaker could not make it at the test conference a week later, then I would volunteer. GDPR is still interesting stuff for testers. In legal terms it is good for the public interest.

Now I had to check my picture. And I hit the wall. It hurt.
Access is denied to the website because of tracking without consent

451 was used for legal demands. I clicked on the link to the official request to add an extra code to the HTTP protocol.
This looked pretty official.

In this case the ministry of justice contacted the internet service provider, which in turn shows a 451 to the user. Sorry access denied.

So this was not about web sites silencing themselves.
So all the hours spent were for nothing. I lost hours of work. I felt miserable. This is part of research.

The weekend before the test conference I looked on the internet. This time I searched on 451 and GDPR. The blog post ‘Is http 451 suitable for GDPR blocking?’ popped up.

So I started my due diligence.

Is it right
What I write?

The author is Terence Dent. That was the guy who had the idea for 451. I looked again in the official proposal for 451. Terence was mentioned. So my sketch note was almost good.

So I only had to change the picture. And I was all set.
Access is sometimes denied to the website because of tracking without consent
I shared my deadline with my kids and they talked about it the next days.

The evening before the conference I checked my sketch note about citizenship. GDPR was quite vague:
“Data subjects who are in the EU” [Article 2]

I could not find something about nationality. So a Dutchman in his own country is a data subject in EU. But a Dutchman in the US is not a data subject in the EU. Did I miss something?

So again I was facing a legal problem in my sketch note.

I used my search engine and found several answers on my question: is it possible to track EU citizens outside the EU?
On Quora there was majority in favour for not tracking. One legal looking website had a complex advice with lots of conditions.

Law is not about democracy, but about sticking to the rules.
Basically I hit the wall again.

Now I am a Dutchman. The big advantage is that the number of Dutch web pages is lower than the number of English web pages.

I entered several Dutch words in my search engine and I found an official web page
“Bedrijven buiten de EU die gegevens van EU-burgers verwerken, moeten een vertegenwoordiger in de EU aanwijzen.”

Please allow me to translate this in English by using the language button on the page:
“Non-EU based businesses processing EU citizen’s data have to appoint a representative in the EU.”

These are the first 2 times I found “EU citizen” on the official EU website pointing to GDPR.
“Is this legal stuff for the court?”
“Sorry no.”
“Really?”

There is a legal notice in the footnote containing a disclaimer. So I am quoting from an interpretation of the EU of GDPR. GDPR is leading and not the interpretation.

The day before first publication date I read article 2 again:
“This Regulation applies to the processing of personal data of data subjects who are in the Union by a controller or processor not established in the Union, where the processing activities are related to:

  • (a) the offering of goods or services, irrespective of whether a payment of the data subject is required, to such data subjects in the Union; or
  • (b) the monitoring of their behaviour as far as their behaviour takes place within the Union.”

The location of the home of the user was not enough. Again I was trying to attempt to tweak this blog post.

Wait. In 2 (a) I found an interesting exception clause. What if an American shop offers products in the EU.
So I drew a shop in the EU.

Okay, here are the promised sketch notes. Sorry for the lengthy introduction.

In the first sketch note I point out that the web site uses the location of the laptop to identify an EU citizen. But this is different from GDPR. The nationality of the user and the location of the shop should be used instead.

Sketch note showing that a web site is denying access based on location instead of nationality and location shop because of tracking.

In the second sketch note there are two situations, which were not intended by the web site owner.

An American cannot access a website in the office in the EU. But GDPR is not applicable.

Suppose your American colleague comes to Germany to help you a hand. Then he wants to go to a website with an expensive subscription. It is not possible: 451. The web site owner will probably state something about GDPR. Hopefully a disclaimer was added for this case.

Looking at GDPR there is no violation. So no privacy penalties are involved.

The second sketch note is really worrying, because an EU citizen is tracked during her or his holidays in the US. That is not right.

EU citizen gets access to American website and gets tracked. This is not always possible according to GDPR because there is a shop in the EU.

The 2nd sketch note looks like

Sketch note which contains the pictures abouthe EU citizen and the American citizen  and emphasizing the differences.

Finally there is a test idea about an American living in the EU, who used to live in the USA and is planning to go back to the USA. I am quite curious whether customer tracking systems can handle all the moves.

For the people who are concerned about money.
Yes, GDPR can have a major impact on your profit.
“Failure to comply with the GDPR may result in significant fines of up to EUR 20 million or 4 % of your company’s global turnover for certain breaches.”

Advice

In some countries the privacy laws outside the EU allow more ways to track users of web sites than General Data Protect Regulation. According to GDPR it is not legal to track the users without their explicit consent in most cases. [Article 7]

A solution is to show an error page 451 that the website cannot be shown because of legal reasons. This is in spirit with the request.

A way to determine the nationality of the user is to use the location of the laptop or PC.  This can be done by determining the internet address.

This is true as long the laptop is used inside the EU. If the laptop is used by an EU citizen outside the EU, then the user might be tracked without consent. This is illegal, if the website offers products in Europe.  [Article 2.a]

According to me the best way is to switch off the immediate tracking of users and ask for explicit consent.

Tips for testing
  • Go as close to the source as possible.
    Read GDPR or find interpretation of the law given by the legislator or representative.
  • Check and double check information and sources.
  • Gamify testing by using different tools.
    I used sketch notes, mind maps, and the internet.
  • Get used to hitting the wall.
Note about experience report

This is my experience report about GDPR testing. I ran in some problems, but I was able to resolve them. I could just skip the problems encountered, but you, the reader, could get a false impression. Learning is stumbling and standing up. And walking again.

GDPR – the Forgotten Tests – Test 2

Black box testing is quite popular: the tester only has to focus on the functions of the system. There is no need to know about things like programming and other techy things.

“But the box in the picture is not completely black.”
“That is a good observation, because it is part of a black box.

Time for a legal break. After the break a pen test.

Disclaimer

I am not a legal expert. So please have a look at my used sources. Or contact a GDPR expert.

I am just a tester finding test ideas about GDPR. Thanks for joining in advance.

The following story has been sanitised by me. Important details have been changed.

Pen test

My wife had bought a gift and she had also found a better gift. So she gave the second gift. And I had the pleasure to return the first gift to the shop. No problem dear.

I went into the shop straight to the counter. After a few sentences I came to my point.
“I want to get my money back.”,
while showing the first gift and the receipt.

The 2 young men went into action. There was a lot of pressing of keys and a new receipt was shown.
“Would you please sign this receipt?”

This was a standard computer generated receipt without a signature field. And I had to leave my signature here. I signed.

I remembered to explore.
“Why do I need to sign this?”
“This way my manager can control, that a customer is returning an article. And not we.”

I ran a quick scenario of returning articles in my head. This sounded reasonable.

But I was still hesitant to leave my signature in the hands of two young men.
“How long will my signature be saved?”
This question led to puzzled faces.

I scribbled the question on a piece of paper. It would be great to have a written answer, so I left my email address.

Then I got my money back and returned 1 week later.

The young man behind the counter recognised me. He went to a pole and pulled my paper with email address off. This was bad.

He dutifully repeated the story about the signature of a customer actually returning an article. The signature would be saved for 1 month. That was fine.

On my way home I was not convinced about the privacy. I had witnessed a breach of my personal data.

Breakdown

In this breakdown I will point to several articles of General Data Protection Regulation or GDPR.

The penalties can be quite big. Let me quote the worst cases
’20 000 000 EUR, or in the case of an undertaking, up to 4 % of the total worldwide annual turnover of the preceding financial year, whichever is higher’ [Article 83 5].

Let me review the most important steps during my visits again. I wanted to return an article and get a refund. Because money is involved, the request for a signature is good [Article 5 1(b)].

The receipt was a bit confusing for me, because there was no clear signature field. I just had to trust what the young men told me [Article 6 1 (f)].

One of the most important things about data is retention period or how long will it be saved. The check of my signature could be executed within a month and then be destroyed. [Article 5 1(e)]

A signature alone is not special. But if I had paid in the online shop, then it is simple to combine my signature with my name and other personal data. This way it is possible for someone else to write letters on behalf of me. It is criminal, but possible.

The note with my email address on a pole was a personal data breach [Article 4 (12)]. It was not intended, but I could get a lot of mails with false promises.

Tips for testing
  • Test the UX or User Experience of the receipt.
    Is it clear to customers that they have to sign a receipt for a refund?
    Can they be specific about any doubts?
  • Ask the people behind the counter, how they explain the refund procedure. Also how they handle personal data like phone numbers and email addresses.
    There are of course managers who will answer the questions flawlessly. Unfortunately they cannot be present in more than 50 shops at the same time all the time.Receipts with signatures should be stored in the same way as money. I did not see how my receipt was stored.

    Small sidestep: after May 25 2018 there were boxes outside shops to collect receipts of customers. If I put a receipt with my name and phone number in the box, then I could be the lucky winner of some fantastic prize. They were cardboard boxes standing on tables.

  • This is an important lesson for myself. If something strange happens, wait to remember it and mention it.

To be continued

GDPR – The Forgotten Tests – Test 1

General Data Protection Regulation or GDPR is all about privacy. If a company handles privacy in the right way, then it can dodge penalties like 20 million Euro or 4 % of the worldwide revenue.

Time for a legal break. Right after this break some idea.

Disclaimer

I am not a legal expert. So please have a look at my used sources. Or contact a GDPR expert.

I am just a tester finding test ideas about GDPR. Thanks for joining in advance.

The following story has been sanitised by me. Important details have been changed.

Bad idea

The job interview was about an agile tester. I thought I could handle that role. The probing questions from the interviewers were increasing. I tried to stay calm and answer the questions in a friendly way.

Then came the expected question about test cases. They should be written beforehand. Time to explore.
“You never know what you will find.”, I remarked.
“Let me give me an example.”

“Your company sent me this mailing.”
I showed a part of the mail.
“At the bottom of the mail I could say, whether I like this mail.”
There were two pictures: one green thumb up and one red thumb down. There was an orange arrow pointing to the thumb up.

“If I hover above the picture of the green thumb, the URL will be shown in the status bar of the mail.” The URL was contained in a red eclipse.

A sketch of a mail with an orange arrow pointing to a thumb up next to a thumb down. The mail also contains a URL in a red eclipse.

“As you notice: the URL is http. This is not secure. If the mail is intercepted, then the reaction of the customer can easily be determined. This is an email about credit, so you can derive that the customer probably has some debts.”

One of the interviewers politely interrupted me:
“Is it possible to intercept mail?”
I gave a technical answer using normal words.
Okay, I got his attention.

Then the exploratory tester awoke in me. And I could not stop him.
“There is a customer number in the mail. This number can be used to get access to an online account.”
I went in full brainstorm mode and described all kinds of product risks or things which could harm the user. I could find information about correspondence about money.

 

I didn’t get the job, but the mailing was fixed afterwards. Obviously 20 million Euros are not enough to qualify as a tester.

But there are retrospectives for.
[On the melody of ‘That’s What Friends Are For’.]

Breakdown

Most of the time primary systems were and are tested for GDPR and national privacy laws. Sometimes this software did not easily support mailings. An easy solution was to use another system outside the company. Specialised in mailings.

All kinds of data like email addresses, names, and profiles were used for mailings. Technical decisions were taken like http instead of https. Somehow the legal department and testers missed something.

According to GDPR the protection of personal data is a fundamental right [ (1) on page 1]. The economic situation of a person can be used for profiling. In turn this can be used to exclude people to get certain services like mortgage [ (75) on page 15].

My tips for testing:

  • become a customer of your own company and use all available channels. Watch for the legal details like the missing s of https. (See last tip)
  • follow security experts on social media. (You know about the last tip)
  • explain legal and security stuff in normal words.
  • let the owner control the flow of information. I should have send my brainstorm on request.
  • read  ‘Here’s Why Your Static Website Needs HTTPS’ by Troy Hunt, a security researcher. It contains an entertaining 25 minute video with several attacks on an http website.
    For people new to security, just watch the video and focus on what you would not like to happen on your website.

Closing note:
At the moment there are browsers showing whether a website is insecure. This was not the case, when I received this mailing.

To be continued.

Explaining exploratory testing with a table

Tables loaded with food and a class of kids playing on a lawn.

Another dad and I picked an all favourite Dutch subject: work.

“What makes a good tester?” the other parent informed.
“A good tester knows about exploratory testing.”

I saw wrinkles on his forehead. This was a bad start for this subject. I had to switch to his context. He was a police agent. Okay, second try.

“Suppose you ask for a driving license.”
I opened my imaginary jacket and pulled out an imaginary object.
“I place a gun on the table and”
I noticed a sudden sharpness.
“then I show you my driving license.”
This time I retrieved a thin imaginary object between my thumb and index finger.

“Would you be interested in the gun?”
“Yes, of course.”
He was constantly switching attention between my hands and the invisible gun on the table.
I continued with
“I would ask questions like
“Do you not feel safe?” or
“Is this your gun?””
He nodded.

Then I explained that a tester adjusts her or his activities based on observations during exploratory testing.

The focus would be on the gun instead of the driving license.

LS In Conf’rence Land

Greetings to the reader or Lectori Salutem.

Texting and talking about diversity

This spring I was invited to speak at a known Dutch test conference. I had a good proposal, so I only had to say: “Yes”. But I had to ponder this carefully. I had a public promise not to speak at a conference with an all male line up.

I also had obliged myself to say: “No”, if there were too few female speakers. Women look different at tech and they need female role models.

This year several male speakers declined to speak at a conference with an all male line up..

There was only one way to find out. Just ask the program committee. I texted my dilemma and asked for the number of female speakers. There were only 2 female speakers selected out of 3. Selection took place on quality of the presentations, theme, and target audience.

I got my dilemma back. Is 2 enough? Looking at the last conference it was an increase of 100% in the number of female speakers. But still it bugged me.

The only way for me to improve the diversity was to make suggestions for the keynote speakers. I texted 3 names of female speakers and subjects fitting to the theme. At the end of the same text message I also agreed to give a workshop.

When I saw the final version of the schedule, I could not suppress a smile on my face: one of my proposed keynote speaker candidates was a speaker with my suggested subject. Yes, mind reading is cool. And there was a female co keynote speaker.

During the conference I saw a tweet about testing of blockchain. There were two speakers and the female one could really explain it. That’s why diversity is so important. Just for the record the tweet was sent by an experienced male tester. And it was not me.

Continuing talking about diversity

Same test conference. There was a representative of a European test conference. One thing about the conf this size fits only 1. And I could not resist the urge to talk about diversity. The answer was of course quality. And the programme committee decided about the talks. Also the names of the submitters were not shown to the reviewers of the proposals.

I was not quite convincing. So the woman offered me her email address to send more information. So I sent information about Karoline Sczcur and a link to  A Balanced Conference Card. I received a polite Thank you.

So what went wrong?
Time for a retrospective. Yes it is an agile thing to do.

I had not prepared some talk. So here is the rebound.
As an organiser you can give guidelines to the programme committee. And you can reach out to female speakers in a positive way. Yes it takes time.

More important is to realise what is diversity about. People who think alike come with solutions alike. This means that these people will fall in the same pitfall.

Back to the conference. If there are a lot of white male speakers, then afterwards the attendees will make similar white male speaker errors. A female perspective can add a different and effective approach.

Also. What works for a white male engineer, might not work for a female engineer. A suggestion from her can easily be ignored or stolen. This can be avoided by using number 10 of survival tips for women in tech from Patricia Aas.

What really baffled me, was that lot of these tips also can be used by people of colour. As a man of colour I have to invest a considerable amount of time in finding and talking with allies. To get things tested.

In the Netherlands the campaign #NietGenoeg was started to get more women in tech.

Jez Humble tweeted about diversity in a refreshing way. You can only make good programs with empathy at the core.
“Empathy is _hard_. It means listening openly and deeply to people with very different perspectives, accepting the truth of those perspectives, questioning and changing your deepest assumptions about the world, and changing your behavior.”

Being Sidetracked – Part 5

Just a few sections to end this blog post serie.

Just store it somewhere safe

Now I had a few productive days. I could easily do Test Driven Development with the help of the junior DevOps engineer. But I left out one important step in the development: the use of the version control system.

Looking at the numbers I think that Windows 10 is better than Windows 8.

One of the advantages of version numbers is that I know which platform the user used. And which version I have to use to pinpoint a problem in production. Versions are great for code.

Writing code is like making a story for a movie. If I made an error, then I reverted the change using Undo. Most of the time several people are involved for making the same movie story. There are a lot of people willing to pay 8 $ for a good movie. Coming soon to this cinema

Let’s take an imaginary superhero movie. Pete knows a lot of action scenes. After a while he describes a scene to the other crew members:
“And then she flies in the air.”
“Sorry Pete, but Lightning Buzzword Angel cannot fly.”

“But she is called Angel and angels can fly.”
“You’ve got a point, Pete. But not every woman called Angel can fly.”
“So I have to rewrite the whole scene.””
“Sorry dude.”
“It took me weeks to figure out this scene.”
“It is great, but we have to stick to the character.”

“So you just have to start from version 0.3 of the Supermarket Fight Scene.”
“But I also changed the First Car Ride Scene and the Milkshake Scene because of the bruises made by Oval Owl.”
“Wait, you say bruises.”
“Yep.”
“But then I have to rewrite my scene.” Amy remarks.
“And I the Milkshake Scene.”, another writer joins in.

Making a story for a movie is like writing code. If a programmer or DevOps engineer changes a method or function, then this can have severe consequences for the code. The trick to detect faulty code as early as possible. There’s absolutely nothing wrong with that.

Using Test Driven Development or TDD in a proper way a DevOps engineer knows that the added code is right.
Using a version control system she or he can merge the added code with the code in the repository. The result of all unit tests for this file is an early indicator for the quality.
Then the integration tests would be added, merged, and executed.

Now comes the most interesting part. If a release was made in this particular company, all unit tests and integration tests for the whole code were executed. All the tests from previous TDD or Red Green Refactor cycles were reused again. Of course some tests would fail, but the DevOps engineer would set things right. This could be the code or the test or both. And yes, this could lead to refactoring or restructuring code with an eye for maintenance.

After a few days of TDD a new file could be checked and processed in case of positive checks.

Just me and the code

During the coding the DevOps engineer mentioned the Boy Scout’s Rule. Now we were not exactly hiking. Not especially with a flatscreen attached to a laptop. The rule basically states that I should leave the place cleaner than when I came. If I found some rubbish, then I would have to put it into a bin.

In case of software it is all about refactoring and adding missing tests. And that was the case. Missing tests could be considered as technical debt. Yes, tests are technical.

There were still some unit tests missing for other input files. And then old patterns emerged again. I started to write Gherkin files and the DevOps engineer was making them operational. After a few days he started to work on a high prio task and I was still fabricating unplugged unit tests. Then I had to turn my attention to a high prio work item. I put all the unit tests in the version control system and almost forgot them.

Just plug and test

“This takes a few weeks.”
This was the initial thought from my scrum master after the request to plug in the remaining unit tests.

Okay, rhyme after me:
I was high
on supply.

There was one simple way to change his view. I put in the whole file with unit tests and the tests were not executable. So I cut a few times until I had only one simple test left over. It was still not usable.

I needed a default input file. My scrum master agreed:
“It takes some time, but it is worthwhile.”
I slowly assembled a file over a few thousands bytes character by character using an ASCII editor. This took a big chunk of one business day. Then I could code to run my first unit test. And I got a successful test, Green.

I slowly added the other unit tests. I was quite pleased with myself until I had a failing test. Red. This must have been a programming error. I checked the knowledge management system. I was right. Next result of the same test was again Red. Of course, because I did not change it a bit. Pun intended.

I slowed down to a crawl. According to the knowledge management I was right. This time I checked the value character by character. Hmm, time for a chat with the Product Owner.

He listened to my story and told me, that my test was right. So I updated the Knowledge Management System. At last a succeeding test. Green.

This happened a few more times.

That’s it?

This serie described my experiences with Test Driven Development or TDD. For me it was often difficult to stick to Red Green Refactor.  I made a lot of errors on my way. These things can happen. Even for experienced testers there are a lot of chances of being sidetracked.

Being Sidetracked – Part 4

Some readers might have noticed that Gherkin is used for unit tests. And this might be strange or unsettling. For me it was normal. In this company the DevOps engineers worked that way. It worked, so as a tester I had to put in more effort to find bugs. Sure no problem.

Me programming

My first action was to write a test to check a business rule for a valid observation date. While I was typing, the Integrated Development Environment aka IDE suggested several options like a search engine like Google. But observation date was not known. So this sentence was highlighted in red. Syntax highlighting is also handy in programming. I don’t mind that at all.

With the help of the DevOps engineer I wrote code in Java, so the IDE could use observation date in Gherkin. The code could be executed, but the test failed. Hey that was bad.

“Now I write the code.” was my next thought.
Of course there was no code, so the test should fail. Definitely Red.
This would be fun.

I had to program in Java and I remembered that the junior DevOps engineer had some useful shortcuts. I looked in the IDE and found the option in a sub menu to add a method. I gave it a name and – yes – I got an empty method in the right class. Now I had to fill in the blanks.

Me writing Gherkin

Months earlier I had written several Gherkin unit tests in the knowledge management system. There were two things wrong with this approach. The tests were not used, so I was high on supply. In plain English I had made something which was not used. On a scale of time it was a waste of time.

The other thing was that it had a web interface. The syntax highlighting did not work. I could make things bold and indent texts, but that slowed me down too much.
The result, the code, was not easily readable.

The dates were in the following format DDMMYYYY. This is programmer s’ language for Day in 2 numbers, followed by Month in 2 numbers, and finished with Year in 4 numbers.
So April 1st 2001 would be written down like 01042001. 01 is the first day of month number 04, which is April. 2001 is of course the year. It is easy to pick good dates like

  • 01042001 (first day of the month),
  • 31032013 (last day of the month), and
  • 29022000 (leap day).

I picked some wrong dates on more rational grounds:

  • 07142011 (for the Americans July 14th 2011),
  • 20132007 (20th day of the 13th month), and
  • 1jan1998 (January 1st 1998 in a hydrated date format.).

After writing date tests for one date field I noticed a pattern. I would check on all these dates again for other dates like expiration date. I thought I was smart. I just copied all the observation date tests and replaced ‘observation date’ by ‘expiration date’. I even copied the tests to a special text file, so I could save time. But I was wrong!
Please read on.

Me at the keyboard again

So I was programming unit tests and on my side was a junior DevOps engineer assisting me. I could finally write a test in Gherkin. In a separate window I opened the knowledge management system. I picked the first right date 01042001.
“Why do you pick this date?”, my personal DevOps engineer informed.
“I want to be sure that the right date is accepted.”
A nod followed by:
“We use Joda-time for that.”

I heard: “Yoda time”
It was not possible for me to link a lightsaber wielding big pointed eared green creature with programming.
“Joda-Time checks on valid dates, so you do not have to test them.”
Java 7 was not safe enough for dates and some programmers made Joda-Time. That saved me a lot of time.

So I only had to test on the wrong dates. Easy. I added tests, which succeeded. Green.

There was a `but` coming up.
“You can skip that date.”, while the DevOps engineer referred to ‘1jan1998’.
“Characters are not allowed in the date. ”

The next morning the DevOps engineer showed me a neat table:

Given the file has observation date <date>
When the file is read
Then the file will not be processed


Examples:
| date     |
| 07142011 |
| 20132007 |

He had improved the test code considerably. Refactor. I forgot to use DRY, Don’t Repeat Yourself.

My scrum master said that it was important to know how to program. This way I could structure my tests in a good way.
But the worst was still to come.

To be continued.

Being Sidetracked – Part 3

“At the moment I am writing a serie of blog posts about Test Driven Development.”
I looked to the recruiter. “You can understand it.”
I moved my view to the manager: “You can also understand it.”

Sticky note: let’s stick to the DevOps engineer.

Vegetable As a Service

The DevOps engineer wrote the unit test in Gherkin. The main advantage is that this language is easy to read. Have a try.
Given the version number is 15
When the file is read
Then the file will be processed

It is also easy to write. This example is in English, but it is also possible to use plain Dutch.
Gegeven een versienummer is 15
Als de file wordt gelezen
Dan wordt de file verwerkt

This is easy for people in Dutch companies. Nothing is lost in translation. It is easy to digest. No mindreading skills are needed.
The tool Cucumber provides a way to translate these sentences to a programming language like Java. A programmer has to code, how the sentences are translated.

When this little story is used by the computer, Java is used to execute this test. Yes, it is time for another cup of coffee. Which is the symbol of this language.

There are of course some people, who want to add some details to it. And yes, this is necessary.
Feature:
As an administrator I only want to process the right file list_20180525.txt, so that marketing managers can still process the data and generate reliable reports.

Background: file A

Scenario: 15 right version number
Given file A has version number 15
When file A is read
Then the file will be processed

Scenario: 16 wrong version number
Given file A has version number 16
When file A is read
Then the file will not be processed

All the text is put in 1 file, so all tests are nicely organised. A very important detail is, that file A is a complete valid file. This takes some time and some byte shuffling, but it is worthwhile.
So after 2 cycles of Red Green Refactor two scenarios were added in a feature file. These scenarios could be executed individually or in a group.

Then the cycle continued. The latest test was used frequently to assure, that the code was modified in the right way. The previous tests were used to assure, that the quality was the same. This led to a massive set of tests.

What’s next?

The DevOps engineer looked for the next feature to program. I saw an impressive table of valid values and validation rules.
“Do we have to test all this?”
“No”, he answered.
I could not believe my ears. There were so many places where things could go wrong.
“We will only test things, which can cause problems in our software. [The postman] will check the data.”

I visualised the data flow in my mind. There was a sender, which gave a file to the postman. This program would check every byte of the file. After a successful check the postman would deliver the
file to the receiver or the program under development. In the past I heard, that sometimes some things were not properly checked by the postman. With the speed of computers today extra checks can be done very fast.

The DevOps engineer continued:
“If data can cause problems in the software, these are checked.”
Version number is a good one. And of course begin date and end date of a period are also important.

Another cycle to start

This looked so easy.
It was like the junior DevOps engineer had some mindreading skills.
“Now you try.”
I was silent for a moment.
I repositioned myself after the keyboard and the mouse.
Pair programming. The real stuff.

Okay, let me start.

Read mindfultester[dot]com
My answer in a job interview

To be continued

Being Sidetracked – Part 2

A few weeks ago I told my wife about the picture for this blog post serie. “There is a red light. And a green light. The rest of the picture is fuzzy.” It was about “Red Green Refactor”.

“You should [tidy up with known product name] the picture.”, she suggested.
I liked the fuzziness. This was about Refactor.

Second cycle

We had just finished the first cycle.
The DevOps engineer wrote another test. Ready for the next modification.
O yeah. Test Driven Development.

Let me sanitise me this example. This means so much like “all confidential information is changed”. On purpose. Everybody happy?

A file can only be processed, if the version number is 15, 20, or 31. After the first cycle every version number was accepted. The first test was:
“Is 15 the right version number?”
“Yes”
But this would lead to some bad side effects:
“Is 16 the right version number?”
“Yes”
Or worse:
“Would you please bring me a cup of coffee?”
“Yes”
What about
“Would you please transfer 1 Million Euro to my bank account?”
[Upset face]
[Whispering] “You are supposed to say: “Yes”.”.

Okay, back to the real example.
The first test was to determine whether 15 was a right version number.
The DevOps engineer added a second test to determine whether 16 was a wrong version number. He performed the last test and 16 was a right version number. This was wrong: a failing test. Red.

The DevOps looked at the invalid value and added a single condition to the code. If the version number is equal to 15, then return “Yes”. Otherwise return “No”. The second test was executed again: this time 16 was a wrong version number. This was right: a succeeding test. Green.

The code was very simple, so no refactoring was necessary. Refactor.

Then the DevOps engineer executed all or both tests and they were both correct. The second cycle was over.

While blogging, I realized old patterns of thinking were still present in my head. Let me answer a few questions.
Question 1: This code is so simple. Why do you need to write all the tests?
Answer 1: During every step the DevOps engineer can use the written tests to determine, whether the code is still good. This is especially handy for complex code. Failed unit tests can point to where things go wrong.

Question 2: why did the DevOps engineer not add a second test for valid version number 20 or 31?
Answer 2: it would not lead to failing test. In other words the code would not have to be modified because of this test. At that particular moment he would have written a redundant test. That is a waste of time.

Question 3: Do all these tests take a lot of time to execute?
Answer 3: No. The tests are unit tests and fast enough to execute. The hardware has been improved considerably, so these tests are executed within a fraction of second.

Question 4: Would you please continue?
Answer 4: Sure. No problem.

In the previous blog post I wrote about my disability to speak Chinese. That is quite confusing, because I look like a Chinese and not like a Dutchman.
Small recap with the restaurant in China:
“Do you speak English?”
“Yes.”
[Half hour later]
“I would like to order dinner.”
[Blank face] (Red)
“Did you understand me?”
“No.”
[I opened the menu.]
[Waitress approaches me.] (Green)
[I just ordered the dishes using my fingers and the menu.] (Refactor)

What I actually did, was trying to find other ways to order dishes. I could have written down my order, but the waitress could follow my hands. So no refactoring was needed.
No is unpleasant answer, but a lot of frustration was avoided. At least I understood, why hand sign language for numbers was included in the pocket book for frequently used phrases in Chinese.

To be continued.

Being Sidetracked – Part 1

Every story has an expiry date.

So I have to hurry up.

While the junior DevOps engineer was programming aloud, I paid attention to all the steps he took. He used Test Driven Development. It is a cycle of Red, Green, and Refactor.

A small recap: he first made a tiny test, which failed. Red is a favourite colour for failing. Then he made code to let this failing test succeed. Green is that other favourite colour for DevOps, testers, and especially managers.

Then he refactored the code. The code became more maintainable and readable. Even for a tester not fluent in Java.

The first test was to check, whether a business rule failed. He wrote only code to let the test succeed.
Before I could think, the method was ready. It had only one return statement with 1 fixed value.

But this would only be the case for very specific situations. I showed my disbelief and he answered that the code had to pass the new test. Right, you are right.

This was a strange situation for me as a tester with a traditional background. Tests should be executed after the implementation and not before. Somehow my brain had pushed the theory about TDD aside. It felt so unnatural to me that I unconsciously switched back to Program First Test Next.

Anyways, the DevOps had a quick look to the code. I did not think that this could be refactored. One line single statement cannot be refactored.
Yes right again. The first cycle was finished. Red Green Refactor.

A return value from a method is like an answer from a human being. What the DevOps basically asked, was: “Is this value right?”
And the method would always answer with “Yes.”

This was strange to me. Now I realised, that this was the most minimal addition to the program.
Without a method the code would have be repeated multiple times and maintained at the same number of places. A recipe for disaster.

Aw I forgot to look in the right low box in the left corner in the room under the stairs.

Now programmers have a small heuristic for this one:
DRY. Don’t Repeat Yourself.

The fact that the answer was always “Yes”, bothered me. While blogging I remembered asking a restaurant in China, whether they could speak English. The answer was “Yes.” My wife and I were delighted until I ordered. O no.

To be continued