Notes on a shelf

An Oracle, Two Systems, And Four Situations

On September 26, 2015 Justin Rohrman ‏tweeted:
“Anyone know of the origin of the word ‘Oracle’ in software testing? Who started using it, what other words were used prior, …”

I answered with:

  • in the past in the Greek city of Delphi there was a special person you could ask question: Oracle.
  • The answer was sometimes cryptic. When the Persians invaded Greece, the answer was to hide behind
  • wooden doors. There was a smart man, who interpreted the answer right. He advised to build
  • a fleet of wooden ships, which beat the Persian fleet.

 

An oracle is a source of information, which I can use during testing. It can be used to determine, whether the observed situation is right. Right?

A dialogue from a movie with John Cleese about choosing a direction (while driving a car):
“Right?”
“Right?!”
“Left?”
“Left!”

Are you sure?
“Hey, I think I found a bug.”
“Okay. Tell me about it.” The other tester requested me.
I told as objective as possible my observations and my conclusion.
“Did you look in the previous version?”

Some systems are so complex. A fast way to sift possible bugs was to look at the system used in production. Of course afterwards it was always possible to read the specs or ask the PO, Product Owner.

One tester was able to pinpoint the version, where a bug was introduced, within minutes. He had the latest versions of the system available.

So I learned,

  • how to install more  than 5 different releases on my PC.
  • how to rollback a release including the database.
  • how to prepare the proper database and configuration for testing.

Where did you see it?
During the test I noticed, that the test version showed status information, which was not shown in the release in production. I asked other testers, who noticed the same. So my test environment was OK. There was still a doubt, whether the customers would miss the information.

The most simple way was to ask help from the help desk.

In the afternoon a blond woman was standing at my door post:
“You are right. The situation occurs in production.”
It was time to make a bug report.

Just don’t let it happen again
“I have not enough information to test the issue.” I complained.
“Did you test in the previous version?”, another tester remarked.
“No.”
“If you reproduce the steps from the issue, then you notice, what will go wrong. The issue has been solved, if the observed situation does not occur.”

During one project I had to test a system with service desk agents and PR colleagues. There were issues to be tested. I did not have time to make test cases for them.  If I let the testers test on their own, there was a chance  that they would check, whether only the bug was removed. I needed information about similar situations and connected cases.

I opened an issue in the defect registration system and read the description. “I would test this …” Then I wrote down the test ideas in the comment. Before the test I gave some instructions to the testers:
“In the comment situations are described, which must be tested. If you are ready, write in a comment, what you did. Then you assign the issue to me.”

After the start of the test the issues came back to me. I looked at the feedback. Sometimes I realised, that more situations had to be tested. Then I added them to the comment of the issue and assigned it back. The nicest thing, that happened, was that extra test ideas were added by the other testers.

Did you have a good look at it?
During the regression test I had to test the export and import function. I had two systems installed with different databases. The only things, what I did, were export data to a file and import the file in the other system.

I had two screens, so it was easy to compare the data: I placed the systems on different screens. If there was a difference between the source and target system, then I made a print screen. The picture contained images of both screens. Hey look mom! With one finger!

Next stop was my word processor, where I pasted the image. Then I used the keyword BUG followed with a small description. Sometimes I analysed the data file to pinpoint the source of the strange situation. If the file was wrong, then the source system might have some glitches. Otherwise it was the target system.

During the stand up I mentioned, that I had found a number of particular situations. This resulted in a session, during which I spoke with a programmer. The following pattern was repeated:

  • Search for “BUG”.
  • Resize the image to show the differences between the systems.
  • If necessary, look in the file.
  • Determine, whether it was a bug.

During the bug removal phase I got two kinds of feedback from the programmer: it was fixed or it cannot be fixed. The latter surprised me, but a study of the specifications changed my view. The format of the file did not support all types of information. There were semantic limitations. Something like: country is not supported, so this cannot be exchanged.