Category Archives: Test automation

Visual TDD in Test Automation – part 2

A month ago I was in the sport school. The manager had a conversation with me:

“You are using the wrong liquid to clean the machine.”
My eyes went to the automatic liquid dispenser.

She followed my view:
“You must use the hand trigger sprayer. You used the spray for the hand instead of the machine.”

Somehow I missed the hand trigger sprayer. It was clearly in view. But I came from the other side, the sprayer was hidden behind a machine.

So what was the problem?

People tend to use the most simple solution for a problem. You know, I am one of them.

For programming I use Test Driven Development. After some impressive demonstrations and real life experiences.
For me the Red Green Refactor cycle of Test Driven Development is simple.

When I used Visual Testing in the preivous blog post, I used a breakpoint.

If I set a breakpoint and debug the program, then it will stop at this breakpoint. Now I have enough time to get useful information from the program, get some coffee, or take a small nap. The latter is an advantage of blogging in free time or being jobless.

Disclaimer: I am not paid by Applitools for writing this article.

Let me visualise the steps from the last blog post:

Start: I have automation code, that opens the Login dialog.
A big rectangle has an arrow with "Automated steps" to a big rectangle containing an empty rectangle!

Red: make a failing test.
I added a CheckWindow method and placed a breakpoint on this line. During the debugging I manually entered the password “Cap” and resumed the program.

What I actually did, was making a screenshot of the desired situation or baseline.

Then I ran the program again. The test failed, because my code did not enter “Cap” in the username.
Why? Cap can be an abbreviation of Captain Marvel or Captain America.

Green: make enough code to pass the failing test.
Then I added code to enter “Cap” in the username textfield. Then I ran the program and it passed.
A rectangle has an arrow with "Automated steps" to a big rectangle containing an empty rectangle. In turn this big rectangle has an arrow with "new automated steps" to a big rectangle with a rectangle containing "Cap". The last big rectangle has an arrow with "snapshot" to a big rectangle with a rectangle containing "Cap"!

Refactor: there was no need to clean up the code, because it was a proof of concept.

Breakpoint overboard

For some people the breakpoint is really strange. So let me try to find a solution with using Test Driven Development while avoiding this point.
A breakpoint to be more precisely.

You are about to read about exploring a new situation in test automation.
Go.

Undesired baseline

And I off went.

For a fast demo I based my code on the Java code on
https://applitools.com/docs/topics/overview/walkthough-example.html#Building

I used the source code TestLogin2.java and pom.xml from this repo.  Notice the name of the class and the test.

The source code of TestLogin2.java is shown with the TestLogin2 class and test name "Login test 12"!

The really interesting coding took place is in the test01 method.

// Go to Login dialog.
driver.get(weburl);

// TODO: fill in the username

// TODO: add test.
// TODO: for making a baseline and
// TODO: comparing the screen with the baseline.
// [edited for blog post]

Now I added the CheckWindow method.

// Go to Login dialog.
driver.get(weburl);

// TODO: fill in the username

// add test.
// for making a baseline and 
// comparing the screen with the baseline.
// [edited for blog post]
eyes.checkWindow("Username filled in");

I left out the breakpoint: no red filled circle is shown.
On line 66 no breakpoint is set, so there is no red filled circle!

Then I ran the program.
On Testlogin2.java the right menu is opened and the Run TestLogin2.main() option is selected!

A message was shown, that a baseline was created.The console shows the following message: "New Baseline Created: 1 steps"!

Time to look in my free Applitools account.
The batch run "Login test 2" has been selected and the result shows a printscreen with a plus sign, thumb up sign, and a green bar!

Now I had done something really strange:
I made an undesired baseline.

Note the danger, that the passed test is accepted as passed, but that was not my intention.

That was not my goal for that day:
adding code to enter “Cap” in username.

There was a simple fix to solve this problem.

Desired baseline

Let me fill in “Cap”.

// Go to Login dialog.
driver.get(weburl);

// fill in the username
driver.findElement(By.id("username")).sendKeys("Cap");

// Test.
// add test. // for making a baseline and 
// comparing the screen with the baseline.
// [edited for blog post]
eyes.checkWindow("Username filled in");

I ran the program again.
On Testlogin2.java the right menu is opened and the Run TestLogin2.main() option is selected!

The test failed.
The console shows the following message: "Test failed : 1 steps matches=0 missing=0 mismatches=1"!

In applitools.com things were not updated yet, so I pressed the Refresh button.

The Refresh button is located under the last 30 batch runs and above Login test 2!

I opened the test result and noticed a difference. But the new screen was right.Button with two images of a form and a double pointed arrow between them is selected. In the left image or baseline; "Cap" in the username field is hightlighted in pink, in the right image or checkpoint several spaces in the username field is hightlighted in pink!

So I pressed the Thumb up button.

A message is shown± `You have successfully resolved all steps. Would you like to continu to the next step or close this editor=` with the Continue button and a Close button!
And the test ruslt became green.
The result shows a printscreen with an unequal sign, thumb up sign, and a green bar. In the upper right corner a Save button or Diskette button is shown!

There was Save button which changing size about every 4 seconds. So I pressed it/

I double checked the test result by removing the hightlighing of the differences.In the menubar a button with a stack of sheets is selected showing a menu with Diffs option which is unchecked. In the left image or baseline; "Cap" in the username field is not hightlighted, in the right image or checkpoint "Cap" in the username field is not hightlighted!

So I ran the program again.On Testlogin2.java the right menu is opened and the Run TestLogin2.main() option is selected!

The test passed this time.The console shows the following message: "All steps passed: 1 steps"!

I went to the Applitools website and pressed the Refresh button.

The Refresh button is located under the last 30 batch runs and above the batch run of Login test 2!

And I selected the last batch run.

The last batch run "Login test 2" has been not selected. The previous batch run "Login test 2" has been selected!

The green test result was shown.

The last batch run "Login test 2" has been selected and the result shows a printscreen with an equal sign, thumb up sign, and a green bar!

There were no differences.

Button with two images of a form and a double pointed arrow between them is selected. In the left image or baseline; "Cap" in the username field is not hightlighted, in the right image or checkpoint "Cap" in the username field is not hightlighted!

Questions

Now I had a test to check whether the username was equal to Cap.
But did I see whether the test fail?
No. Can I trust a test, which I did not see failing?
Probably not.

Another question:
is it good to trust a test, which status went from Green to Red?
I don’t think so.
If someone takes over the coding  of my test, then this would be really confusing: “A failed test is a good sign?!”

According to me this is the problem:
I have a useful pattern or way of working. It is Red Green Refactor.
Then I changed it into an anti pattern or bad way of working. This is Green Red Refactor.
This happened, because I wanted to avoid a breakpoint.

There is another way to explain this.
I have the habit to make pictures of the state of the game or baseline. Please have a look at a game in progress.

Same hotel board game in progress from another angle!

A hotel board game in progress!

Notice that I took pictures from different angles to be sure that the positions of all game pieces were clear.
Also to be sure that there was no hand trigger sprayer hidden somewhere somehow.

Suppose one of my kids would protest: “We were playing another game.” This is actually a bad baseline.

Then I could changed to this picture:A game of Settlers of Catan in progress!

Right game, but right state?
Looking at the boats I don’t think so.A board game of Settlers of Catan is shown with land and and sea tiles. At the edges of the sea files there are ships transporting people and spices!

Visual steps without a breakpoint

Let me visualise the steps taken in this very blog post:

Start: I have automation code, that opens the Login dialog.
A big rectangle has an arrow with "Automated steps" to a big rectangle containing an empty rectangle!

Green: make enough code to pass the failing test.
I added a CheckWindow method and ran the program.

What I actually did, was making a screenshot of the undesired situation or baseline.

When I ran the program again, the test passed.
A rectangle has an arrow with "Automated steps" to a big rectangle containing an empty rectangle. In turn this big rectangle has an arrow with "snapshot" to a big rectangle containing an empty rectangle!

Red: make a failing test.
Then I added code to enter “Cap” in the username textfield. Then I ran the program and the test failed.
A rectangle has an arrow with "Automated steps" to a big rectangle containing an empty rectangle. In turn this big rectangle has an arrow with "new automated steps" to a big rectangle with a rectangle containing "Cap". The last big rectangle has an arrow with "snapshot" to a big rectangle with a rectangle containing "Cap"!

Refactor: there was no need to clean up the code, because it was a proof of concept.

The used anti pattern is Green Red Refactor.
Highly unrecommended.

Wrap up

In this blog post I described a way to use Visual Testing without a breakpoint. I also did not really use Test Driven Development.

There were two drawbacks to this approach:

  1. The test CheckWindow was not tested. So an extra step was needed.
  2. I assumed that the change in the test result would be a sign of progress.

Visual TDD in Test Automation – part 1

A few weeks ago I was in my sport school.
Minding my own business. Keeping my distance.

I was called by the manager.
Minding her own business. Closing her distance to the allowed 1.5 meters.

“You are using the wrong liquid to clean the machine.”
My eyes went to the automatic liquid dispenser.

She followed my view:
“You must use the hand trigger sprayer. You used the spray for the hand instead of the machine.”

So what is the problem?

In the previous blog posts about Test Driven Development in test automation I showed how to slowly grow the code. I used the case of a log in form using Red Green Refactor:

  • Red: Make a failing test for entering a user name.
  • Green: Make enough code to let the test succeed.
  • Refactor: Clean up the code.

There is also a password field, so I have to repeat these steps. There are probably more text fields, which lead to similar steps.

Big side step.
At the moment I use the Page Object Model or POM to represent my application. It is a popular model in test automation. This basically means, that every webpage has unique web elements like text fields.

An advantage of this model is that a webpage has his own class or all code in one place. A disadvantage is that code is copied and pasted and therefore harder to maintain.

“You used the same code for all 200 textfields in this web application. Now you have to change at all those 200 places.“
This can go wrong at 200 places and counting.

I wouldn’t be surprised, if it is possible to make some nice class for a text field. But I try to avoid this because of the complications.

Big side step back.

But entering the right username is not very exciting stuff to check. What about no username or a user name with spaces? So I have to check whether error messages are shown. A message can be shown in different ways under the text field, after the text field, or in a dialog.

Filling in a user name in text field could lead to 2 checks:

  • does the right text field contain the user name?
  • is the right error message shown?

The code to check for the right error message is of course different.

Unfortunately there are many elements on a website like checkboxes and buttons. For every check I need new code to check whether Selenum WebDriver executed the right steps and whether the right error message is shown. That is a lot of code.

It is as cumbersome like using 2 different liquids in sport school to prevent spreading of the Corona virus. For me this was confusing.

So my test automation problem is that I want to check every step in my code, then I need to write new code every time. For the people who are worried about the increasing number of checks I have a spoiler: it is not always needed. See the FAQ in the previous blog post.

For the people who really need to code check for every test automation action; there is a viable alternative.

A solution for the problem

A baseline is an old state which is used to be compared with the new state. This is handy for testing purposes.

I like to play board games with my kids. There is a problem: the kitchen table is also used for dinner. In case of game time shortage I make pictures. That is our baseline. When we want to continue the game we use the pictures to restore the state of game. In case of any arguments the pictures are right.

A board game of Settlers of Catan is shown with land and and sea tiles. At the edges of the sea files there are ships transporting people and spices!

At the moment there is software to make pictures of my screen during an automated test. Applitools is a tool which uses AI or Artificial Intelligence to compare these pictures.

The advantage of artificial intelligence is that pixel perfect comparisons are not always needed. If the new state differs a few pixels from the baseline, then the test passes. In case of a failed test a human tester must verify the test result. This can save time and energy.

It can be compared with restoring the state of a board game.A ship must lie on the edge of sea tile. If the ship is close enough to 1 edge of a blue tile, then the exact location of the game piece on the tile does not matter.

Disclaimer: I am not paid by Applitools for writing this article.

The first time the code is used this will happen:

  • the code fills the username field.
  • the code makes a picture of the screen and suggest this as the baseline.

The second time the code will be executed this will happen:

  • fill the username field.
  • make a picture of the screen.
  • compare this picture with the baseline.
  • determine whether any differences are acceptable. Only the last step must be executed by a human tester.

In this example I will show the most basic test using Visual Test Driven Development in Test Automation.

Minimal Viable Project

I will describe the steps using Java. You are free to switch to your favourite language and programing tool. Applitools supports other programming languages. On Test Automation University there are several courses for Applitools including installation instructions.

At the moment I will only focus on the combination of visual testing and Test Driven Development. This means that the code can be improved in many ways. Let’s call it a proof of concept.

Java code

For a fast demo I used the Java code on
https://applitools.com/docs/topics/overview/walkthough-example.html#Building

For this blog post I used the following the files pom.xml and TestLogin.java, which can be found in my github. I made a Maven project, which used the XML file. You can use the code to follow along.

I made some adjustments and minimised the code to demonstrate Visual TDD:

  • I changed the appName and the testName.
  • Because of some execution problem I changed the resolution in viewportSizeLandscape.
  • I changed the application under test to http://the-internet.herokuapp.com/login.
  • I put the method test01Changed in comment
  • I removed some code in the method setup.

In the main method 3 things are done:

  1. Set up the test.
    Applitools is initialised and the URL of the website to be tested is set.
  2. Execute the actual test.
    This code is in the method test01.
  3. Tear down of the test.
    At the end Applitools is stopped in orderly way.

Red: make a failing test

Let’s start.

// Go to Login dialog.
driver.get(weburl);

// TODO: fill in the username

// TODO: add test. // TODO: In other words add code for making a baseline and
// comparing the screen with the baseline
// [edited for blog post]

I used the checkWindow method.
For me this was confusing. This method has two possible outcomes:

  1. If there is no baseline, make a screenprint and suggest it as a baseline.
    I could not derive this from the method name.
  2. Compare the screenprint with the baseline and report the test result.
    This was the expected outcome of the method.

A better name wouuld be something like compareScreenWithBaselineIfPresentOtherwiseSuggestBaseline.
There is an advantage: I only need 1 method.

So I added checkWindow

// Go to Login dialog.
driver.get(weburl);

// TODO: fill in the username

// add test.
// In other words add code for making a baseline and
// comparing the screen with the baseline
// [edited for blog post] eyes.checkWindow("Username filled in");

and a breakpoint.

On line 65 a breakpoint is set which is shown as a red filled circle!

Then I started the debugger .

The Run Menu is openened and the Debug TestLogin is selected!

The Debugger pane is shown with information about the program at the breakpoint!

At the breakpoint I entered Cap as user name. Why? Cap can be an abbreviation of Captain Marvel or Captain America.

Then I resumed the program.

The button with the thin green filled rectangle and the green filled triangle poining to the right is selected showing a menu with the Resume Program option!

Now I was curious about the result. I logged in in the Applitools website with my free account,:

In the Applitools website the Login Batch run is shown with the minimised screen with an orange bar on the left!

My last batch run showed an orange test result. I clicked on the test result.

A message is shown, that the baseline is not present and a picture of the screenshot. In the right upper corner there is a thumbs Up button!

But there was no baseline. So I selectedt he Thumb up icon in the right upper corner.

A message is shown± `You have successfully resolved all steps. Would you like to continu to the next step or close this editor=` with the Continue button and a Close button!

There was only 1 step I had to resolve.

Then I clicked on the Save button or the Diskette button..

What happened?
I made a script which does 2 things:

  • Go to login page and
  • make a picture of the login with the filled in user name field.

Then I went to a website to set the expected baseline.
Now I had a test with a baseline. But would it fail?

I removed the breakpoint and ran the the program again.
The test failed.

The console shows the following message: "Test failed : 1 steps matches=0 missing=0 mismatches=1"!

Back to the Applitools portal.
I pressed the Refresh button or the button with the two pointed arrows above the list of batch runs.

After the selection of the Refresh button or the button with the two arrows followed by the selection of the last Test Login batch a screen print with a orange bar to the left is shown!

I clicked on the orange test result. The differences were highlighed in pink, because the the button with the Stack of sheets had Diffs checked.

In the menubar a button with a stack of sheets is selected showing a menu with Diffs option which is checked. In the print screen "Cap"; in the username is highlighted in pink!

I unchecked the Diffs, so I could see screen print without any pink highlighting.

It was possible to show both screen printes at the same time.

Button with two images of a form and a double pointed arrow between them is selected and showing a menu is shown with the Show both option which is checked. In the left image or baseline "Cap" in the username field is hightlighted in pink, in the right image or checkpoint several spaces in the username field is hightlighted in pink!

It was also possible to compare the picture without the pink highlighting.

In the left image or baseline "Cap" in the username is shown, in the right image or checkpoint the username is empty. In the right upper corner there is a thumbs down button!

This was expected: a failing test,I clicked on the refresh button.

The refresh button is located under the last 30 batch runs and above login test!

Then I selected the Thumb down button,

The latest Test Login batch is selected and a screen print with a red bar to the left and a Thumbs down are shown!

So I had my failing test.

Green: make enough code to make the test succeed

// Go to Login dialog.
driver.get(weburl);

// fill in the username
driver.findElement(By.id("username")).sendKeys("Cap");

// Test.
// In other words add code for making a baseline and
// comparing the screen with the baseline
// [edited for blog post] eyes.checkWindow("Username filled in");

Remember: checkWindow can also compare the screen with a baseline, if present.

Let’s run the code.
And it passed.

The latest Test Login batch is selected and a screen print with a green bar to the left is shown!

I went back to the Applitools website and pressed the Refresh button.

The refresh button is located under the last 30 batch runs and above login test!

The website gave the same green test result.

The latest Test Login batch is selected and a screen print with a green bar to the left is shown!

Refactor: clean up the code

For a proof of concept the code is good enough.

Recap

  • Red: make a failing test by adding the username test and using a breakpoint.
  • Green: add code to fill the username and execute the new script.
  • Refactor: clean up the code.

Problem solved?

Did I reduce the number of lines of code to check whether my test automation scripts work?
Probably.

Some people might think:
“One line to rule them all.”
checkWindow for the win.

But you might have notice that extra code is needed for setup, tear down, and report. So that is a trade off.

For this demonstration I did not have to pay for Applitools.
This tool must be paid, if you use a lot of image comparisons.

There is an extra price tag for storing the images on your preferred server instead of the server of Applitools. Something which should be discussed with the legal department.

This case looks complicated with the breakpoint.
In the next blog post I will look to a different solution without using the breakpoint.

Fast Forwarding Act

Kent Beck used an old programming technique. This way he rediscovered Test Driven Development.

Decades later I was combining Test Driven Development with test automation. According to me it was feasible.
Like the British would say:
“The proof is in the program.”

Recap

In order to make this test automation code work I used Arrange Act Assert.

  • First I had to Arrange, that given my Selenium web driver is in the Form Authentication of https://the-internet.herokuapp.com/.
  • I had to Act: when my Selenium web driver will enter the user name and password, and pressing the Enter button.
  • I had to Assert: then I got a message that I had entered the right credentials.

I had finished Arrange and Assert. The next step was to get Act in place.

Inserting Act between Arrange and Assert

Now I had made code for a situation with no credentials. I had not entered any user name and password, so I did not get access. This seems fair enough.

I programmed my steps using Red Green Refactor, which are used in Test Driven Development.
Under the rectangle Arrange there are two groups of a arrow pointing to the right and three rectangles Red, Gree, and Refactor. The left group is higher than the right group!

Ready Set Code.

Do You Want To Make Some Test Code?
[On the melody of “Do You Want to Build a Snowman?”]

My first step for Act was to enter the user name.

Red:
the failing test was that no user name was entered. I could see, that the user name field was empty.

Green:
I added code in the test code.

loginPage.setUsername("tomsmith");

and additional code for setUsername.

public class LoginPage {

  private WebDriver driver;
  // NB: edited for readbility
  private By usernameField = By.id("username");  

  public void setUsername(String username){
    driver.findElement(usernameField).sendKeys(username);
  }
}

I executed the code. The user name field contained “tomsmith”, so I passed my test.

Refactor:
clean up code was not needed.

My second step for Act was to enter the password. Another round of Red Green Refactor was needed.

Red:
the failing test was that no password was entered. I could see, that the password field was empty.

Green:
I added code in the test code.

loginPage.setPassword("SuperSecretPassword!");

Then I put in additional code for setPassword in the LoginPage class:

public class LoginPage {

  private WebDriver driver;
  private By usernameField = By.id("username");
  private By passwordField = By.id("password");
  private By loginButton = By.cssSelector("#login button");

  public LoginPage(WebDriver driver){
    this.driver = driver;
  }

  public void setUsername(String username){
    driver.findElement(usernameField).sendKeys(username);
  }

  public void setPassword(String password){
    driver.findElement(passwordField).sendKeys(password);
  }

  public SecureAreaPage clickLoginButton(){
    driver.findElement(loginButton).click();
    return new SecureAreaPage(driver);
  }
}

When I used the test code, the password field was filled. I made enough code to pass my failing test.

This piece of code had another effect: a green mark was shown in my programming tool. My Assert was also right: my Selenium web driver was successfully logged in.

I also most had my Act together. One more step.

Refactor:
clean up code was not really needed. Maybe I could have deleted the empty line.

The complete code looked like this:

@Test
public void testSuccessfulLogin(){
  LoginPage loginPage = homePage.clickFormAuthentication();
  loginPage.setUsername("tomsmith");
  loginPage.setPassword("SuperSeretPassword!");
  SecureAreaPage secureAreaPage =
    loginPage.clickLoginButton();
  assertTrue(secureAreaPage.getAlertText()
             .contains("You logged into a secure area!"),
             "Alert text is incorrect");
}

Let Me Code.
[On the melody of “Let It Go”]

Automatic please

There are people who would suggest to automatically check every step in Arrange. Of course this is possible. Here is another piece of code:

public class UploadTests extends BaseTests {
  @Test
  public void testUploadFile(){
    // Arrange
    FileUploadPage fileUploadPage =
      homePage.clickFileUploadPage();

    // Act
    // Upload file
    fileUploadPage.selectFileToBeUploaded(
      "D:\\Users\\Han Toan\\Downloads\\test.txt");
    FileUploadPage.FileUploadedPage fileUploadedPage =
       fileUploadPage.triggerUpload();

    // check header.
    // var result = "ran349454lk";
    var result = fileUploadedPage.getHeader();
    Assert.assertEquals(result, "File Uploaded!", 
      "Wrong page is shown");

    // Assert
    // Check file name
    // result = "ran3495840";
    result = fileUploadedPage.getUploadedFiles();
    Assert.assertEquals(result, "test.txt", 
      "Wrong file uploaded");
  }
}

So the Test in Test Driven Development can be performed on view or automatically.

Overview of TDD in Test Automation

At a low level I have small building steps: Red Green Refactor.
These steps orginated from TDD or Test Driven Development.

Red:
make a failing test.

  • What is the failing test in this particular case? Which text or information is shown to the tester?
  • Does the test provide useful information? Is the message specific enough? “Fail” is not an option, “Wrong screen” is a better one.
  • How do I determine whether the test fail? With my own eyes or automatically?

Green:
make enough code to pass the failing test.

  • Does the test pass?
  • Did I write minimal code to pass this failing test? Did I not add too much in the code
  • Does the code test the application and not my test code?

Refactor:
cleanup the code.

  • Did I take time to look at the code?
  • Do I know the right design pattern to optimise the code? E.g. POM, Page Object Model.
  • Does my programming tool offer options to refactor code? E.g. options in Refactor menu.
  • Do I know a resource, where I can find design patterns? E.g. Test Automation University, “Design Patterns” by Martin Fowler
  • Can I delete redundant automatic tests?
  • Did I execute the refactored code?

At a high level there are three construction phases for automatic tests:

  • Arrange: make sure, that everything is in order before a function is tested in a specific situation.
  • Act: perform the function to be tested.
  • Assert: determine, whether the tested function can be used in the right way in this specific situation.

For a construction phase I could use one or more cycles of building steps: Red, Green, and Refactor.

Most of the time I use them in a chronological order. I start at the beginning of Arrange and work slowly to the end of Arrange. It is the same with Act.

here are two groups of a arrow pointing to the right and three rectangles Red, Gree, and Refactor. The left group is higher than the right group!

For Assert I change the order, because Assert works as expected in a specific situation. In this case I start at the end and work my way back to the beginning.

There are two groups of a arrow pointing to the right and three rectangles Red, Gree, and Refactor. The left group is lower than the right group!

My whole approach for TDD in Test Automation can be summarised as follows:

Arrange followed by an arrow pointing to the right Assert followed by an arrow pointing to the left Act followed by an arrow pointing to the right!

The main reason to make an Assert before the Act is to assure that my Assert can provide me information about the wrong situation. More information can be found in my previous blog post.

FAQ or Frequently Asked Questions

Here are some questions, which I asked myself after several sessions of deliberate practice. Some of my answers are based on practice. Other answers are theoretical, which have to be tested in practice.

I ‘m a tester after all.
[On the melody of “It’s a small world after all.”]

Q: If I have a form with 9 fields, do I have to check whether Selenium Webdriver has entered the right values?
A: Some reasons for checking the contents of the fields are a change of the user interface and screen flow by filling the key fields properly. Examples are extra entry fields popping up or warnings about format shown to the user.

Q: Are there reasons to skip the checking of all 9 fields?
A: If the user interface will not change much. Another reason is that you do not want to test the Selenium code. A third reason is time saving.

Q: Is there an alternative for an automated check of field entry?
A: Yes, I can view changes with my own eyes. Or demonstrate my code to someone else.

Q: You use Arrange, Act, and Assert. You could also use Given, When, and Then. Why do you prefer the first ones?
A: I expected the question. For the people unfamiliar with this construction I edited the first section

  • Given I am in the Form Authentication of https://the-internet.herokuapp.com
  • When I enter a valid user name and password combination
  • Then I get a message that I had entered the right credentials
    The description of the steps are at a higher level than the steps in the first section, but it basically leads to the same result. The reason I use Arrange, Act, and Assert is to put myself in action. I have code to write and it will not write it self.

Q: You mentioned time saving. Why would it save time to skip tests?
A: If I give directions for a walk in Amsterdam, it is along the lines:
“First street right, 3rd street left, etc.”

I do not tell:
“You see a house with number 12. Continue walking to the red brick house with number 34, then turn to the right. At your right you see a small house with number 45b. Continue walking to the brick house with number 85a, then turn to the left.”
This costs a lot of time to tell and to remember. You can give it a try.

End of example

Let me write a little more about the directions to walk from the Rijksmuseum to the Palace in Amsterdam.
The Palace is situated on a dam. The Dam – yes seriously – is a place with buildings around it.

My next instructions would be like:
Go to the Monument on The Dam. This looks like a big needle with a sculpture at the bottom.
Look over The Dam with your back to the Monument on The Dam.
Then you can see the royal palace.

Of course readers might object:
“How are you sure you are looking at the right building?”
In my previous blog post I wrote, how the tourist could Assert this:
it look like a grey huge townhall with a dome.

For the people with normal vision there is no white castle with rank towers.
For the people with X-ray vision there is no Sleeping Beauty in the palace.

Last quote

Why should I not practice TDD in test automation?
It is a weird idea. Like TDD.

“No one is stupid enough to be a competitor. For about a year.”
Kent Beck about Explore at YOW! Conference 2018.

Fast Forwarding Assert

Recap

In order to make this test automation code I used Arrange Act Assert.

  • First I had to Arrange, that given my Selenium web driver is in the Form Authentication of https://the-internet.herokuapp.com/.
  • I had to Act: when my Selenium web driver will enter the user name and password, and pressing the Enter button.
  • I had to Assert: then I got a message that I had entered the right credentials.

I had finished Arrange. The next step was to add Assert. I know that this seems a bit of order, but you are allowed to read the section “Trust”. It would be great that you would return for the technical stuff in the next section.

Connecting Arrange and Assert

My first attempt to make Assert was a failure. It was code which tested my code instead of the code under test.

Now I needed to look again to the Red Green Refactor cycle of Test Driven Development or TDD in Assert.
In the upper part there is a rectangle "Assert". Under this rectangle there are two groups rectangles "Red", "Green", "Refactor". The left group is lower than the right group!

What was Assert supposed to do?
It was supposed to show some message. I had to translate Assert to Red Green Refactor.

  • Red: no message is shown.
  • Green: a message is shown.
  • Refactor: cleanup code.

If it was too difficult for one cycle, I could always use additional Red Green Refactor cycles.

The first step is Red or make a failing test. How did I determine whether the message was not shown?
I looked with my own eyes. Guess what? I had already a failing test.

The second step is Green or make enough code to pass the failing test,
It was time for my rebound. I had enough code in Arrange to make useful code for Assert.

 
   assertTrue(secureAreaPage.getAlertText()
              .contains("You logged into a secure area!"),
              "Alert text is incorrect");

I could plug in my code like this:

@Test
public void testSuccessfulLogin(){
  LoginPage loginPage = homePage.clickFormAuthentication();
  assertTrue(secureAreaPage.getAlertText()
             .contains("You logged into a secure area!"),
             "Alert text is incorrect");
}

But this would lead to code, which could not be compiled. For starters secureAreaPage was not initialised and did not have a method getAlertText. Code which is not executable can not be tested. The line with assertTrue was unusable.

The test could not be performed, so it is not really a proper Assert.

Now I had to introduce a secureAreaPage.

public class LoginPage {

  private WebDriver driver;
  private By loginButton = By.cssSelector("#login button");

  public LoginPage(WebDriver driver){
    this.driver = driver;
  }

  public SecureAreaPage clickLoginButton(){
    driver.findElement(loginButton).click();
    return new SecureAreaPage(driver);
  }
}

Painful discoveries

While blogging my code looked perfect to me, but my memory was not that perfect. When I coded, I did not use version control system for my personal project at that moment. Luckily I had made short notes in charters. I missed some notes, but I was still able to reconstruct my steps.

Another thing I noticed was, that the code for testBackspacePressed was written before testSuccessfulLogin. I could change flash forward to flash backward. But that would be quite confusing for the reader. So this blog post serie is based on real stories.

Connecting Assert and Act Continued

A way to introduce secureAreaPage in the code is to insert a new line of code:
“SecureAreaPage secureAreaPage = loginPage.clickLoginButton();”

@Test
public void testSuccessfulLogin(){
  LoginPage loginPage = homePage.clickFormAuthentication();
  loginPage.setUsername("tomsmith");
  loginPage.setPassword("SuperSecretPassword!");
  SecureAreaPage secureAreaPage = loginPage.clickLoginButton();
  assertTrue(secureAreaPage.getAlertText()
             .contains("You logged into a secure area!"),
             "Alert text is incorrect");
}

Now secureAreaPage was initialised, but now I had to code a method clickLoginButton.

public SecureAreaPage clickLoginButton(){
  driver.findElement(loginButton).click();
  return new SecureAreaPage(driver);
}

There was still something missing. I missed a class for SecureAreaPage. So I coded a class with a constructor SecureAreaPage and the long awaited method getAlertText. For the newbie programmers a constructor is a method to make object. If there is no constructor, then there is no object to manipulate or test.

public class SecureAreaPage {
  private WebDriver driver;
  private By statusAlert = By.id("flash");

  public SecureAreaPage(WebDriver driver){
    this.driver = driver;
  }

  public String getAlertText(){
    return "ran394594";
  }
}

Notice, that I put in some random text in the method getAlertText. So every time getAlertText is invoked, it will return “ran394594”.

Now comes the interesting part. Let us have a look what happens, if the following line is executed:

assertTrue(secureAreaPage.getAlertText()
             .contains("You logged into a secure area!"),
             "Alert text is incorrect");

secureAreaPage.getAlertText() returns “ran394594”.
This is different from “You logged into a secure area!”,
so the following error message was shown to me:
“Alert text is incorrect”.

So I had finished Green: a message is shown.

The third step of the cycle in TDD is Refactor. The code was already well structured. End of cycle.

Time for a flash forward for another example

@Test
public void testBackspacePressed(){
  // Arrange
  // String header  = "Random text";
  var keyPressesPage = homePage.clickKeyPresses();

  String header  = keyPressesPage.getHeader();
  Assert.assertEquals(header, "Key Presses", "You entered the wrong page.");

  // Assert
  String result = keyPressesPage.getResult();
  Assert.assertEquals(result, "You entered: BACK_SPACE", "Wrong key detected");
}

Stumbling around

Some readers might prefer a straight forward description of TDD in test automation. And yes, I will describe it more concisely in the next blog post. My message is that it took time to adjust my mind to a new way of developing. I had to unlearn things. That is difficult.

Let me clarify this a bit more. Test Driven Development in test automation sounds like programming and testing of programming and testing. It took me a lot of effort to keep the Arrange, Act, and Assert apart from the Red, Green, and Refactor.

Another thing is that blogging this way gives me a better understanding how it works. It is no theory; it is real practice; it does actually work.

Trust

“Never trust a test you have never seen fail.”

I have seen talks of Angie Jones and Bas Dijkstra, two good test automation experts. Their recurring warning is always to see whether the test actually worked. They used Arrange, Act, and Assert in this order. Then they changed Act to see whether the Assert would show an error.

By using the order Arrange, Assert, and Act I know that Assert is testing the right thing. If I will finish my Act in the next blog post, I do not have to change Act to verify my Assert.

A small story to end this blog post

Maybe you remember the story about the walk from the Rijksmuseum to the palace in Amsterdam from the previous blog post. It is not some palace, but a real royal palace is used by the Dutch king and queen.

Lots of kids would go like
“Awwww, white castle with rank towers.
We want to see it!
Now!
Please.”

Only the royal palace is different from the one from a fairy tale. If you are on The Dam, there is no single building looking like that castle of a Sleeping Beauty.

How do I describe this palace to a tourist?
It looks like a grey huge townhall with a dome. This is a way to assert that a tourist is looking to the right building.
Sorry kids for spoiling the fun.

My next blog post is about inserting Act between Arrange and Assert.

Fast Forwarding Arrange

At that moment I had a test, which would always pass. This was not the right starting point.

Putting

The first phase of a test is to arrange, that things can be tested. Somehow I had to land on the Login Page on https://the-internet.herokuapp.com/. In other words put the automation tool on the same page. And off I coded.

What is a way to identify this screen?
Okay, the header is “Login Page”.
Now I can use Red, Green, and Refactor to make the final step to this screen.

Red or making a failing test was simple, because nothing could be seen.

@Test
public void testSuccessfulLogin(){
  assertTrue(loginPage.getHeader()
               .contains("Login page"),
               "You entered the wrong page.");
}

Green: now I had to make enough code to pass the test. The code was quite simple.
In LoginPage.java I made a method for the class LoginPage:

public String getHeader(){
  return "ran394594";
}

Refactor: the code looked simple. So there was no need for cleanup.

There was a drawback. I had a continuous passing test. This reminded me of the last unsuccessful attempt. In that case I started with a failing test of the Assert. Also in this case I ended with test code unconnected with the code under test.

Things

Then I realised that I had repeated the same wrong steps. This walk was clumsy. I started at the end of the Arrange and wanted to walk back with Red, Green, and Refactor.

In the upper part there is a rectangle "Arrange". Under this rectangle there are two groups rectangles "Red", "Green", and "Refactor" under an arrow pointing to the left. The left group is lower than the right group!

A lot of people state that you should start with the desination and then try to figure out what you need to get there. I had the feeling that I was rather awkward.

Let us assume that I have to provide some information to a wondering visitor of The Netherlands.
Imagine me walking backwards from the Palace on the Dam to the Rijksmuseum in Amsterdam.
They might mistake me for being a tourist. I hope.
Better to take some extra pictures. A habit of tourist. I think.

In the right order

It was better to start with Red, Green, and Refactor at the beginning of the Arrange. And from there I would slowly walk in the right direction.
Under the rectangle Assert there are two arrows RGR pointing to the right. The left RGR is higher than the right RGR!

So I needed to have code to get to the home page. But LoginPage.java extends BaseTests.java. This basically means that all code in BaseTests.java also applies to LoginPage. I discovered that the code to get to the Home Page was already present.

@BeforeClass
public void setUp(){
  System.setProperty("webdriver.chrome.driver", "resources/chromedriver.exe");
  driver = new ChromeDriver();
  driver.get("https://the-internet.herokuapp.com/");

  homePage = new HomePage(driver);
}

My first step was to put the code with getHeader in a comment.

@Test
public void testSuccessfulLogin(){
  /*
  assertTrue(loginPage.getHeader()
               .contains("Login page"),
               "You entered the wrong page.");
  */		   
}

Red: now I had to test that the Login Page was not shown. There was no code in place, so my test would always fail. I checked with my own eyes, that the Login dialog was not shown

Green: then I made some code to get there

@Test
public void testSuccessfulLogin(){
  /*
  assertTrue(loginPage.getHeader()
               .contains("Login page"),
               "You entered the wrong page.");
  */
  LoginPage loginPage = homePage.clickFormAuthentication();
}

I also needed some code for the method clickFormAuthentication in HomePage.java.

private By formAuthenticationLink = By.linkText("Form Authentication");

public LoginPage clickFormAuthentication(){
  driver.findElement(formAuthenticationLink).click();
  return new LoginPage(driver);
}

I executed the code again. And I saw the Login dialog.

Refactor: the code contained some comment with old code.

@Test
public void testSuccessfulLogin(){
  /*
  assertTrue(loginPage.getHeader()
               .contains("Login page"),
               "You entered the wrong page.");
  */
  LoginPage loginPage = homePage.clickFormAuthentication();
}

I deleted the comment:

@Test
public void testSuccessfulLogin(){
  LoginPage loginPage = homePage.clickFormAuthentication();
}

Also I noticed, that there were only manual checks and no automatic tests in the code of login page. Somehow I let my attention slip away. Again.

It is time for a flash forward. A few practice runs later:

@Test
public void testBackspacePressed(){
  // Arrange
  // String header  = "Random text";
  var keyPressesPage = homePage.clickKeyPresses();

  String header  = keyPressesPage.getHeader();
  Assert.assertEquals(header, "Key Presses", "You entered the wrong page.");
}

Why should I bother to use Red, Green, and Refactor so intensively?
This was the only way for me to fully experience this new way of developing. Later on I could always change things.

Okay, back to the point where I was trying to test a login form.
There I was. Ready to make actual tests.

This exercise was so small, that I started at the wrong point twice.

  1. At the end of the Assert
  2. At the end of the Arrange.

The whole way of programming in the right direction was still not a natural thing for me.

Somehow I forgot the time being a test coordinator for performance tests. My team members had to make scripts for complete unknown websites. My task was to simplify this by asking for click paths.
“Would you please provide screenshots, which are shown to to the user, and the input of the users?”

Another side note

In music there is a saying that the tape does not lie. If you are a good musician, then the recording will prove it. For software development I could translate it to “the code does not lie”. For doubtful people I could change it to the “the version control system does not lie”.

I could write about an error free test automation experience, which is not real.
My coding in TDD in test automation was still in progress. I was learning.

My next blog post is about fixing the Assert.