All posts by Mindful tester

XML Injection For Beginners

Suppose I sent one of my kids to the supermarket with a shopping list. Afterwards I got all the things on the list and the change. How would I know that everything went right?

If there was a discount on an article, then I would get more money back. Or my loyalty card could have been used to get free candy.

The best way is to look at the bill, the money, and the bought items. This is audit in a nutshell.

If you use a screen reader, please configure the screen reader to read interpunctions and symbols aloud.

XML uses special characters like the greater sign (>) and the less than sign (<).

Info about XML basics

The problem with an experience report is, that I make huge jumps in my thoughts. So I put in some chapters with more details. For people unfamiliar with XML, I put “Info about” in the title.

Let me introduce an imaginary small company.
Grey Pizza Pasta is an Italian restaurant which recycles pizzas in pasta. To make it special the pasta is grey for some special reason. It began as a joke and now it is a business.

A big scoop of pasta will cost 3 Euro. What is a scoop?
If it looks like a spoon, can I put a top of pasta on it? What kind of tool can I use to measure it? How do you determine the right amount of pasta?

Mister Grey would like make a row of pasta on the shelf, but which are the first and last ones? How could you tag them?

Maybe it would be better to make small heaps of pasta. The space between the heaps is like a boundary. The pasta between the spaces has been weighted.

According to Wikipedia XML or eXtensible Markup Language is a markup language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable.

In XML a heap of pasta could be described as follows:

<heap>pasta</heap>

<heap> is called a begin tag. You might call it a starting point.

  • It starts with< or smaller than sign.
  • heap is the description, what it represents.
  • > or greater than sign ends the tag.

</heap> is called an end tag. You might call it an end point. It almost looks like the begin tag. Only a forward slash or / is used before heap.

So everything between <heap> and </heap> is contained in the heap. This is called an element in XML. <heap>pasta</heap> can be described as a heap of pasta.

Is every tag name allowed?
Not really, because it must be defined in an XML Schema. In most cases an XML file can be read using common sense or domain knowledge. The last one is knowledge about the product or service being offered.

It is very important to write the end tag.

<heap>pasta chocolate milk

would be converted to a heap of pasta, chocolate, and milk. This is a tough one to imagine. Even for me.

Maybe Mister Grey would have intended a heap of pasta, a heap of chocolate, and a cup of milk.

This would lead to the following code:

<heap>pasta</heap>
<heap>chocolate</heap>
<cup>milk</cup>

Even a kid can imagine these things.

Stumbling over XML

The following story is based on a real story. All references to real companies have been changed.

During a regression test I was requested to look at the audit log. So I opened the VIP Cinema app and bought a ticket for Monsters Unlimited, Potato chips, and a Cola.

My next step was to look at the audit log. I opened the file and saw plain English. So it was easy to understand. It contained my movie ticket, my snack, and my drink.

Being curious I scrolled down. The se­­­­cond part of the file contained XML or Extensible Markup Language. And my head filled with one big question mark.

<item>
  <qty>1</qty>
  <name>potato chips</name>
  <remarks></remarks>
</item>
<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks></remarks>
</item>

At least I recognized my bought items. I ordered 1 cola. The quantity is 1, so qty is short for quantity. So the quantity is shown between <qty> and </qty>.

Now I had an audit log with the first part in plain English and the second part in XML. It looked like that the XML was translated to plain English for the audit. If an auditor would read this, then she or he would understand what happened.

I wanted to test my assumption, that the XML was translated to plain English.

Info about nesting XMl elements

There is a problem with the heaps of pasta. It looks nice, but people would like to have a box or bag containing pasta. Mister Grey being minimalistic again wanted to make a grey paper box. The colour grey was chosen was for an obvious reason.

<box>pasta</box>

This XML code is quite confusing. What does the box contain actually? What is the weight of the content of the box? What are the ingredients?

This could lead to the following XML code

<box>pasta</box>
<weight>230 gram</weight>
<ingredients>flour, salt, water, cheese, tomato sauce, pepperoni</ingredients>

Looking at the shelf I would expect to see a box, a weight, and ingredients. But there is only a box on the shelf.

In XML it is possible to nest elements in other elements. E.g. on the box a name of the product, a weight, and ingredients are shown.

<box>
  <name>pasta</name>
  <weight>230 gram</weight>
  <ingredients>flour, salt, water, cheese, tomato sauce, pepperoni</ingredients>
</box>

SQL Injection

My Inner Exploratory Tester woke up. I looked at the code and saw <remarks>. Remarks are used for extra information, which contains requests from the customer.

I would like my Cola in a special cup by adding “a cup with a blue furry print” in the Remarks field.

SQL or Structured Query Language is frequently used language to change data. My order would be like:
“Computer, add 1 ticket for Monster Unlimited, 1 Potato chips, and 1 Cola to my order.”

Remarks can also be used by hackers for bad things. I remembered SQL injection. A hacker could add the command in the Remarks Field:
“Computer, add 1 lemonade to my order.”
On my arrival in the cinema I get a free lemonade. This is SQL injection in a nutshell.

SQL injection is adding SQL code which will add wrong information to the system. In turn this can be misused.

SQL is a language like XML. So the injection should also be possible with XML.

Info about adding code

This was the code I would like to modify.

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks></remarks>
</item>

If I would enter “XML code” in the Remarks field, then the following code would be generated:

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>XML code</remarks>
</item>

Now I used known XML code to put in the Remarks field. This piece of code has the right structure.

<item>
  <qty>2</qty>
  <name>potato chips</name>
  <remarks></remarks>
</item>

If I would put the XML code for the potato chips in the remarks element of the cola, then I would get something like:

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>
    <item>
      <qty>2</qty>
      <name>potato chips</name>
      <remarks></remarks>
    </item>
  </remarks>
</item>

The program would try to match begin tags with end tags.

If an end tag is found, then it will be matched with the corresponding begin tag before it.
A simple case is the following line:

<qty>1</qty>

</qty> is an end tag. The first begin tag is <qty>. qty or quantity is 1, because 1 is between the <qty> and </qty>.

Now it was time to match <item> with the right </item>.

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>
    <item>
      <qty>2</qty>
      <name>potato chips</name>
      <remarks></remarks>
    </item>
  </remarks>
</item>

The code had 2 elements: an element about cola and an element about potato chips. Let me try to tell me what this means: 1 cola has remarks containing 2 potato chips. This looked promising. Let’s have a closer look at the Remarks element of the cola Item element.

 <remarks>
  <item>
    <qty>2</qty>
    <name>potato chips</name>
    <remarks></remarks>
  </item>
</remarks>

Is it possible that a Remarks element has a nested item element?
Probably not. Most of the time this is bad syntax and the XML code will not be processed. A Remarks field should contain text and should not nest elements in XML.

Info about writing the proper code

This section has been added to illustrate my train of thoughts which took a few seconds during the test. It took me some time to get the experience for writing the XML code.

What I wanted to order

For my test I wanted to change the XML code. I really wanted to have this piece of XML code in the audit log:

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>
  </remarks>
</item>
<item>
  <qty>1</qty>
  <name>lemonade</name>
  <remarks>
  </remarks>
</item>

My first order attempt

A Remarks field can contain text. So the first line in the remarks field would be </remarks>. I marked this with the line:

<!-- Begin added code -->

This is a XML comment with the text ” Begin added code “. A program which processes XML code, will ignore this line. Comments are useful for programmers and testers.

<item>
<qty>1</qty>
  <name>cola</name>
  <remarks>
  <!-- Begin added code -->
  </remarks>
</item>
<item>
  <qty>1</qty>
  <name>lemonade</name>
  <remarks>
  </remarks>
</item>

Time for trial and error. I had to select a piece of code, I picked the line before to the line with a remarks begin tag. I marked this with the line <!– End added code –>. This is also an XML comment.

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>
    <!-- Begin added code -->
  </remarks>
  </item>
  <item>
    <qty>1</qty>
    <name>lemonade</name>
    <!-- End added code -->
    <remarks>
    </remarks>
</item>

My thought was to put the following text in the Remarks field:

<!-- Begin added code -->
  </remarks>
  </item>
  <item>
    <qty>1</qty>
    <name>lemonade</name>
    <!-- End added code -->
</item>

In turn this should lead to the following code:

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>
    <!-- Begin added code -->
  </remarks>
  </item>
  <item>
    <qty>1</qty>
    <name>lemonade</name>
    <!-- End added code -->
    <remarks>
    </remarks>
</item>

But the generated code was different:

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>
  <!-- Begin added code -->
  </remarks>
  </item>
  <item>
    <qty>1</qty>
    <name>lemonade</name>
    <!-- End added code -->
  </remarks>
</item>

I noticed that <remarks> under the line <!– End added code –> was not contained in the generated code. I counted only 1 <remarks> instead of 2 <remarks> or begin remarks tag.

At that moment there was only 1 proper pair of <remarks> and </remarks>. This was wrong. For good XML code the number of begin tags is equal to the number of end tags.

My second order attempt

Another trial and error. I could choose the line before the line with the end tag of remarks.

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>
  <!-- Begin added code -->
  </remarks>
  </item>
  <item>
    <qty>1</qty>
    <name>lemonade</name>
    <remarks>
   <!-- End added code -->
  </remarks>
</item>

A simple way to determine whether the XML code is right, that the piece of code has the same number of end tags as the number of begin tags:

<!-- Begin added code -->
</remarks>
</item>
<item>
  <qty>1</qty>
  <name>lemonade</name>
  <remarks>
<!-- End added code -->

For people with more time on their hands, there are other ways to derive the code.

XML injection in action

Because of the desert scene of Monsters Unlimited, visitors usually order extra drinks. I had the right XML code for this purpose:

<!-- Begin added code -->
</remarks>
</item>
<item>
  <qty>1</qty>
  <name>lemonade</name>
  <remarks>
<!-- End added code -->

In order to show the use of this code I added the XML comment with “Begin added code” or <!– End added code –>.

I placed this code in the Remarks field of my Cola order and the App made the following XML code:

<item>
  <qty>1</qty>
  <name>cola</name>
  <remarks>
  <!-- Begin added code -->.
  </remarks>
</item>
<item>
  <qty>1</qty>
  <name>lemonade</name>
  <remarks>
  <!-- End added code -->.
  </remarks>
</item>

This XML injection led to the following piece in plain English in the audit trail:
1 cola
1 lemonade

I was quite pleased with my successful data manipulation on my first try. For the price of 1 cola I got 1 cola and 1 lemonade according the audit log. Two drinks for the price of one.

In the cinema I only would get a cola. This was the only drink in my order. The lemonade mentioend in the remarks was not included in the order. But a lemonade was still in the audit log.

What is the deal?
In this case someone would have a lemonade for free. The drink was already included in the log.

What would happen, if an expensive collectible gold cup is added to my order?
Someone could take this cup home for free. One cup for the price of none.

Solutions to prevent XML injection

As a tester I had two solutions:

  1. Limit the number of characters in the Remarks field.
    This way the bad XML code would be reduced or even prevented.
  2. Train the employees of the cinema.
    If they could detect strange text in the Remarks field, then there was a chance on an attack of a hacker.

The developers were quite strict: prevent the use of < and > in the Remarks field.

If you use a screen reader, please set the screen reader to the normal reading mode.

Summary

Data is not about what is stored, but how it is used,

XML injection is adding XML code which will add wrong information to the system. In turn this can be misused.

Advanced blogging – The Usual Rushed Texts

The first time I noticed the speed of the screen reader of a blind person I was overwhelmed. I barely could catch each word. The speed was at least twice the speaking speed of a person.

Imagine a friend who excitedly talks about a great job offer.
It sounds like a rushed text.

“Are you a member of the Guild?”
“Last time I checked.”

Show me the code

The default editor in WordPress is the visual editor. For beginners this is a great way to begin. The text is shown in the same way the visitors will read it. There is a catch. Sometimes the editor will change the look and feel.

For me the visual editor in WordPress hided too much information, so I used the code editor. I was editing HTML files and checked the changes in the visual editor.

For fast writing, HTML provides a nice structure: headings. I just put some keywords or short sentences in the file. Jerry Weinberg called them fieldstones. I kept the rest of the words in my mind and added them later on.

Let me show and tell

I always thought that it was a good idea to show what I did. Pictures can clarify a lot. A lot of pictures can clarify a lot more.

in the visual editor the picture had a bad contrast. I thought that I shrunk the picture too much. The picture looked good in the preview mode.

No alternative text

If I did not use an alternative text, then the screen reader would read the following aloud:
“Just to be sure I turned off the highlighting.
https://mindfultester.com/wp-content/uploads/2021/01/mindful-tester-p3-1.20-no-highlightingboxes.jpg”

In this case NVDA, a screen reader, used the path and file name instead of the empty alternative text.

“This is not according to the code of the Guild.”

Non descriptive alternative text

If I did use a short alternative text, then the screen reader would read the following aloud:
“Just to be sure I turned off the highlighting.
Dialog without highlighting”

This sounds like a rushed text. In the picture there are more details worth mentioning.

“This is not part of the deal.”

A complete alternative text

In my blog post I used a descriptive alternative text, then the screen reader would read the following aloud:
“Just to be sure I turned off the highlighting.
In the left image or baseline “Cap” in the username is shown, in the right image or checkpoint the username is empty. In the right upper corner, there is a thumb down button.”

“This is the way.”

Summary

Let me break down the structure. First, I wrote a text with a short introduction to the picture. Then the alternative text was a detailed description of the picture. A screen reader user would hear a normal story. This way I told a story with normal texts and alternative texts in a natural flow.

For the third blog post about Visual TDD in Test Automation I had a separate file for alternative texts. This was handy for the consistency of the text. But I disliked the continuously switching of files: open WordPress, go to the location of the picture, open the editor with the alternative texts, copy the desired text, and paste the text at the right spot in WordPress.

So I put all my alternative texts into the HTML file. This saved me some application switching. It might have some impact on the consistency of the alternative texts.

Even worse, I needed to do some spell checking.

I have a spell on you

The problem with blogging is that the mind is faster than the hands. This often leads to spelling and punctuation errors.

For the creative mind, speed is essential. I could lose my train of thoughts. This is why I gladly accept any typos.
It sounds like a rushed text.

WordPress uses the browser spell check. This is focused on the right spelling of the words. The syntax of the sentences is ignored.

If I spell check my blog post, then I go to the visual editor. I copy the text into my favourite word processor and start the spell checking. Because I was worried about the layout, I change the text in both WordPress and the text editor.

Fine, but what about the alternative text?
A simple solution is:

  • go to the picture
  • copy the alternative text
  • paste in your favourite word pressor
  • do a spell check

There is one place where all the alternative texts are shown and that is the HTML file. The code editor shows all the lines including the alternative texts, which are not shown in the visual editor.

Simple solution

My advice would be: search for alt in the html file and copy the stuff.

Difficult solution

For the selection of my alternative texts, I chose a complicated solution.
This paragraph is meant for people with an IT background. You are free to skip this.

If you are using a screen reader, please set the interpunction level to all.

I opened baregrep, a shareware tool which can filter lines of codes with the text “alt”. This basically means: give me all the lines with alternative texts. I was lucky that I did not use words like alternative or salt.

Baregrep shows all the lines of the files with the file name ending with "v1t.txt" in the subdirectory D:\Users\Han Toan\Mijn documenten\Blog\2020 Blog containing the text alt!

I copied the output.

The Export Test Results Dialog has a To Clipboard button!

The next step is to use VIM, a free software program with a typical interface. I pasted the text in my word processor.

VIM shows the pasted lines of the test results!

The file contained one picture with an alternative text on a different line.

:%s/^.*alt="//
  • : means, that I wanted to execute an ed command. ed is an editor command.
  • % is for all lines in the file
  • s is short for substitute.
  • /^.*alt="/ is the pattern, which I looked for between / and /.
    • ^ is from the beginning of the line.
    • .* is zero or more arbitrary characters.
    • alt=" is the literal text “alt=””
  • // is the text between / and /, which must be used for the replacement. In this case this is an empty string.

The whole command removed all the text from the beginning of the line until and including “alt=”in every line.


:%s/".*//

  • : means, that I wanted to execute an ed command. ed is an editor command. What I basically write, is “ed, I am talking to you. The following command is for you.”
    It is like a home assistant: “VIP Home app, I am talking to you. The following command is for you.”
  • % is for all lines in the file
  • s is short for substitute.
  • /".*/ is the pattern, which I looked for between / and /.
    • " is the literal text “.
    • .* is zero or more arbitrary characters.
  • // is the text between / and /, which must be used for the replacement. In this case this is an empty string.

The whole command removed all the text from ” until the end of the line in every line.

The last step is to copy the lines into the word processor for the final spell checking.

For the people who like to quit VIM without saving:
:q!

  • : means, that I wanted to execute an ed command. ed is an editor command. What I basically write, is “ed, I am talking to you. The following command is for you.”
    It is like a home assistant: “VIP Home app, I am talking to you. The following command is for you.”
  • q is quit.
  • ! means without saving, because I am sure.

Back to Media for the change

Then I updated the alternative texts in Media, where all my pictures are stored.
If you are using a screen reader, you can set the interpunction level to your favourite setting.

Here is the change

A few days later I noticed that the wrong alternative texts were included in the blog post. I tried to figure out what happened.
Who was the usual suspect?

The moment I inserted a picture from the Media in my blog post, a copy of the HTML code was used. Any later changes in Media remained there. So I had to change the text again in the code editor.

One of my kids summarised my story as follows:
if you send your homework to school, then you send a copy of the file.

I could have chosen to replace the old pictures in the blog post with the latest pictures in Media. I was afraid to add the wrong picture. Then I should not forget the resizing.

So I edited the alternative texts in the visual editor.

Listen carefully. I only tell this once.

Pro tip: use the screenreader to read your text aloud. Some typos are difficult to see, but are easy to hear.
Just hover with your mouse over the text and hear it.

“I have spoken.”

Advanced blogging – Honey, I Shrunk The Pictures.

Small recap

It took me three days and three takes to make 26 pictures. Now I had to shrink the pictures to a smaller size. This would lead to a faster download of the blog post and hopefully a better performance of my blog post.

Warning:
This is not the standard look-how-great-I-am blog post. It is more like look-how-I-stumbled-forward.

Reminder to myself:
As a tester I built this website to share information, but also to get a better understanding of building websites. The more I blog, the better I test.

Here, have some notes

Writing a blog post is creating an introduction, a good end, and something in between. The last part was still missing.

I only had notes and pictures in a charter. As an experienced tester I made my observations including date and time. I used the abbreviation PIC for Picture.

20201012 blogging
PIC 10:37 Screen print TEstLogin3.java 
Input: Cap / Secret.
PIC 10:44 debug menu
PIC 10:46 first breakpoint.
PIC 10:47 Cap in screen.

All my pictures had names like “20201012 – 1.2 debug program.jpg”:

  1. “20201012” is the date when the picture was taken. The format is year in 4 digits, the month in 2 digits, and the day in 2 digits.
  2. “1.2” means Take 1 and Step 2.
  3. “debug program” describes the picture.
  4. “.jpg” describes how the picture is stored. I chose this file format, because it is great for editing. That is a bit of a spoiler.

The big advantage of this name is that I can order the pictures on name.
I must have looked like a Holywood director. Day 1, take 1.

Now I had to make a link between my actions and the pictures. Luckily, I had used descriptive names for the screen pictures. These pictures had a day and time stamp, so it was easy to place the file names at the right places in my blog post.

20201012 blogging
PIC 10:37 Screen print TEstLogin3.java -  20201012 - 1.1 three breakpoints.jpg
Input: Cap / Secret.
PIC 10:44 debug menu - 20201012 - 1.2 debug program.jpg 
PIC 10:46 first breakpoint. - 20201012 - 1.3 - first breakpoint.jpg
PIC 10:47 Cap in screen.- 20201012 - 1.4 - fill in username.jpg

Processing a picture

A 288 kb picture is too big for my blog post. I figured out several steps to cut down the size.

Emphasizing

The first step is to determine whether a box should be placed in the picture. If this was the case, I used a green bordered and transparent rounded rectangle for highlighting.I would add “+box”:
e.g. “mindful-tester-p3-20201012 – 1.4 – fill in username+box.jpg”.

Let me break down the name again:

  1. “mindful-tester” is the name of my blog.
  2. “p3” is the third part of TDD in Test Automation.
  3. “20201012” is the date when the picture was taken.
  4. “fill in username” is the description of the picture.
  5. “+box” states that a rectangle is placed in the picture for highlighting an important detail.
  6. “.jpg” describes how the picture is stored.

In case of no box I did not add anything at the end of the name.
This would lead to:
“mindful-tester-p3-20201012 – 1.4 – fill in username+no+box.jpg”
After a while, I did not use the “+no+box” any more, because it did not add value.

I stored the old version of the file in case of any mistakes.
So far, so good.

Zooming in

The second step was that I cropped the picture: I would cut a rectangle out of the picture and save it. My favourite shareware tool is irfanview. Less pixels means smaller size and better performance.

Shrinking

The third step was to shrink the picture also using irfanview.
This way a picture of 288 kb could be shrunk to 40 kb.

In the Resize/Resample image dialog of irfanview the "Set new size as percentage of original" is selected. The Width and Height are set to 100%!

Note: for this blog post I accidentally used a png file. png is like jpg a way to store pictures. Only irfanview would not let me resize the png picture using percentages.

In WordPress I resized the picture to maximum size. I selected a corner of the picture and stretched the picture. This way the user does not have to click on the picture to see the details.

The new file name was:
“mindful-tester-p3-1.4-fill-in-username+box.jpg”.
I left out the date and the spaces in the name.

Was this a joyful experience for me? Not really.
So the time of processing pictures became longer and longer, until I forgot my process.

Processing pictures

The status of the picture was not clear in the file name. Also, the location of the pictures had its own logic. So the time between the editing of one picture and the editing of another picture became longer and longer.

So I made a nice file with all kind of instructions, “Instructions printscreens programming.txt”.
This included files, their contents, and their locations.
Exploratory blogging in a nutshell.

Only to discover that I had already “Overview files.txt” in place.
Exploratory blogging takes practice.

Because a lot of time was spent on editing the pictures, I did not remember my steps well enough while writing. Now pictures are handy, because pictures do not lie. So I had to change my story a few times.

Looking at some pictures I missed the essential point of the picture. So I added the transparent rectangle and went through the whole editing process again. Even after cropping the picture the important details were too small in some cases. So I split pictures in certain cases.

Placing a picture in a blog post is quite tiresome. I had to go to the right location, search the right picture, and upload it. Not enjoyable at all.

But I already knew what I wanted to upload. So I opened Media and uploaded all the pictures using one drag and drop.

Then I searched for the first picture to be included using PIC for Picture. All my files had an unique name like
“mindful-tester-p3-1.4-fill-in-username+box.jpg”.
Using the filter I searched for the file with 1.4 in the name. Then I selected the file.
In the Add media dialog of WordPress the Search Field is set to "1.4". The result of the search is a single picture containing a part of the Login Page!

Uncomfortable observations afterwards

  • At the end I shrunk the total size of my pictures in the blog post to a quarter: from 8 Mb to 2 Mb, which is still big. Is it still possible to shrink the pictures without losing the important details?
  • I did not check the size of my blog post with the preview version.
  • Chrome Devtools contains Lighthouse, an accessibility tool. My site scored 58 on a scale from 1 to 100 for performance. This is not really high.
  • The setting of “the most number of blogs posts show on page” was still on 5. I forgot to save this setting. Saving a setting in WordPress was not consistent with saving an alternative text for picture.
  • The trouble with pictures is, that it can cost a lot of time. This will decrease my enjoyability. It makes my blogging slower. On the other hand it will increase the enjoyability of the readers. At the end this will increase my enjoyability. I like to read my own blog posts. I might call this the Enjoyability Paradox. Things which I did, were no fun, but finishing the blog post was good for my mood.

Coming up

At the end of the picture processing, I had a bunch of pictures without alternative texts. For blind people or people with bad sight this had to be fixed. A screen reader needed these texts to describe the pictures aloud. This was about accessibility.