ZebraTester Blog

Everything Web Performance


seo3
cloud1
mobile6
seo4
ideoa5



Extending ZebraTester with Plug-ins

ZebraTester ships with a diverse range of built-in functions intended to cover the most common use cases. ZebraTester is also highly expandable, capitalizing on the power and flexibility of the ZebraTester plug-in architecture. In this article, we will demonstrate how to write a ZebraTester plug-in to handle dynamically changing data.

Case study:
Suppose we have a website with a list of headlines that refreshes itself minute-by-minute. What is the best approach to load testing for this type of site? Typically, changing data is handled with input files. But in the case of a breaking news website, input files will not be enough, as news items will quickly become outdated and fall off the front page. A test relying on file input data will not only require frequent updates, but will also fail to catch news items that are added while the performance test is administered.


Legend: Examples of “breaking news” updates frequently refreshed throughout the day
http://www.zebratester.com/wp-content/uploads/2016/02/1.png
The input file approach is insufficient in this case. What is needed is a way of parsing the headlines from the response content while the load test is running. To do this, we will need to find a simple way of working with HTML. It is, of course, possible to write our own HTML parser, with all the pitfalls that entails—but thanks to the vastness of the existing Java ecosystem, we can rely on an existing library.

Enter Jsoup:
Jsoup is a flexible HTML parsing library in Java that allows us to find and extract data using standard CSS selectors. Using Jsoup to parse the response content, we have a robust, well-tested solution that is very unlikely to break in the way that an ad-hoc HTML parser might.

How can we add Jsoup support to ZebraTester? you ask. Is this a massive undertaking that will set our project back days?

Far from it. We can write a simple plug-in using Jsoup in a matter of minutes.

Let us first explore the structure of the data we need to extract:

Using Safari’s built-in web development tools (similar tools exist for Chrome, Firefox, and Opera), we can see that headlines live in an “h3” element of class “cd__headline”.

http://www.zebratester.com/wp-content/uploads/2016/02/2.png

The full HTML of the element is:

<h3 class="cd__headline" data-analytics="Top%20stories_list-hierarchical-xs_article_"><a href="/2016/01/24/health/zika-cases-uk/index.html"><span class="cd__headline-text">Zika virus reaches UK </span><span class="cd__headline-icon"></span></a></h3>
As we can see from the HTML of the element, we are interested in the href attribute of the “a” child element. In other words, we can extract all such links from the document using the selector document.querySelectorAll(‘h3[class=cd__headline] > a’). This will return a list of links. By selecting one such element at runtime, we will accurately simulate a user clicking on any of the top headlines on the news site.

Let’s take a look at how this can be implemented in ZebraTester. The first step is to open the ZebraTester Project Navigator and then navigate to the Plugin folder. We then click the “Create New Load Test Plug-in” button.

As we can see from the HTML of the element, we are interested in the href attribute of the “a” child element. In other words, we can extract all such links from the document using the selector document.querySelectorAll(‘h3[class=cd__headline] > a’). This will return a list of links. By selecting one such element at runtime, we will accurately simulate a user clicking on any of the top headlines on the news site.

Let’s take a look at how this can be implemented in ZebraTester. The first step is to open the ZebraTester Project Navigator and then navigate to the Plugin folder. We then click the “Create New Load Test Plug-in” button.

http://www.zebratester.com/wp-content/uploads/2016/02/3.png

In the resulting pop-up, we enter the plug-in class name, GUI label, and description. For input parameter, we can define one parameter called “inputHTML” and an output parameter called “outputHREF”. The “inputHTML” parameter will contain the HTML to parse, and the “outputHREF” will contain the link to follow.

http://www.zebratester.com/wp-content/uploads/2016/02/missing_image_per_article.png

We then click “Save Template and Continue” followed by “Generate Plug-In Code”. ZebraTester will now generate the boilerplate code needed. What is left to add is code implementing our specific task: to find and extract the headline link.

http://www.zebratester.com/wp-content/uploads/2016/02/4.png

After the code has generated, we then save the template code by pressing the “Save Plug-In Code” button. Notice that some 250 lines of Java code have already been generated for us. By adding just a few lines, we will now be able to implement the required functionality. http://www.zebratester.com/wp-content/uploads/2016/02/5.png

Feel free to open the generated file “News_parse.java” in your favorite text editor. We will note that, by default, the plug-in imports a lot of ZebraTester-internal classes. Since we intend to use Jsoup, we will need to add the following lines to the end of the import definition:

	import org.jsoup.Jsoup;
	import org.jsoup.select.*;
	import org.jsoup.nodes.*;
	import java.util.Random;
Believe it or not, we are already almost done! The only step left is to modify the execute method. The execute method is called when the plug-in is executed, and will contain the code necessary to implement our required functionality. Remove the “ vvv — sample code — vvv” code statements from the execute method (they exist for example purposes and are not needed for our plug-in).

We will now need to add some code to:

1) Parse the HTML
2) Extract the “a” child element of the “h3.cd__headlines” elements
3) Select one such element at random and extract the href attribute

With Jsoup we can accomplish this in just these few lines of code:

	int sz;
	Random r;
	Document doc = null;
	Random rnd = new Random();

	doc = Jsoup.parse(inputHTML, "https://edition.cnn.com");
		
	Elements elts = doc.select("h3[class=cd__headline] > a");
	sz = elts.size();
	if (sz > 0)
	{
		Element elt = elts.get(rnd.nextInt(sz));
		outputHREF = elt.attr("href");
	}
	else
	{
		logVector.log("No headlines available!");
		outputHREF = null;
	}

Now we have a fully functional plug-in that uses Jsoup for the heavy lifting.

To use the plug-in:

First, we must ensure that the compiler will be able to find the Jsoup library. Click the “Declare External Resources For Self-Developed Plug-ins” button and add the absolute file path. Make sure the “Add File to Java CLASSPATH on the Exec Agent(s)” option is selected, then click “Add Absolute Path”. Since our $CLASSPATH variable is now correct, we can then compile our plug-in by pressing the “Compile” button next to the “News_parse.java” file.

http://www.zebratester.com/wp-content/uploads/2016/02/9.png

First, the plug-in needs to be compiled. Click the “Compile” button next to the “News_parse.java” file in the Project Navigator. Assuming no errors are found, we will then get a pop-up confirmation that the file has been successfully compiled.

http://www.zebratester.com/wp-content/uploads/2016/02/10.png

Final steps:

1) The HTML response content must be extracted to a variable.
2) The plug-in must be invoked to get the target URL.
3) The recorded URL must be replaced with the correct target URL.

For step one, click on the HTTP request in the GUI and select the “Extract Whole Response Content to Var” option.

http://www.zebratester.com/wp-content/uploads/2016/02/12.png

For step two, we will use the “Add Load Test Plug-in” option with the variable we just created as an input variable, and with an output variable whose name we can choose freely.

http://www.zebratester.com/wp-content/uploads/2016/02/14.png

For step three, we need to tell ZebraTester to rewrite the “Load Headline” step using the “Assign Var to Request-File Pattern” with the output variable of step two.

http://www.zebratester.com/wp-content/uploads/2016/02/16.png

And we’re done! We have a load test program that will always work with the latest headlines, even if the headlines change while the load test is running. This kind of outcome would be impossible to achieve with standard record-replay techniques and URL replacement using data files only. By capitalizing on ZebraTester’s plug-in architecture and leveraging the existing libraries in the vast Java ecosystem, we are able to achieve more flexible and dynamic testing—all with minimal effort.

The source code for this plug-in can be found at our public github repository.

Written by Per Hjeltman, Senior Performance Engineer, Apica

Leave a Reply

Your email address will not be published. Required fields are marked *