SoapUI for web service testing

A popular tool for web service testing is SoapUI. The following blog post describes how to use it for testing a simple web service, as well as setting up a series of tests and even load/stress testing. Without a doubt similar documentation is available on the SoapUI website; my post gives my spin on the product that maybe useful to others.

In this post we'll look at:

1) A simple web service example
2) Creating a simple test in SoapUI
3) Creating a test suite
4) Loast/stress testing your web services

Our web service

The web service we'll use here for testing is a simple Hello World example. It is comprised of the following solutions:

HelloWorldExample.xsd

xmlns="http://www.sagecomputing.com.au"
            targetNamespace="http://www.sagecomputing.com.au" elementFormDefault="qualified">
  
  

The inputElement and the outputElement will constitute the incoming and outgoing payloads of a simple HelloWorld web service.

HelloWorldExample.wsdl

             xmlns:tns="urn: HelloWorldExample.wsdl" xmlns:xsd="http://www.w3.org/2001/XMLSchema"
             xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
             xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:wsca="http://www.sagecomputing.com.au">
  
    
      
    
  
  
    
  
  
    
  
  
    
      
      

    
  
  
    
    
      
      
        
      
      

        
      
    
  
  
    
      
    
  

Finally the web service is served at the following address:

http://www.sagecomputing.com.au:7101/SoapUIWebServiceExample /HelloWorldPortTypePort

...and the WSDL at this location...

http://www.sagecomputing.com.au:7101/SoapUIWebServiceExample /HelloWorldPortTypePort?WSDL

Starting SoapUI 3.0

On starting SoapUI 3.0 you'll see this screen:


Note in the left navigator the Projects node.

Creating your first web service test

We want to test the web service we recently created. To do follow these steps:

Right click on the Projects node and select New soapUI Project from the context menu. This will display the New soapUI Project dialog.

We'll enter an arbitrary Project Name. For the Initial WSDL/WADL we enter the complete URL of our web service's WSDL:


Leave the default check boxes, where Create Requests is checked, and the rest unchecked.

On completing the dialog the navigator will now look as follows:


Note how SoapUI has created the project, a node to represent the WSDL binding, a node to represent the WSDL operations (one in this case), and a dummy request.

If you double click on the request, this shows where SoapUI begins to sing:


The new window shows a potential request to send to your web service. You'll note thanks to the WSDLs and XSDs defined within the web service, SoapUI has automatically created a dummy payload with the required XML structure. We could overtype the inputElement's value, and then press the green arrow button to send the request to the web service, seeing this result:


The beauty of SoapUI is its taken care of virtually everything but the actual data values for you. Network communications, formation of the SOAP payloads, and even validating the XML structures. For the developer this is a boon as you can test one of your web services to see what's happening without writing any client side code.

Creating a test suite

The previous steps were good for one of adlibbed tests on our web service, but what if you wanted to create a series of tests, replay them, and add assertions based around the results? SoapUI offers the Test Suite.

In the navigator right click the dummy request (Request 1) and select Add to Test Case. In the Create Test Suite dialog, and the Create Test Case dialog, accept the default names. In the Add Request to Test Case dialog accept the defaults as following:


On accepting the dialog you'll have the following structure in the navigator and a new window for the Test Suite:


From the navigator you can guess that from the Test Suite you can create a number of Test Cases, with a sequenced set of Step Tests made up of requests. In our example the request has been copied from our previous dummy request:


Also unlike our original dummy request, with the test request open, looking at the bottom of the window we have an Assertions option, of which when we click on it (the word Assertion) we see the following:


An Assertion is a test to run against the web service response to ensure we get the result we require. The current Assertion SOAP Response – UNKNOWN is simply saying we should get a valid SOAP response. We could alternative add another Assertion to ensure the result is not a SOAP Fault by selecting the Add button then the "Not a SOAP Fault" option:


Once we've set up our tests and assertions, we want to run them. We return to the test case in the project navigator, open it and select the green arrow button to run the complete test suite. Assuming all things go well, we should see the following result:


If one of our assertions on our test case fails we'll see the following result:


Load/stress testing your web services

While running a test suite of tests can prove your web services are functionally correct, they don't tell you:

a) What's the maximum transactions per second (ie. load testing)
b) Does your program fail under multiple requests (ie. stress testing)

Load and stress testing are an integral part of ensuring your application will work on hitting productions.

Strictly speaking load and stress testing are proving different things, but for the remainder of this post I'll assume they're one and the same as they use the same facilities in SoapUI.

You'll have noted in the project navigator under the Test Suite an additional node Load Tests was created:


Right clicking on the Load Tests node, select New Load Test, accepting the default name in the New Load Test dialog, you'll see the Load Test window:


The easiest thing to do to understand this screen is just hit the green arrow. So what's happening?:


In the above picture you can see that the web service is being called multiple times. The process bar tells you the percentage completeness of the tests. The Limit field is the number of seconds the test will run for. The Threads is the number of threads (essentially simulated users) that SoapUI will spawn to send requests to the web service. Finally the Test Delay is the milliseconds between each response and next requests.

On completion of the test you'll see the collated statistics in the table:


Different figures are useful in assessing the performance of your web services, including:

Min and max shows you how fast and how slow your web service could process a request and send a response. Typically your worst time is on startup as the server initializes the web services.

Tps – the average transactions per second, which gives you a rough idea of the concurrent sessions you can support (remembering that at startup you will service less, but once the server is initialized generally you can support more)

Err – number of errors detected such as SOAP Faults.

Now an average of 6.71 transaction per second doesn't look that healthy. In this case it's SoapUI that's throttling your web services as the number of threads and the delay between requests means your web services can't fly, they're just not receiving enough work to do. If we change the numbers, increasing the threads, and reducing the delay as per the following screen shot we see we can really put our servers under some pressure:


(Nice bit about this screenshot is we're using an Oracle WebLogic Server behind the scenes, a fairly expensive server all round, but it certainly cranks along with this simplistic test case).