February 17, 2016

SOAP Web Service Testing with SoapUI: Simple Yet Powerful

Testing web services was always a challenge for me. Doing functional testing of applications having UI is quite fun I must admit. You have pages, buttons, labels, text-box, images, etc. You know the underlying logic. Just enter the test data, click some buttons to call the underlying functions and voila, you are good to go.

Now, when you have web services to test, there is no UI involved (most of the time). You have to deal with XML files, XSDs, WSDLs, request and their responses and many more which for a functional tester might be overwhelming sometimes.

I used to create the XML requests manually. I would do this by getting the requested samples and functional specifications from the developers and product team and I would just change the attributes and/or their values in any text editor and then post them using Poster (a good add-on for Mozilla Firefox) to test SOAP requests and their responses. It was a no frills way to test my SOAP requests. Each test condition requires at least one SOAP request, so if I have let’s say 50 such conditions then you can imagine the burden of creating at least 50 different requests. Even though I could have saved them in different files, managing them was then quite a cumbersome task and it does not include that if there is a change needed even for a single attribute, I have to update all 50 files.

Then I got to know about an open source tool designed specifically to test the APIs, i.e. SoapUI. It has been developed and distributed by a company called SMARTBEAR and has free as well as paid versions. For the basic testing, even the free version provides robust support and most of the time your testing requirement should have been fulfilled by the free version.

Without any further ado, let’s dive into the SoapUI and see how it can help us with our web service testing in a simple way.

Main features:

  1. Mocking the services → With the help of SoapUI, we can easily mock the web services and test them thoroughly. We can use the WSDL file and it will auto generate the services and all the methods it contains. So the hassle to create the services is now automated. It allows SOAP as well as REST services to mock.
  2. Testing Security → SoapUI allow us to test the security features too. We can test the database by SQL injections to test the database vulnerability. Likewise it allows us to test the stack overflow by bombing the xml files. We can also test the Cross Site Scripting by SoapUI.
  3. Load Testing → One of the great features of SoapUI is to create the load test which can be seemingly integrate in LoadUI. We can set the SLA and verify it. It also lets you select the built-in load strategies like simple, fixed-Rate etc.
  4. Support wide variety of Technologies/Protocols → It support a wide variety of protocols as well as technologies to make the life of a tester easier. List comprises like SOAP, REST, Web/HTTP(s), JMS to name a few.
  5. Automation Integration support → You can bundle your SoapUI test and integrate them with Maven, Hudson, Bamboo, JUnit, Ant etc without any hassle. You can even run your test with any task scheduler.
  6. Powerful Analytics → Analytics gives us the ability to make the decisions and the real picture of AUT. SoapUI provide us various reports which are comprehensive in terms of the data yet not overwhelming for the user to understand. It also lets you to export the reports in any standard format and also gives us the option to customize them according to our needs.
    Every report has some standard metrics which can be grouped as per need basis too.
  7. Recording → SoapUI allows to record the all the data that is being sent and received between client and server. It even allows you to record the HTTP traffic. You can use the recorded messages to convert them into test cases.
  8. Great Open Source Support → It allows anyone to develop the SoapUI plugins for their application. Widely used open source softwares like IntelliJ, Net Beans, Eclipse has developed the plugins for SoapUI and your test can be seemingly integrated with them.

Now the very basic question that needs to be answered is how will it help me testing a Soap Web Service?

All you have to do is download this free tool and install it. The good thing is that it’s available for Winodws, Linux and Mac X OS. Once it’s installed and ready to use, you can start creating the Test Suite as well as test cases to test your web service. You can save all your test cases and use them any time in future. You can even parameterize your test cases so that you can test with different data sets without any over burden.

First Step (Installation)-> Visit the mirror site : http://sourceforge.net/projects/soapui/files/soapui/ and please download the latest version of SoapUI . Once you have the .exe file downloaded, run it. The Installation should be smooth.

First Interaction -> Well once the application will starts running, you will see the following GUI.

WSTesting 1
Here I have renamed the workspace as “testSoap”. You can right click the workspace and start creating your SOAP project.
Import WSDL: Every web service has to tell what services its offering, kind of arguments it can take, kind of response it will return, endpoint etc. It has been defined in a XML file with wsdl extension. WSDL stands for “Web Services Description Language”.

To test a SOAP web service, you have to add the corresponding wsdl file to the project. I have added a sample wsdl file to my project (refer below screen shot).

WSTesting 2

Once you’ve added the valid wsdl file and clicked “OK”, it will create the Project with the sample requests which are defined in the attached wsdl file. Then you can start adding your test suites, test cases and steps as per you testing needs.

Below is the screenshot that should help you understand how Workspace, Projects etc are organized in SoapUI.

WSTesting 3

Sample Request/Response -> Hereunder is the screenshot which is showing a sample request with sample response and the endpoint on which this request is sending.

WSTesting 4

Assertions: When we get the response of the request, it may contain some codes, some values etc., which we have to verify for each step to confirm whether the step is pass or fail based on the requirement. We can define assertion into each test step and soapUI will automatically mark the test step as pass/fail based on the assertion. We can define multiple assertions on a single step.

When you open the step into edit mode, at the bottom, you’ll find the “Assertions” beside “Request Log”. Click on this will open the assertion window.

WSTesting 5


The  WSTesting sign  sign lets you add the assertions. When you click on it, it opens the assertions drop down available for the test step. You can select one and click OK. It will pop up that assertion window with the relative options.

WSTesting 6

Like the above assertion is “Contains” and it lets you define any key word which the response should contain for marking it to pass. You can also define the regex and it will match the regex with the response.
Parameterization: while testing web services, you might want to have some values which would be same across for a test suite or even a test project for a given environment like the URL. So if you use hard coded values for them, you have to change those in all the requests every time you need to change. Parameterization of such values comes handy in those circumstances. We can define the variable at Project level, at Test suite level, Test case level or at Test step level based on you requirement.

Suppose you want to parameterize the URL and define that at Test suite level then just click on the desired test suite and then click on the “Properties” at the bottom left corner. It will show the properties window consisting two tabs “TestSuite Properties” and “Test Properties”.

TestSuite Properties – It consist the system defined properties of the test suite e.g. Name etc.

Test Properties – Here it shows the user defined properties. You can add/remove/change order/clear all/import/export properties. To add a new property, click on the “+” sign to open the model window where you can define the variable name and its value. I’ve added the variable “QA24-URL” and the url value to this variable.

WSTesting 7

Now where ever you want to use this variable, you have to use the following: ${#TestSuite#QA24-URL}

Running entire test suite in one GO and get the report: When you want to run the entire test suite in just one go and verify the result later, all you need to do is to select the desired test suite and click “Enter”. It will open the test suite editor window consisting all the test cases of the test suite with some options.

WSTesting 8

Just click on the green play icon and it will execute all the test cases one by one and display the results. You can click on the “TestSuite Log” button at the bottom and it will display the execution log of the test suite. If you have assert the test steps then it will display them in either “Red” of “Green” based on the assertion is failed or passed. If there is no assertion is defined then it will simply shows them as “Green” with “Finished” and you have to verify manually whether its pass or fail.

Hereunder is one of the suite and its execution status:
The main execution status lien is “Red” with FAILED as all the assertion defined is failed in the test suite.

The first two test cases have been marked as “Red” with FAILED because I’ve defined the “Contains” assertion to them with “OK” value. So it will check whether the response contains “OK” text or not. As both the response didn’t contain OK hence they both marked as failed. They are partially RED because of multiple test step defined into each test case and when first step failed then it didn’t run the next steps.

The last four test cases has been marked as “Green” with FINISHED as there is no assertion defined hence they are simple FINISHED. User has to check manually each response to verify whether it’s pass or fail.

Also you can see the execution log at the bottom and infer the possible root cause of the failure.

WSTesting 9

In my opinion, SoapUI is very vast in terms of functionalities its offering. To cover everything in one blog might not be a good idea.

I am planning to write at least couple of blogs to give a good hold on the SoapUI. My next blog, “Load Testing with SoapUI,” will give you a glimpse of load test capabilities of soapUI.