In this blog, I’ll describe how to load test web services using the SoapUI. But first let me explain the different types of load testing that are executed in AUT.
Load testing, or performance testing, is actually simulating multiple users load on the AUT and see how the application fares under such loads.
Here are some performance testing types that we generally come across.
Now let’s start using the SoapUI (Free version) and see how we can use it to do a load testing of web service.
Adding the Load Test to your existing test case → Open the SoapUI and select any existing testcase, then right-click on it. Select “New Load Test” from the options (or press “Crtl + N”). It will show a dialogue box asking you to enter the name of load test.
Once you click “OK” it will add this test and you’ll see the following screen.
As you can see, there are 4 steps added under “Test Step” because the test case that I chose has 4 steps. Also notice all steps except the first are marked as disabled because I disabled those in the test case. If you want to run all or some steps in your load test, you can simply enable/disable them in the test case by right clicking and selecting the appropriate option.
Now let’s check out the different buttons/options present in the window.
The first icon in the top left corner is a “Play” button icon that runs the load test. The second icon is used to stop the current execution. The next 4 icons are related to Statistics graphs, and then there is the option icon. Following those icons is the icon used to convert this test into a loadUI test case, and the last icon is help.
Limit will allow you to set the limit in seconds, total runs, or runs per thread, so you can choose how your test will run.
The next Box shows the percentage of the current test execution.
Threads allow us to set the number of virtual users we want to run this test. The default value is 5.
Strategy – This is how we want to run the load test and it should be chosen based on the requirements. Let’s see the different strategies and their significance.
When the test is running, the statistics are collected and displayed under the following headers:
Load Test Assertions: Assertions measure the actual values against the defined values for the test step and log the number of assertions errors under “err”. We can add assertions to the test steps by right clicking on the step and selecting “Add Assertions”. We have to choose the assertion type while adding the assertion from the given list. The following are the assertion types:
1. Step TPS: This assertion is used to verify the Transaction per second of the test step. When you select this type, a model window opens for this assertion and has the following options—
2. Max Errors: This assertion is used to verify the total errors that the test step should not exceed the specified value. When you select this type, a model window opens for this assertion with the following options (skipping the common options mentioned above) —
3. Step Average: This assertion is used to verify that the averages of the test step should not exceed the specified value. When you select this type, a model window opens for this assertion with the following options (skipping the common options mentioned above) —
4. Step Maximum: This assertion is used to verify that the time taken by the test step should not exceed the specified value. When you select this type, a model window opens for this assertion having the following options (skipping the common options mentioned above) —
5. Step Status: This assertion is used to verify that the test step is not failed with an error. When you select this type, a model window opens for this assertion with the following options (skipping the common options mentioned above) —
There are Log sub windows in this window in which you can see the logs.
Let me walk you through a real time example that I used in my testing. My application lets me add a new entity through the create request and I can update that entity via an update request. I have 3 update steps, so I’ve disabled 2 update steps because they have nothing different in terms of functionality.
I am just doing the baseline test with 1, 5, 10, 20, and 50 users for a 1-minute duration with simple strategy and 1-second delay (I have not randomized the test).
Step 1: Execute with 1 thread. Here is the result:
So we can see that create took an average of 2.5 seconds and update took around 2 seconds. This looks OK with 1 user load and with the AUT running on the VM, but this is not the actual load test environment.
Step 2: Execute with 5 threads. Here is the result:
So we can see that create took an average of 5 seconds and update also took around 5 seconds. This is almost double the result of 1 user.
Step 3: Execute with 10 threads. Here is the result:
Now the average time to create has gone to around 11 seconds and for update its 10 seconds. Also notice that the max time for create and update has gone up to 18 and 14 seconds respectively. Now we can see that even for 10 seconds, the time is very high as compared to 1 and 5 users.
Step 4: Execute with 20 threads. Here is the result:
Here the average time to create is around 16 seconds, but the average time to update has gone up to 40 seconds. This gives us an understanding that our application update function is not performing well under even 20 user loads as compared to the create function.
At this point, we know that we should not run the test for 50 users and we can see the performance degradation with 20 users for update. We can identify these kinds of issues earlier with baseline test, as shown above, which really helps developers to fix the major performance issues at this stage itself.