TestNG - multithreaded testing

1, Foreword

With the continuous accumulation of automation assets within the enterprise and the promotion of continuous integration, the requirements for the running speed of automated testing are becoming higher and higher. There may be thousands of automated test cases for some applications. How to accelerate the running speed of these automated test cases and implement automation into each version change and continuous integration is a problem to be solved.

2, Multithreading posture in TestNG

1. Concurrency in test methods

Sometimes, we need to perform concurrent tests on a test case, such as an http interface, that is, repeated calls of an interface. TestNG also provides elegant support.

@Test(threadPoolSize = 5,invocationCount = 10)
    public void test1(){
        Assert.assertEquals(2,2);
        System.out.println("this is thread"+ Thread.currentThread().getId());
    }

Description:   threadPoolSize indicates the thread pool capacity used to call the method. In this example, five threads execute the method in parallel at the same time; invocationCount indicates the total number of times the method needs to be executed.

In this example, five threads execute simultaneously. When the total number of executions reaches 10, it stops. The execution results are as follows:

  2. Concurrency in XML

Generally, during the execution of TestNG, the test level can be divided into Suite - > test - > class - > method from top to bottom. The relationship between the left element of the arrow and the right element is a one to many inclusion relationship.

The Test here refers to the test tag in testng.xml, not a @ Test in the Test class. A @ Test in the Test class actually corresponds to the method here. Therefore, when we use @ BeforeSuite, @ BeforeTest, @ BeforeClass, @ BeforeMethod, their actual execution order is also based on this level.

1) Test, class and method level concurrency can be set under suite tag in testng.xml, such as:

<suite name="Testng Parallel Test" parallel="tests" thread-count="5">
<suite name="Testng Parallel Test" parallel="classes" thread-count="5">
<suite name="Testng Parallel Test" parallel="methods" thread-count="5">
Note: what they have in common is that up to five threads can execute different use cases at the same time.

Their differences are as follows:

  • tests level: use cases under different test tags can be executed in different threads, and use cases under the same test tag can only be executed in the same thread.
  • classes level: use cases under different class tags can be executed in different threads. Use cases under the same class tag can only be executed in the same thread.
  • methods level: all use cases can be executed in different threads.

It is very important to understand the level of concurrency. It can help us reasonably organize use cases. For example, non thread safe test classes or group s can be unified into one test. In this way, we can ensure that the use cases in these classes are executed by single thread while concurrency. You can also set class level concurrency as needed to let the use cases in the same test class execute in the same thread.

2) Example demonstration

Test code: TestMethods.java

package com.test.extent;
import org.testng.Assert;
import org.testng.Reporter;
import org.testng.annotations.Test;

public class TestMethods {

    @Test
    public void test1(){
        Assert.assertEquals(2,2);
        System.out.println("this is thread"+ Thread.currentThread().getId());
    }

    @Test
    public void test2(){
        Assert.assertEquals(1,1);
        System.out.println("this is thread"+ Thread.currentThread().getId());
    }

    @Test
    public void test3(){
        Assert.assertEquals("sss","sss");
        System.out.println("this is thread"+ Thread.currentThread().getId());
    }

}

xml settings: tests level concurrency example

<?xml version="1.0" encoding="UTF-8" ?>

<suite name="Suite1" parallel="tests" thread-count="2">
    <test name="Test case 1">
        <classes>
            <class name="com.test.extent.TestMethods"></class>
        </classes>
    </test>
    <test name="Test case 2">
        <classes>
            <class name="com.test.extent.TestMethods"></class>
        </classes>
    </test>
    <test name="Test case 3">
        <classes>
            <class name="com.test.extent.TestMethods"></class>
        </classes>
    </test>
</suite>

Note: during the execution of the current test plan, separate threads are used for the execution of each test case (referring to < test > in xml) (the test methods in the test case share one thread),

At most two threads are concurrent. The execution results are as follows: < test name = "test case 1" > the thread IDs of the three methods are 16

xml settings: classes level concurrency example

<?xml version="1.0" encoding="UTF-8" ?>

<suite name="Suite1" parallel="classes" thread-count="2">
    <test name="Test case 1">
        <classes>
            <class name="com.test.extent.TestMethods"></class>
            <class name="com.test.extent.TestMethods2"></class>
        </classes>
    </test>
</suite>

Note: during the execution of the current test plan, separate threads are used for the execution of each test class (referring to < class > in xml) (the test methods in the test class share one thread), with a maximum of 2 concurrent threads. The execution results are as follows:

xml settings: methods level concurrency example

<?xml version="1.0" encoding="UTF-8" ?>

<suite name="Suite1" parallel="methods" thread-count="2">
    <test name="test1">
        <classes>
            <class name="com.test.extent.TestMethods"></class>
        </classes>
    </test>
</suite>

Note: during the execution of the current Test plan, separate threads are used for the execution of each Test method (referring to each @ Test), with a maximum of 2 concurrent threads. The execution results are as follows:

 

Posted on Sun, 05 Dec 2021 10:16:54 -0500 by mitwess