testng-xml

Learn Quickly on How to Work With TestNG XML

What is TestNG XML? how does it look?

TestNG XML being the configuration file for TestNG is used to initiate TestNg tests. The structure of this XML file follows the standard DTD. As per this issue thread in StackOverflow, the easier way to generate a testng.xml is to convert the project to TestNG.

testng.xml is a conventional name and is by default generated by TestNG itself. However, we can create the testng.xml as per our need and rename as per our requirements. Once we become an expert, we do not need to rename it.

testng.xml can be used to-

  • define test suites and tests.
  • pass parameters to our test methods.
  • include packages, classes, independent test methods.
  • execute multiple tests in a multithreaded or single-threaded environment
  • use regular expression for inclusion or exclusion.

How to create testng.xml?

generate testng in xml
generate testng in XML
testng xml auto generation
TestNG XML auto-generation

The main code will look like below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
    <classes>
      <class name="com.com.test.NewTest"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

we can specify the package name as well. TestNG will check all the classes of the specified package and honors only classes that have TestNG Annotation.

The structure of testng.xml

The structure of testng xml
The structure of TestNG XML
  1. The suite is provided into one XML file and it can have one or multiple tests. A suite is defined as <suite> tag.
  2. A test is denoted with <test> tag . It may contain one or more TestNG classes.
  3. A Class is denoted by <Class> tag. Mainly it is a java class that has TestNg annotations in it. A TestNG class can contain one or several test methods.

Once we have got the basic structure of the testng.xml, we can now play around on this.

How to add N different tests in TestNG?

There are several ways to add N different tests in testng.xml depending on the conditions.

Add tests coming from different classes

In this case, we have tests written in different classes. We need to accumulate test methods and execute them via testNG.

let us create test classes:
MyGroupOfGroups class

package com.test;

import org.testng.annotations.Test;

public class MyGroupOfGroups {
	@Test(groups = { "slowRun" })
	public void f1() {
		System.out.println("F1- slow Run");
	}

	@Test(groups = { "slowRun" })
	public void f2() {
		System.out.println("F2- slow Run");
	}

	@Test(groups = { "fastRun" })
	public void f3() {
		System.out.println("F3- fast Run");
	}

	@Test(groups = { "fastRun" })
	public void f4() {
		System.out.println("F4- fast Run");
	}
}

let us create a dev code:

package com.dev;

public class TestString {
	private String token;
	public TestString(String token) {
		this.token=token;
	}
	public String printToConsole() {
		System.out.println(token);
		return token;
	}

}

the equivalent test code:

package com.test;

import org.testng.Assert;
import org.testng.annotations.Test;

import com.dev.TestString;

public class MyTestStringClass {
	String token="Test1";
	TestString ts=new TestString(token);
    @Test
    public void testPrintToConsole() {
    	Assert.assertEquals(token, ts.printToConsole());
    }
}

let us create one more class in development environment:

package com.dev;

import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;

public class FileLoader {
	private String filepath;
	public FileLoader() {}
	public FileLoader(String path) throws IOException {
		filepath=path;
		
	}
	public void loadAFile(String filepath) throws IOException {
		// TODO Auto-generated method stub
		BufferedReader br = new BufferedReader(new FileReader(filepath)); 
		  
		  String st; 
		  while ((st = br.readLine()) != null) 
		    System.out.println(st); 
		  } 
	

}

The test class is as follows:

package com.test;

import java.io.FileNotFoundException;
import java.io.IOException;

import org.testng.annotations.Test;

import com.dev.FileLoader;

public class MyExpectedExceptionTest {
	
String filePath="E:\\Test.txt";

@Test(expectedExceptions=FileNotFoundException.class)
public void testFileLoad() throws IOException {
	FileLoader fl= new FileLoader(filePath);
	fl.loadAFile(filePath);
}
}

the testng.xml will look like:

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MySuite" verbose="1"> 
    <test name = "MyGroupofGroupTest"> 
         <classes> 
           <class name = "com.test.MyGroupOfGroups"/> 
       </classes> 
    </test>
        <test name = "MyTestStringClass"> 
         <classes> 
           <class name = "com.test.MyTestStringClass"/> 
       </classes> 
    </test>
        <test name = "MyExpectedExceptionTest"> 
         <classes> 
           <class name = "com.test.MyExpectedExceptionTest"/> 
       </classes> 
    </test>
 </suite>

if we run the testng.xml we will get the below report
[RemoteTestNG] detected TestNG version 7.0.0
PASSED: test1
PASSED: test2
===============================================
Tests run: 2, Failures: 0, Skips: 0
===============================================

READ  Learn How to Work on Groups in TestNG Effectively

===============================================
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================
This is how we can add N number of tests into testng.xml.

Alternatively, we can add multiple test classes under one test tag. like:

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MySuite" verbose="1"> 
    <test name = "MyTest"> 
         <classes> 
           <class name = "com.test.MyGroupOfGroups"/> 
           <class name = "com.test.MyTestStringClass"/> 
           <class name = "com.test.MyExpectedExceptionTest"/> 
       </classes> 
    </test>
 </suite>

the output of the testng.xml execution will be as follows:
[RemoteTestNG] detected TestNG version 7.0.0
PASSED: test1
PASSED: test2

===============================================
Tests run: 2, Failures: 0, Skips: 0
===============================================

===============================================
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================

Add tests coming from different classes and they are from different packages

classes from different packages
classes from different packages

The testng can be configured as per our need

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MySuite" verbose="1"> 
    <test name = "MyTest"> 
         <classes> 
           <class name = "com.test.Test1Class"/> 
           <class name = "com.test2.Test2Class"/> 
           <class name = "com.test3.Test3Class"/> 
       </classes> 
    </test>
 </suite>

The output of the execution will be as follows:
[RemoteTestNG] detected TestNG version 7.0.0
This is test1
This is test2
This is test3

===============================================
MySuite
Total tests run: 3, Passes: 3, Failures: 0, Skips: 0
===============================================

Create tests by packages in TestNG

In this way, we only provide the package information to testng.xml to execute all methods of the packages.

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MyPackageTest" verbose="1"> 
    <test name = "MyPackageTest"> 
	<packages>
           <package name = "com.test2"/> 
           <package name = "com.test3"/> 
	</packages>
    </test>
 </suite>

The output of the execution is as follows:
[RemoteTestNG] detected TestNG version 7.0.0
This is test2
This is test3

===============================================
MyPackageTest
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================

TestNG execution report
TestNG execution report

In this case, all tests having @Test annotation in the classes of the packages.

How to use regular expression in package name?

We can use regular expression in the package naming in the testng.xml too!.

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MyPackageTest" verbose="1"> 
    <test name = "MyPackageTest"> 
	<packages>
           <package name = "com.test.*"/> 

	</packages>
    </test>
 </suite>

The execution result will look like below:
[RemoteTestNG] detected TestNG version 7.0.0
Before Suite
Before Test
F1- slow Run
F2- slow Run
F3- fast Run
F4- fast Run
Test1 is part of smoke, regression and normal groups
Test2 is part of smoke and normal
Test3 is part of regression and normal
Test4 is part of normal group
Test1
Before Class
Before Method
After Method
Before Method
After Method
Before Method
P1 test
After Method
Before Method
P2 test
After Method
Before Method
P3 test
After Method
After Class
This is test1
After Test
After suite

===============================================
MyPackageTest
Total tests run: 16, Passes: 16, Failures: 0, Skips: 0
===============================================

It has executed all @Test methods from all the packages.

execute packages in testng
execute packages in testng

Note: If there are any sub-packages inside these packages and if they have any @Test methods, they will also be executed.

How to add specific methods to testng.xml?

We can add specific methods to testng.xml too!!. For that, we need to provide the class name from where the method will come and the method names that need to be executed.

Also, we need to use include tag to get the method added to the execution list.

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MyMethod" verbose="1"> 
    <test name = "MyMethodTest"> 
	  <classes>
	   <class name="com.test2.Test2Class">
	      <methods>
	      <include name="myTest2"/>
	      </methods>
	   	</class>
	</classes>
    </test>
 </suite>
add method names in testng xml
add method names in testng XML

The report looks like below:

READ  Learn How To Install TestNG for IDEs Quickly
testNG specif method execution report
testNG specif method execution report

How to customize packages, classes, test methods in a single go in TestNG?

TestNG provides the ability to execute full package or classes or selective methods to execute via testng.xml.

In this case as my com.test package is having a lot of classes we want to execute the com.test.Test1Class and under that myTest1 method.

In com.test2 package, we have Test2Class. we want to execute all methods that are having @Test tag.

For the com.test3 package, we want to execute the test cases at the package level.

The testng.xml will look like below:

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MyTestForAll" verbose="1"> 
    <test name = "MyTestForAll"> 
    <packages>
       <package name="com.test3"/>
    </packages>
	  <classes>
	   <class name="com.test2.Test2Class"/>
	    <class name="com.test.Test1Class">
	      <methods>
	      <include name="myTest1"/>
	      </methods>
	   	</class>
	</classes>
    </test>
 </suite>
testng customize execution report
testng customize execution report

So testNG provides all sorts of flexibilities to customize testng.xml to add required details to it.

How to use include and exclude tags to add and remove packages in TestNG?

During the test suite creation, TestNG provides options to include or exclude certain tests. These tags are helpful to include or exclude certain sets of test cases.

Include package or packages

In this example, we will look at how to include all sub-packages of a parent package (com.*) with a regular expression. Then we will include only the com.test2 subpackage to execute.

The testng.xml will look like-

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MyIncludeTestPackage" verbose="1"> 
    <test name = "MyTestIncludePackage"> 
    <packages>
       <package name="com.*">
           <include name="com.test2"/>
       </package> 
    </packages>
	</test>
 </suite>

likewise, we can add multiple sub-packages.
The execution report will look like:

execution report of include tag
execution report of include tag

The console output is as follows:
[RemoteTestNG] detected TestNG version 7.0.0
This is test2

===============================================
MyIncludeTestPackage
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

Note include tag has a mandatory attribute called name. The corresponding value is the particular package name we want to include.

include attribute includes only the package or packages that have been asked to include. Hence the report only shows the methods excluded from com.test1 and com.test3 packages. All other methods belong to other packages are ignored.

Exclude package or packages

Similarly, we can exclude packages from test execution.The tag to exclude a package is <exclude name=”xxx”>. Once again name is the mandatory attribute of this tag.

The value of this attribute is the package or packages name or names, we want to skip during our test execution.  In that case, the testng.xml will be as follows:

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MyExcludeTestPackage" verbose="1"> 
    <test name = "MyTestExcludePackage"> 
    <packages>
       <package name="com.*">
           <exclude name="com.test"/>
       </package> 
    </packages>
	</test>
 </suite>

once we execute the test we will get the below report:
[RemoteTestNG] detected TestNG version 7.0.0
This is test2
This is test3

===============================================
MyIncludeTestPackage
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================

The screenshot is as follows:

exclude package execution report
exclude package execution report

Here, testng.xml accumulates all sub packages of com.* by using the regular expression but during execution, it ignores the com.test package.

 

How to use include and exclude tags to add and remove methods in TestNG?

During the test suite creation, TestNG provides options to include or exclude certain tests. These tags are helpful to include or exclude certain sets of test cases. Interestingly TestNG also supports to include or exclude test methods during test execution.

The cherry on the top feature is that it supports pattern patching feature to select or deselect methods by using a regular expression.

Include the method or methods

In order to include a method, we need to use <include name=”yyy”> tag where yyy stands for the method name that needs to be included. By using this tag we can include a particular test method from the class.

READ  Learn How to Work on Groups in TestNG Effectively

If we want to add some other method, we can create another <include name=”xxx”> tag to add the method in the execution list.

The testng.xml will look like-

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MyIncludeMethodTest" verbose="1"> 
    <test name = "MyIncludeMethodTest"> 
       <classes>
            <class name="com.test2.Test2Class">
       		<methods>
       			<include name="myTest2"/>
       		</methods>
       </class>
       </classes>
	</test>
 </suite>

The execution report will look like:

include method execution report
include method execution report

The console output is as follows:

[RemoteTestNG] detected TestNG version 7.0.0
This is test2

===============================================
MyIncludeMethodTest
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

As we have requested TestNG to include myTest2() method to execute. It has successfully executed the same.

Exclude the method or methods

In order to exclude method, we need to use <exclude name=”yyy”> tag where yyy stands for the method name that needs to be excluded. By using this tag we can exclude a particular test method from the class.

If we want to omit some other method, we can create another <exclude name=”xxx”> tag to add the method in the exclusion list.

The testng.xml will be as follows:

<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> 
 <suite name = "MyExcludeMethodTest" verbose="1"> 
    <test name = "MyExcludeMethodTest"> 
       <classes>
            <class name="com.test2.Test2Class">
       		<methods>
       			<exclude name="myTest2"/>
       		</methods>
       </class>
       </classes>
	</test>
 </suite>

The execution result will be as follows:

exclude method execution report
exclude method execution report

The console output will be as follows:
[RemoteTestNG] detected TestNG version 7.0.0
This is test4

===============================================
MyExcludeMethodTest
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

Note In the same class we have two methods myTest2() and myTest4(). But with the exclude tag we have requested TestNg to omit myTest2() method. Hence only myTest4() method got executed.

How to add or remove groups in testNG.xml?

We can define new groups in testng.xml with additional details attributes (like- if we want to run them in parallel, how many threads we will be using, if we are using JUnit tests etc.)

here is the how-to guide and discussion on groups.

How to Modify the testng.xml programmatically?

Sometimes, we need to update testng.xml dynamically via our code. TestNG provides the IAlterSuiteListener interface to trap the original testng.xml before even TestNG starts it execution.

We need to create a hook to implement this trap.

When it is useful?

Below are the few scenarios where the IAlterSuiteListener interface comes handy:

  1. Add listeners in the background.
  2. Add additional groups to execute.
  3. Configure advanced Jenkins executions.

In this way, we create an instance of the XmlSuite then by using code, we add remove the attributes we need.

package com.test3;
import java.util.List;
import org.testng.IAlterSuiteListener;
import org.testng.xml.XmlSuite;

public class MyListenerTest implements IAlterSuiteListener{
	public void alterSuite(List suites)
	{
		for(XmlSuite suite:suites) {
		suite.addIncludedGroup("smoke");
		}
	}
}

Conclusion

testng.xml is a very powerful feature of TestNG. With testng.xml we can perform various configurations. We have so far discussed the following topics

How to create testng.xml?
The structure of testng.xml
How to add N different tests in TestNG?
Add tests coming from different classes and they are from different packages
Create tests by packages in TestNG
How to use regular expression in package name?
How to add specific methods to testng.xml?
How to customize packages, classes, test methods in a single go in TestNG?
How to use include and exclude tags to add and remove packages in TestNG?
How to use include and exclude tags to add and remove methods in TestNG?
Exclude the method or methods.

If you enjoyed this post, please share it on social media ?What am I missing here? Let me know in the comments and I’ll add it in!”

Share and Enjoy !

Leave a Comment

Your email address will not be published. Required fields are marked *