Concurrency testing with tempus-fugit

Development for a highly concurrent environment is a tough technique. A programme with a good coverage of passing unit tests can fail miserably in production due to concurrency issues.

In this post I’ll be discussing a pragmatic approach to concurrency testing with Junit tests and the tempus-fugit library.

This library provides us with many utilities that help dealing with typical concurrency aspects. You can read about those in the official documentation, but now I’m going to talk about a technique that I find particularly useful and easy to apply: what they call load/soak tests.

One of the typical characteristics of concurrency errors is the indeterminism. Errors are hard to reproduce because they depend on the threads running in a particular order.

The load/soak tests provided by tempus-fugit allow us to run a number of threads high enough to maximise the likelihood of raising any concurrency issues.

Let’s illustrate this with an example.

First of all, let start with a naïve class that performs a basic XML schema validation on an XML string  [1]:

public class MySchemaValidator {

	private Validator validator;

	public MySchemaValidator() {

		final String simpleSchema = "<?xml version='1.0'?>"
				+ "<xs:schema xmlns:xs='http://www.w3.org/2001/XMLSchema'>"
				+ "<xs:element name='note'>"
				+ "	<xs:complexType><xs:sequence>"
				+ "		<xs:element name='to' type='xs:string'/>"
				+ "		<xs:element name='from' type='xs:string'/>"
				+ "		<xs:element name='heading' type='xs:string'/>"
				+ "		<xs:element name='body' type='xs:string'/>"
				+ "	</xs:sequence></xs:complexType>"
				+ "</xs:element></xs:schema>";
		final StreamSource svgXsdSource = new StreamSource(new StringReader(simpleSchema));
		try {

			validator = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI)
							.newSchema(svgXsdSource).newValidator();

		} catch (final SAXException e) {
			// Not expected to happen
			e.printStackTrace();
		}
	}

	public boolean isValid(final String xml) {
		try {

			validator.validate(new StreamSource(new StringReader(xml)));
			return true;

		} catch (final Exception e) {
			e.printStackTrace();
			return false;
		}
	}
}

This class exhibits some characteristics that leave to potentical concurrency issues:

  • Mutable state. The class has a mutable instance variable (validator).
  • These data are shared and non-thread-safe. As an instance variable, it can be accessed by multiple threads. Furthermore, the Validator class is not thread-safe, as it’s stated in its javadoc.

Before digging deeper into the concurrency issues, let’s write some basic JUnit tests for the class:

  1. A test that make sure that our isValid method returns true when a valid XML string is received
  2. A test that make sure that our isValid methos returns false when an invalid XML string is received
public class MySchemaValidatorTest {

	private MySchemaValidator validatorUnderTest;

	@Before
	public void setUp() throws Exception {
		validatorUnderTest = new MySchemaValidator();
	}

	@Test
	public void testIsValidInvalidXml_shouldReturnFalse() throws IOException {

		final String invalidXml = "<html></html>";

		Assert.assertFalse(validatorUnderTest.isValid(invalidXml));
	}

	@Test
	public void testIsValidValidXml_shouldReturnTrue() throws IOException {

		final String validXml = "<?xml version=\"1.0\"?>"
				+ "<note>"
				+ "	<to>Tove</to>"
				+ "	<from>Jani</from>"
				+ "	<heading>Reminder</heading>"
				+ "	<body>Don't forget me this weekend!</body>"
				+ "</note>";

		Assert.assertTrue(validatorUnderTest.isValid(validXml));
	}
}

After running these tests, we confirm that everything looks OK. Or so we think.
(By the way, if anyone is intrigued about the name I gave to the test methods, I wholeheartedly recommend reading this gem about XUnit testing: xUnit Test Patterns by G. Meszaros.)

Unfortunately, If we were to use our class in a concurrent environment, we’d start to see some weird behaviour.

This is the moment to write some load/soak tests with the help of the tempus-fugit library. This involves just two easy steps:

  1. Including in our test class a JUnit rule provided by tempus-fugit: ConcurrentRule.
  2. Annotating our test methods with @Concurrent, which receives the number of desired concurrent threads as argument.

If we’re using Maven, we’ll need the following dependency to use the library:

    <dependency>
      <groupId>com.google.code.tempus-fugit</groupId>
      <artifactId>tempus-fugit</artifactId>
      <version>1.0</version>
      <scope>test</scope>
    </dependency>

Now we’re ready to write our concurrent tests:

public class MySchemaValidatorConcurrencyTest {

	@Rule
	public ConcurrentRule concurrently = new ConcurrentRule();

	private MySchemaValidator validatorUnderTest = new MySchemaValidator();

	@Test
	@Concurrent(count = 100)
	public void testIsValidInvalidXml_shouldReturnFalse() throws IOException {

		final String invalidXml = "<html></html>";

		Assert.assertFalse(validatorUnderTest.isValid(invalidXml));
	}

	@Test
	@Concurrent(count = 100)
	public void testIsValidValidXml_shouldReturnTrue() throws IOException {

		final String validXml = "<?xml version=\"1.0\"?>"
				+ "<note>"
				+ "	<to>Tove</to>"
				+ "	<from>Jani</from>"
				+ "	<heading>Reminder</heading>"
				+ "	<body>Don't forget me this weekend!</body>"
				+ "</note>";
		Assert.assertTrue(validatorUnderTest.isValid(validXml));
	}
}

You may have noticed that we have initialised the validatorUnderTest inlined instead of using a set-up method. The reason is to have all the 100 threads to access the same instance, instead of having a different object for each thread.

If we run our newly created test class, we’ll realise that the tests are failing now, as we’ve been fearing. The console will have shown messages like this one:

        FWK005 parse may not be called while parsing.

It’s time to change our original class in order to synchronise the access to the shared mutable field (validator).

public class MySchemaValidator {

	private volatile Validator validator;

	public MySchemaValidator() {

		final String simpleSchema = "<?xml version='1.0'?>"
				+ "<xs:schema xmlns:xs='http://www.w3.org/2001/XMLSchema'>"
				+ "<xs:element name='note'>"
				+ "	<xs:complexType><xs:sequence>"
				+ "		<xs:element name='to' type='xs:string'/>"
				+ "		<xs:element name='from' type='xs:string'/>"
				+ "		<xs:element name='heading' type='xs:string'/>"
				+ "		<xs:element name='body' type='xs:string'/>"
				+ "	</xs:sequence></xs:complexType>"
				+ "</xs:element></xs:schema>";
		final StreamSource svgXsdSource = new StreamSource(new StringReader(simpleSchema));
		try {

			validator = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI)
							.newSchema(svgXsdSource).newValidator();

		} catch (final SAXException e) {
			// Not expected to happen
			e.printStackTrace();
		}
	}

	public boolean isValid(final String xml) {
		try {
			final StreamSource xmlSource = new StreamSource(new StringReader(xml));

			synchronized (validator) {

				validator.validate(xmlSource);

			}
			return true;

		} catch (final Exception e) {
			e.printStackTrace();
			return false;
		}
	}
}

The most relevant changes are highlighted. Explaining the reasons behind these changes is not the goal of this post. If you need more information about it, this subject is clearly addressed on books like Effective Java or Concurent Programming in Java.

That’s all. Hopefully our concurrent tests are passing now and our class is ready for a highly concurrent environment.

I hope you find this post useful. Please, don’t hesitate in sharing your thoughts with the rest of us.

 

1. The sample XML schema and XML document are extracted from http://www.w3schools.com/xml/schema_intro.asp

Advertisements

3 thoughts on “Concurrency testing with tempus-fugit

  1. Howdy! This post couldn’t be written much better!
    Looking through this post reminds me of my previous roommate!

    He constantly kept preaching about this. I most certainly will send
    this article to him. Fairly certain he will have a good read.
    Thank you for sharing!

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s