Achieving 100% code coverage. Testing getters and setters.

There’s been much debate about how much code should be tested. In particular, whether to test accessor methods (here, I’ll call accessor methods both getters and setters, i.e. methods to read or write an object’s property) has often been the subject of such debate.

While I myself think that testing effort should be devoted to more interesting things, others differ. Whether we agree or not, we might be working for a customer whose test coverage requirements are unsatisfiable without testing them. These requirements can be as high as 100%.

In such circumstances there’s an easy way to have your accessor methods tested, thanks to OpenPojo. Please, take a look at their wiki to have an idea of what this library is about.

Now let’s see a way to take advantage of it for our particular scenario.

First of all, we’ll need the following dependency if we’re using maven:

<dependency>
	<groupId>com.openpojo</groupId>
	<artifactId>openpojo</artifactId>
	<version>${openpojo.version}</version>
</dependency>

I’ll also include the Cobertura Maven plugin to check the test coverage later on:

<reporting>
	<plugins>
		<plugin>
			<groupId>net.sourceforge.cobertura</groupId>
			<artifactId>cobertura</artifactId>
			<version>${cobertura.version}</version>
		</plugin>
	</plugins>
</reporting>

Now, suppose we have a couple of data transfer objects like these:

public class Attachment {

	private String filename;
	private String base64Content;
	private String contentType;

	public String getFilename() {
		return filename;
	}

	public String getBase64Content() {
		return base64Content;
	}

	public String getContentType() {
		return contentType;
	}

	public void setFilename(final String filename) {
		this.filename = filename;
	}

	public void setBase64Content(final String contentBytes) {
		this.base64Content = contentBytes;
	}

	public void setContentType(final String contentType) {
		this.contentType = contentType;
	}
	[...]
public class MailMessage {

	private String from;
	private List<String> bcc;
	private List<String> cc;
	private String replyTo;
	private String subject;
	private String text;
	private List<String> to;
	private List<Attachment> attachments = new ArrayList<Attachment>();

	public String getFrom() {
		return from;
	}

	public List<String> getBcc() {
		return bcc;
	}

	public List<String> getCc() {
		return cc;
	}

	public String getReplyTo() {
		return replyTo;
	}

	public String getSubject() {
		return subject;
	}

	public String getText() {
		return text;
	}

	public List<String> getTo() {
		return to;
	}

	public void setFrom(final String from) {
		this.from = from;
	}

	public void setBcc(final List<String> bcc) {
		this.bcc = bcc;
	}

	public void setCc(final List<String> cc) {
		this.cc = cc;
	}

	public void setReplyTo(final String replyTo) {
		this.replyTo = replyTo;
	}

	public void setSubject(final String subject) {
		this.subject = subject;
	}

	public void setText(final String text) {
		this.text = text;
	}

	public void setTo(final List<String> to) {
		this.to = to;
	}

	public List<Attachment> getAttachments() {
		return attachments;

	}
	[...]

We’re going to write an OpenPojo accessor validator and call it from the JUnit test classes for both Data Transfer Objects.
In order to reuse code, we can write a single validator method in a utility class. This method will receive a single parameter: the class whose accessors will be tested.

public class PojoTestUtils {

	private static final Validator ACCESSOR_VALIDATOR = ValidatorBuilder.create()
													.with(new GetterTester())
													.with(new SetterTester())
													.build();

	public static void validateAccessors(final Class<?> clazz) {
		ACCESSOR_VALIDATOR.validate(PojoClassFactory.getPojoClass(clazz));
	}
}

Test classes can now use it:

public class AttachmentTest {

	@Test
	public void testAccesors_shouldAccessProperField() {

		PojoTestUtils.validateAccessors(Attachment.class);
	}
	[...]
public class MailMessageTest {

	@Test
	public void testAccesors_shouldAccessProperField() {

		PojoTestUtils.validateAccessors(MailMessage.class);
	}
	[...]

Time to check our test coverage:

mvn clean cobertura:cobertura

Finally, if you check the generated report you’ll see our longed-for 100% coverage in all its greenness.

Coverage Report

That’s all for today. Make sure you read OpenPojo’s documentation. There are more use cases you might be interested in.

Testing a cacheable method with Mockito

The Spring framework provides a simple way to cache the value returned by a method.
Basically, you annotate a method with @Cacheable and next time you call it with the same arguments, you get the cached value avoiding re-executing the method’s code.
In the example we’re going to use, the Spring cache infrastructure will be backed by Ehcache. These are the basic Maven dependencies we’ll need:

	<dependency>
		<groupId>org.springframework</groupId>
		<artifactId>spring-context</artifactId>
		<version>3.1.2.RELEASE</version>
	</dependency>
	<dependency>
		<groupId>net.sf.ehcache</groupId>
		<artifactId>ehcache-core</artifactId>
		<version>2.5.0</version>
		<scope>runtime</scope>
	</dependency>
	<dependency>
		<groupId>cglib</groupId>
		<artifactId>cglib</artifactId>
		<version>2.2.2</version>
		<scope>runtime</scope>
	</dependency>
	<dependency>
		<groupId>org.springframework</groupId>
		<artifactId>spring-test</artifactId>
		<version>3.1.2.RELEASE</version>
		<scope>test</scope>
	</dependency>
	<dependency>
		<groupId>junit</groupId>
		<artifactId>junit</artifactId>
		<version>4.11</version>
		<scope>test</scope>
	</dependency>
	<dependency>
		<groupId>org.mockito</groupId>
		<artifactId>mockito-all</artifactId>
		<version>1.9.5</version>
		<scope>test</scope>
	</dependency>

The method we’re going to cache is a service method that returns some (possibly expensive) database record count:

import org.springframework.cache.annotation.Cacheable;

public class CacheableService {

	private EntityDao entityDao;

	public int someEntityCount(final CriteriaDto criteria) {
		return entityDao.count(criteria);
	}
	[...]

The CriteriaDto class is as follows:

public class CriteriaDto {

	private String criterion1;
	private String criterion2;
	private String otherProperty;
	
	[...]

We want that method to return the cached value whenever it’s repeatedly invoked with the same criteria. To complicate matters further, let’s suppose that only the fields criterion1 and criterion2 affect the returned record count.
Those requirements can be met by preceding the method with the following annotation:

	@Cacheable(value = "entityCount", key = "{ #criteria.criterion1, #criteria.criterion2 }")
	public int someEntityCount(final CriteriaDto criteria) {
	[...]

As you can see, the relevant criteria for the cache can be specified via a SpEL expression in the key argument. The other argument, value is the name of the cache that will be used.

Assuming you’re using the venerable XML Spring configuration, you can set up the cache service with the following XML fragment in your application context:

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:aop="http://www.springframework.org/schema/aop"
	xmlns:context="http://www.springframework.org/schema/context"
	xmlns:jee="http://www.springframework.org/schema/jee"
	xmlns:tx="http://www.springframework.org/schema/tx"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:cache="http://www.springframework.org/schema/cache"
	xsi:schemaLocation="http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd   http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd   http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd   http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.1.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd
	http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd">

	 [...]
	 
	<cache:annotation-driven cache-manager="countCacheManager"/>

	<bean id="countCacheManager"
		  class="org.springframework.cache.ehcache.EhCacheCacheManager">
		  <property name="cacheManager" ref="ehcache"></property>
	</bean>

	<bean id="ehcache"
		  class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
		  <property name="configLocation" value="META-INF/count-ehcache.xml"></property>
	</bean>
	 [...]

Lastly, we need to provide the referenced Ehcache configuration file (count-ehcache.xml in this case):

<?xml version="1.0" encoding="UTF-8"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd">
	<cache 
	       name="entityCount"
	       maxElementsInMemory="10"
	       timeToLiveSeconds="300"
	       logging="true"
	       statistics="true"
	        />    
</ehcache>

The important configuration item here is the cache’s name attribute, which must match the value specified in the @Cacheable annotation.
Now it’s time to test the correct behaviour of our cached method.
The cache will work as expected if the actual method’s code is executed only the first time the method is called with equivalent criteria.
First of all, let’s prepare a test with two different but equivalent CriteriaDTO objects.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = "/META-INF/applicationContext.xml")
public class CacheableServiceTest {
	
	@Autowired
	private CacheableService serviceUnderTest;
	
	@Test
	public void testSomeEntityCountCalledTwice_shouldCallDaoMethodOnce() {
		
		// Local fixture set-up
		CriteriaDto criteria1 = new CriteriaDto();
		criteria1.setCriterion1("c1");
		criteria1.setCriterion2("c2");
		criteria1.setOtherProperty("c3a"); // Should be ignored by Cacheable annotation
		
		CriteriaDto criteria2 = new CriteriaDto();
		criteria2.setCriterion1("c1");
		criteria2.setCriterion2("c2");
		criteria2.setOtherProperty("c3b"); // Should be ignored by Cacheable annotation
	[...]

The class under test, CacheableService has a collaborating class: EntityDao. We’re going to substitute it with a mock.

	@Test
	public void testSomeEntityCountCalledTwice_shouldCallDaoMethodOnce() {
		
		// Local fixture set-up
		CriteriaDto criteria1 = new CriteriaDto();
		criteria1.setCriterion1("c1");
		criteria1.setCriterion2("c2");
		criteria1.setOtherProperty("c3a"); // Should be ignored by Cacheable annotation
		
		CriteriaDto criteria2 = new CriteriaDto();
		criteria2.setCriterion1("c1");
		criteria2.setCriterion2("c2");
		criteria2.setOtherProperty("c3b"); // Should be ignored by Cacheable annotation
		
		// Mock installation
		EntityDao mockEntityDao = Mockito.mock(EntityDao.class);
		serviceUnderTest.setEntityDao(mockEntityDao);
		[...]

Lastly, we’re going to call the method under test (someEntityCount) with both criteria objects. If everything works as expected, the second call will return the value cached in the first invocation, avoiding the call to the potentially expensive DAO method.
Since the DAO object used in the test is actually a mock object, we can use Mockito.verify to check its use. More specifically, we’ll use the optional VerificationMode parameter to make sure that the method has only been called once.
Here’s the full test method:

	@Test
	public void testSomeEntityCountCalledTwice_shouldCallDaoMethodOnce() {
		
		// Local fixture set-up
		CriteriaDto criteria1 = new CriteriaDto();
		criteria1.setCriterion1("c1");
		criteria1.setCriterion2("c2");
		criteria1.setOtherProperty("c3a"); // Should be ignored by Cacheable annotation
		
		CriteriaDto criteria2 = new CriteriaDto();
		criteria2.setCriterion1("c1");
		criteria2.setCriterion2("c2");
		criteria2.setOtherProperty("c3b"); // Should be ignored by Cacheable annotation
		
		// Mock installation
		EntityDao mockEntityDao = Mockito.mock(EntityDao.class);
		serviceUnderTest.setEntityDao(mockEntityDao);
		
		// First call with criteria1
		serviceUnderTest.someEntityCount(criteria1);
		
		// Second call with criteria2
		serviceUnderTest.someEntityCount(criteria2);
		
		// Verification
		Mockito.verify(mockEntityDao, Mockito.times(1)).count(Mockito.any(CriteriaDto.class));
	}

As you can see, we’ve used the method times to specify the number of times we expect the method to have been called during the test and any to tell Mockito that we don’t care about the actual parameters passed in the method invocations.

This is an easy way to test the behaviour of a cached method. It could be further refined in several ways: verifying the call with the right parameters, adding a method to test that the cached value is not returned when the criteria objects differ, etc.
I hope you find this article useful and, as usual, I’ll be glad to read your comments.

Mocking a constructor. A unit test for a factory method.

Introduction

In our previous post we have learnt the basics of using Mockito to write true unit tests.
Now we are going to unit test a factory method. The particular implementation used here is the one described in the traditional Data Access Object pattern for the DAO Factory.
In the process we’re going to stumble upon some Mockito’s limitations and we’ll see a way to overcome them.

The factory method

Let’s suppose we have a typical DAOFactory class exposing a factory method with the following signature:

public static DAOFactory getDAOFactory(Type type)

The method is meant to be called by clients in order to get a DAO Factory implementation for a particular database vendor. For instance, by calling DAOFactory.getDAOFactory(DAOFactory.Type.MARIADB) we’d get an instance of the class MariaDbDAOFactory, suited to work with a MariaDB database.

The test

Then let’s write a unit test for that scenario: obtaining a MariaDbDAOFactory object by calling the factory method with the type MARIADB.
First of all, we can establish the following success condition for the test: it will pass if and only if the method creates and return a MariaDbDAOFactory instance when it’s called with a MARIADB type as the parameter.
By following the naming conventions advised in xUnit Test Patterns, we can start writing our method like this:

@Test
public void testGetDAOFactoryMariaDb_shouldCreateAndReturnMariaDbDAOFactory() throws Exception {

	// XUnit Phase 1: Setup

	// XUnit Phase 2: Exercise

	// XUnit Phase 3: Verification

}

I have also written some comments as placeholders to better organise the testing code according to the XUnit phases (teardown phase is not necessary here).
Since we want a true unit test, we should mock the collaborating classes, like MariaDbDAOFactory.
When writing tests, most of the time we’ll be dealing with regular method calls on an instance of a collaborating class. But this case is different, since the interaction consists of invoking a constructor of the class to be mocked.
This is where the difficulty lies, since Mockito cannot mock constructors. We must adopt, therefore, a different approach.

Enter PowerMock

The framework PowerMock extends other mocking libraries with advanced features, like mocking constructors, static methods, final methods, etc.
One of the extended libraries happens to be Mockito. This allows us to utilise PowerMock while leveraging our experience with Mockito.
Now that we have the tool we need to mock constructors, let’s see how to use it.
Before going further, if you use Maven, these are the dependencies you’re going to need:

  	<dependency>
  		<groupId>org.powermock</groupId>
  		<artifactId>powermock-api-mockito</artifactId>
  		<version>1.5</version>
  		<scope>test</scope>
  	</dependency>
  	<dependency>
  		<groupId>org.powermock</groupId>
  		<artifactId>powermock-module-junit4</artifactId>
  		<version>1.5</version>
  		<scope>test</scope>
  	</dependency>

Now, let’s prepare the test class. PowerMock tests run with their own JUnit runner: PowerMockRunner.
Also, since PowerMock’s extended features requires byte-code manipulation, it’s also necessary to specify which classes will be prepared in that way. The annotation @PrepareForTest allows us to do it.
Now, our test class looks like this:

@RunWith(PowerMockRunner.class)
@PrepareForTest(DAOFactory.class)
public class DAOFactoryTest {
[...]

Note that the class to be prepared is the class invoking the constructor, and not the class to be instantiated.
Next, we need to record the mock behaviour, which we’ll do in two steps:

  1. Creating the mocked new instance of MariaDbDAOFactory. For this purpose we’ll use PowerMockito.mock instead of Mockito.mock
  2. Specifying how the constructor invocation is going to behave. Basically, we are going to tell PowerMock to return our recently created mocked instance whenever the constructor is called.

This is the actual code:

		// XUnit Phase 1: Setup
		MariaDbDAOFactory mockedDAOFactory = PowerMockito.mock(MariaDbDAOFactory.class);

		PowerMockito.whenNew(MariaDbDAOFactory.class)
					.withNoArguments()
					.thenReturn(mockedDAOFactory);

Now we can exercise the method under test (DAOFactory.getDAOFactory):

		// XUnit Phase 2: Exercise
		DAOFactory daoFactory = DAOFactory.getDAOFactory(DAOFactory.Type.MARIADB);

Lastly, we need to verify that the tested method has actually called the MariaDbDAOFactory constructor. We’ll use PowerMockito.verifyNew for that.
Additionally, we can verify that the object returned by the tested method is indeed the mocked DAO factory we created in the setup phase. A simple JUnit assertEquals will suffice.
The complete test class will look like this:

@RunWith(PowerMockRunner.class)
@PrepareForTest(DAOFactory.class)
public class DAOFactoryTest {

	@Test
	public void testGetDAOFactoryMariaDb_shouldCreateAndReturnMariaDbDAOFactory() throws Exception {

		// XUnit Phase 1: Setup
		MariaDbDAOFactory mockedDAOFactory = PowerMockito.mock(MariaDbDAOFactory.class);

		PowerMockito.whenNew(MariaDbDAOFactory.class)
					.withNoArguments()
					.thenReturn(mockedDAOFactory);

		// XUnit Phase 2: Exercise
		DAOFactory daoFactory = DAOFactory.getDAOFactory(DAOFactory.Type.MARIADB);

		// XUnit Phase 3: Verification
		PowerMockito.verifyNew(MariaDbDAOFactory.class).withNoArguments();
		Assert.assertEquals(mockedDAOFactory, daoFactory);
	}
}

Final remarks

In this article we’ve learnt how to mock a constructor with the framework PowerMock using a real-life but admittedly simple example. The reasons behind this choice are purely educational. In your day-to-day work you may judge a factory method too simple and well-established to be worth testing. Or you may prefer some integration testing (instead of unit testing) for this method, avoiding any mocking at all.
Secondly, I’ve been using the term mock object instead of test double, which, although widely used, would be consider incorrect by purists. I recommend checking this site or its associated book for more accurate definitions.
Lastly, don’t hesitate in using the comments section to share your thoughts with the rest of us.

Unit Testing with Mock Objects

Unit Testing

A Java application consists (roughly) of several classes interacting.
The first step towards a reliable application is, therefore, to have well tested individual classes. This is precisely the purpose of unit testing: testing units of code in isolation from other units. In our case, we’ll consider the Java class as the unit of code.
The main advantage of testing the class in isolation is making sure that the potential errors we can stumble upon are due to defects in the class under test and not caused by defects on collaborating classes.

Achieving the isolation

However, classes work in cooperation with other classes, so, how can we isolate the tested class?
Well, the idea is substituting the collaborating classes with compatible classes whose implementation is so trivial that we can be confident on their correctness.
The instances of those classes are known as test doubles and, very often, they’re also called mock objects even if this is a little bit of a misnomer.

What should we substitute

Substituting every single collaborating object with a test double would certainly be overkilling. I guess no one would even think of replacing a String object with a mock object.
Choosing the candidates to be replaced is a matter of common sense. Most people would not use test doubles in the following cases:

  • Classes with deterministic behaviour from trusted third parties. The aforementioned String object would certainly fall into this category.
  • Classes with a trivial implementation. Think of data transfer objects that are mere bunches of properties.

Easing the pain of writing test doubles

Once we have identified the candidates to be replaced, it’s time to write the test doubles. But wait!, thanks to several available mocking frameworks, we can avoid most of the work.
Among those frameworks, my favourite is Mockito, which I find very flexible and feature-rich.
Let’s see what Mockito can do for us in this first phase.
First of all, it can create a mock object from an existing type like this:

CustomerDao customerDaoDouble = Mockito.mock(CustomerDao.class);

After this line, methods from customerDaoDouble either do nothing (void methods) or just return a trivial value (null, false, 0, etc.).
Sometimes this is enough but usually we’ll need a specific behaviour for our test to work. This is another situation in which Mockito can help us. We can easily specify the behaviour of our test doubles’ methods like this:

Mockito.when(customerDaoDouble.findOne(nonexistentCustomerId)).thenReturn(null);
[...]
Mockito.when(customerDaoDouble.findOne(existingCustomerId)).thenReturn(existingCustomer);

Now, whenever we invoke findOne with the parameter nonexistentCustomerId we will get null, and whenever we call findOne with the parameter existingCustomerId we will get the object existingCustomer.
The action of specifying the test doubles’ behaviour is commonly known as setting the expectations.

Replacing the real object

Now we have our test double ready. We just need to use it in place of the real object. Let’s see how.
First of all, our class under test can cooperate with other classes in several ways:

  • An object of the collaborating class is used as a parameter in a method of our class.
    In this case we can just pass the test double as a parameter:

    	stackUnderTest.push(mockedItem);
    	
  • An object of the collaborating class is used as a field in our class.
    Then we’ll have to install our test double using some method, usually with a setter or a constructor. Or even assign it directly if the field is not private.
    But what, might you ask, if there is no accessible method to set the field? Well, in this case I’d suggest to just write a setter method or a constructor to set it.
    If you’re concerned about the field being publicly writeable by any class, you can always make the method protected and place the test class in the same package under the test tree.
    Lastly, some may argue about the suitability of writing a method just to be used by test classes, but remember that designing for testability is widely seen as a good development practice.

    	[...]
    	CustomerDao customerDaoDouble = Mockito.mock(CustomerDao.class);
    	// Installing the test double in place of the real customer DAO
    	customerServiceUnderTest.setCustomerDao(customerDaoDouble);
    	// Exercise some customerServiceUnderTest's method
    	[...]
    	
  • An object of the collaborating class is created inside a method.
    This kind of objects cannot be replaced by test doubles. If we need to do it, some refactoring will be necessary: promote the local object to field and provide a setter method for it.

Verification

The purpose of tests is to verify the correctness of our software. There are two ways we can do that:

  1. State verification. Since the execution of our software under test changes the state of the system, a straightforward verification strategy lies on checking the state of the modified objects. A typical example of this strategy is the verification through assertions of the object returned by a method.
    	Assert.assertEquals(expectedObject, stackUnderTest.pop());
    	
  2. Behaviour verification. A different approach to verification consists in ensuring that our tested class interacts with its collaborating classes in the expected way.
    This approach is of particular interest in this article, since it’s another area where Mockito can help us. In particular, the provided method verify makes this kind of verification pretty easy.
    In its simplest use, this method is used like this:

    	Mockito.verify(testDouble).expectedMethod(expectedParameters);
    	

    A JUnit test with such a verification will pass only if the method expectedMethod is invoked with the parameters expectedParameters.
    Let’s illustrate this with an example. Suppose we want to test a service class which uses a DAO in order to create a new customer.
    If we want our test to be a real unit test, we should isolate our service class by “mocking” the DAO object.
    In this scenario, our only possible verification strategy seems to be checking that the DAO’s creation method is invoked with the right parameters. Let’s see how:

    	[...]
    	// Creating the test double
    	CustomerDao customerDaoDouble = Mockito.mock(CustomerDao.class);
    	
    	// Installing the test double in place of the real customer DAO
    	customerServiceUnderTest.setCustomerDao(customerDaoDouble);
    	
    	// Call the service's creation method
    	customerServiceUnderTest.create(CUSTOMER_CODE, CUSTOMER_NAME);
    	
    	// Verification phase
    	Mockito.verify(customerDaoDouble).save(new CustomerDTO(CUSTOMER_CODE, CUSTOMER_NAME));
    	

    The previous test will fail if save is not invoked as a result of calling create or if it’s invoked with a parameter different [1] from the specified parameter (new CustomerDTO…).

Summary

The technique of testing with mock objects is often misunderstood.
A common mistake is forgetting to install the test double in place of the real object. In this scenario the test double remains unused and it’s the real object that gets called.
I’ve also seen tests exercising the mocked methods instead of the methods in the class under test. Since the implementation of the mocked methods is trivial, such tests are of no value.
But succeeding is really easy if you follow these steps in your use of Mockito:

  1. Create the test double (or mock object, stretching a little bit the meaning).
  2. Set up the double’s behaviour. Optional step, usually done when the methods return some value.
  3. Replace the real object with its corresponding test double.
  4. Exercise the software under test.
  5. Verify. It can be a traditional JUnit/Hamcrest assertion, a Mockito verification, or even a combination of both.

Lastly, some subjects have only been briefly addressed. I expect to elaborate a little bit on them in forthcoming articles. Stay tuned!

1. different according to equals, more on this in forthcoming articles

Concurrency testing with tempus-fugit

Development for a highly concurrent environment is a tough technique. A programme with a good coverage of passing unit tests can fail miserably in production due to concurrency issues.

In this post I’ll be discussing a pragmatic approach to concurrency testing with Junit tests and the tempus-fugit library.

This library provides us with many utilities that help dealing with typical concurrency aspects. You can read about those in the official documentation, but now I’m going to talk about a technique that I find particularly useful and easy to apply: what they call load/soak tests.

One of the typical characteristics of concurrency errors is the indeterminism. Errors are hard to reproduce because they depend on the threads running in a particular order.

The load/soak tests provided by tempus-fugit allow us to run a number of threads high enough to maximise the likelihood of raising any concurrency issues.

Let’s illustrate this with an example.

First of all, let start with a naïve class that performs a basic XML schema validation on an XML string  [1]:

public class MySchemaValidator {

	private Validator validator;

	public MySchemaValidator() {

		final String simpleSchema = "<?xml version='1.0'?>"
				+ "<xs:schema xmlns:xs='http://www.w3.org/2001/XMLSchema'>"
				+ "<xs:element name='note'>"
				+ "	<xs:complexType><xs:sequence>"
				+ "		<xs:element name='to' type='xs:string'/>"
				+ "		<xs:element name='from' type='xs:string'/>"
				+ "		<xs:element name='heading' type='xs:string'/>"
				+ "		<xs:element name='body' type='xs:string'/>"
				+ "	</xs:sequence></xs:complexType>"
				+ "</xs:element></xs:schema>";
		final StreamSource svgXsdSource = new StreamSource(new StringReader(simpleSchema));
		try {

			validator = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI)
							.newSchema(svgXsdSource).newValidator();

		} catch (final SAXException e) {
			// Not expected to happen
			e.printStackTrace();
		}
	}

	public boolean isValid(final String xml) {
		try {

			validator.validate(new StreamSource(new StringReader(xml)));
			return true;

		} catch (final Exception e) {
			e.printStackTrace();
			return false;
		}
	}
}

This class exhibits some characteristics that leave to potentical concurrency issues:

  • Mutable state. The class has a mutable instance variable (validator).
  • These data are shared and non-thread-safe. As an instance variable, it can be accessed by multiple threads. Furthermore, the Validator class is not thread-safe, as it’s stated in its javadoc.

Before digging deeper into the concurrency issues, let’s write some basic JUnit tests for the class:

  1. A test that make sure that our isValid method returns true when a valid XML string is received
  2. A test that make sure that our isValid methos returns false when an invalid XML string is received
public class MySchemaValidatorTest {

	private MySchemaValidator validatorUnderTest;

	@Before
	public void setUp() throws Exception {
		validatorUnderTest = new MySchemaValidator();
	}

	@Test
	public void testIsValidInvalidXml_shouldReturnFalse() throws IOException {

		final String invalidXml = "<html></html>";

		Assert.assertFalse(validatorUnderTest.isValid(invalidXml));
	}

	@Test
	public void testIsValidValidXml_shouldReturnTrue() throws IOException {

		final String validXml = "<?xml version=\"1.0\"?>"
				+ "<note>"
				+ "	<to>Tove</to>"
				+ "	<from>Jani</from>"
				+ "	<heading>Reminder</heading>"
				+ "	<body>Don't forget me this weekend!</body>"
				+ "</note>";

		Assert.assertTrue(validatorUnderTest.isValid(validXml));
	}
}

After running these tests, we confirm that everything looks OK. Or so we think.
(By the way, if anyone is intrigued about the name I gave to the test methods, I wholeheartedly recommend reading this gem about XUnit testing: xUnit Test Patterns by G. Meszaros.)

Unfortunately, If we were to use our class in a concurrent environment, we’d start to see some weird behaviour.

This is the moment to write some load/soak tests with the help of the tempus-fugit library. This involves just two easy steps:

  1. Including in our test class a JUnit rule provided by tempus-fugit: ConcurrentRule.
  2. Annotating our test methods with @Concurrent, which receives the number of desired concurrent threads as argument.

If we’re using Maven, we’ll need the following dependency to use the library:

    <dependency>
      <groupId>com.google.code.tempus-fugit</groupId>
      <artifactId>tempus-fugit</artifactId>
      <version>1.0</version>
      <scope>test</scope>
    </dependency>

Now we’re ready to write our concurrent tests:

public class MySchemaValidatorConcurrencyTest {

	@Rule
	public ConcurrentRule concurrently = new ConcurrentRule();

	private MySchemaValidator validatorUnderTest = new MySchemaValidator();

	@Test
	@Concurrent(count = 100)
	public void testIsValidInvalidXml_shouldReturnFalse() throws IOException {

		final String invalidXml = "<html></html>";

		Assert.assertFalse(validatorUnderTest.isValid(invalidXml));
	}

	@Test
	@Concurrent(count = 100)
	public void testIsValidValidXml_shouldReturnTrue() throws IOException {

		final String validXml = "<?xml version=\"1.0\"?>"
				+ "<note>"
				+ "	<to>Tove</to>"
				+ "	<from>Jani</from>"
				+ "	<heading>Reminder</heading>"
				+ "	<body>Don't forget me this weekend!</body>"
				+ "</note>";
		Assert.assertTrue(validatorUnderTest.isValid(validXml));
	}
}

You may have noticed that we have initialised the validatorUnderTest inlined instead of using a set-up method. The reason is to have all the 100 threads to access the same instance, instead of having a different object for each thread.

If we run our newly created test class, we’ll realise that the tests are failing now, as we’ve been fearing. The console will have shown messages like this one:

        FWK005 parse may not be called while parsing.

It’s time to change our original class in order to synchronise the access to the shared mutable field (validator).

public class MySchemaValidator {

	private volatile Validator validator;

	public MySchemaValidator() {

		final String simpleSchema = "<?xml version='1.0'?>"
				+ "<xs:schema xmlns:xs='http://www.w3.org/2001/XMLSchema'>"
				+ "<xs:element name='note'>"
				+ "	<xs:complexType><xs:sequence>"
				+ "		<xs:element name='to' type='xs:string'/>"
				+ "		<xs:element name='from' type='xs:string'/>"
				+ "		<xs:element name='heading' type='xs:string'/>"
				+ "		<xs:element name='body' type='xs:string'/>"
				+ "	</xs:sequence></xs:complexType>"
				+ "</xs:element></xs:schema>";
		final StreamSource svgXsdSource = new StreamSource(new StringReader(simpleSchema));
		try {

			validator = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI)
							.newSchema(svgXsdSource).newValidator();

		} catch (final SAXException e) {
			// Not expected to happen
			e.printStackTrace();
		}
	}

	public boolean isValid(final String xml) {
		try {
			final StreamSource xmlSource = new StreamSource(new StringReader(xml));

			synchronized (validator) {

				validator.validate(xmlSource);

			}
			return true;

		} catch (final Exception e) {
			e.printStackTrace();
			return false;
		}
	}
}

The most relevant changes are highlighted. Explaining the reasons behind these changes is not the goal of this post. If you need more information about it, this subject is clearly addressed on books like Effective Java or Concurent Programming in Java.

That’s all. Hopefully our concurrent tests are passing now and our class is ready for a highly concurrent environment.

I hope you find this post useful. Please, don’t hesitate in sharing your thoughts with the rest of us.

 

1. The sample XML schema and XML document are extracted from http://www.w3schools.com/xml/schema_intro.asp

Presentation

I am a regular software developer who has realised the potential of software testing to make reliable software. I intend to share with you some (hopefully) useful pieces of advice, which are based on my own experience and aimed at solving real-life problems. You are more than welcome to comment on them and enrich them with your own ideas.
My programming background is mostly Java-based and therefore most of the posts will be related to that language. Many of the techniques should be general enough to be widely applied, though.

I hope you find it useful.