Testing JMS bridge to IBM MQ with Spring Boot

Not invented here syndrome gained recognition in software development, because it’s relatively common phenomenon. It has its roots in psychology, where it was identified as knowledge communication problems of decision makers (Martin, E. (2007). Knowledge Communication Problems between Experts and Decision Makers: an Overview and Classification. Electronic Journal of Knowledge Management, 5(3), pp.291-300.). Specific technical choices of other people comprise an easy target for complaints, especially when artifacts that were picked are not state-of-the-art anymore.
The fact that architectural decisions were made elsewhere is a lousy excuse for writing low-quality code. Bare in mind that low-quality comes in different shapes and sizes. One of them is related to tests. Writing poor tests or no tests at all is a repugnant practice. But it’s tempting to blame proprietary middleware messaging system for missing integration tests – setting up additional broker is costly, mundane, cumbersome, etc. A simple solution can be to rely on standardized API for communication and then replace target system with the one you have control over.
Nelson Elhage recently described how he writes tests. In his post he encourages to write lots of fakes, meaning to mock complex things that are not crucial to current testing subject. As an example he provides an idea of in-memory mock for Amazon S3 storage. The same concept can be applied to messaging middleware.
Apache Kafka has an embedded version of broker included in test module out of the box. You can set up your tests to communicate with full-blown Kafka broker which happens to reside in memory. No such facility exists for IBM Websphere MQ, however equipped with JMS interfaces one can take advantage of a system that is a bit more flexible.
In one of my previous posts I shortly described, how to get started with IBM MQ with Spring Boot and JMS. Having all classes designed for testability greatly reduces the effort needed to wire up a working example. Some hints on testing IBM Websphere MQ with Spring Boot were put together previously, however using HornetQ with Spring Boot became deprecated. The current post can be treated as an update.
Configuration is divided into two classes. One is tightly coupled with IBM Websphere MQ and defines a connection factory. The other configuration class defines a number of messaging components that depend on aforementioned factory.
There’s no need for running IBM MQ instance in tests. An in-memory queuing solution can be used, for example Apache ActiveMQ Artemis in embedded mode. Spring Boot provides a starting point for Artemis as starter dependency, which allows automatic detection of dependencies in classpath, important components instantiation and handful of autowiring capabilities. Excellent for testing purposes with low plumbing overhead.
To get started with Artemis the following dependencies have to declared.

dependencies {
	// ...
	testCompile 'org.springframework.boot:spring-boot-starter-artemis'
	testCompile 'org.apache.activemq:artemis-jms-server:1.3.0'
	testCompile 'org.springframework.boot:spring-boot-starter-test'
	testCompile 'org.awaitility:awaitility:2.0.0'
}

As a next step the code must somehow indicate that Artemis is supposed to run in embedded mode. To achieve the goal, an entry in application.properties must be added.

spring.artemis.mode=embedded
# ...

When it comes to configuration classes, only non-IBM-related one can be picked. It will rely on Spring Boot to provide default connection factory to an in-memory queue. We can even extend the configuration class and replace specific beans with the implementation that fits tests better.

@Configuration
static class TestConfiguration extends MQConfiguration {
    List<String> receivedMessages = new CopyOnWriteArrayList<>();

    @Override
    @Bean
    public Consumer<String> messageConsumer() {
        return receivedMessages::add;
    }

    public boolean hasReceivedMessages() {
        return !receivedMessages.isEmpty();
    }

}

SpringBootTest annotation allows limiting set of classes which contains beans definition, so that a single test class populates Spring context only with significant components. It makes tests faster and less error-prone.

@RunWith(SpringRunner.class)
@SpringBootTest(classes = { MQGatewayIntegrationTest.TestConfiguration.class, MQGateway.class, MQProperties.class })
@EnableAutoConfiguration
@EnableJms
public class MQGatewayIntegrationTest {
    @Inject
    JmsTemplate jmsTemplate;

    @Inject
    TestConfiguration configuration;

    @Value("${pl.ciruk.blog.mq.incoming-queue}")
    String queue;

    @Test
	public void shouldReceiveMessageInListener() throws Exception {
        String message = "This is a test message";

        jmsTemplate.convertAndSend(queue, message);
        await().atMost(5, TimeUnit.SECONDS)
                .until(configuration::hasReceivedMessages);

        assertThat(configuration.receivedMessages, contains(message));
    }

    // ...
}

Complete code can be found on Github.

What can’t JIT do?

Just-In-Time compiler is a part of HotSpot’s execution engine responsible for compiling hot methods to native code. It comes in two flavours which differ in compilation pace and optimization methods.
Client compiler (C1) starts compiling sooner to achieve better performance in short term. However, it makes decision based on constrained data – which can potentially degrade performance in the long run.
Server compiler (C2) collects data longer, thus it’s able to detect hot spots with better probability. Although, it’s not well-suited for short-running applications.
Commencing the release of Java 7, tiered compilation was introduced to combine client and server compilers. Such combination does not solve all the performance problems related to compilation. In specific scenarios better results are obtained by having this feature disabled.
You can take a peek under the hood of JIT by instrumenting the JVM to be a bit more verbose. The following set of parameters will result in compilation log file creation:

-XX:+UnlockDiagnosticVMOptions
-XX:+LogCompilation
-XX:+TraceClassLoading
-XX:+PrintAssembly

The file can be read by talented individuals or interpreted and presented to mere humans by handy tool called JITwatch.

Generated binary code resides in non-heap memory region called code cache. The size of code cache is limited, which means that JVM sometimes has to remove already compiled methods in order to make room for the hotter ones.
Shrinking free space is, however, only one reason for de-optimization. JIT can decide on its own, that optimization techniques applied to given method are in fact not helpful. Most optimizations introduced by C2 are speculative.
Having said that, JIT is robust and comprehensive – it has nearly 100 optimizations under its belt. Among them there is an entire group dedicated to removing instructions: autobox elimination, dead code elimination, null check elimination, etc. What JIT cannot do is to eliminate loop when termination expression (i.e. boolean condition in the loop) involves 64-bit value comparison. In that case, JIT fails to eliminate even empty loops.

For loops are directly translated to bytecode instructions. The following listings present bytecode for sample empty loops when iterated over integer and long value.
They seem rather straightforward. In case of iteration which uses integer as an incremented value, JIT has no problems eliminating empty loop. Consider the listing below. The code was run with -XX:+PrintCompilation flag, which produces peculiar output.
After the first iteration, a call to intLoop is removed.

@Test(timeout = 100_000L)
public void shouldEliminateEmptyIntLoop() throws Exception {
	for (int i = 0; i < 100; i++) {
		runAndMeasureTime(this::intLoop);
	}
}

private void intLoop() {
	for (int i = 0; i < 20_000_000; i++) {
		// no-op
	}
}
    236  382 %     3       pl.ciruk.blog.jitest.EmptyLoopTest::intLoop @ 2 (15 bytes)
    236  383       3       pl.ciruk.blog.jitest.EmptyLoopTest::intLoop (15 bytes)
    236  384 %     4       pl.ciruk.blog.jitest.EmptyLoopTest::intLoop @ 2 (15 bytes)
    237  382 %     3       pl.ciruk.blog.jitest.EmptyLoopTest::intLoop @ -2 (15 bytes)   made not entrant
    237  384 %     4       pl.ciruk.blog.jitest.EmptyLoopTest::intLoop @ -2 (15 bytes)   made not entrant
2 ms
    237  385 %     4       pl.ciruk.blog.jitest.EmptyLoopTest::intLoop @ 2 (15 bytes)
    238  386       4       pl.ciruk.blog.jitest.EmptyLoopTest::intLoop (15 bytes)
1 ms
    238  383       3       pl.ciruk.blog.jitest.EmptyLoopTest::intLoop (15 bytes)   made not entrant
0 ms   
0 ms
0 ms
0 ms
...

However, when long values come into play, JIT seems helpless.

@Test(timeout = 100_000L)
public void shouldEliminateEmptyLongLoop() throws Exception {
	for (int i = 0; i < 100; i++) {
		runAndMeasureTime(this::longLoop);
	}
}
private void longLoop() {
	for (long i = 0; i < 20_000_000L; i++) {
		// no-op
	}
}
    231  391 %     3       pl.ciruk.blog.jitest.EmptyLoopTest::longLoop @ 2 (18 bytes)
    231  392       3       pl.ciruk.blog.jitest.EmptyLoopTest::longLoop (18 bytes)
    231  393 %     4       pl.ciruk.blog.jitest.EmptyLoopTest::longLoop @ 2 (18 bytes)
    231  391 %     3       pl.ciruk.blog.jitest.EmptyLoopTest::longLoop @ -2 (18 bytes)   made not entrant
    242  393 %     4       pl.ciruk.blog.jitest.EmptyLoopTest::longLoop @ -2 (18 bytes)   made not entrant
12 ms
    243  394 %     4       pl.ciruk.blog.jitest.EmptyLoopTest::longLoop @ 2 (18 bytes)
    243  395       4       pl.ciruk.blog.jitest.EmptyLoopTest::longLoop (18 bytes)
    244  392       3       pl.ciruk.blog.jitest.EmptyLoopTest::longLoop (18 bytes)   made not entrant
10 ms
6 ms
7 ms
7 ms
6 ms
8 ms
...

The same situation with JIT struggling to help, but without any result, can be observed when the value in a loop is an integer, but the termination clause contains comparison to a long one.

@Test(timeout = 100_000L)
public void shouldEliminateEmptyIntToLongLoop() throws Exception {
	for (int i = 0; i < 100; i++) {
		runAndMeasureTime(this::intToLongLoop);
	}
}
private void intToLongLoop() {
	for (int i = 0; i < 20_000_000L; i++) {
		// no-op
	}
}

This strange misbehavior is really hard to understand. Bytecode generated by javac in case of integer in the loop is similar to the one generated for iterating over long.

void loop();
    descriptor: ()V
    flags:
    Code:
      stack=2, locals=2, args_size=1
         0: iconst_0
         1: istore_1
         2: iload_1
         3: ldc           #2                  // int 20000000
         5: if_icmpge     14
         8: iinc          1, 1
        11: goto          2
        14: return

The only difference is the way local variable gets incremented. There’s no single bytecode instruction for incrementing a long value, so it needs to be replaced with load var, load one, add them and store the result. Still, it operates only on local variable.

void loopOnlyLongs();
    descriptor: ()V
    flags:
    Code:
      stack=4, locals=3, args_size=1
         0: lconst_0
         1: lstore_1
         2: lload_1
         3: ldc2_w        #3                  // long 20000000l
         6: lcmp
         7: ifge          17
        10: lload_1
        11: lconst_1
        12: ladd
        13: lstore_1
        14: goto          2
        17: return