Terracotta and the Master/Worker Pattern

Terracotta enables applications to be deployed on multiple JVMs, yet interact with each other as if they were running on the same JVM. The user nominates objects that are to be shared between JVMs and Terracotta transparently makes them accessible to all. Terracotta provides a number of pre-configured solutions for clustering particular Java enterprise environments – for example, Tomcat, JBoss or Spring. However, in our case we weren’t using a container so needed to configure Terracotta ourselves.

Fortunately we found an approach that was both appropriate to our application and had already been proven to work well with Terracotta: the Master/Worker design pattern. Its usage with Terracotta was first described by Jonas Bonèr in his excellent blog entry How to Build a POJO-based Data Grid using Open Terracotta.

For those not familiar with the Master/Worker pattern, the ‘Master’ is responsible for determining the items of work that need to be done and places each item onto a shared queue. It then continues to monitor the completion status of each item and completes when all have completed. The ‘Workers’ pick the work items off the queue and process them, setting the item’s completion status when the work is finished.

In our implementation, both the Master and Workers runs in their own JVMs. The JVMs are distributed across a number of machines. Any shared objects (such as the work queue and work item statuses) are managed (and effectively synchronized) by the Terracotta server.

There are a number of useful ways to visualise this architecture. The first is a purely physical one, showing how machines, JVMs and Master/Workers relate to one another on the network.

Note that it is possible to have multiple JVMs on a single machine. You might want to do this if the heap for a JVM has hit its maximum allowable size, but there is additional physical memory on the machine that another JVM could use.

So where does Terracotta fit into this physical architecture? Well, it oversees all of the JVMs – something perhaps best illustrated with a second, more logical visualization:

We see that Terracotta instruments the JVMs so that they can transparently share a queue of work items across the physical network. It’s even possible to add additional Masters and have them share the queue as well.