Saturday, May 01, 2010

A Couple Minutes With Terracotta Toolkit Nightly

A couple days ago I wrote a short blog about our goal of creating a Terracotta Toolkit for others to easily build simple to use scaled out frameworks, tools and in some cases apps. This same exact toolkit is used to build Ehcache, Hibernate, Http Sessions and Quartz with Scale and HA just by adding 2 lines of config. I was pretty shocked and excited about what a tremendous reaction this blog and the toolkit received. Usually when I write a blog most of it's traffic comes from awesome sites like DZone but this blog was different. I actually got considerably more direct traffic from things like e-mails and twitter than anything conventional. That kind of viral excitement shows a much deeper level of curiosity and is tremendously motivating for the team here. Anyway, I digress.

Me being the curious type I decided to do something a little dangerous yesterday. I asked Geert, one of the leads on the Terracotta toolkit project, if we were in a state where I could play around with it a bit and get a sense of how hard it is to get started. We discussed the usual caveats, "We haven't agreed on some of the naming and factoring stuff yet?" which I clearly understood. Knowing that fact and that I'd be working off a nightly(this link downloads it) I set out to follow his simple instructions and see just how easy it was. Here is what I did.

Geert gave me a simple cyclic barrier sample to work off of. This is a sample I've whipped up my self a few times over the years. Many of our earliest users leveraged Terracotta strictly as a distributed lock manager and that's what this simple sample does as well. You start the app with a barrier name and a node count and it just hangs until the node count is reached. Seems like as good of a sample as any to highlight how to get started with the toolkit.

tar -xzvf terracotta-trunk-nightly-rev15534.tar.gz

  • Grabbed a quick sample app

import org.terracotta.api.ClusteringToolkit;

import org.terracotta.api.TerracottaClient;

import org.terracotta.coordination.Barrier;

public class PlayingWithExpressBarrier {

public static void main(String[] args) {

final String barrierName = args[0];

final int numberOfParties = Integer.parseInt(args[1]);

//Start the Terracotta client

ClusteringToolkit clustering = new TerracottaClient(


//Get an instance of a barrier by name

Barrier barrier = clustering.getBarrier(barrierName,


try {

System.out.println("Waiting ...");

int index = barrier.await();

System.out.println("... finished " + index);

} catch (Exception e) {





  • I worked in eclipse so at this point all I had to do is add the toolkit jar to the classpath to get it to compile

  • Now kickoff the Terracotta server

  • And run the sample 3 time
java -cp .:terracotta-toolkit-1.0-runtime-1.0.0-SNAPSHOT.jar barrierName 3

The first two nodes should hang waiting for the 3rd node. Once the 3rd node is started all three should run to completion. This is just a little taste of what is coming.

The toolkit will have highly concurrent maps, locks, counters, queues, evictors and more so stay tuned. All of the toolkit classes will have tests in our TCK. The toolkit will run on any 1.5/1.6 JVM and requires no boot-jars, agents or container specific code. Given versions of the TCK will allow people to just drop in that toolkit jar at runtime for any app developed using that version of the API provided by the version of the TCK. I'll talk a bit more about how versioning the toolkit API will work in a future blog but it will work like any other spec. You can use any implementation that implements the spec. Enjoy, much more to come.


pveentjer said...

If Terracotta is purely used as distributed lock manager, is there any functionality for detecting deadlocks? Deadlocks are easy to create, especially in a distributed environment.

Peter Veentjer
Multiverse: Software Transactional Memory for Java

Sergio Bossa said...


doesn't the same happen if you use an instrumented java.util.concurrent.CyclicBarrier (the old style)?

What's the difference between the two?

Steve Harris said...
This comment has been removed by the author.
Steve Harris said...
This comment has been removed by the author.
Steve Harris said...

There are a few benefits to the express toolkit over how things worked in the past:
1) JVM Indepedence - can work on Jrocket, IBM JDK etc
2) Container independent - just a jar, works in any container
3) No agents or boot-jars

Steve Harris said...

One other thing is that a JVM can talk to multiple L2 clusters with the express toolkit

Sergio Bossa said...

Everything sounds great :)

Steve Harris said...

No deadlock detector right now except for cluster wide thread dumps

Unknown said...

This is interesting. In the project I'm working on, I'm trying to do something similar to what's in this example. I'm retrieveing a BlockingQueue int this way:

BlockingQueue queue=clustering.getBlockingQueue("queue");

I have several clients inserting elements in the queue (queue.put(object);), and one client reading them (queue.take();).

With one reader and one writer, no problem, but starting a third client writing in the queue, it seems that all clients block. I think it's a problem in locking configuration. Do I have to configure the locking in the tc-config.xml file or it must be managed through api contained in this express runtime jar?

Anonymous said...

Alessio, this is actually a bug that's still in the nightly version of the toolkit, the queue is in a very experimental stage right now and we're working on getting this to work. Sorry about this, it should be fixed soon.

Steve Harris said...

BTW, if your interested in the toolkit would be great if you could attend tomorrow's gathering on the topic. We'll be chatting about where to go with the API and what not.
e-mail me at steve at the company that makes the product if you want an invite.

Anonymous said...

Alessio, we just committed a fix to the toolkit's clustered queue, so your problem should be fixed with the next nightly. Do let us know how it goes, we're very interested in any suggestions you might have!

pveentjer said...

Hi Geert,

how can one distribute objects using the express api? Normally terracotta instrumentation makes sure that code is transformed so that the right api calls are done. But if you bypass instrumentation...

I have created a question about this on the terracotta forum some some time ago:

Anonymous said...

With the express API you have two options:

1. write the required code in the advanced 'behind the scenes' toolkit API to support these types and have them instrumented in the express classloader's context, this is still a bit hokey and we're working out how to improve the experience with this at the moment

2. serialize your types and store byte arrays if identity or fine-grained changes aren't important

Hope this helps,


Unknown said...

I tried the last nightly build, and the fix solved the problem.

Unknown said...

Hi. Me again.
Is it possible to shutdown a client without quitting the application?
I saw there's a shutdown() method in class, but it's empty.

Steve Harris said...

Unfortunately shutdown of the client doesn't currently work

Radhakrishnan D said...

Hi, steve,

How to Instrument our classes while we are using expressToolKit.

For example ::
ClusteringToolkit clusteringToolkit = new TerracottaClient("").getToolkit() ;

Map toolKitMap = clusteringToolkit.getMap("toolkitSampleMap") ;

While we are putting into toolKitMap , the classes ClassA and ClassB must be instrumented right /
how to achieve that ?


Steve Harris said...

Good question... Yes it is possible to instrument your own classes but it is not simple. People who use the toolkit generally rely on some for of self serialization to cluster objects that we haven't provided. It is however possible if you really need it. 2 options:

1) Use the toolkit as a tim and then you can instrument anything you want.
2) Post to tc-dev and maybe some of the developers can help you through how to do it in express mode because it's too big a topic for this comment.

We do plan to add a simple way to instrument classes for express mode but it just isn't there yet.

Radhakrishnan D said...

Hi, steve.
can we use dso mode for some use cases and express tool kit for some usecases with terracotta-3.3


Steve Harris said...

If we are talking about the L1 or application process then you can use the Toolkit in the same process with DSO but not Express. The reason for this is DSO is a JVM wide level of sharing while the Express toolkit is scoped by the TerracottaClient. You can have many of these per JVM. The Server Array can talk to both at the same time.