I recently had occasion to use a trick with test execution that is simple and very helpful, but that I forget about frequently, so I thought I’d pass it along here.

We use Maven to execute our unit (and sometimes functional, but that’s another story) tests for a suite of applications my team works on. Some of our apps have quite a few tests, and even though each unit test is very fast (as good unit tests ought to be), running that many of them still takes a bit of time. We’re also burdened with a very slow CI server, unfortunately, which aggravates that situation.

Unit tests, by definition, do not rely on the operation of any other test – they are self-contained, stateless and repeatable. This means they’re ideally suited to running in parallel, and fortunately, that’s very easy to do with Maven.

Maven uses the Surefire Plugin to do it’s test-running, and it’s configurable to run tests in parallel. Our project uses a parent pom (different from an aggregator pom) to specify common configuration, so our pom files stay lean. In our parent pom, we just add the following:

  org.apache.maven.plugins
  maven-surefire-plugin
  2.5
  
methods
20
  

And we’ve told Maven to run our tests in a maximum of 20 threads, and to make each method run in parallel. (There are other options here for only running classes in parallel, etc, which might be helpful for functional tests).

Just doing that took a minute off the time of a couple of our builds, and a bit off the time of even some of the smaller ones. Of course, it’s even more effective if you’re running on a multicore machine, but even the feeble VM running our CI was faster with it there.

Enjoy!

One of the ways to ensure quality in a software project is to find a set of coding standards that your team can agree on then put automated checks in place to ensure they are adhered to.

In this post I’d like to take a very small example of such a stanard, and show how you can use several different tools to help ensure and measure compliance.

The Example
The simple rule we wil use is the use of curl braces on if statements and similar blocks. Our team reached a concensus that we would always prefer this form

if (condition) {
   block
}

over this

if (condition)
    block

even when there is only one line in the block, where the braces are optional. I’m in the habit of not doing this, so I kept doing it the wrong way, making code just that little bit harder for the rest of the team to read and maintain.

We have an existing codebase of hundreds of thousands of lines, some of which adhere to this rule, some of which doesn’t.

First, I wanted to measure adherence to the rule, so I chose the Sonar quality assurance tool.

Sonar
Sonar installation is straightforward: download the distribution, unzip it and run one script. Sonar runs as a web application, so you can immediate go to your browser and see the initial page, like so:

I’m using Sonar (for now) in it’s default mode with a bundled Derby database. If I were going to use it more heavily, I could install a separate MySQL database, and run Sonar itself inside a container of my choice (it comes bundled with a basic web container so you can get going right away).

For my machine, I ran

bin/macosx-universal-64/sonar.sh console

“Console” runs Sonar without backgrounding the task. Normally you’d use “start”, which works just like a Unix service, and is suitable for running from an init.d-type script immediately. The first time around, Sonar takes a few moments to fire up, as it creates all it’s required database tables and populates them with an initial set of data to get your started.

Sonar can measure many hundreds of different metrics, but for the purposes of this simple experiment I only want the one, so I create a custom profile, called “team”.

I set this profile to be the default, then click to edit it.

Now I can search for the braces rule in checkstyle by setting filter criteria:

You can click the rule name to see a description, and to set any optional parameters on the rule.

I activate the rule, and leave it at the default “Minor” priority.

If I now return to my profile, I have a link to access the generated checkstyle.xml file, the configuration file that is used by checkstyle to configure its rules. This will be important later on.

Now I can go to my project and analyze it by running “mvn clean install sonar:sonar”. With the default configuration, this works only when the build is taking place on the same box that Sonar is running, but it’s simple to set up a settings.xml file to tell the Sonar plugin where to find the Sonar instance, so you can have a single shared Sonar instance for the whole team. This is covered nicely in the Sonar documentation.

One minor hiccup: The released version of the Sonar plugin for Maven doesn’t work with Maven version 3 (still in Beta), which I was using. You can build the source version if you need this support, or I’m guessing it’ll be released about the time Maven 3 might be.

Now that I’ve analyzed my project, I can go back into Sonar and see how well I adhered to my rule:

As I see I have a violation, I can drill down and see the violation in context right in the code:

This is handy for checking the state of a rule against an existing codebase, but I’d like to have more immediate feedback. No problem, I can integrate Sonar into both my IDE and my build process very easily.

IntelliJ IDEA: Immediate Feedback

There’s also a new plugin for IntelliJ IDEA for Sonar. To install, just go to the “Available” tab in your plugin configuration within IntelliJ, right-click and select Download and Install. You’ll have to restart IntelliJ to activate the plugin.

Now you can find the Sonar entry in configuration, and set up a URL to point to your Sonar installation, along with a username and password to log in to Sonar. The default is “admin/admin”.

Having done that, I can now see violations directly in my code, as I open each file, like so:

In the above, I’m hovering over the little “price tag” annotation in the left gutter to see the message. You can also see the usual IntelliJ “yellow bar” on the right side of the screen, which shows the same message.

This lets me know of a rule violation before I even attempt to build, so when I forget my braces, I don’t have to wait to be chastized, IntelliJ can do it for me immediately

Maven: Ensuring Conformance

There is a Maven Checkstyle Plugin that allows us to make a run of checkstyle part of the build process of our project. I configured this plugin in my POM like so:

    4.0.0
    com.jglobal
    experiment
jar
    Experiment
    1.0.0-SNAPSHOT

UTF-8
    

    

        
            junit
            junit
            4.5
            test
        

    

    


                maven-compiler-plugin
                
                    1.6
                    1.6
                
            

                org.apache.maven.plugins
                maven-checkstyle-plugin
                2.5
                
                    
                        org.codehaus.plexus
                        plexus-resources
                        1.0-alpha-7
                    
                
                
                    http://localhost:9000/rules_configuration/export/java/team/checkstyle.xml
                    
                    true
                    true
                
            

                org.codehaus.sonar
                sonar-maven-plugin
                1.8M1
            
        
    


            codehaus
            Maven Plugin Repository
            http://repository.codehaus.org/
            default
            
                false
            
            
                never
            
        
    

As you can see, instead of providing a file system based location for the configuration file for Checkstyle, I just pointed the Checkstyle plugin at my Sonar-generated checkstyle.xml, this way my IDE, Sonar and my build stay in perfect sync, and I have a nice GUI for adjusting my checkstyle rules, instead of having to hand-edit XML.

Now when I run my build, Maven will analyze my project against the checkstyle rules, and report any violations. With code as we’ve seen above, it of course reports a violation, but the build still succeeds.

This is fine, but now that I’ve verified we are adhering to this rule, I’d like to actually fail the build if we slip back. Also easily done – in Sonar, I just change the level of violation for my one and only rule to “Critical” in my profile. Now when I run my Maven build, sure enough, the build fails, noting the fact that I’m not adhering to a Critical rule.

As my codebase gets cleaner and cleaner, I can up the priority of the rules I really care about until there are no more violations – then Maven and Sonar can guarantee it stays that way. Of course, in a multimodule build (e.g. with an aggregator POM), I might want to create a parent POM, and specify all this configuration there. (It’s generally not a good idea to have the aggregator and the parent in one pom, use two).

This is of course a trivial example of Sonar, IntelliJ and Maven working together. Sonar can apply over 700 rules right out of the box, and more can be added via plugins easily.

What I find good is that I can synchronize all my tools off one common and agreed-upon rulebase, without having to tweak XML files all over the place to gradually increase my quality standards over time.

In a project I was tinkering with lately I found the complexity getting to the point where I could use component dependency injection. There are in fact a number of techniques to do this in “plain” Scala, but they generally seem more painful to me than just dropping in a lightweight DI framework, so I went with Guice.

As it happens, Guice works with Scala very nicely, and is very unobtrusive. There’s a few minor things I haven’t quite figured out yet, but I’d like to share what I’ve discovered so far.

After adding this section to my project POM:

 
            com.google.inject
            guice
            2.0
        

I had all I needed of Guice embedded in my app. Now I can scratch up a series of tests that exercise all of the different ways use Guice from Scala. Let’s start with the simplest, and work our way up:

 @Test
  def basicDependencyInjection {
    class ScalaModule extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).to(classOf[SomeService])
      }
    }

    class ScalaModule2 extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).to(classOf[SomeOtherService])
      }
    }

    val injector = Guice.createInjector(new ScalaModule)
    val component = injector.getInstance(classOf[MyComponent])
    assertEquals("someService", component.callTheService)

    val injector2 = Guice.createInjector(new ScalaModule2)
    val component2 = injector2.getInstance(classOf[MyComponent])
    assertEquals("someOtherService", component2.callTheService)
  }

So let’s explore the test a bit. I’m using Scala to build a JUnit test, as shown by the annotation on line 1. One of the advantages of this is that it’s easy to use Scala’s namespace capabilities to build test classes that are only visible within the test itself, which is what I’m doing starting line 3. I create a class call ScalaModule with extends Guice’s base class AbstractModule. I override one method called “configure”, which sets up the bindings of interfaces to implementations for this Guice module. (This is all covered in detail in Guice’s doc).

On line 6 I bind the interface “AService” to an implementation of AService called “SomeService”. Here’s what they look like (these classes are defined later in the file – I don’t embed them inside the test class as Guice didn’t let me).

Here’s the trait for the service:

trait AService {
  def service: String
}

And a simple implementation:

class SomeService extends AService {
  def service(): String = "someService"
}

Now I have a component that uses that service, called MyComponent. It looks like this:

class MyComponent @Inject()(val service: AService) {
  def callTheService(): String = service.service
}

Note the annotation on the first line. This is how you put an annotation on the constructor in Scala, Annoyingly, the () is apparently required. This Inject annotation indicates that when this component is requested from Guice, it should have the appropriate implementation of Service injected.

So now to return to our test, on line 10 you’ll see we are building a second module, this time binding the AService interface (trait, actually) to the SomeOtherService implementation, which looks like this:

class SomeOtherService extends AService {
  def service(): String = "someOtherService"
}

Then we get into the meat of the test in line 17. First, we ask Guice for our injector, the “god object” for Guice, from which you get all the other object you’ll need. In the first case, we use our ScalaModule class. Then we ask this injector for our component, and it constructs the component for us, injecting the dependency as necessary. In this situation, we have not told Guice that MyComponent is a singleton (we’ll deal with that later – in several different ways), so it builds a new one for us every time.

On line 18, we make the assertion that proves that our MyComponent instance has in fact been built with the “SomeService” instance, as opposed to any other instance, by verifying our string that SomeService responds with.

Then we do the whole thing again, except this time asking for the ScalaModule2. ScalaModule2 is where we have the SomeOtherService implementation of AService wired up, so when we assert on line 23, we now see the “someOtherService” string returned instead, which means that we’re properly wired, as expected.

Now let’s give Guice a little harder work, exploring some scenarios we’ll see in more sophisticated applications.

In many situations, we’ll want to avoid creating a new instance of our service object every time we use it. There are a couple of ways to do this with Guice.

@Test
  def instanceBindingExample {
    // if you want to bind to a specific instance of a class...
    class InstanceExampleModule extends AbstractModule {
      @Override
      protected def configure() {
        val instance1 = new InstanceService("instance1")
        bind(classOf[AService]).toInstance(instance1)
      }
    }
    val injector = Guice.createInjector(new InstanceExampleModule)
    val service = injector.getInstance(classOf[AService])
    assertEquals("instance1", service.service)
  }

In the code above, we’re configuring our module, called InstanceExampleModule, with a binding that specifies our AService trait uses a specific instance of the class called InstanceService.

Here’s InstanceService:

class InstanceService(val value: String) extends AService {
  def service(): String = value
}

Back in our test, line 13 ensures that we’re actually talking to the right service, as we fed the service it’s return string (“instance1″) when it was created in the module.

Ok, so now we can inject services, and specify an instance of a service instead of a newly-created one every time. That’s a good start, but we want to do more.

What if we have to do some more work to instantiate our singleton instance? E.g. what happens if it takes a whole method to create the service?

Guice has that covered as well, like so:

Here you see a new module being built, but this time we don’t actually say anything in the “configure” method. Instead, we annotate another method in the module with the @Provides annotation – this means to Guice “whenever you need a thing of the type this method returns, call this method”. Then we verify that that’s in fact what’s happening by creating the injector and asking for the service.
Ok, so now we can create instances of services however we want, but we’ve still only created a single instance of a given trait. What if we’ve got a whole whack of services that implement our interface? Ok, that’s also easy:
@Test
  def bindingWithNameAnnotationExample {
    // what if I have two implementations of the same interface, how do I say which one I want?
    class ScalaModule extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).annotatedWith(Names.named("foo")).to(classOf[SomeServiceNamedFoo])
        bind(classOf[AService]).annotatedWith(Names.named("bar")).to(classOf[SomeServiceNamedBar])
      }
    }

    val injector = Guice.createInjector(new ScalaModule)
    val barComponent = injector.getInstance(classOf[MyBarComponent])
    val fooComponent = injector.getInstance(classOf[MyFooComponent])
    assertEquals("bar", barComponent.callTheService)
    assertEquals("foo", fooComponent.callTheService)
  }

Now we’ve got a couple of services to declare:
class SomeServiceNamedFoo extends AService {
  def service(): String = "foo"
}
class SomeServiceNamedBar extends AService {
  def service(): String = "bar"
}

The services themselves both implement our trait, of course, but now we have two different components, one which needs the Foo service, other other one needs the Bar:
class SomeServiceNamedFoo extends AService {
  def service(): String = "foo"
}
class SomeServiceNamedBar extends AService {
  def service(): String = "bar"
}

In our test, you can see we ask the injector for the two components. What’s very interesting to me is that the component themselves are not listed in the module. Why that’s important is that unlike Spring (to pick an example many people know), you don’t need any configuration provided at all for stuff that Guice can work out on it’s own. This is a good thing.
Anyway, you can see by our asserts that in this situation the Foo service gets the Foo implementation of the AService (SomeServiceNamedFoo), and the Bar service gets SomeServiceNamedBar instead.
Now that we can handle multiple implementations, let’s go back and consider the singleton vs. new instance issue from another angle: Scala has no concept of the idea of “static” classes, but it does have companion objects, which are, in all cases I’ve seen, better.
If we declare a service like so:
object SingletonService extends AService {
  def service(): String = "singleton"
}

and another (non-singleton) implementation, like so:
class NonSingletonService extends AService {
  def service(): String = "nonsingleton"
}

Now we can write a test that tries this out like so:
  @Test
  def singletonOrPrototype {
    class ScalaModuleSingleton extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).toInstance(SingletonService)
      }
    }

    class ScalaModuleNonSingleton extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).to(classOf[NonSingletonService])
      }
    }

    val singletonInjector = Guice.createInjector(new ScalaModuleSingleton)
    val singleton = singletonInjector.getInstance(classOf[AService])
    assertEquals("singleton", singleton.service)
    val secondInstance = singletonInjector.getInstance(classOf[AService])
    assertSame(singleton, secondInstance)

    val nonSingletonInjector = Guice.createInjector(new ScalaModuleNonSingleton)
    val nonsingleton = nonSingletonInjector.getInstance(classOf[AService])
    assertEquals("nonsingleton", nonsingleton.service)
    val secondNonSingletonInstance = nonSingletonInjector.getInstance(classOf[AService])
    assertNotSame(nonsingleton, secondNonSingletonInstance)
  }

The money here comes at the line that binds the classOf[AService] to a singleton – in this case, the SingletonService. This isn’t a class, so we don’t use to(classOf[SingletonService]), we say toInstance(SingletonService) instead.
As you can see from the test, this works very nicely, and gives us a much more Scala-canonical way to do singleton.
However, there are times when we want to declare a regular Scala class, and have Guice only instantiate one of em for us, and handle it as a singleton that way (some examples might be when we’re using a constructor to inject other dependencies – although we can use field injection for this, which we’ll look at later).
We can annotate our class explicitly to tell Guice it’s a singleton like so:
@com.google.inject.Singleton class SingletonClassService extends AService {
  def service(): String = "singletonClass"
}

Now we can write a test like so to prove that we get the same instance of our service, even if we ask the injector for it multiples times:
@Test
  def singletonClassExample {
    class ScalaModuleSingleton extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).to(classOf[SingletonClassService])
      }
    }

    val singletonInjector = Guice.createInjector(new ScalaModuleSingleton)
    val singleton = singletonInjector.getInstance(classOf[AService])
    assertEquals("singletonClass", singleton.service)
    val secondInstance = singletonInjector.getInstance(classOf[AService])
    assertSame(singleton, secondInstance)
  }

The assertSame in JUnit asserts that we have the exact same instance of an object, not that they’re only equal.
One last scenario: What is the service isn’t the singleton, but the component is? E.g. what if we have an object instead of class that needs a service injected into it? Turns out Guice can handle that nicely as well. The only extra work we need is to define a trait for the binding of our component, like so, with the service injected into the object:
trait SingletonComponentInterface {
  def callTheService:String
}
object SingletonComponent extends SingletonComponentInterface {
  @Inject val service:AService = null
  def callTheService(): String = service.service
}

Now we can write a test to make sure this is doing what we want:
@Test
  def injectIntoSingletonExample {
    // if you want to inject into an object instead of a class
    class InstanceExampleModule extends AbstractModule {
      @Override
      protected def configure() {
        val instance1 = new InstanceService("instance1")
        bind(classOf[AService]).toInstance(instance1)
        bind(classOf[SingletonComponentInterface]).toInstance(SingletonComponent)
      }
    }
    val injector = Guice.createInjector(new InstanceExampleModule)
    val component = injector.getInstance(classOf[SingletonComponentInterface])
    assertEquals("instance1", component.callTheService)
  }

That’s it for my experiments so far. Here’s my conclusions, please comment with your experiences:
The Good, The Bad, and the Ugly
The Good

There was a lot I liked about Guice in Scala. First, no XML was injured in the making of these examples – everything was code, and fully type-safe (I mentioned it was Scala, right?) I always find myself writing a “smoke test” for my Spring apps to verify I haven’t made a typo in my “beans.xml” file – here I didn’t need to. Yes, I know you can do XML-free Spring as well, but it’s the norm with Guice, and it feels a lot lighter weight.
I don’t need to configure anything for classes that just get injected. This means that the configuration I do need to write is very short, and because it’s a class, it’s extensible. In my “real” project, for instance, I was able to write a module that created services that use static lists for their data (instead of reading and writing to a real database, for instance), then extend that class with an OSGi-aware version that used the “real” services from other modules. No repeated work, no wiring, no config – it just worked.
Auto-handling of singletons, both with companion objects and classes, as I prefer: It was nice to be able to write Scala in a very Scala-like way, and just have the DI framework slot right in whichever way I needed it to, as described above.
The Bad

I have to annotate my classes with Guice-specific annotations, although there is a JSR on the way to standardize these, which makes it a bit less objectionable. What happens if I need to inject to a class I don’t have source for (e.g. something in an existing Java library that I’m calling? I also haven’t tried this in a Servlet environment, although I understand there’s a Guice extension for that, so I don’t expect any trouble there.
The Ugly

I have to (it seems) use the @Inject() format, not just @Inject, and I have to insert it in what looks like an odd place in the declaration. Not a big deal, and I suspect I’ll get used to it.
            

I’ve been sold for some time on the benefits of highly modular software development (and deployment, for that matter, almost equally importantly).

However, I’m aware that not everyone is on the same page, or has the same definition of “modularity” in the first place, so I thought I’d collect and enumerate the things that have convinced me, as best I can.

I’ve been in the software business for a very long time, and sometimes I have a preference or leaning towards a certain technique or technology. Sometimes that preference has clear causes behind it, but sometimes not. Modularity is one of the latter, where I know it’s not only good, but critically important, but it’s hard for me to immediately say why or how I know that in a succinct way.

So, I apologize for the length of this explanation, as I’ve not yet taken the time to make it shorter

Modularity Defined
First things first: What do I mean by modularity?

In short, I mean a system comprised of relatively small self-contained units. The idea of modularity has caught on much more completely in many other disciplines, but far less so in software. In manufacturing, especially mechanical and electronic, in business, especially finance, the idea of modularity is well established. Modularity is used to increase and maintain quality, and reduce cost while increasing output. In the automotive industry, for example, extensive use of standardization and modularity allows us all to afford a car, as opposed to just the wealthy or the mechanically inclined.

In the software world, the use of modularity is still woefully small, however. When it is employed, though, some spectacular successes have arisen. One of the oldest examples is Unix, arguably the most successful operating system ever. Since 1969, the “do one thing, do it well, and play well with others” modular design of Unix has been the basis of it’s success and flexibility, allowing it to span everything from embedded devices to mainframes.

The Java Virtual Machine (JVM) is another realm where modularity is on the rise, and starting to have an impact. Many different approaches are available, and some of them work with any of the large number of languages available on this platform.

Increase overall success rate of projects
In the many software projects I’ve been involved with over the years, I’ve noticed a few trends. Smaller, well-contained projects have a higher success rate than larger and more poorly contained projects. (By contained, I mean the number of places and ways the project interacts with other projects is small). I’m sure there are a host of reasons for this, but part of it is simple comprehension – small projects are generally easier to understand than big one, so the number of times things get done wrong is reduced.

A system comprised of well-isolated modules tends to be more like a set of small projects that happen to work together, so I find this reason alone a plus for modularity.

But this hides much of the detailed reasoning behind why modularization works, so let’s dig a bit deeper…

Makes the API between components explicit, not hidden
Sometimes I heard the argument that modularity is complex, that it actually increases the difficulty of working with a system. I find that doesn’t hold up under examination: when modularity seems complex, it’s probably because it’s being done wrong, or being retrofitted to a system that was not designed with modularity in mind. What you’re really seeing in this situation is that the attempt to modularize is exposing the complexity that was already there.

When a system is properly designed in a modular fashion, the APIs between modules are small and well-defined, with a limited number of cross-module dependencies.

Reduce unnecessary complexity
By reducing a large complex system into a series of small and better understood modules, we’ve reduced the complexity we have to deal with at any one time. This means that the overall large system only has to deal with how the exposed APIs of our modules will be fitted together, treating the modules themselves as “black boxes” that perform specific functions, without worrying about how they perform it.

In both cases, we’ve reduced the complexity we need to deal with.

I specify here “unnecessary” complexity, as of course the complexity of the issue itself doesn’t get any simpler just because we modularize it – but if we don’t have to deal with a lot of tangled plumbing, we can concentrate on the real issue.

Increase testability
A module that performs a single function is much easier to test exhaustively than an entire monolithic system. The number of interdependence is limited, meaning we can do more of our testing at the unit and orchestration level than at the functional or integration level, meaning our feedback time is improved and our coverage likely increased.

This means our development velocity can increase, as the impact of changes can be evaluated more easily and more quickly.

Support Composition
When you have a number of smaller building blocks, it’s relatively easy to reshuffle those blocks to change the behaviour and functionality of your overall system. This is making use of composition – connecting blocks together to do more than they can on their own, and its a pattern that is supported well in a modular environment.

Although composition is often thought of in the context of re-usable modules, when building a new system, it’s also valuable in modules that are only used within one system, as it allows the large-grained behavior of the system to change much more rapidly than without modules.

Composition makes large complex systems easy to understand and work on, and vastly increase the maintainability of the overall system.

Impact of changes isolated
The isolation of changes to an effected module is a major benefit of modularity, especially as it supports maintainability. If you’re constantly concerned with your changes affecting parts of the system you’re not actually working on, you’re not able to go as fast as you could if you knew that your changes would only apply to a small area of the whole system – the current module.

This is related to and similar to the support for refactoring that unit tests give – if you’re tests are all green after your refactor, you know you haven’t changed the functionality of your system. In modularity, if you’re isolated from other modules in the system, this goes a step further – you can change the functionality of the module freely without concern about other bits of your system breaking. This is true if you’re doing anything that’s not in the one package you’ve “exported” as the service definition.

Allows physical design to reflect logical design
Modularity also allows developers to build systems where the logical design actually corresponds as needed to the physical design, an often overlooked abstraction. Kirk’s blog post on the topic describes this issue better than I could, so I refer you there.

Supports Refactoring Better
Refactoring is the process of changing the implementation of a system without changing it’s behavior. One of the primary reasons to write unit tests, for instance, is to support and allow refactoring, so we know the system does the same thing when we’re done as when we started.

Modularization supports this ability by making clear the portions of a system that affect other modules, and the portions which do not. When you’re refactoring in a monolithic system, even with IDE support, the scope of your changes is the entire project. When you’re refactoring in a modularized system, if you’re “inside” the module, you have no need to consider scope on the outside, as there isn’t any. And when you’re refactoring the exported interfaces, you know ahead of time your scope is cross module, and can take it into account.

In short, modularization lets you refactor more freely within a module, and lets you be intentional and organized when refactoring between modules.

A colleague recently suggested that modularization would interfere with the ability to see a system as a whole. It’s been my experience that the exact opposite is true – a well structured set of modules can still be considered as a whole when necessary (an aggregator POM is any easy way to set this up in the Maven universe), but it also allows modules to be worked on “safely” in isolation, which a monolithic system does not. This ability to only look at one piece at a time is all the more critical the larger the system becomes.

The need to refactor “across” modules is a warning sign, in my opinion – it generally indicates that the API between modules is changing, and this is a change you want to be more aware of then any amount of changes within modules. If it happens often, it probably indicates that the module boundaries are not yet stabilized, and you have more design work to do.

Scalable Development
A modular system lends itself well to many hands being involved in it’s development and maintenance. It’s not necessary for a team or a pair to understand the whole system well, or even at all – they can still work on a small well-defined and decoupled module, and can do it in parallel with development on other modules.

Helps prevent Design Rot
As discussed in this blog post, design tends to degrade over time, especially in large monolithic systems. Modularity helps to prevent this rot, as pieces that become superseded by better ways of doing things can be replaced individually, and the impact to the overall system managed and isolated. This is akin to removing a brick from the wall and replacing it with a better brick, rather than tearing down the wall.

Solves the classpath hell problem entirely
Classpath hell is the name for the condition where a large number of dependencies are “in scope” at once. It’s not unique to Java, or even the JVM. It is apparently called “DLL hell” on Windows environments, but I’m happy to report I know very little about that. I do know it can happen on platforms other than the JVM, however.

By isolating the influence of dependencies, modularization, especially the way OSGi does it, makes it an explicit process to import and export exactly what packages you wish to and from your modules. What the module uses in it’s private implementation under the hood is no longer relevant to the system at large.

What happens in the bundle stays in the bundle, in other words, with appropriate apologies to Las Vegas.

Management of dependencies is the cure for classpath hell, and modularization is key to this, as described in this excellent article.

Allows smaller pieces of the system to be versioned
A key element in continuous releasability is the careful versioning of both individual modules and their dependency requirements. Modules give you the ability to release pieces of your system, not the whole system, better supporting the idea of continuous releasability and enhancing maintainability.

An article about software maintenance that a colleague recently sent me describes the problem perfectly – and independently versioned modules is a large part of the answer to this problem.

Re-Use
Although re-use it touted often as a major benefit of modularity, I’d categorize it as the least important benefit, not the most. Although the idea of re-using well-designed modules in new projects is indeed very powerful, it’s beyond the technical and design capabilities of many teams, often causing them to reject the need for modularity because they see no reason to build for re-use.

In a way, they’re right: you don’t build for re-use. You build for just the specific case at hand, but when you separate concerns properly and adhere to modularity best practices, you end up with a module that may lend itself to later re-use much more easily. If such a re-use case emerges, well and good, then you can reap the benefit. It is a lot like a framework – you can’t design a good one in isolation from it’s use cases – a framework is an emergent artifact from the re-use of code on many different (although similar) projects. The same is true of re-usable modules: you don’t set out to build a module explicitly to be re-usable.

Allows dependencies to be isolated: e.g. all Weblogic-specific code in one module, or all XMLMill-specific code in one module, increasing ability to refactor safely

Suggested Reading:
Agile Architecture Requires Modularity
Modularity Patterns
Runtime and Development time Modularity

In a later post we’ll look at some of the counter-arguments against modularity, and why I think they’re not especially valid or convincing in my experience.

A few weeks ago I had an opportunity to go to The Server-Side Java Symposium, an annual event for Java aficionados held in Las Vegas, Nevada. What happened there will not stay there, however.

I found a few interesting things out, and I’m going to try to share them over the course of a few blog posts, in case anyone out there is interested, and so I can find them later for myself as well

TSSJS is not about vendors, it’s about technical info, so there were only a few “booths” from the handful of commercial sponsors of the event, one of which was, inevitably, Oracle. Given the Sun acquisition, a question on the mind of many developers has been “what’s Oracle going to do for/with/to Java?”.

It’s a legitimate concern, and indeed a complex question with a number of complex answers – but one of my reasons for being in Vegas was to get some of the answers.

The keynote presentation at TSSJS was by James Gosling, the “father” of Java. As everyone no doubt reading this already knows, he’s subsequently left Oracle, for undisclosed reasons. That, to me, lessens the impact of his remarks somewhat, as I suspect he knew by the time we heard him speak in Vegas that he’d be on the way out the door, but he still had a lot of interesting things to say.

About 800 people were in the room, I estimated, to hear the keynote. Interestingly enough, Gosling’s position at Oracle was “Client Software Group” VP. I generally don’t think of Java as having much to do with “client” software, personally, so that in itself was informative.

He had an image of the Java mascot (the little delta-shaped guy) wearing a snorkle in a fish tank, a play on the “Soracle” name mash-up.

In any case, the keynote touched on what’s happening in the Java world. He emphasized the wide spectrum of devices on which Java is found, and how networking ties all of these devices more and more into a cohesive whole.

Some stats were mentioned: there are about 15 million downloads per week of Java, and something like 10 billion (with a “B”) Java-enabled devices of one sort or another out there in the wild, perhaps 1 billion of which are Java-enabled desktops. By any metric, Java is a huge success, and is in many ways more popular in the rest of the world, especially Europe, than it is in North America.

100 million of those devices are are TV devices, and on top of all it are about 2.6 billion Java-enabled cell phones.

Conclusion: It’s not going away, soon or quickly

A site like the Apple App Store existed for Java apps 7 years before Apple’s app store – but it was in Japan.

There are 5.5 billion smart card devices Java-enabled out there – the V3 standard actually has a servlet container in it – U.S. military dog-tags apparently now use this standard.

The Amazon Kindle is a Java device, under the skin.

Gosling proffered a definition of success: being completely invisible, people just get stuff done.

There are something like 6.5 million professional Java developers out there, 800 thousand of which are in India. Most colleges have a Java course requirement, modifying the old saw of “write once, run anywhere” to include “learn once, work anywhere”.

After this deluge of interesting stats, Gosling moved on to discuss the next big thing in Java, in his opinion: Java EE 6, the foundation for next-generation enterprise software. The spec was approved November 30th, 2009.

There are two major flavors of JEE6 – the “full” profile, with everything, and the “web” profile, with EJB3 “Lite” and just the most frequently used specs, not the more obscure stuff.

A major emphasis in next-generation Java development is modularity: making hard problems easier by breaking them into a number of less-hard problems. Major endorsement here for OSGi.

As far as Gosling’s concerned, the correct amount of XML required is exactly none Many developers in the room agreed with him, and it’s now possible with JEE6 and it’s powerful annotations.

Much of the benefit of dependency injection and resolution that Spring provides has been baking into the core Java EE platform now, and much of the pain of EJB has gone away. In fact, one of the most painful parts of EJBs, CMP Entity Beans, is no longer supported by the spec at all, in favor of JPA.

Glassfish V3 is the JEE6 reference implementation, and it’s OSGi-based.

Then it was demo time: Gosling (with some help from a couple of developer types) fired up Netbeans and Glassfish and did some show and tell of the coolness of JEE6.

We saw a webapp deployed – with no web.xml, no WEB-INF directory at all, everything annotation-based and dependency-injected at the proper times. We saw a stateless EJB auto-deployed to via an OSGi bundle into the container in less than a second, direct from the IDE. No boilerplate, no config, no reams of XML files to bolt it all together, just code and go.

We saw that same app preserve session state even when it was redeployed, which was very cool (a counter was showing the number of hits to the page – code was changed and redeployed, and the counter kept incrementing – and no, it wasn’t stored in a client-side cookie, it was in the session).

I noticed that all the Oracle guys were using Macs, incidentally

Gosling notes that JEE6 shouldn’t just be one version number past JEE5 – it’s more like JEE 60 in comparison

Saw the @Schedule annotation, where a bean can be scheduled for execution with cron-like ease, but all within a JVM.

Gosling noted that Felix was the web container under the skin of Glassfish.

We saw the @Observes annotation being used – a way to register listeners without any code or configuration. It was noted that Swing could benefit enormously by this pattern.

Then we saw a demonstration of Glassfish’s web console. The “web profile” of JEE6 doesn’t include JMS by default, so the example shown was adding that support by installing a new bundle in the running container. Two clicks and it was done. A GUI admin console can be added for any bundle, including your own bundles.

Gosling then wrapped up with an overview of some of his personal projects, and then took some questions. He also mentioned a new “app store” for Java that’s in the works, to be coming on “store.java.com”. It’s in the final stages of work now, apparently.

That was the end of the first keynote of TSSJS. In the next few posts I’ll talk about some additional sessions that my associate and I attended, and some of the other good stuff we learned.

In my ongoing experiments with Apache ServiceMix, I recently installed the Felix Web Console project, and it was simple and helpful.

OSGi containers typically have a “console” for administering the bundles you’re deploying into our container via the command-line. ServiceMix has a nice “remote” feature, that allows me to get to the console of our ServiceMix instance from my local machine easily, without having to actually ssh to the ServiceMix box.

It looks like this:

You can list your bundles, install, uninstall, restart, etc, all from here.

What Felix Web Console brings to the table is a nice webapp front-end on all this functionality.

Installation is the easiest thing possible: you go to your existing console and type:

    <dependency>
smx@root:osgi> install http://mirror.switch.ch/mirror/apache/dist/felix/org.apache.felix.webconsole-2.0.2.jar

It reports that a new bundle is installed, and gives you the number (237 in my case).

Then I say:

 start 237

And I can immediately surf to the following URL in my browser: http://servicemix.point2.com:8080/system/console

and see the following page:

Now I can browse my bundles, install, uninstall – and all the same good stuff as via the console, even view the output of the log service, except all from the comfort of my web browser.

Like many things in the OSGi world, it just works like it’s supposed to, and is much easier to do than it is to explain

In Part 1 of our OSGi adventures, we described how to build a nice Ajax-aware Vaadin UI app, and couple it to a generic back-end OSGi service we called the service dispatcher.

Now we’ll take the adventure to the next step, show how to deploy that webapp into our OSGi container and get it up and running, then go through the functionality of the Dispatch Service, and show how it routes requires through to the first of our domain-specific application services.

In our UI, we built a form that allowed the user to enter and save a “Dealer”, with some fields like name, phone number, etc., so the first service we’ll build for the service dispatcher to talk to will be our “dealer” service.

First, though, let’s see how to get our webapp into our OSGi container.

As we’re building our Vaadin app with Maven, we can easily add the small bits of additional configuration to turn our project into an OSGi-friendly WAR file.

OSGi deploys “bundles”, but a bundle is just a jar file (or a war, which is after all just a special kind of jar) with a bit of extra meta-data. The META-INF/MANIFEST.MF file is where the magic happens. We add the following to our POM:

              <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.0</version>
                <configuration>
                  <archive>

                      org.springframework.osgi.web.context.support,\
                       org.springframework.web.servlet,\
                       org.springframework.web.servlet.handler,\
                       org.springframework.web.servlet.mvc,\
                       org.springframework.web.servlet.view,\
                       dwmj.domain,\
                       org.springframework.web.servlet.mvc.annotation,\
                       org.springframework.web.context

                    <manifestEntries>
                      <Bundle-ManifestVersion>2</Bundle-ManifestVersion>
                        <Bundle-SymbolicName>com.point2.Admin</Bundle-SymbolicName>
                        <Bundle-Name>Admin</Bundle-Name>
                        <Bundle-Version>1.0.0</Bundle-Version>
                        <Bundle-Activator>com.point2.ServiceClient</Bundle-Activator>
                        <Import-Package>org.osgi.framework,com.point2.services.dispatch,javax.servlet;version="2.4.0",
javax.servlet.http;version="2.4.0",org.osgi.service.http;version="1.2.0",
org.osgi.util.tracker;version="1.3.2"</Import-Package>
                        <Webapp-Context>admin</Webapp-Context>
                        <Bundle-ClassPath>WEB-INF/classes, WEB-INF/lib/service-dispatch-api-1.0.0.jar, WEB-INF/lib/vaadin-6.1.3.jar</Bundle-ClassPath>
                    </manifestEntries>
                  </archive>
                </configuration>
              </plugin>
     

This bit of magic allows Maven to build us a MANIFEST.MF like this:

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: mnash
Build-Jdk: 1.6.0_15
Extension-Name: admin
Implementation-Title: admin
Implementation-Version: 1.0-SNAPSHOT
Bundle-Activator: com.point2.ServiceClient
Bundle-ClassPath: WEB-INF/classes, WEB-INF/lib/service-dispatch-api-1.
 0.0.jar, WEB-INF/lib/vaadin-6.1.3.jar
Bundle-ManifestVersion: 2
Bundle-Name: Admin
Bundle-SymbolicName: com.point2.Admin
Bundle-Version: 1.0.0
Import-Package: org.osgi.framework,com.point2.services.dispatch,javax.
 servlet;version="2.4.0",javax.servlet.http;version="2.4.0",org.osgi.s
 ervice.http;version="1.2.0",org.osgi.util.tracker;version="1.3.2"
Webapp-Context: cadmin

Now we have our OSGi-friendly .war file, sitting in our target directory. We can then connect to the console of our OSGi container (in this case, ServiceMix), and say:

install war:file:/Users/mnash/experiments/admin/target/admin-1.0.0-SNAPSHOT.war

Our container honors the “Webapp-Context” tag in our manifest, so we can then surf to http://localhost:8181/admin/ to see our application. 8181 is the default port for the Jetty HTTP service in ServiceMix – it can easily be changed to another number as required, of course.

Dispatch Service
Now that we can see how to get the wepapp into ServiceMix, let’s look at the Dispatch Service in detail.

Our Dispatch Service is going to be a regular OSGi bundle, so we have a separate Maven project that produces a .jar file in this case, not a WAR file.

Our MANIFEST.MF needs the following magic for this project:

Bundle-ManifestVersion: 2
Bundle-SymbolicName: com.point2.services.dispatch.DispatchService
Bundle-Name: DispatchService
Bundle-Version: 1.0.0
Bundle-Classpath: .,lib/commons-beanutils-1.8.0.jar,lib/commons-collections-3.2.1.jar
Bundle-Activator: com.point2.services.dispatch.impl.DispatchServicePublisher
Import-Package: org.osgi.framework
Export-Package: com.point2.services.dispatch

You can see we’re again specifying a “Bundle-Activator” class, which, much like the ServiceClient class in our webapp, is called by the OSGi framework when the bundle containing this service is started.

One slight oddity: The dispatch service needs the two jars listed under “Bundle-Classpath:” to do it’s business – because we are building an OSGi bundle, we “embed” these jars in our bundle (which is, yes, another jar) by putting them in the src/main/resources/lib directory. We refer to them in that location in the POM, and Maven automatically includes them in the finished jar, where they’re available when our bundle needs them. The other alternative is to install the dependencies as their own bundles, but that’s a whole ‘nother post

In the case of DispatchService, we’ve got an activator like this:

public class DispatchServicePublisher implements BundleActivator {
    private ServiceRegistration registration;

    public void start(BundleContext context) throws Exception {
        registration = context.registerService(DispatchService.class.getName(),
                new DispatchServiceImpl(), null);
        DispatchServiceImpl.setBundleContext(context);
        System.out.println("Dispatch Service registered");
    }

    public void stop(BundleContext context) throws Exception {
        registration.unregister();
        System.out.println("Dispatch Service unregistered");
    }
}

Which simply takes a reference to the BundleContext on startup and passes it to the DispatchServiceImpl, which implements the DispatchService interface, like so:

public interface DispatchService {
   List<Map<String, Object>> call(String serviceName, String serviceOperation, Map<String, Object> parameters,
                                  String versionPattern, String securityToken) throws Exception;
}

The big JuJu with OSGi is that only this interface is “exposed” from the bundle. No other service can see the innards of our service, unlike Jars on a classpath.

The implementation of the DispatchService is equally straightforward:

public class DispatchServiceImpl implements DispatchService {

    private static BundleContext context;

    public List<Map<String, Object>> call(String serviceName, String serviceOperation, Map<String, Object> parameters, String versionPattern, String securityToken) throws Exception {

        ServiceReference reference = getServiceNamed(serviceName);
        if (reference == null)
            throw new RuntimeException("There is no service with service-name " + serviceName);
        Object service = context.getService(reference);
        Adapter adapter = new Adapter(service);
        List<Map<String, Object>> response = adapter.callList(serviceName, serviceOperation, parameters);
        context.ungetService(reference);
        return response;
    }

    private ServiceReference getServiceNamed(String serviceName) throws Exception {
        ServiceReference[] references = context.getAllServiceReferences(null, null);
        System.out.println("There are " + references.length + " services available");
        for (int i = 0; i < references.length; i++) {
            ServiceReference reference = references[i];
            Object serviceNameValue = reference.getProperty("service-name");
            if (serviceNameValue != null) {
                System.out.println("I see service with name " + serviceNameValue);
                if (serviceNameValue.toString().equalsIgnoreCase(serviceName))
                    return reference;
            }
        }
        throw new RuntimeException("There is no service available with service-name " + serviceName);
    }

    public static void setBundleContext(BundleContext newContext) {
        context = newContext;
    }
}

The dispatch service implementation simply looks through the “published” services in the OSGi context, and looks at the “service-name” property of each service to find the one specified in the call. Assuming it finds an appropriate service, it then uses another class, Adapter, to do the type conversion of the generic Map of parameters to the specific types needed for the service being called, then uses Java reflection to actually make the call and return the response as a List of Maps, again converting the service-specific types to a generic Map in order to pass the response back to the user interface.

Why do we go through those gyrations, instead of just having access to the domain-specific beans in our UI? If we want a full decoupling, and the advantages that come with it, the type-independence of this approach gives us that. It also allows parallel development of the UI and the back-end services without an ever-increasing number of binary dependencies as well – for that matter, with the static data adapter in the Vaadin app allows us to develop on our UI entirely without the back-end services. That’s pretty decoupled.

The Dispatch Service by itself, though, doesn’t give us any functionality. It acts much like a router on a network, simply moving the request from the client to the proper service on the back-end, so let’s build such a service – in this case, a service for handling Dealers.

Dealer Service
The bulk of our application-specific code will be prepared as independent OSGi services, just like this one. In later postings, I’ll describe how to set up dependencies between services (and how to use Spring DM to make doing that kind of thing far simpler).

First, let’s look at our MANIFEST.MF for our new service (again, we can use Maven to produce this for us, but the result is the same):

Bundle-ManifestVersion: 2
Bundle-SymbolicName: com.point2.services.dealer.DealerService
Bundle-Name: DealerService
Bundle-Version: 1.0.0
Bundle-Activator: com.point2.services.dealer.impl.DealerServicePublisher
Import-Package: org.osgi.framework
Export-Package: com.point2.services.dealer

The two key elements above are the Bundle-Activator, a class called DealerServicePublisher, and the Export-Package.

OSGi best practices dictate that the package that is exported should only contain the interface for the class (or classes) you wish to make available from your service. In our case, that comprises a bean class, called “Dealer”, and the interface to our actual service, DealerService.

The Dealer bean is very simple, just your basic property-holding JavaBean:

public class Dealer {
    private String name;
    private String phone;
    private String contactLast;
    private String contactFirst;
    private String address1;
    private String address2;

    public String getContactLast() {
        return contactLast;
    }

    public void setContactLast(String contactLast) {
        this.contactLast = contactLast;
    }

    public String getContactFirst() {
        return contactFirst;
    }
   .....

I’ve ommitted the rest of the getters and setters here for brevity (in case you’re wondering, yes, this is much easier to express in Scala, and yes, OSGi works just fine with Scala…)

The interface for our service is even simpler:

public interface DealerService {
    public void save(Dealer dealer);
    public Dealer findById(int id);
}

Of course, a fully fleshed-out service might in fact have more methods, but again, you get the idea.

I won’t bore you with the actual implementation of the DealerServiceImpl class, but it’s important to note that it’s not in the same package as the Dealer bean and the DealerService implementation. They are the only two classes (well, one class, one interface) in that package, as the entire package is “exported” by OSGi, and therefore visible to other services and clients.

The DealerServicePublisher is the last piece to examine, and it’s pretty straightforward as well:

package com.point2.services.dealer.impl;

import com.point2.services.dealer.DealerService;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import org.osgi.framework.ServiceRegistration;

import java.util.Dictionary;
import java.util.Hashtable;

public class DealerServicePublisher implements BundleActivator {
    private ServiceRegistration registration;

    public void start(BundleContext context) throws Exception {
        System.out.println("Registering dealer service");
        Dictionary dictionary = new Hashtable();
        dictionary.put("service-name", "dealer");
        registration = context.registerService(DealerService.class.getName(),
                new DealerServiceImpl(), dictionary);
    }

    public void stop(BundleContext context) throws Exception {
        System.out.println("Unregistering dealer service");
        registration.unregister();
    }
}

Essentially all we do in this activator is “register” our service, making it visible to the OSGi container and other services or clients that need it. We add an extra property via the “Dictionary” object, which allows us to specify arbitrary properties to be associated with our service. Because we want to look up our services from the dispatcher by name, rather than by class or interface name, we use the string “dealer” and associate it with the key “service-name”. If you examine the code for our DispatchService, you’ll see that it uses this property to find services.

Now we can build our new service with a simple “mvn package” command, and install the resulting jar into our OSGi runtime with “install file:/Users/mnash/experiments/dealer/target/dealer-1.0.0.jar” (again, you can do an install directly from the Maven repository, or from a URL, as opposed to a file).

That’s it – our “dealer” service is now available, and our “dispatch” service is fired up and ready to locate it.

If we deploy our Vaadin application in our OSGi container (as described in the previous post), you’ll find that calls to the “call” method of the ServiceClient now return the “real” data from our service.

In the next few posts we’ll examine an even easier way to define new services, and explore the power of OSGi for updating and working with our decoupled services. Then we’ll look at how to hook services together, allowing services to call other services to do their jobs as required.

In my experience, the future of modularity on the JVM is OSGi.

Many developers don’t seem to recognize the need for it, but other bloggers have covered this in great detail (see my Resources page for some links), so I’m not going to try to “sell” OSGi here at all – I assume you’re already sold, and looking at how to put OSGi to good use in your development and deployment process.

Extending my recent experiments with the Vaadin framework, I decided I wanted to have a Vaadin front-end talking to a set of OSGi services on the back end. Initially, these will be running within the same OSGi container, which in this case is FUSE 4, the commercially supported variant of Apache ServiceMix.

One of my goals was to acheive a loose coupling between the Vaadin webapp and the backing services, so that new services can readily be added, started, stopped, and updated, all without any impact on the running Vaadin app. I also wanted to maintain the convenience of being able to run and tinker with the UI portion of my app by just doing a “mvn jetty:run”, so the app needed to be able to start even if it wasn’t inside the OSGi container.

Fortunately, doing all this is pretty easy, and in the next series of articles I’ll describe how I went about it, and where the good parts and bad parts of such an approach became obvious.

In this part, we’ll start by describing the Vaadin app, and how it calls the back-end services. In later parts, I’ll describe the evolution of the back-end services themselves, as I experimented with more sophisticated techniques.

Vaadin Dependency
I’m building all my apps with Apache Maven, so the first step was to create a POM file suitable for Vaadin. Fortunately, Vaadin is a single jar file, and trivial to add to the classpath. My POM needed this dependency:

       <dependency>
            <groupId>vaadin</groupId>
            <artifactId>vaadin</artifactId>
            <version>6.1.3</version>
            <scope>system</scope>
            <systemPath>${basedir}/src/main/webapp/WEB-INF/lib/vaadin-6.1.3.jar</systemPath>
        </dependency>
     

Here I’m using the trick of specifying a systemPath for my jar, instead of retrieving it on demand from a local Nexus repository or from the internet, but that’s just one way of doing it – the main thing is to get this one Vaadin jar on your path.

web.xml
Next I proceeded to tweak my web.xml to have my top-level Vaadin application object available on a URL. The main Vaadin object is an extension of a Servlet, so this is also very easy – here’s my web.xml in it’s entirety:

     <?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5">
  <display-name>Admin</display-name>
  <servlet>
    <servlet-name>Admin</servlet-name>
    <servlet-class>com.vaadin.terminal.gwt.server.ApplicationServlet</servlet-class>
    <init-param>
      <param-name>application</param-name>
      <param-value>Admin</param-value>
    </init-param>
  </servlet>
  <servlet-mapping>
    <servlet-name>Admin</servlet-name>
    <url-pattern>/*</url-pattern>
  </servlet-mapping>
</web-app>

In this situation my “Admin” class is in the default package, which is not generally a good practice, but you get the idea.

Menu and MenuDispatcher
I then used the “Tree” class in Vaadin to build myself a nice tree-style menu, and added that to the layout on my main page. My main page has some chrome regions for a top banner and other assorted visual aids, then a left-side area where the menu lives, and a “main” area, where all the real action in the application will happen.

A class I called MenuDispatcher “listens” for events on the menu (e.g. the user clicking something), and does the appropriate action when a certain menu item is clicked.

Here’s the interesting bits from the MenuDispatcher – as you can see, it’s constructed with a reference to the “mainArea” layout when it’s first initialized.

     public class MenuDispatcher implements ItemClickEvent.ItemClickListener {

    private VerticalLayout mainArea;

    public MenuDispatcher(VerticalLayout mainArea) {
        this.mainArea = mainArea;
    }

    public void itemClick(ItemClickEvent event) {
        if (event.getButton() == ItemClickEvent.BUTTON_LEFT) {
            String selected = event.getItemId().toString();

            System.out.println("Selected " + selected + " from menu");
            if (selected.equals("create dealer")) {
                createDealer();
            } else if (selected.equals("edit dealers")) {
                editDealer();
            }
...
            }
            System.err.println("Selected " + event.getItemId());
        }
    }

    private void createDealer() {
        mainArea.removeAllComponents();
        Component form = new CreateDealerForm();
        mainArea.addComponent(form);
        mainArea.setComponentAlignment(form, Alignment.MIDDLE_CENTER);
        mainArea.requestRepaint();
    }

    private void editDealer() {
...
    }

...
}

Again, this code can be made more sophisticated – I’m thinking a little Spring magic could make adding new forms and such very convenient, but this gets us started.

Submitting the Form
The “CreateDealerForm” object referred to in the Dispatcher then builds a Vaadin “Form” class, much like the example form built in the “Book of Vaadin”. The only interesting part of my form was that I chose not to back it with a Bean class, which is an option with Vaadin forms. If you back with a bean, then you essentially bind the form to the bean, and the form fields are generated for you from the bean.

If I wanted to then send the corresponding bean to the back-end service, then binding the bean to the form would be a good way to go. Instead, however, I don’t want my back-end services to be sharing beans with my UI application at all. I’ll explain why and how later on in more detail.

The interesting part of my form, then, is how I handle the submission of the form:

Assuming I have a button on my form, defined like so:

      Button okbutton = new Button("Submit", dealerForm, "commit");

I can add a listener to this button (again, using Vaadin’s magic ability to route the Ajax events to my Java code) like thus:

      okbutton.addListener(new Button.ClickListener() {
            public void buttonClick(Button.ClickEvent event) {

                Map<String, Object> parameters = new HashMap<String, Object>();
                for (Object id: dealerForm.getItemPropertyIds()) {
                    Field field = dealerForm.getField(id);
                    parameters.put(id.toString(), field.getValue());
                }

                System.out.println("*** Calling dealer service via dispatcher");
                ServiceClient.call("dealer", "save", parameters, null, null);
                getWindow().showNotification("Dealer Saved");
            }
        });

I’m using an anonymous inner class to listen to the event, and the “buttonClick” method gets called when the user says “Submit”.

The next steps are where the form meets the back-end service: First, I iterate over the form and build a map containing all the field values. The map is keyed with a string, for the name of the field or property, and the value in the map is the value the user entered. Note that these values are already typed – e.g. a checkbox can have a boolean, a TextField can have a string, a calendar field can have a java.util.Date. We retain these types, and wrap everything up in a map.

Now (after a quick println so I can see what’s going on), I call a static method on class called ServiceClient, sending along the name of the service I want to call, the operation on that service, and the parameter map I just built from my form.

The last line just shows a nice “fade away” non-modal notification to the user that the dealer he entered has been saved (assuming the call to ServiceClient didn’t throw an exception, which we’re assuming for the moment for simplicity).

So, now we have our “call” method to consider, where the map of values we built in our Vaadin front-end gets handed off to the appropriate back-end service.

Calling the Service

The only job of the ServiceClient object is to route a call from somewhere in our Vaadin app (in this case, the submission of a form) to the proper back-end service, and potentially return a response.

We identify our back-end services by a simple string, the “service name” (our first argument). The second argument to call tells us the “operation” we want from this service, as a single service can normally perform several different operations. For instance, our dealer service might have the ability to save a dealer, get a list of dealers, find a specific dealer, and so forth.

In Java parlance, a service operation might correspond to a method on the service interface, but we’re not coupling that tightly at this point – our service, after all, might not be written in Java for all we know at this point, and I prefer to keep it that way.

This is the “loose joint” in the mechanism between the UI and the back-end. To keep the joint loose, we don’t send a predefined Bean class to the back-end service to define the parameters to the service operation, we send a map, where that map contains a set of key/value pairs. The keys are always Strings, but the values can be any type – possibly even another map, for instance, which would allow us to express quite complex structures if required, in a type-independent fashion.

Let’s have a look at ServiceClient:

     public class ServiceClient implements BundleActivator {

    private static DispatchService dispatchService = null;

    public void start(BundleContext context) throws Exception {
            ServiceReference reference = context.getServiceReference(DispatchService.class.getName());
            if (reference == null)
                throw new RuntimeException("Cannot find the dispatch service");
            dispatchService = (DispatchService) context.getService(reference);
            if (dispatchService == null)
                throw new RuntimeException("Didn't find dispatch service");
    }

    public void stop(BundleContext context) throws Exception {
        System.out.println("Stopping bundle");
    }

    public static List<Map<String, Object>> call(String serviceName, String operation, Map<String, Object> params, String versionPattern, String securityToken) throws Exception {
            if (dispatchService == null) {
                System.out.println("No OSGi dispatch service available - using dummy static data");
                return StaticDataService.call(serviceName, operation, params, versionPattern, securityToken);
            }
            return dispatchService.call(serviceName, operation, params, versionPattern, securityToken);
    }
}

Let’s go through that class piece by piece. First, you’ll notice that the class implements the BundleActivator interface – this tells the OSGi container that it is possible to call this class when the OSGi bundle containing it is started and stopped. During the start process, you can have the class receive a reference to the BundleContext. This is somewhat analagous to the Spring ApplicationContext, in that it gives our class access to the other services also deployed in OSGi. The Spring DM framework lets you do this kind of thing more declaratively, but it’s good to know how the low-level works before engaging the autopilot, I always find.

In order to allow BundleActivator to be found, we need another couple of things on our classpath, so we add this to our POM:

       <dependency>
            <groupId>org.osgi</groupId>
            <artifactId>osgi_R4_core</artifactId>
            <version>1.0</version>
        </dependency>

        <dependency>
            <groupId>org.osgi</groupId>
            <artifactId>osgi_R4_compendium</artifactId>
            <version>1.0</version>
        </dependency>

This defines the BundleActivator interface and other OSGi requirements which we use.

As you can see, we use our reference to the OSGi context to get a ServiceReference to an interface called “DispatchService”. We’ll examine dispatch service in detail in my next posting, but for now you can see we hold on to our reference as an instance variable.

When the call to the “call” method happens, our DispatchService reference is already available if we’re running inside an OSGi container, all wired up and ready to go. To give us flexibility, though, we also allow for the case where dispatch service is null, meaning we’re not running inside and OSGi container.

Instead of crashing and burning, however, we simply redirect our call to a “StaticDataService” class, which does just what you might expect. For every call it understands, it returns a static “canned” response. This allows us to build and test our UI without actually having written any of the back-end services, and to continue to run our Vaadin app with a simple “mvn jetty:run”, when all we’re working on is look and feel, or logic that only affects the UI.

This means my “cycle time” to test a change in the UI code is a matter of seconds – when I do a “mvn jetty:run” my code is compiled and up and running in my browser in about 5 seconds, and that’s on my 5 year-old macbook laptop, so there’s no penalty for not having a dynamic language in the mix here, from my point of view.

If DispatchService is not null, however, then we’re running “for real” inside an OSGi container, so we use our stored reference to the dispatch service to “forward on” our call. The dispatch service works it’s magic, which we’ll examine in a later post, and returns a List of Map objects with our response. This list might only contain one Map, of course, if the service was a simple one.

The response is then returned to the caller elsewhere in the Vaadin application, to do whatever is necessary from a UI point of view – perhaps populate a form or table with the response data.

As we’ll see in detail in my next post, the dispatch service in this case acts as a “barrier” between the UI-oriented code in our Vaadin app and our domain-specific application logic contained in our services. It is responsible for mapping our generic map of parameters into whatever domain beans are used by our back-end services, and for figuring out which of those services should be called, and which operation on that service is required. Those services then return whatever domain-specific objects they return, and the dispatcher grinds them into non-type-bound maps, or lists of maps, if there is a whole series of returns.

This means our Vaadin app only ever has to talk to one thing: the dispatcher. We only change our Vaadin code for one reason: to change the UI, never in response to a change of service code, even if the beans and classes of that service change significantly.

Next time we’ll look at the dispatch service and the first of our application-specific domain-aware services, and see how they come together.

I’ve had the chance over a recent weekend to have a first crack at a web framework called Vaadin.

I was originally browsing for news about the latest release of Google’s GWT framework when I stumbled on a reference to Vaadin, and decided to go take a look. What I found intrigued me, and I decided to take it for a test drive, as I was down sick for a couple of days with a laptop nearby…. My back became annoyed with me, but it was worth it, I think.

First Look
First off, the practicalities: Vaadin is open source, and with a reasonable license, the Apache License. The essential bits of Vaadin are contained in a single JAR, and it’s both Ant and Maven friendly right out of the box.

The next thing that struck me about Vaadin was the documentation. The first unusual thing about it’s documentation was the fact of it’s existance, as open source projects are downright notorious for poor documentation. Vaadin is a pleasant exception, with tons of examples, a well-organized API doc, in the usual JavaDoc format, and even the “Book of Vaadin”, an entire PDF book (also available in hardcopy) that takes you through Vaadin in enough detail to be immediately productive.

Given that surprisingly pleasant start, I dug deeper, creating a little app of my own.

Just the Java, Ma’am
The main thing that kept me interested in Vaadin once I started digging further was that it’s pure Java. Many frameworks talk about writing your UI work in Java, such as Wicket, but there’s still a corresponding template and some wiring that happens to put the code and the template together. Not so with Vaadin.

When they say “just Java”, they mean it – your entire UI layer is coded in Java, plain and simple. No templates, no tag libraries, no Javascript, no ‘nuthin. It’s reminiscent of the Echo framework, except in Vaadin’s case the Javascript library that your code automatically produces is Google’s GWT, instead of Echo’s own Core.JS library.

Unlike GWT, though, the Vaadin approach doesn’t bind you to any specific source code though, it’s just a binary jar you put on your classpath.

The only thing in my sample app, other than 2 Java files, was a web.xml and a css stylesheet, both of which were only a few lines long. And this was no “Hello, World”, either, but a rich AJAX webapp with a tree menu, fancy non-modal “fading” notifications, images, complex layouts, and a form with build-in validation. And it took maybe 4 hours of total work to produce – and that was from a standing start, as I’d never heard of Vaadin before last Thursday. Not bad, not bad at all.

I found I was able to get a very capable little webapp up and running with no need to invent my own components, even though I had trees and sliders and menus and other assorted goodies on the page. It worked in every browser I was able to try it in, which is certainly not the case for my own hand-rolled JavaScript most of the time

I haven’t yet tried creating my own custom components, but it certainly looks straightforward enough.

I did try linking to external resources, and included non-Vaadin pages in my app, with no difficulties, so it appears that Vaadin plays well with others, and can be introduced into an existing project that uses, for instance, a whack of JSP’s that one might want to obsolete.

Webapps
I think Vaadin warrants more exploration, and I intend to put it further through its paces in the next few weeks. It appears extremely well-suited to web applications, as opposed to websites with a tiny bit of dynamic stuff in them.

It offers an interesting alternative to some of the patterns I’ve seen for advanced dynamic webapp development so far.

One approach I’ve seen a lot is to divide the duties of creating an app into the “back end” services and the “UI”. Generally the UI is written in either JavaScript, or uses Flex or some other semi-proprietary approach. The “back end” stuff is frequently written to expose it’s services as REST, then the two are bolted together. The pain point here happens when the two meet, as it’s common and easy to have minor (or major!) misunderstandings between the two teams. This usually results in a lot of to-and-fro to work out the differences before the app comes all the way to life.

The other approach, more common on smaller or resource-strapped teams, is to have the same group responsible for both UI and back-end services. This reduces the thrash in the joints a bit, but doesn’t eliminate it, because the two technologies on the two sides of the app aren’t the same. You can’t test JavaScript the same way you write Java, for instance, and they’re two different languages – one of which (Java) has far better tooling support than the other. IDE support, for instance, is superb for Java, and spotty at best for JavaScript.

With Vaadin, both of these approaches become unnecessary, as its the same technology all the way through (at least, what you write is – technically it’s still using JavaScript, but because that’s generated, I don’t count it).

You get to use all of the tools you know and love for the back-end services to write the code for the UI, which you can then unit and functional test to your heart’s content.

The temptation to mix concerns between UI code and back-end service code must still be resisted, of course, but at least that code isn’t buried somewhere in the middle of a JSP page, ready to leap out and bite you later.

Because you’re using dynamic layouts, the app always fits properly on the screen without any extra work, addressing a pet peeve of mine, the “skinny” webapp, restraining itself to the least common denominator of screen size, thus rendering impotent my nice wide monitors.

Scala
Just because Vaadin is a Java library doesn’t restrict you to using Java to drive it, however. I made another little webapp where the whole UI was defined in Scala, calling the Vaadin APIs, and it worked like a charm. In some ways, Scala is an even better fit for Vaadin than straight Java, I suspect. I haven’t tried any other JVM compatible language, but I see no reason they wouldn’t work equally well.

Deployment and Development Cycle
As I was building the app with Maven, I added a couple of lines to my POM and was able to say “mvn jetty:run” to get my Vaadin app up and running on my local box in a few seconds. My development cycle was only a few seconds between compile and interactive tests, as I was experimenting with the trial-and-error method.

TDD would be not only possible, but easy in this situation.

I successfully deployed my little Vaadin app to ServiceMix, my OSGi container of choice, without a hitch.

Performance appeared excellent overall, although I haven’t formally tested it with a load-testing tool (yet).

Summary
So far, I’m impressed with Vaadin. I’m more impressed with any web framework I’ve worked with in a number of years, in fact. I’m sure there are some warts in there somewhere, but for the benefits it brings to the table, I suspect they’re easily worth it. I think the advantages to teams that already speak fluent Java is hard to overstate, and the productivity to produce good-looking and functioning webapps is quite remarkable.

One of the most common, and, in my opinion, valid criticism’s of Apache Maven is the way it handles dependencies.

The default way of doing dependencies in Maven is for a build to look for dependency jars in one of a specified number of places. If you don’t specify a location, it will start with your local Maven repository, then a stock set of online repositories (such as ibiblio).

What this means is that you can quickly and easily add a dependency to your project by just listing it in the POM, and Maven will frequently just go get it for you as required. For dependencies that don’t change rapidly (which is generally a bad idea for entirely separate reasons), the local copy is always used, so after the first download you don’t go get the jar again unless it changes.

The downside of this (and hence the basis for the criticism) is that your local repository is considered a cache, and is not checked in to your source control system. Maven aficionados believe that source control should be for what the name implies: source, not jars (which are an artifact, not source code). This is good, except in situations where the local repository is removed (as it’s a cache, and not “backed up” by being in source control) and you want to rebuild your project. Normally, no problem: Maven will simply go back out into the wilds of the internet and get your jars for you again. Except when it can’t, which is where the problem starts.

Let’s say your project depends on foo-1.0.jar, for the sake of discussion. Foo-1.0.jar is found in a repository at http://baz.com. No problem. Two months from now, you come back and make a change to your project, and your local repo doesn’t have foo-1.0.jar in it. Maven goes to get it for you, but baz.com has gone offline. Oops. Now you can’t build your project at all. Worse, let’s say you *do* find foo-1.0.jar, but it’s been updated. Granted, this is a gross violation of the version scheme, but as baz.com is not under your control, it *can* happen – and Murphy was an optimist.

The first level of defense against this is to have a local jar proxy that *is* backed up, like Nexus. Nexus not only provides a safe haven for the exact versions of Jars you need to build your project, but also proxies new jars automatically, when set up correctly. That way, once you use that external dependency, it’s available on the local proxy immediately forever more, in precisely the correct version.

This solution is so good, that I always consider Nexus and Maven to be two parts of one solution – as you don’t really have reliable and repeatable builds without Nexus in the mix.

Another way to go, however, is perhaps even more straightforward, and might be appropriate if you have a limited number of “internal” binary dependencies – that is, binary dependencies that you generate within your organization between projects.

This is the option I want to describe in this blog, and it takes a form very similar to the old Ant style of having a “lib” directory in your project, and checking jars into source control.

You simply create a directory (called lib or whatever you wish, but “lib” might be more standard), and put your jar dependencies in it. Then you refer to each dependency in your pom.xml file with a special syntax, like so:

    <dependency>
      <groupId>mygroup</groupId>
      <artifactId>myjar</artifactId>
      <version>1.0.0</version>
      <scope>system</scope>
      <systemPath>${basedir}/src/main/resources/lib/myjar-1.0.0.jar</systemPath>
    </dependency>

Now the myjar-1.0.0.jar will be on my classpath during compilation, easy as that. I can then check in that nicely versioned jar file, and have a fully self-contained project, while at the same time being able to leverage the power of Maven and it’s rich ecosystem of plugins. If you’re using the shade plugin, this is an excellent way to create a self-contained executable jar file containing all of your necessary dependencies in one shot, rather than having to worry about classpaths at runtime.

In web applications, you can do even better. By placing the dependency in the correct spot in your source tree, you can have it automatically included in the finished .war file, while at the same time finding it in your classpath during compile time, like so:

    <dependency>
      <groupId>xstream</groupId>
      <artifactId>xstream</artifactId>
      <version>1.3.1</version>
      <scope>system</scope>
      <systemPath>${basedir}/src/main/webapp/WEB-INF/lib/xstream-1.3.1.jar</systemPath>
    </dependency>

This includes the xstream jar in my webapp. Every IDE that can digest a POM file (which include my 3 main ones, IntelliJ IDEA, Eclipse IDE and Netbeans) also automagically find the dependencies.

One minor constraint here is that you must use a *versioned* jar file – e.g. it must be named according to the artifact-version.jar pattern. I consider this a feature, not a limitation, as I’m a firm believer that all artifacts should be properly versioned, so I can tell what source code originally created them. Yes, I know that can be done internally, in a manifest or such, but I like to see the label on the outside of the box, so to speak.

As one colleague put it, now you can be assured of coming back to your project months or years later, even if you don’t have internet connectivity, and be able to build the darn thing again, without going off on a wild jar chase all over the internet.

This pattern can overcome a serious and legitimate objection to Maven, and might be appropriate in many situations. Give it a try, let me know what you think!