»
S
I
D
E
B
A
R
«
OSGi Adventures, Part 1 – A Vaadin front-end to OSGi Services
Nov 16th, 2009 by Mike

In my experience, the future of modularity on the JVM is OSGi.

Many developers don’t seem to recognize the need for it, but other bloggers have covered this in great detail (see my Resources page for some links), so I’m not going to try to “sell” OSGi here at all – I assume you’re already sold, and looking at how to put OSGi to good use in your development and deployment process.

Extending my recent experiments with the Vaadin framework, I decided I wanted to have a Vaadin front-end talking to a set of OSGi services on the back end. Initially, these will be running within the same OSGi container, which in this case is FUSE 4, the commercially supported variant of Apache ServiceMix.

One of my goals was to acheive a loose coupling between the Vaadin webapp and the backing services, so that new services can readily be added, started, stopped, and updated, all without any impact on the running Vaadin app. I also wanted to maintain the convenience of being able to run and tinker with the UI portion of my app by just doing a “mvn jetty:run”, so the app needed to be able to start even if it wasn’t inside the OSGi container.

Fortunately, doing all this is pretty easy, and in the next series of articles I’ll describe how I went about it, and where the good parts and bad parts of such an approach became obvious.

In this part, we’ll start by describing the Vaadin app, and how it calls the back-end services. In later parts, I’ll describe the evolution of the back-end services themselves, as I experimented with more sophisticated techniques.

Vaadin Dependency
I’m building all my apps with Apache Maven, so the first step was to create a POM file suitable for Vaadin. Fortunately, Vaadin is a single jar file, and trivial to add to the classpath. My POM needed this dependency:

       <dependency>
            <groupId>vaadin</groupId>
            <artifactId>vaadin</artifactId>
            <version>6.1.3</version>
            <scope>system</scope>
            <systemPath>${basedir}/src/main/webapp/WEB-INF/lib/vaadin-6.1.3.jar</systemPath>
        </dependency>
     

Here I’m using the trick of specifying a systemPath for my jar, instead of retrieving it on demand from a local Nexus repository or from the internet, but that’s just one way of doing it – the main thing is to get this one Vaadin jar on your path.

web.xml
Next I proceeded to tweak my web.xml to have my top-level Vaadin application object available on a URL. The main Vaadin object is an extension of a Servlet, so this is also very easy – here’s my web.xml in it’s entirety:

     <?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5">
  <display-name>Admin</display-name>
  <servlet>
    <servlet-name>Admin</servlet-name>
    <servlet-class>com.vaadin.terminal.gwt.server.ApplicationServlet</servlet-class>
    <init-param>
      <param-name>application</param-name>
      <param-value>Admin</param-value>
    </init-param>
  </servlet>
  <servlet-mapping>
    <servlet-name>Admin</servlet-name>
    <url-pattern>/*</url-pattern>
  </servlet-mapping>
</web-app>

In this situation my “Admin” class is in the default package, which is not generally a good practice, but you get the idea.

Menu and MenuDispatcher
I then used the “Tree” class in Vaadin to build myself a nice tree-style menu, and added that to the layout on my main page. My main page has some chrome regions for a top banner and other assorted visual aids, then a left-side area where the menu lives, and a “main” area, where all the real action in the application will happen.

A class I called MenuDispatcher “listens” for events on the menu (e.g. the user clicking something), and does the appropriate action when a certain menu item is clicked.

Here’s the interesting bits from the MenuDispatcher – as you can see, it’s constructed with a reference to the “mainArea” layout when it’s first initialized.

     public class MenuDispatcher implements ItemClickEvent.ItemClickListener {

    private VerticalLayout mainArea;

    public MenuDispatcher(VerticalLayout mainArea) {
        this.mainArea = mainArea;
    }

    public void itemClick(ItemClickEvent event) {
        if (event.getButton() == ItemClickEvent.BUTTON_LEFT) {
            String selected = event.getItemId().toString();

            System.out.println("Selected " + selected + " from menu");
            if (selected.equals("create dealer")) {
                createDealer();
            } else if (selected.equals("edit dealers")) {
                editDealer();
            }
...
            }
            System.err.println("Selected " + event.getItemId());
        }
    }

    private void createDealer() {
        mainArea.removeAllComponents();
        Component form = new CreateDealerForm();
        mainArea.addComponent(form);
        mainArea.setComponentAlignment(form, Alignment.MIDDLE_CENTER);
        mainArea.requestRepaint();
    }

    private void editDealer() {
...
    }

...
}

Again, this code can be made more sophisticated – I’m thinking a little Spring magic could make adding new forms and such very convenient, but this gets us started.

Submitting the Form
The “CreateDealerForm” object referred to in the Dispatcher then builds a Vaadin “Form” class, much like the example form built in the “Book of Vaadin”. The only interesting part of my form was that I chose not to back it with a Bean class, which is an option with Vaadin forms. If you back with a bean, then you essentially bind the form to the bean, and the form fields are generated for you from the bean.

If I wanted to then send the corresponding bean to the back-end service, then binding the bean to the form would be a good way to go. Instead, however, I don’t want my back-end services to be sharing beans with my UI application at all. I’ll explain why and how later on in more detail.

The interesting part of my form, then, is how I handle the submission of the form:

Assuming I have a button on my form, defined like so:

      Button okbutton = new Button("Submit", dealerForm, "commit");

I can add a listener to this button (again, using Vaadin’s magic ability to route the Ajax events to my Java code) like thus:

      okbutton.addListener(new Button.ClickListener() {
            public void buttonClick(Button.ClickEvent event) {

                Map<String, Object> parameters = new HashMap<String, Object>();
                for (Object id: dealerForm.getItemPropertyIds()) {
                    Field field = dealerForm.getField(id);
                    parameters.put(id.toString(), field.getValue());
                }

                System.out.println("*** Calling dealer service via dispatcher");
                ServiceClient.call("dealer", "save", parameters, null, null);
                getWindow().showNotification("Dealer Saved");
            }
        });

I’m using an anonymous inner class to listen to the event, and the “buttonClick” method gets called when the user says “Submit”.

The next steps are where the form meets the back-end service: First, I iterate over the form and build a map containing all the field values. The map is keyed with a string, for the name of the field or property, and the value in the map is the value the user entered. Note that these values are already typed – e.g. a checkbox can have a boolean, a TextField can have a string, a calendar field can have a java.util.Date. We retain these types, and wrap everything up in a map.

Now (after a quick println so I can see what’s going on), I call a static method on class called ServiceClient, sending along the name of the service I want to call, the operation on that service, and the parameter map I just built from my form.

The last line just shows a nice “fade away” non-modal notification to the user that the dealer he entered has been saved (assuming the call to ServiceClient didn’t throw an exception, which we’re assuming for the moment for simplicity).

So, now we have our “call” method to consider, where the map of values we built in our Vaadin front-end gets handed off to the appropriate back-end service.

Calling the Service

The only job of the ServiceClient object is to route a call from somewhere in our Vaadin app (in this case, the submission of a form) to the proper back-end service, and potentially return a response.

We identify our back-end services by a simple string, the “service name” (our first argument). The second argument to call tells us the “operation” we want from this service, as a single service can normally perform several different operations. For instance, our dealer service might have the ability to save a dealer, get a list of dealers, find a specific dealer, and so forth.

In Java parlance, a service operation might correspond to a method on the service interface, but we’re not coupling that tightly at this point – our service, after all, might not be written in Java for all we know at this point, and I prefer to keep it that way.

This is the “loose joint” in the mechanism between the UI and the back-end. To keep the joint loose, we don’t send a predefined Bean class to the back-end service to define the parameters to the service operation, we send a map, where that map contains a set of key/value pairs. The keys are always Strings, but the values can be any type – possibly even another map, for instance, which would allow us to express quite complex structures if required, in a type-independent fashion.

Let’s have a look at ServiceClient:

     public class ServiceClient implements BundleActivator {

    private static DispatchService dispatchService = null;

    public void start(BundleContext context) throws Exception {
            ServiceReference reference = context.getServiceReference(DispatchService.class.getName());
            if (reference == null)
                throw new RuntimeException("Cannot find the dispatch service");
            dispatchService = (DispatchService) context.getService(reference);
            if (dispatchService == null)
                throw new RuntimeException("Didn't find dispatch service");
    }

    public void stop(BundleContext context) throws Exception {
        System.out.println("Stopping bundle");
    }

    public static List<Map<String, Object>> call(String serviceName, String operation, Map<String, Object> params, String versionPattern, String securityToken) throws Exception {
            if (dispatchService == null) {
                System.out.println("No OSGi dispatch service available - using dummy static data");
                return StaticDataService.call(serviceName, operation, params, versionPattern, securityToken);
            }
            return dispatchService.call(serviceName, operation, params, versionPattern, securityToken);
    }
}

Let’s go through that class piece by piece. First, you’ll notice that the class implements the BundleActivator interface – this tells the OSGi container that it is possible to call this class when the OSGi bundle containing it is started and stopped. During the start process, you can have the class receive a reference to the BundleContext. This is somewhat analagous to the Spring ApplicationContext, in that it gives our class access to the other services also deployed in OSGi. The Spring DM framework lets you do this kind of thing more declaratively, but it’s good to know how the low-level works before engaging the autopilot, I always find.

In order to allow BundleActivator to be found, we need another couple of things on our classpath, so we add this to our POM:

       <dependency>
            <groupId>org.osgi</groupId>
            <artifactId>osgi_R4_core</artifactId>
            <version>1.0</version>
        </dependency>

        <dependency>
            <groupId>org.osgi</groupId>
            <artifactId>osgi_R4_compendium</artifactId>
            <version>1.0</version>
        </dependency>

This defines the BundleActivator interface and other OSGi requirements which we use.

As you can see, we use our reference to the OSGi context to get a ServiceReference to an interface called “DispatchService”. We’ll examine dispatch service in detail in my next posting, but for now you can see we hold on to our reference as an instance variable.

When the call to the “call” method happens, our DispatchService reference is already available if we’re running inside an OSGi container, all wired up and ready to go. To give us flexibility, though, we also allow for the case where dispatch service is null, meaning we’re not running inside and OSGi container.

Instead of crashing and burning, however, we simply redirect our call to a “StaticDataService” class, which does just what you might expect. For every call it understands, it returns a static “canned” response. This allows us to build and test our UI without actually having written any of the back-end services, and to continue to run our Vaadin app with a simple “mvn jetty:run”, when all we’re working on is look and feel, or logic that only affects the UI.

This means my “cycle time” to test a change in the UI code is a matter of seconds – when I do a “mvn jetty:run” my code is compiled and up and running in my browser in about 5 seconds, and that’s on my 5 year-old macbook laptop, so there’s no penalty for not having a dynamic language in the mix here, from my point of view.

If DispatchService is not null, however, then we’re running “for real” inside an OSGi container, so we use our stored reference to the dispatch service to “forward on” our call. The dispatch service works it’s magic, which we’ll examine in a later post, and returns a List of Map objects with our response. This list might only contain one Map, of course, if the service was a simple one.

The response is then returned to the caller elsewhere in the Vaadin application, to do whatever is necessary from a UI point of view – perhaps populate a form or table with the response data.

As we’ll see in detail in my next post, the dispatch service in this case acts as a “barrier” between the UI-oriented code in our Vaadin app and our domain-specific application logic contained in our services. It is responsible for mapping our generic map of parameters into whatever domain beans are used by our back-end services, and for figuring out which of those services should be called, and which operation on that service is required. Those services then return whatever domain-specific objects they return, and the dispatcher grinds them into non-type-bound maps, or lists of maps, if there is a whole series of returns.

This means our Vaadin app only ever has to talk to one thing: the dispatcher. We only change our Vaadin code for one reason: to change the UI, never in response to a change of service code, even if the beans and classes of that service change significantly.

Next time we’ll look at the dispatch service and the first of our application-specific domain-aware services, and see how they come together.

Acceptance Testing
Jun 30th, 2009 by Mike

Recently I’ve had the opportunity to consider the right qualities of a tool and/or framework for acceptance testing.

Acceptance tests can be found at a number of different levels, depending on how the story criteria is expressed. (See my previous post on levels of testing) . Often they’re called “executable specifications” if they’re written in such as way as to describe the behavior of a system in a given scenario.

Often they are functional or integration tests, and generally they are best expressed as “black box” tests, that is, tests that have no awareness of the internals of the code or component being tested – all they see is what goes in and what comes out, and they assert their success or failure based on those elements alone.

An acceptance test must be comprehensible to the story author, or whatever domain expert is going to actually do the “accepting” that the story has been satisfied. If they simply take the word of a developer that this big ball of code they see in front of them is testing what they said it should, that’s not as good as if they can actually read and understand the executable specification (test) themselves. Ideally, they should even be able to author their own acceptance tests, without a developer involved – perhaps using existing tests as an example.

So, some of the criteria for a good tool might be:

  1. High Visibility: Something that’s easy to see by all stakeholders
  2. Easily Understood: Something that’s easy to at least read by any non-developer domain expert
  3. Easily Authored: Something that’s easy to learn to write
  4. “Black Box”: Something that has no knowledge of the internals of the code under test, that only interacts with it from “outside the box”
  5. Realistic: Something that tests a version of the application deployed in a realistic environment, as close to production as possible.

Given these criteria, should the tests be part of the build process for the piece of code under test? For acceptance tests that span modules, this is not practical – you can’t actually run the test when you build a module, you can only run it after a certain set of modules has been built (and perhaps even deployed).

I would propose that acceptance tests don’t even belong with the build they’re testing. I think they belong elsewhere, in an independent location where they can be updated as stories are written and verified. Ideally, there should be a shared space for stories for an entire system or suite of applications, as often a piece of criteria spans multiple components – so organizing the tests by component is artificial at best.

Another question that comes up when considering tools for these kinds of tests is whether or not they must be stored and versioned with the code they validate. As a developer, my immediate reaction is “yes” – until I think about it a bit. How does it help me to know what tests a previous version of my code passed or did not pass? All that tells me is where I was in the past, not where I am now. How much inconvenience am I willing to put up with on the part of test authors to get this capability? Dealing with any version control system is extra work, especially for a domain expert/BA who’s not also a developer. I’m now convinced that where I am now is more important than where I was, and being able to easily write and run and make visible tests is more important to me than knowing what happened back in history.

Let’s examine a few different tools and ways of doing acceptance tests and look at the pros and cons:

JUnit
Developers working with Java have a choice of a number of excellent test frameworks, including TestNG and JUnit. They are likely already familiar with them, and the tests fit nicely into the build cycle of Java applications built with any of the more popular project build and management tools, such as Ant, Buildr, Maven, and so forth.

Our acceptance tests are written just like a functional test, in that they fire up whatever context our application should run within, then provide input to it and verify the output with assertions.

This kind of test is not well-suited for acceptance tests for a number of reasons. First, it’s hard to separate the code from the test to ensure we’re truly doing black-box testing. If, for example, we’re within the same VM as the thing we’re testing, it’s very tempting to manipulate objects directly in our test, as opposed to going through, lets say, the publish REST api. This also makes it more difficult to initialize a new testable instance of the application – to be truly independent from it, our test should spawn a whole new JVM from the system under test – which is non-trivial from within Java, although of course re-usable fixtures can be written to take the sting out of it.

This doesn’t help us much when our test spans multiple modules or components, however – then we must either create stubs for each of the components we depend on, or work in a much more complex deployment process to create a “live” version of each of our dependencies.

Finally, a JUnit test is not particularly readable by a non-developer, and not particularly visible, other than as a green or red bar in an IDE or CI server somewhere, so it fails a couple of our most important criteria.

JBehave, easyb, Specs, RSpec
BDD frameworks such as those listed above take our testing power in a different direction. They mostly rely on sophisticated DSL’s, enabling us to write our tests in a much more english-like fashion, often quite readable to the non-developers who have taken the time to learn a bit of the DSL.

They still suffer from some of the other problems described above, however – although some of these tools do offer better output formats to make their results more visible (such as the excellent Forms capability in specs, which can show test results in HTML table formats).

They of course still have a learning curve, as the DSLs in each case are another whole language to be learned.

FitNesse
A different approach can be seen in tools like FitNesse, which allows tests to be created in a Wiki, by editing web pages and inserting special markup to call test “fixtures”. These fixtures still have to be customized to the situation, of course, but with FitNesse we have the potential of being de-coupled from a single module or system under test, and of developing our tests independently of the code altogether.

The table approach of FitNesse still requires the test author to understand the fixtures available to FitNesse – this is essentially FitNesse’s DSL – but of course only a fairly small number of fixtures need be learned to be able to write a wide variety of tests.

Some projects, however, in an attempt to retain the ability to use FitNesse tests to verify previous versions of their application, take the step of checking the FitNesse tests in to their version control system, thereby losing some of that independence.

FitNesse has it’s own ability to track changes to it’s Wiki pages (and thus it’s tests), but this is not tied to the checkins or releases of the software under test.

FitNesse makes it easy for a non-developer domain expert to author tests by using existing tests as templates, then changing the inputs to the fixtures being used to create new tests from the existing building blocks.

These tests and their results are highly visible, especially if the FitNesse wiki is hosted and available to anyone (including developers) to use at any time to verify against a specific deployed instance of the entire suite of applications.

Of course, you’ve still got the issue of deploying a testable instance of your system to deal with in FitNesse. One approach I’ve seen to solve this is to actually have a fixture that can deploy the testable instance as part of the set up for the whole test suite, using versions to indicate the revision of code to be tested – e.g. you have a fixture that says “deploy module X version 123 to test environment 1″, “deploy module Y version 345 to test environment 1″ and so forth, then executes its tests against those deployed instances.

Of course, any database cleanup/reset to get to known state can happen before or just after the deployment, so your tests always start from a known point.

An important part of making this fully automatable is a mutex service: a way to make a call to a known location and say “check out test environment 1″, basically saying that test environment 1 is now busy until it’s released by another call. This ensures that you don’t start another test run on the same environment while it’s still in a transition state.

The mutex can also of course report on who has what environment in use, and since when, to detect failures that might not release a lock.

FitNesse has it’s warts, however, even in a scenario like this, but overall I’ve seen it succeed more often that other approaches for the high-visibility acceptance tests that many projects need, while tools such as easyb, specs, and RSpec are better for describing behaviours within a single module, and executing as part of that module’s build process.

High Ceremony doesn’t have to mean High Cycle Time
Apr 23rd, 2009 by Mike

Oftentimes languages such as Java are called “high ceremony” languages compared to languages like Ruby or Python. This refers to the fact that there’s generally a bit more plumbing involved in firing up a Java application – particularly a web application – than there is with the scripting languages.

Of course, Java is compiled (to byte-code at least), so it’s not quite a 1 to 1 comparison with a more interpreted language such as Ruby, but still, even in a “high ceremony” language it’s important not to get too high a “cycle time” for developers, IMO.

By “cycle time” I mean the time between making a change and seeing it working – either in a test, or, ideally, in a running application. Most modern IDEs made the cycle time for tests pretty darn low (and great tools like Inifinitest can take all the manual work out of it, no less), but to see a running application and be able to excercise your changes deployed in a container is a bit more of a grind.

That’s where a tool like Jetty can come in handy. Jetty is a lightweight web app container that can be easily added to your development cycle in place of a heavier-weight solution to allow you a faster cycle time, and, often, greater productivity and interactivity.

Especially in combination with it’s integration with Maven, Jetty can get your app deployed far faster than with other solutions. For most webapps, it’s just a matter of saying:

mvn jetty:run

And you’ve got a container up and running with your app in it within a few seconds.

Jetty can even do a certain amount of “hot update”: modify a JSP (or even some code – although there are limits) and the running webapp is updated, and you’re able to test, edit… cycle away without the painful wait for a deployment any more often than necessary.

You can pass required system properties to your app via maven’s -D mechanism, and they’ll be available to your app:

mvn -Dsome.property=someValue jetty:run

And even control the port your application binds to on the fly (or via the handy jetty.xml file if you want to set it more permanently).

Jetty and maven also give you the ability to script, for example, if you need to run a test utility on your running webapp to ping a series of REST calls, for example, you can:

mvn clean package # Build the webapp
mvn jetty:run & # start jetty, spawning it in the background
java -jar mytestutility.jar # Run my test jar, which pings the URLs for all my rest services, maybe does performance checks, etc
mvn jetty:stop # Stop the jetty instance we fired up in the background

Lightweight containers such as Jetty are just one way to help crank down the “cycle time” for developers, of course. Some other possibilities I’ll leave for a later entry.

»  Substance: WordPress   »  Style: Ahren Ahimsa