In Part 1 of our OSGi adventures, we described how to build a nice Ajax-aware Vaadin UI app, and couple it to a generic back-end OSGi service we called the service dispatcher.

Now we’ll take the adventure to the next step, show how to deploy that webapp into our OSGi container and get it up and running, then go through the functionality of the Dispatch Service, and show how it routes requires through to the first of our domain-specific application services.

In our UI, we built a form that allowed the user to enter and save a “Dealer”, with some fields like name, phone number, etc., so the first service we’ll build for the service dispatcher to talk to will be our “dealer” service.

First, though, let’s see how to get our webapp into our OSGi container.

As we’re building our Vaadin app with Maven, we can easily add the small bits of additional configuration to turn our project into an OSGi-friendly WAR file.

OSGi deploys “bundles”, but a bundle is just a jar file (or a war, which is after all just a special kind of jar) with a bit of extra meta-data. The META-INF/MANIFEST.MF file is where the magic happens. We add the following to our POM:

              <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.0</version>
                <configuration>
                  <archive>

                      org.springframework.osgi.web.context.support,\
                       org.springframework.web.servlet,\
                       org.springframework.web.servlet.handler,\
                       org.springframework.web.servlet.mvc,\
                       org.springframework.web.servlet.view,\
                       dwmj.domain,\
                       org.springframework.web.servlet.mvc.annotation,\
                       org.springframework.web.context

                    <manifestEntries>
                      <Bundle-ManifestVersion>2</Bundle-ManifestVersion>
                        <Bundle-SymbolicName>com.point2.Admin</Bundle-SymbolicName>
                        <Bundle-Name>Admin</Bundle-Name>
                        <Bundle-Version>1.0.0</Bundle-Version>
                        <Bundle-Activator>com.point2.ServiceClient</Bundle-Activator>
                        <Import-Package>org.osgi.framework,com.point2.services.dispatch,javax.servlet;version="2.4.0",
javax.servlet.http;version="2.4.0",org.osgi.service.http;version="1.2.0",
org.osgi.util.tracker;version="1.3.2"</Import-Package>
                        <Webapp-Context>admin</Webapp-Context>
                        <Bundle-ClassPath>WEB-INF/classes, WEB-INF/lib/service-dispatch-api-1.0.0.jar, WEB-INF/lib/vaadin-6.1.3.jar</Bundle-ClassPath>
                    </manifestEntries>
                  </archive>
                </configuration>
              </plugin>
     

This bit of magic allows Maven to build us a MANIFEST.MF like this:

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: mnash
Build-Jdk: 1.6.0_15
Extension-Name: admin
Implementation-Title: admin
Implementation-Version: 1.0-SNAPSHOT
Bundle-Activator: com.point2.ServiceClient
Bundle-ClassPath: WEB-INF/classes, WEB-INF/lib/service-dispatch-api-1.
 0.0.jar, WEB-INF/lib/vaadin-6.1.3.jar
Bundle-ManifestVersion: 2
Bundle-Name: Admin
Bundle-SymbolicName: com.point2.Admin
Bundle-Version: 1.0.0
Import-Package: org.osgi.framework,com.point2.services.dispatch,javax.
 servlet;version="2.4.0",javax.servlet.http;version="2.4.0",org.osgi.s
 ervice.http;version="1.2.0",org.osgi.util.tracker;version="1.3.2"
Webapp-Context: cadmin

Now we have our OSGi-friendly .war file, sitting in our target directory. We can then connect to the console of our OSGi container (in this case, ServiceMix), and say:

install war:file:/Users/mnash/experiments/admin/target/admin-1.0.0-SNAPSHOT.war

Our container honors the “Webapp-Context” tag in our manifest, so we can then surf to http://localhost:8181/admin/ to see our application. 8181 is the default port for the Jetty HTTP service in ServiceMix – it can easily be changed to another number as required, of course.

Dispatch Service
Now that we can see how to get the wepapp into ServiceMix, let’s look at the Dispatch Service in detail.

Our Dispatch Service is going to be a regular OSGi bundle, so we have a separate Maven project that produces a .jar file in this case, not a WAR file.

Our MANIFEST.MF needs the following magic for this project:

Bundle-ManifestVersion: 2
Bundle-SymbolicName: com.point2.services.dispatch.DispatchService
Bundle-Name: DispatchService
Bundle-Version: 1.0.0
Bundle-Classpath: .,lib/commons-beanutils-1.8.0.jar,lib/commons-collections-3.2.1.jar
Bundle-Activator: com.point2.services.dispatch.impl.DispatchServicePublisher
Import-Package: org.osgi.framework
Export-Package: com.point2.services.dispatch

You can see we’re again specifying a “Bundle-Activator” class, which, much like the ServiceClient class in our webapp, is called by the OSGi framework when the bundle containing this service is started.

One slight oddity: The dispatch service needs the two jars listed under “Bundle-Classpath:” to do it’s business – because we are building an OSGi bundle, we “embed” these jars in our bundle (which is, yes, another jar) by putting them in the src/main/resources/lib directory. We refer to them in that location in the POM, and Maven automatically includes them in the finished jar, where they’re available when our bundle needs them. The other alternative is to install the dependencies as their own bundles, but that’s a whole ‘nother post

In the case of DispatchService, we’ve got an activator like this:

public class DispatchServicePublisher implements BundleActivator {
    private ServiceRegistration registration;

    public void start(BundleContext context) throws Exception {
        registration = context.registerService(DispatchService.class.getName(),
                new DispatchServiceImpl(), null);
        DispatchServiceImpl.setBundleContext(context);
        System.out.println("Dispatch Service registered");
    }

    public void stop(BundleContext context) throws Exception {
        registration.unregister();
        System.out.println("Dispatch Service unregistered");
    }
}

Which simply takes a reference to the BundleContext on startup and passes it to the DispatchServiceImpl, which implements the DispatchService interface, like so:

public interface DispatchService {
   List<Map<String, Object>> call(String serviceName, String serviceOperation, Map<String, Object> parameters,
                                  String versionPattern, String securityToken) throws Exception;
}

The big JuJu with OSGi is that only this interface is “exposed” from the bundle. No other service can see the innards of our service, unlike Jars on a classpath.

The implementation of the DispatchService is equally straightforward:

public class DispatchServiceImpl implements DispatchService {

    private static BundleContext context;

    public List<Map<String, Object>> call(String serviceName, String serviceOperation, Map<String, Object> parameters, String versionPattern, String securityToken) throws Exception {

        ServiceReference reference = getServiceNamed(serviceName);
        if (reference == null)
            throw new RuntimeException("There is no service with service-name " + serviceName);
        Object service = context.getService(reference);
        Adapter adapter = new Adapter(service);
        List<Map<String, Object>> response = adapter.callList(serviceName, serviceOperation, parameters);
        context.ungetService(reference);
        return response;
    }

    private ServiceReference getServiceNamed(String serviceName) throws Exception {
        ServiceReference[] references = context.getAllServiceReferences(null, null);
        System.out.println("There are " + references.length + " services available");
        for (int i = 0; i < references.length; i++) {
            ServiceReference reference = references[i];
            Object serviceNameValue = reference.getProperty("service-name");
            if (serviceNameValue != null) {
                System.out.println("I see service with name " + serviceNameValue);
                if (serviceNameValue.toString().equalsIgnoreCase(serviceName))
                    return reference;
            }
        }
        throw new RuntimeException("There is no service available with service-name " + serviceName);
    }

    public static void setBundleContext(BundleContext newContext) {
        context = newContext;
    }
}

The dispatch service implementation simply looks through the “published” services in the OSGi context, and looks at the “service-name” property of each service to find the one specified in the call. Assuming it finds an appropriate service, it then uses another class, Adapter, to do the type conversion of the generic Map of parameters to the specific types needed for the service being called, then uses Java reflection to actually make the call and return the response as a List of Maps, again converting the service-specific types to a generic Map in order to pass the response back to the user interface.

Why do we go through those gyrations, instead of just having access to the domain-specific beans in our UI? If we want a full decoupling, and the advantages that come with it, the type-independence of this approach gives us that. It also allows parallel development of the UI and the back-end services without an ever-increasing number of binary dependencies as well – for that matter, with the static data adapter in the Vaadin app allows us to develop on our UI entirely without the back-end services. That’s pretty decoupled.

The Dispatch Service by itself, though, doesn’t give us any functionality. It acts much like a router on a network, simply moving the request from the client to the proper service on the back-end, so let’s build such a service – in this case, a service for handling Dealers.

Dealer Service
The bulk of our application-specific code will be prepared as independent OSGi services, just like this one. In later postings, I’ll describe how to set up dependencies between services (and how to use Spring DM to make doing that kind of thing far simpler).

First, let’s look at our MANIFEST.MF for our new service (again, we can use Maven to produce this for us, but the result is the same):

Bundle-ManifestVersion: 2
Bundle-SymbolicName: com.point2.services.dealer.DealerService
Bundle-Name: DealerService
Bundle-Version: 1.0.0
Bundle-Activator: com.point2.services.dealer.impl.DealerServicePublisher
Import-Package: org.osgi.framework
Export-Package: com.point2.services.dealer

The two key elements above are the Bundle-Activator, a class called DealerServicePublisher, and the Export-Package.

OSGi best practices dictate that the package that is exported should only contain the interface for the class (or classes) you wish to make available from your service. In our case, that comprises a bean class, called “Dealer”, and the interface to our actual service, DealerService.

The Dealer bean is very simple, just your basic property-holding JavaBean:

public class Dealer {
    private String name;
    private String phone;
    private String contactLast;
    private String contactFirst;
    private String address1;
    private String address2;

    public String getContactLast() {
        return contactLast;
    }

    public void setContactLast(String contactLast) {
        this.contactLast = contactLast;
    }

    public String getContactFirst() {
        return contactFirst;
    }
   .....

I’ve ommitted the rest of the getters and setters here for brevity (in case you’re wondering, yes, this is much easier to express in Scala, and yes, OSGi works just fine with Scala…)

The interface for our service is even simpler:

public interface DealerService {
    public void save(Dealer dealer);
    public Dealer findById(int id);
}

Of course, a fully fleshed-out service might in fact have more methods, but again, you get the idea.

I won’t bore you with the actual implementation of the DealerServiceImpl class, but it’s important to note that it’s not in the same package as the Dealer bean and the DealerService implementation. They are the only two classes (well, one class, one interface) in that package, as the entire package is “exported” by OSGi, and therefore visible to other services and clients.

The DealerServicePublisher is the last piece to examine, and it’s pretty straightforward as well:

package com.point2.services.dealer.impl;

import com.point2.services.dealer.DealerService;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import org.osgi.framework.ServiceRegistration;

import java.util.Dictionary;
import java.util.Hashtable;

public class DealerServicePublisher implements BundleActivator {
    private ServiceRegistration registration;

    public void start(BundleContext context) throws Exception {
        System.out.println("Registering dealer service");
        Dictionary dictionary = new Hashtable();
        dictionary.put("service-name", "dealer");
        registration = context.registerService(DealerService.class.getName(),
                new DealerServiceImpl(), dictionary);
    }

    public void stop(BundleContext context) throws Exception {
        System.out.println("Unregistering dealer service");
        registration.unregister();
    }
}

Essentially all we do in this activator is “register” our service, making it visible to the OSGi container and other services or clients that need it. We add an extra property via the “Dictionary” object, which allows us to specify arbitrary properties to be associated with our service. Because we want to look up our services from the dispatcher by name, rather than by class or interface name, we use the string “dealer” and associate it with the key “service-name”. If you examine the code for our DispatchService, you’ll see that it uses this property to find services.

Now we can build our new service with a simple “mvn package” command, and install the resulting jar into our OSGi runtime with “install file:/Users/mnash/experiments/dealer/target/dealer-1.0.0.jar” (again, you can do an install directly from the Maven repository, or from a URL, as opposed to a file).

That’s it – our “dealer” service is now available, and our “dispatch” service is fired up and ready to locate it.

If we deploy our Vaadin application in our OSGi container (as described in the previous post), you’ll find that calls to the “call” method of the ServiceClient now return the “real” data from our service.

In the next few posts we’ll examine an even easier way to define new services, and explore the power of OSGi for updating and working with our decoupled services. Then we’ll look at how to hook services together, allowing services to call other services to do their jobs as required.

I’ve had reason recently to do some thinking on the various “levels” of software testing. I think there’s a rough hierarchy here, but there’s some debate about the naming and terminology in some cases. The general principals are pretty well accepted, however, and I’d like to list them here and expound on what I think each level is all about.

An important concern in each of these levels is to achieve as high a level of automation as possible, along with some mechanism to report to the developers (or other stakeholders, as required) when tests are failing, in a way that doesn’t require them to go somewhere and look at something. I’m a big fan of flashing red lights and loud sirens, myself

Unit
Unit testing is one of the most common, and yet in many ways, misunderstood levels of test. I’ve got a separate rant/discussion in the works about TDD (and BDD), but suffice it to say that unit-level testing is a fundamental of test-driven development.

A unit test should test one class (at most – perhaps only part of a class). All other dependencies should be either mocked or stubbed out. If you are using Spring to autowire classes into your test, it’s definitely not a unit test – it’s at least an integration test. There should be no databases or external storage involved – all of those are external and superfluous to a single class that you’re trying to verify is doing the right thing.

Another reason to write comprehensive unit tests is that it’s the easiest place to fix a bug: there are fewer moving parts, and when a simple unit tests breaks it should be entirely clear what’s wrong and what needs to be fixed and how to fix it.

As you go up the stack to more and more complex levels of testing, it becomes harder and harder to tell what broke and how to fix it.

Generally unit tests for any given module are executed as part of every build before a developer checks in code – sometimes this will also include some functional tests as well, but it’s generally a bad idea for any higher-level tests to be run before each and every check-in (due to the impact on developer cycle-time). Instead, you let your CI server handle that, often on a scheduled basis.

Functional
Some people suggest that functional and integration are not two separate types, but I’m separating them here. The key differentiation is that a functional test will likely span a number of classes in a single module, but not involve more than one executable unit. It likely will involve a few classes from within a single classpath space (e.g. from within a single jar or such). In the Java world (or other JVM-hosted languages), this means that a functional test is contained within a single VM instance.

This level might include tests that involve a database layer with an in-memory database, such as hypersonic – but they don’t use an *external* service, like MySQL – that would be an integration test, which we explore next.

Generally in a functional test we are not concerned with the low-level sequence of method of function calls, like we might be in a unit test. Instead, we’re doing more “black box” testing at this level, making sure that when we pour in the right inputs we get the right outputs out, and that when we supply invalid input that an appropriate level of error handling occurs, again, all within a single executable chunk.

Integration
As soon as you have a test that requires more than one executable to be running in order to test, it’s an integration test of some sort. This includes all tests that verify API contracts between REST or SOAP services, for instance, or anything that talks to an out-of-process database (as then you’re testing the integration between your app and the database server).

Ideally, this level of test should verify *just* the integration, not repeat the functionality of the unit tests exhaustively, otherwise they are redundant and not DRY.

In other words, you should be checking that the one service connects to the other, makes valid requests and gets valid responses, not comprehensively testing the content of the request or response – that’s what the unit and functional tests are for.

An example of an integration test is one where you fire up a copy of your application with an actual database engine and verify that the operation of your persistence layer is as expected, or where you start the client and server of your REST service and ensure that they exchange messages the way you wanted.

Acceptance
Acceptance tests often take the same form as a functional or integration test, but the author and audience are usually different: in this case an acceptance test should be authored by the story originator (the customer proxy, sometimes a business analyst), and should represent a narrative sequence of exercising various application functionality.

They are again not exhaustive in the way that unit tests attempt to be in that they don’t necessarily need to exercise all of the code, just the code required to support the narrative defined by a series of stories.

Fitnesse, RSpec and Green Pepper are all tools designed to assist with this kind of testing.

Concurrency
If your application or service is designed to be used by more than one client or user, then it should be tested for concurrency. This is a test that simulates simultaneous concurrent load over a short period of time, and ensures that the replies from the service remain successful under that load.

For a concurrency test, we might verify just that the response contains some valid information, and not an error, as opposed to validating every element of the response as being correct (as this would again be an overlap with other layers of testing, and hence be redundant).

Performance
Performance, not to be confused with load and scalability, is a timing-based test. This is where you load your application (either with concurrent or sequential requests, depending on it’s intended purpose) with requests, and ensure that the requests receive a response within a specified time frame (often for interactive apps a rule is the “two second rule”, as it’s thought that users will tolerate a delay up to that level).

It’s important that performance tests be run singly and on an isolated system under a known load, or you will never get consistency from them.

Performance can be measured at various levels, but is most commonly checked at the integration or functional levels.

Load/Scalability
A close relative of, but not identical with concurrency tests are load and/or scalability tests. This is where you (automatically) pound on the app under a simulated user (or client) load, ideally more than it will experience in production, and make sure that it does not break. At this point you’re not concerned with how slow it goes, only that it doesn’t break – e.g. that you *can* scale, not that you can scale linearly or on any other performance curve.

Quality Assurance
Many Agile and Lean teams eschew a formal quality assurance group, and the testing such a group does, in favor of the concept of “built in” QA. Quality assurance, however, goes far beyond determining if the software perform as expected. I have a detailed post in the works that talks about how else we can measure the quality of the software we produce, as it’s a topic unto itself.

Alpha/Beta deployments
Not strictly testing at all, the deployment of alpha or beta versions of an application nonetheless relates to testing, even though it is far less formalized and rigorous than mechanized testing.

This is a good place to collect more subjective measures such as usability and perceived responsiveness.

Manual Tests
The bane of every agile project, manual tests should be avoided like the undying plague, IMO. Even the most obscure user interface has an automated tool for scripting the testing of the actual user experience – if nothing else, you should be recording any manual tests with such a tool, so that when it’s done you can “reply” the test without further manual interaction.

At each level of testing here, remember, have confidence in your tests and keep it DRY. Don’t test the same thing over and over again on purpose, let the natural overlap between the layers catch problems at the appropriate level, and when you find a problem, drive the test for it down as low as possible on this stack – ideally right back to the unit test level.

If all of your unit test are perfect and passing, you’d never see a failing test at any of the other levels, theoretically. I’ve never seen that kind of testing nirvana achieved entirely, but I’ve seen projects come close – and those were projects with a defect rate so low it was considered practically unattainable by other teams, yet it was done at a cost that was entirely reasonable.