In my ongoing experiments with Apache ServiceMix, I recently installed the Felix Web Console project, and it was simple and helpful.

OSGi containers typically have a “console” for administering the bundles you’re deploying into our container via the command-line. ServiceMix has a nice “remote” feature, that allows me to get to the console of our ServiceMix instance from my local machine easily, without having to actually ssh to the ServiceMix box.

It looks like this:

You can list your bundles, install, uninstall, restart, etc, all from here.

What Felix Web Console brings to the table is a nice webapp front-end on all this functionality.

Installation is the easiest thing possible: you go to your existing console and type:

    <dependency>
smx@root:osgi> install http://mirror.switch.ch/mirror/apache/dist/felix/org.apache.felix.webconsole-2.0.2.jar

It reports that a new bundle is installed, and gives you the number (237 in my case).

Then I say:

 start 237

And I can immediately surf to the following URL in my browser: http://servicemix.point2.com:8080/system/console

and see the following page:

Now I can browse my bundles, install, uninstall – and all the same good stuff as via the console, even view the output of the log service, except all from the comfort of my web browser.

Like many things in the OSGi world, it just works like it’s supposed to, and is much easier to do than it is to explain

In Part 1 of our OSGi adventures, we described how to build a nice Ajax-aware Vaadin UI app, and couple it to a generic back-end OSGi service we called the service dispatcher.

Now we’ll take the adventure to the next step, show how to deploy that webapp into our OSGi container and get it up and running, then go through the functionality of the Dispatch Service, and show how it routes requires through to the first of our domain-specific application services.

In our UI, we built a form that allowed the user to enter and save a “Dealer”, with some fields like name, phone number, etc., so the first service we’ll build for the service dispatcher to talk to will be our “dealer” service.

First, though, let’s see how to get our webapp into our OSGi container.

As we’re building our Vaadin app with Maven, we can easily add the small bits of additional configuration to turn our project into an OSGi-friendly WAR file.

OSGi deploys “bundles”, but a bundle is just a jar file (or a war, which is after all just a special kind of jar) with a bit of extra meta-data. The META-INF/MANIFEST.MF file is where the magic happens. We add the following to our POM:

              <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.0</version>
                <configuration>
                  <archive>

                      org.springframework.osgi.web.context.support,\
                       org.springframework.web.servlet,\
                       org.springframework.web.servlet.handler,\
                       org.springframework.web.servlet.mvc,\
                       org.springframework.web.servlet.view,\
                       dwmj.domain,\
                       org.springframework.web.servlet.mvc.annotation,\
                       org.springframework.web.context

                    <manifestEntries>
                      <Bundle-ManifestVersion>2</Bundle-ManifestVersion>
                        <Bundle-SymbolicName>com.point2.Admin</Bundle-SymbolicName>
                        <Bundle-Name>Admin</Bundle-Name>
                        <Bundle-Version>1.0.0</Bundle-Version>
                        <Bundle-Activator>com.point2.ServiceClient</Bundle-Activator>
                        <Import-Package>org.osgi.framework,com.point2.services.dispatch,javax.servlet;version="2.4.0",
javax.servlet.http;version="2.4.0",org.osgi.service.http;version="1.2.0",
org.osgi.util.tracker;version="1.3.2"</Import-Package>
                        <Webapp-Context>admin</Webapp-Context>
                        <Bundle-ClassPath>WEB-INF/classes, WEB-INF/lib/service-dispatch-api-1.0.0.jar, WEB-INF/lib/vaadin-6.1.3.jar</Bundle-ClassPath>
                    </manifestEntries>
                  </archive>
                </configuration>
              </plugin>
     

This bit of magic allows Maven to build us a MANIFEST.MF like this:

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: mnash
Build-Jdk: 1.6.0_15
Extension-Name: admin
Implementation-Title: admin
Implementation-Version: 1.0-SNAPSHOT
Bundle-Activator: com.point2.ServiceClient
Bundle-ClassPath: WEB-INF/classes, WEB-INF/lib/service-dispatch-api-1.
 0.0.jar, WEB-INF/lib/vaadin-6.1.3.jar
Bundle-ManifestVersion: 2
Bundle-Name: Admin
Bundle-SymbolicName: com.point2.Admin
Bundle-Version: 1.0.0
Import-Package: org.osgi.framework,com.point2.services.dispatch,javax.
 servlet;version="2.4.0",javax.servlet.http;version="2.4.0",org.osgi.s
 ervice.http;version="1.2.0",org.osgi.util.tracker;version="1.3.2"
Webapp-Context: cadmin

Now we have our OSGi-friendly .war file, sitting in our target directory. We can then connect to the console of our OSGi container (in this case, ServiceMix), and say:

install war:file:/Users/mnash/experiments/admin/target/admin-1.0.0-SNAPSHOT.war

Our container honors the “Webapp-Context” tag in our manifest, so we can then surf to http://localhost:8181/admin/ to see our application. 8181 is the default port for the Jetty HTTP service in ServiceMix – it can easily be changed to another number as required, of course.

Dispatch Service
Now that we can see how to get the wepapp into ServiceMix, let’s look at the Dispatch Service in detail.

Our Dispatch Service is going to be a regular OSGi bundle, so we have a separate Maven project that produces a .jar file in this case, not a WAR file.

Our MANIFEST.MF needs the following magic for this project:

Bundle-ManifestVersion: 2
Bundle-SymbolicName: com.point2.services.dispatch.DispatchService
Bundle-Name: DispatchService
Bundle-Version: 1.0.0
Bundle-Classpath: .,lib/commons-beanutils-1.8.0.jar,lib/commons-collections-3.2.1.jar
Bundle-Activator: com.point2.services.dispatch.impl.DispatchServicePublisher
Import-Package: org.osgi.framework
Export-Package: com.point2.services.dispatch

You can see we’re again specifying a “Bundle-Activator” class, which, much like the ServiceClient class in our webapp, is called by the OSGi framework when the bundle containing this service is started.

One slight oddity: The dispatch service needs the two jars listed under “Bundle-Classpath:” to do it’s business – because we are building an OSGi bundle, we “embed” these jars in our bundle (which is, yes, another jar) by putting them in the src/main/resources/lib directory. We refer to them in that location in the POM, and Maven automatically includes them in the finished jar, where they’re available when our bundle needs them. The other alternative is to install the dependencies as their own bundles, but that’s a whole ‘nother post

In the case of DispatchService, we’ve got an activator like this:

public class DispatchServicePublisher implements BundleActivator {
    private ServiceRegistration registration;

    public void start(BundleContext context) throws Exception {
        registration = context.registerService(DispatchService.class.getName(),
                new DispatchServiceImpl(), null);
        DispatchServiceImpl.setBundleContext(context);
        System.out.println("Dispatch Service registered");
    }

    public void stop(BundleContext context) throws Exception {
        registration.unregister();
        System.out.println("Dispatch Service unregistered");
    }
}

Which simply takes a reference to the BundleContext on startup and passes it to the DispatchServiceImpl, which implements the DispatchService interface, like so:

public interface DispatchService {
   List<Map<String, Object>> call(String serviceName, String serviceOperation, Map<String, Object> parameters,
                                  String versionPattern, String securityToken) throws Exception;
}

The big JuJu with OSGi is that only this interface is “exposed” from the bundle. No other service can see the innards of our service, unlike Jars on a classpath.

The implementation of the DispatchService is equally straightforward:

public class DispatchServiceImpl implements DispatchService {

    private static BundleContext context;

    public List<Map<String, Object>> call(String serviceName, String serviceOperation, Map<String, Object> parameters, String versionPattern, String securityToken) throws Exception {

        ServiceReference reference = getServiceNamed(serviceName);
        if (reference == null)
            throw new RuntimeException("There is no service with service-name " + serviceName);
        Object service = context.getService(reference);
        Adapter adapter = new Adapter(service);
        List<Map<String, Object>> response = adapter.callList(serviceName, serviceOperation, parameters);
        context.ungetService(reference);
        return response;
    }

    private ServiceReference getServiceNamed(String serviceName) throws Exception {
        ServiceReference[] references = context.getAllServiceReferences(null, null);
        System.out.println("There are " + references.length + " services available");
        for (int i = 0; i < references.length; i++) {
            ServiceReference reference = references[i];
            Object serviceNameValue = reference.getProperty("service-name");
            if (serviceNameValue != null) {
                System.out.println("I see service with name " + serviceNameValue);
                if (serviceNameValue.toString().equalsIgnoreCase(serviceName))
                    return reference;
            }
        }
        throw new RuntimeException("There is no service available with service-name " + serviceName);
    }

    public static void setBundleContext(BundleContext newContext) {
        context = newContext;
    }
}

The dispatch service implementation simply looks through the “published” services in the OSGi context, and looks at the “service-name” property of each service to find the one specified in the call. Assuming it finds an appropriate service, it then uses another class, Adapter, to do the type conversion of the generic Map of parameters to the specific types needed for the service being called, then uses Java reflection to actually make the call and return the response as a List of Maps, again converting the service-specific types to a generic Map in order to pass the response back to the user interface.

Why do we go through those gyrations, instead of just having access to the domain-specific beans in our UI? If we want a full decoupling, and the advantages that come with it, the type-independence of this approach gives us that. It also allows parallel development of the UI and the back-end services without an ever-increasing number of binary dependencies as well – for that matter, with the static data adapter in the Vaadin app allows us to develop on our UI entirely without the back-end services. That’s pretty decoupled.

The Dispatch Service by itself, though, doesn’t give us any functionality. It acts much like a router on a network, simply moving the request from the client to the proper service on the back-end, so let’s build such a service – in this case, a service for handling Dealers.

Dealer Service
The bulk of our application-specific code will be prepared as independent OSGi services, just like this one. In later postings, I’ll describe how to set up dependencies between services (and how to use Spring DM to make doing that kind of thing far simpler).

First, let’s look at our MANIFEST.MF for our new service (again, we can use Maven to produce this for us, but the result is the same):

Bundle-ManifestVersion: 2
Bundle-SymbolicName: com.point2.services.dealer.DealerService
Bundle-Name: DealerService
Bundle-Version: 1.0.0
Bundle-Activator: com.point2.services.dealer.impl.DealerServicePublisher
Import-Package: org.osgi.framework
Export-Package: com.point2.services.dealer

The two key elements above are the Bundle-Activator, a class called DealerServicePublisher, and the Export-Package.

OSGi best practices dictate that the package that is exported should only contain the interface for the class (or classes) you wish to make available from your service. In our case, that comprises a bean class, called “Dealer”, and the interface to our actual service, DealerService.

The Dealer bean is very simple, just your basic property-holding JavaBean:

public class Dealer {
    private String name;
    private String phone;
    private String contactLast;
    private String contactFirst;
    private String address1;
    private String address2;

    public String getContactLast() {
        return contactLast;
    }

    public void setContactLast(String contactLast) {
        this.contactLast = contactLast;
    }

    public String getContactFirst() {
        return contactFirst;
    }
   .....

I’ve ommitted the rest of the getters and setters here for brevity (in case you’re wondering, yes, this is much easier to express in Scala, and yes, OSGi works just fine with Scala…)

The interface for our service is even simpler:

public interface DealerService {
    public void save(Dealer dealer);
    public Dealer findById(int id);
}

Of course, a fully fleshed-out service might in fact have more methods, but again, you get the idea.

I won’t bore you with the actual implementation of the DealerServiceImpl class, but it’s important to note that it’s not in the same package as the Dealer bean and the DealerService implementation. They are the only two classes (well, one class, one interface) in that package, as the entire package is “exported” by OSGi, and therefore visible to other services and clients.

The DealerServicePublisher is the last piece to examine, and it’s pretty straightforward as well:

package com.point2.services.dealer.impl;

import com.point2.services.dealer.DealerService;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import org.osgi.framework.ServiceRegistration;

import java.util.Dictionary;
import java.util.Hashtable;

public class DealerServicePublisher implements BundleActivator {
    private ServiceRegistration registration;

    public void start(BundleContext context) throws Exception {
        System.out.println("Registering dealer service");
        Dictionary dictionary = new Hashtable();
        dictionary.put("service-name", "dealer");
        registration = context.registerService(DealerService.class.getName(),
                new DealerServiceImpl(), dictionary);
    }

    public void stop(BundleContext context) throws Exception {
        System.out.println("Unregistering dealer service");
        registration.unregister();
    }
}

Essentially all we do in this activator is “register” our service, making it visible to the OSGi container and other services or clients that need it. We add an extra property via the “Dictionary” object, which allows us to specify arbitrary properties to be associated with our service. Because we want to look up our services from the dispatcher by name, rather than by class or interface name, we use the string “dealer” and associate it with the key “service-name”. If you examine the code for our DispatchService, you’ll see that it uses this property to find services.

Now we can build our new service with a simple “mvn package” command, and install the resulting jar into our OSGi runtime with “install file:/Users/mnash/experiments/dealer/target/dealer-1.0.0.jar” (again, you can do an install directly from the Maven repository, or from a URL, as opposed to a file).

That’s it – our “dealer” service is now available, and our “dispatch” service is fired up and ready to locate it.

If we deploy our Vaadin application in our OSGi container (as described in the previous post), you’ll find that calls to the “call” method of the ServiceClient now return the “real” data from our service.

In the next few posts we’ll examine an even easier way to define new services, and explore the power of OSGi for updating and working with our decoupled services. Then we’ll look at how to hook services together, allowing services to call other services to do their jobs as required.

Apache’s ServiceMix 4 is a unique combination of OSGi container and JBI-compliant ESB, but in addition to this it’s also a capable web application container. More specifically, it can contain a webapp container, but the effect is the same…

My current favorite tool for cranking out MVC-style webapps is the Lift framework, so I thought I’d document here the process for deploying a Lift webapp into ServiceMix 4.

The good news is that it’s very straightforward.

First, build the .war for your Lift app using mvn package. This will result in a .war file in the target directory of your project, and is well-covered by the Lift documentation if you need more details. In my case, I started with a slightly modified version of the PocketChange example app for Lift.

Now start servicemix with

bin/servicemix

Done from the root directory of your ServiceMix install. You should see the ServiceMix banner after a few moments, followed by a prompt.

In the console, type the “osgi” command to enter OSGi mode.
then

install war:file:///tmp/admin-0.1.war?Webapp-Context=admin

You’ll see some startup messages as your webapp is digested by ServiceMix, turned automagically into a bundle, and processed by various intestinal organs such as the HTTP service.

Now surf to

http://localhost:8080/admin/

And up comes your application!!

The Apache Group have recently released version 4 of the ServiceMix ESB. (Enterprise Service Bus), and I had a chance to work with it a bit over the weekend.

As we develop more and more services, the plumbing is starting to get complicated. We’re starting to ask questions like what versions of what services do we have? What dependencies do we have between services? How can/should we deploy services? Should we use REST or JMS for this service? How can we easily manage and monitor all these services in a fully clustered, scalable and high-availability environment?

In short, how can we develop powerful scalable services fast and not worry about the plumbing?

We’re not the only people to have asked these questions – and one powerful answer is to use an ESB.

ESB’s have come a long way, and the JBI standard and OSGi finally get together in this version of ServiceMix, and some pretty powerful magic happens as a result.

Just about every organization that writes sophisticated applications, particularly Java applications, have run into some of the problems that an ESB provides solutions for – the same is true of OSGi.

As Anthony Juckel put it on his blog, “Any sufficiently complicated Java program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of OSGi.“. This is quite true, in my experience, and can even be extended to include ESB’s, in that if you’ve got a number of JVM services interacting on a network to solve a problem, you’ve got some kind of an ESB going on, whether you call it that or not.

A common practice recently is to write software that provides “services”, then to combine these services into a complete system – in other words, we write services, but we compose systems (from these services). The awkward part comes (well, one awkward part) when we try to write the service in such a way that it can be used in many different ways – in other words, to maximize the potential for re-use – while at the same time trying to do the simplest thing that can possibly work.

No matter what service mechanism we choose, we lock ourselves in to some degree. If we write our service to use JMS (Java Messaging System), so we can support nice events and queues and other asynchronous goodness, we can’t easily then use our service from, let’s say, a client that looks for a REST service. If we choose SOAP, we can’t easily talk to our service from a client (which could be another service) that wants to post an event, instead of call a SOAP interface.

Normalized Messages
The Java JBI (Java Business Integration) standard answers this problem: it provides a single message-oriented interface for services, called the Normalized Message. This Normalized Message contains some header info, some properties, and XML payload, and optionally, binary attachments. It is not specific to any one protocol – in other words, it’s not SOAP, it’s not REST, it’s not JMS, it’s not SMTP, it’s not any of those.

JBI is a JSR for defining Java Business Integration, a way to cut through the complexity of different types of communications mechanisms for component message-passing (e.g. things such as REST, JMS, SOAP, RPC, email, FTP, HTTP, file system, and so on).

ServiceMix is one of several choices – once we have services written to the JBI standard, we can deploy them in any JBI-compliant ESB. Glassfish’s OpenESB is another option, for example.

Instead of worrying about the plumbing of the specific protocol we want to use, we can concentrate on writing our business logic, then rely on the ESB to “wire up” the messages between our service-providing components – whether they’re local (e.g. running in the same ESB, in which case it’s a simple in-memory message), or distributed, on another box in our cluster, or halfway around the world. The ESB provides adapters for each of the different protocols – so we can take our one service and talk to it via REST, JMS, SOAP, Email, carrier pigeon (ok, maybe not the last one – but you could write an adapter!).

This means I can write my service without even knowing what kind of protocol I’m going to be talking to it with – maybe it receives some messages via JMS, some via REST calls, a couple of SOAP services, and one in a while via an email. All that is the problem of the ESB – I just write one simple method in one simple interface to swallow the appropriate NormalizedMessage, do my processing whatever it is, and spit back out another NormalizedMessage.

This can significantly speed up the development of new services, by taking decisions about the plumbing away from the process of writing the correct business logic. We don’t even need to decide up-front if our service will be used by other services or not – we can wire it up as required.

The other important thing about a Normalized Message is that it refers to services by an identifier, but not by a specific location – in other words, a message from a client might say “I need this service to process this message”, but it’s up to the bus to figure out how to get the message to the service – this provides the opportunity to decouple services from each other. Let’s say you need to write a service that takes some kind of business message and sends an email. You don’t have to worry about passing your service information to let it find the email service at all – you just send the message to a service name (maybe “email”), and the ESB figures out the rest.

This means you can swap the email service out for another service that provides the same mechanism without worrying about the details.

JBI and OSGi Together
ServiceMix is one example of such an ESB, which marries the modularization advantages of OSGi with the above-described benefits of an ESB. This means each of our components can run in it’s own classpath space, with no interference with other components – even other components that might use different versions of the same jars. How many times have you discovered you’ve got 3 different versions of log4j on your classpath? This can lead to all sorts of weird and wonderful problems that only occur in certain circumstances, and are all-but-impossible to fix. By providing true modularization, OSGi solves this problem properly. Other solutions that I’ve seen applied in the past lead directly to the quote about every complex system having a half-baked OSGi implementation inside them

ServiceMix also provides a whack of existing components in two different JBI flavors: Service Engines and Binding Components. Services Engines are the ones that do the actual business logic, and Binding Components are the adapters, the pieces that route messages to and from various protocols. Many common tasks can be achieved by configuring the existing pieces, without writing a line of code.

When it is necessary to write some code, ServiceMix includes adapters to make that even easier as well – for instance, it provides a Service Engine to allow any POJO (plain old java object) to be used as a service, without that object having to actually implements the interface for handling NormalizedMessages directly. It doesn’t get any easier than that to write a service – just write the bean, plug in a bit of configuration, and you’re done – instance REST (JMS, email, SOAP, etc etc) service!

Spring integration is built into ServiceMix, so SE’s can have their dependencies injected automagically on deployment.

SE’s can be hot-deployed and hot-removed. Multiple versions of a single SE can be running without conflict, so “hot” upgrades can be done easily. You can have one version of a service that requires version 1.1 of another service running at the same time as some other service that requires 1.2 of the same service. This is flexibility in deployment, as it means that there’s no need to “drag along” other services when upgrading if you don’t want/need to.

Services can easily be managed and monitored via the administrative capabilities of the bus.

Binding Components
Binding components are how we talk outside the backbone – they take messages from the backbone and transmit or receive via an external protocol to non-ESB services – e.g. via REST, JMS, HTTP, file system, etc etc. There are dozens of existing binding components to support just about every protocol we’d care about.

Of course, all this power comes with a price: you have to learn to manage and deploy your ESB of choice, and even the simplest ESB is still a fairly complex beast. ServiceMix is no exception. The good news, however, is that it’s possible to pretty much ignore much of the available complexity in order to get started small, then learn more as you need it to start to leverage the power available.

A caveat, however: It’s important to use an ESB the way it’s intended, and not try to shoehorn things that don’t fit into it’s structure. If you find yourself struggling to write services, or having to write a lot of Binding Components, chances are you’re making inappropriate design decisions, or that you haven’t separated the concerns of a Service Engine from a Binding Component fully. This is not uncommon, as in many environments these two concerns are handled by a single component. If you’re using to writing REST services, for instance, with things such as JRA or Jersey, you’ll find it very odd to separate the processing from the “presentation” (even when that presentation is simply into XML or JSON to the REST client).

Once this technique becomes second nature, however, the true power of an ESB becomes clearer.

A whole fleet of buses…
Multiple ESB instances can be deployed and clustered, providing highly scalable and fault-tolerant system. The communication between ESB instances is handled entirely by the ESB itself – nothing about our components needs to be aware that they’re working in a clustered environment.

From my latest look at ServiceMix and it’s new release, I suspect I’ll be taking a deeper dive in the near future.

(Thanks to Craig Walls for pointing out that excellent quotation – and for all his help in understanding OSGi!)