I’ve recently been doing a lot of my own projects in Scala, and, as many other developers have learned, have been finding it harder and harder to “revert” back to Java when Java is mandated for a specific project.
Don’t get me wrong, I still use a lot of other languages, and languages come and go – I’ve learned not to get too attached. Yesterday (literally) I was coding in Ruby, today in Scala, Java and a lick of Python (well, Jython, technically).
Still, I find myself more and more reaching for Scala as my preferred weapon. So I started asking myself why that is, and thought I’d share the answers I’ve come up with so far. I’d be pleased to hear anyone other experiences as well, as I’m pretty sure this is not a complete list. It’s also not in any particular order, just the order I’ve remembered them.
REPL The REPL for Scala is a huge plus for me. I can quickly experiment with a few lines of code, either with or without the rest of my project on the classpath, and interactively work my way towards a solution. The closest in Java was by writing a throw-away unit test in my IDE, and that’s clumsy by comparison. The REPL is also an excellent learning tool, as anyone who’s used a language with one can attest.
Actors Actors are a library, not an in-built feature of Scala per se, but both “regular” Actors and alternatives such as Akka provide much of the power of languages like Erlang, while retaining all the other benefits of Scala. It’s difficult to grasp just how much power this can mean to a development project until you’ve worked with them a while, then they’re very hard to live without.
Speed of Evolution Scala evolves significantly faster than Java (or many other languages, for that matter). I won’t go into the many reasons why, they’ve been explored elsewhere, but this is a huge plus for me compared to the glacial pace of Java language evolution.
Open Source Scala is actually open source. Not nearly or mostly or kinda or somewhat. Not it’s libraries, or everything except it’s mobile version, or all the bits someone doesn’t think they can make money off of. It’s released under a very open license, rather like a BSD license, with basically no restrictions. Here’s the license itself, so have a read. It won’t take more than a moment.
By contrast, here’s the page discussing the licenses for Java. Bring lunch. And a lawyer, preferably.
JVM and CLR I don’t use the CLR, and in general despise everything ever produced by Microsoft, but it’s interesting and valuable in some situations to note that Scala doesn’t only run on the JVM, but also the CLR.
Scripting Ant is an obsolete build system and a worse cross-platform scripting language. Shell is OK, but showing it’s age a bit. Scala runs on every platform I care about, and is every bit as easy as shell to script with, so I’m finding I use it more and more for both quick-and-dirty scripts where I might have used shell before, and for things that will live longer, such as deployment and installation.
Expressiveness You can say a lot, quickly, with Scala. It’s not just terse and brief, but highly expressive. Examples abound out there, so I won’t waste space trying to prove it here, but in general I find I’m writing perhaps one fifth as many lines to do any given job when compared with Java. It’s at least on par with Ruby.
Not just speed of writing the code, although that’s good. It’s more about speed of reading the code. I can read good Scala and understand it significantly faster than I can wade through the equivalent code in Java.
Community I can’t deny Java has a huge and helpful community as well, but Scala’s community compared to that of other “newcomer” languages is very diverse and helpful. It’s not a reason to pick Scala over Java, but it might be enough to pick Scala over, say, Python or Ruby, in some cases.
Functional/OO Blend Scala brings the capabilities of functional languages like Erlang, Haskell, and others to an approachable platform that lets you incorporate these concepts as you’re ready for them, as opposed to an “all or nothing” approach, as a Java programming moving to, say, OCaml might feel. It encourages immutability, but gently and easily, giving you time to learn the power of functional development, while allowing the many advantages of object-orientation to be retained at the same time. No other languages does this quite as well as Scala, in my view.
Options Had enough of NullPointerException ruining your day? Have a look at Option in Scala. It’s not a silver bullet, but it’s a heck of a lot better than Null
Productivity The expressiveness of Scala means I can get a lot done, quickly, yet still have high-quality functional and performant code that I can read and maintain when I’m finished. Scala is approachable and easy to learn, and you don’t have to learn all the fancy stuff it can do at first – you can just write what looks like extremely expressive Java, then get into the cool stuff as you go.
Not Oracle I have very little faith in Oracle’s stewardship of Java, to be honest, and I don’t think Oracle can sue me for using Scala. IANAL, however, and they may try
One of my primary reasons for this worry is Oracle’s about-face on the issue of open source Java. Do a google on the “Apache Harmony Project” for the gory details.
Traits Multiple inheritance without the pain, basically. Interfaces and abstract classes with all the warts of both taken away. This is one area where Scala doesn’t just top Java, it blows it out of the water.
Advanced Type System Scala’s type system starts with the pure object-orientation principle (everything is an object, there are no primitives), and goes much, much further. Case classes alone are worth the price of admission, and they’re only the tip of the iceberg. See High Wizardry in the Land of Scala for a taste of what the type system in Scala can do.
Java compatability Conversely, Java itself is a reason for my preference for Scala. Instead of Ruby or Haskell or OCaml, if I choose Scala I basically get the entire JVM ecosystem available to me – and that’s a lot. I get OSGi, JDBC, all the drivers for every SQL and NoSQL database you’ve every heard – the works. Except with a language that’s decades ahead of plain Java in other areas.
Tool support Not compared to Java, which of course has amazing tool support, but tool support for Scala is now up to the point where it’s quite acceptable, even good in some places. I prefer IntelliJ IDEA with the Scala plugin, but good support also exists for Scala now in most of the major IDEs, and more come along every day, it seems.
Collections Scala’s collection classes are a long way ahead of Java’s, even when you add libraries like Google Collections to Java, and allow you an easy way to go back and forth to existing Java libraries, especially in Scala 2.8.
DSLs Scala’s flexible syntax makes internal DSLs almost trivial, a huge boon when working with many frameworks. My own little DSL for Vaadin UI development puts a whole new aspect on creating UI’s, I’ve found, and tools like Squeryl and others put this DSL capability to incredible use, things that just aren’t easily possible in Java.
If you’re interested in learning more about Scala, try my resource page. It’s chock-full of links to many excellent Scala resources on the web.
I am pleased to report that the campaign goes well thus far.
We met the enemy at the border of Weblogic, and battle was joined, SVN commits clashing with intractable lengths of binary spaghetti at every turn.
Our original plan was to crush all remote beans entirely, liberating the entire application from this particular scourge. Much progress has been made towards this goal, although the enemy is powerful and resourceful.
We can confirm that at least five of our opponents fell early in the glorious battle, with minimal losses on our side, despite the heavy resistance. At several points in the battle the WTF/minute meter was rising to dangerous levels. One particularly brave refactor was rolled back in a blaze of glory, still muttering “stay in target directory… stay in target directory”…
Our initial attempt at penetration of the enemy’s lines was at the JMX pass. Despite valiant attempts by our brave tests, we were eventually defeated in this attack by proprietary “extensions” to what should have been well-accepted standards. The enemy knows no shame, and will stoop to perverting even innocent RMI for it’s nefarious purposes. In fact, they dredged up what should have been long-forgotten and discarded protocols to throw us off the track, even resorting to JMX over IIOP, with proprietary client libraries, which was too much for us – we could not withstand that level of firepower. We had to find another way.
We were repelled by the evil machinations of the enemy’s classpath conflicts at the battle of the EAR, but were able to outflank them and retreat to a previous revision number with minimal losses. We then regouped and attacked again in the region of the remote stateless session bean, and only one of the enemy escaped our wrath. We will be watching this one remaining combatant carefully, and at his first slipup he is sure to be doomed as well. We have assigned our best integration test to watch his every move.
Despite attempts to deceive our troops with obscure and fiendish JNDI naming practices, we were able to ensnare many of the enemy in our powerful web of functional tests, where they can do no further harm.
At least four entire modules of the enemy’s forces did not survive the encounter!
We have won this conflict for now, yet many challenges still lie ahead in future battles. The enemy used many clever tactics this time, but we have not yet encountered their ultimate weapon: the greatly-feared container-managed entity bean! Weapons of mass deprecation of this nature must not be allowed to continue to intimidate future generations. Massive as these weapons are, they are slow and no match for our lightweight DDD techniques – we will lure them into shallow memory and watch them go aground, tangled in their own distributed transactions.
For now, though, our troops have withdrawn to the safety of our own home OSGi modules, where order, structure and modularity reign supreme.
Stay tuned for future reports….
I recently had occasion to use a trick with test execution that is simple and very helpful, but that I forget about frequently, so I thought I’d pass it along here.
We use Maven to execute our unit (and sometimes functional, but that’s another story) tests for a suite of applications my team works on. Some of our apps have quite a few tests, and even though each unit test is very fast (as good unit tests ought to be), running that many of them still takes a bit of time. We’re also burdened with a very slow CI server, unfortunately, which aggravates that situation.
Unit tests, by definition, do not rely on the operation of any other test – they are self-contained, stateless and repeatable. This means they’re ideally suited to running in parallel, and fortunately, that’s very easy to do with Maven.
Maven uses the Surefire Plugin to do it’s test-running, and it’s configurable to run tests in parallel. Our project uses a parent pom (different from an aggregator pom) to specify common configuration, so our pom files stay lean. In our parent pom, we just add the following:
org.apache.maven.plugins maven-surefire-plugin 2.5 methods 20
And we’ve told Maven to run our tests in a maximum of 20 threads, and to make each method run in parallel. (There are other options here for only running classes in parallel, etc, which might be helpful for functional tests).
Just doing that took a minute off the time of a couple of our builds, and a bit off the time of even some of the smaller ones. Of course, it’s even more effective if you’re running on a multicore machine, but even the feeble VM running our CI was faster with it there.
Enjoy!
I’ve been sold for some time on the benefits of highly modular software development (and deployment, for that matter, almost equally importantly).
However, I’m aware that not everyone is on the same page, or has the same definition of “modularity” in the first place, so I thought I’d collect and enumerate the things that have convinced me, as best I can.
I’ve been in the software business for a very long time, and sometimes I have a preference or leaning towards a certain technique or technology. Sometimes that preference has clear causes behind it, but sometimes not. Modularity is one of the latter, where I know it’s not only good, but critically important, but it’s hard for me to immediately say why or how I know that in a succinct way.
So, I apologize for the length of this explanation, as I’ve not yet taken the time to make it shorter
Modularity Defined First things first: What do I mean by modularity?
In short, I mean a system comprised of relatively small self-contained units. The idea of modularity has caught on much more completely in many other disciplines, but far less so in software. In manufacturing, especially mechanical and electronic, in business, especially finance, the idea of modularity is well established. Modularity is used to increase and maintain quality, and reduce cost while increasing output. In the automotive industry, for example, extensive use of standardization and modularity allows us all to afford a car, as opposed to just the wealthy or the mechanically inclined.
In the software world, the use of modularity is still woefully small, however. When it is employed, though, some spectacular successes have arisen. One of the oldest examples is Unix, arguably the most successful operating system ever. Since 1969, the “do one thing, do it well, and play well with others” modular design of Unix has been the basis of it’s success and flexibility, allowing it to span everything from embedded devices to mainframes.
The Java Virtual Machine (JVM) is another realm where modularity is on the rise, and starting to have an impact. Many different approaches are available, and some of them work with any of the large number of languages available on this platform.
Increase overall success rate of projects In the many software projects I’ve been involved with over the years, I’ve noticed a few trends. Smaller, well-contained projects have a higher success rate than larger and more poorly contained projects. (By contained, I mean the number of places and ways the project interacts with other projects is small). I’m sure there are a host of reasons for this, but part of it is simple comprehension – small projects are generally easier to understand than big one, so the number of times things get done wrong is reduced.
A system comprised of well-isolated modules tends to be more like a set of small projects that happen to work together, so I find this reason alone a plus for modularity.
But this hides much of the detailed reasoning behind why modularization works, so let’s dig a bit deeper…
Makes the API between components explicit, not hidden Sometimes I heard the argument that modularity is complex, that it actually increases the difficulty of working with a system. I find that doesn’t hold up under examination: when modularity seems complex, it’s probably because it’s being done wrong, or being retrofitted to a system that was not designed with modularity in mind. What you’re really seeing in this situation is that the attempt to modularize is exposing the complexity that was already there.
When a system is properly designed in a modular fashion, the APIs between modules are small and well-defined, with a limited number of cross-module dependencies.
Reduce unnecessary complexity By reducing a large complex system into a series of small and better understood modules, we’ve reduced the complexity we have to deal with at any one time. This means that the overall large system only has to deal with how the exposed APIs of our modules will be fitted together, treating the modules themselves as “black boxes” that perform specific functions, without worrying about how they perform it.
In both cases, we’ve reduced the complexity we need to deal with.
I specify here “unnecessary” complexity, as of course the complexity of the issue itself doesn’t get any simpler just because we modularize it – but if we don’t have to deal with a lot of tangled plumbing, we can concentrate on the real issue.
Increase testability A module that performs a single function is much easier to test exhaustively than an entire monolithic system. The number of interdependence is limited, meaning we can do more of our testing at the unit and orchestration level than at the functional or integration level, meaning our feedback time is improved and our coverage likely increased.
This means our development velocity can increase, as the impact of changes can be evaluated more easily and more quickly.
Support Composition When you have a number of smaller building blocks, it’s relatively easy to reshuffle those blocks to change the behaviour and functionality of your overall system. This is making use of composition – connecting blocks together to do more than they can on their own, and its a pattern that is supported well in a modular environment.
Although composition is often thought of in the context of re-usable modules, when building a new system, it’s also valuable in modules that are only used within one system, as it allows the large-grained behavior of the system to change much more rapidly than without modules.
Composition makes large complex systems easy to understand and work on, and vastly increase the maintainability of the overall system.
Impact of changes isolated The isolation of changes to an effected module is a major benefit of modularity, especially as it supports maintainability. If you’re constantly concerned with your changes affecting parts of the system you’re not actually working on, you’re not able to go as fast as you could if you knew that your changes would only apply to a small area of the whole system – the current module.
This is related to and similar to the support for refactoring that unit tests give – if you’re tests are all green after your refactor, you know you haven’t changed the functionality of your system. In modularity, if you’re isolated from other modules in the system, this goes a step further – you can change the functionality of the module freely without concern about other bits of your system breaking. This is true if you’re doing anything that’s not in the one package you’ve “exported” as the service definition.
Allows physical design to reflect logical design Modularity also allows developers to build systems where the logical design actually corresponds as needed to the physical design, an often overlooked abstraction. Kirk’s blog post on the topic describes this issue better than I could, so I refer you there.
Supports Refactoring Better Refactoring is the process of changing the implementation of a system without changing it’s behavior. One of the primary reasons to write unit tests, for instance, is to support and allow refactoring, so we know the system does the same thing when we’re done as when we started.
Modularization supports this ability by making clear the portions of a system that affect other modules, and the portions which do not. When you’re refactoring in a monolithic system, even with IDE support, the scope of your changes is the entire project. When you’re refactoring in a modularized system, if you’re “inside” the module, you have no need to consider scope on the outside, as there isn’t any. And when you’re refactoring the exported interfaces, you know ahead of time your scope is cross module, and can take it into account.
In short, modularization lets you refactor more freely within a module, and lets you be intentional and organized when refactoring between modules.
A colleague recently suggested that modularization would interfere with the ability to see a system as a whole. It’s been my experience that the exact opposite is true – a well structured set of modules can still be considered as a whole when necessary (an aggregator POM is any easy way to set this up in the Maven universe), but it also allows modules to be worked on “safely” in isolation, which a monolithic system does not. This ability to only look at one piece at a time is all the more critical the larger the system becomes.
The need to refactor “across” modules is a warning sign, in my opinion – it generally indicates that the API between modules is changing, and this is a change you want to be more aware of then any amount of changes within modules. If it happens often, it probably indicates that the module boundaries are not yet stabilized, and you have more design work to do.
Scalable Development A modular system lends itself well to many hands being involved in it’s development and maintenance. It’s not necessary for a team or a pair to understand the whole system well, or even at all – they can still work on a small well-defined and decoupled module, and can do it in parallel with development on other modules.
Helps prevent Design Rot As discussed in this blog post, design tends to degrade over time, especially in large monolithic systems. Modularity helps to prevent this rot, as pieces that become superseded by better ways of doing things can be replaced individually, and the impact to the overall system managed and isolated. This is akin to removing a brick from the wall and replacing it with a better brick, rather than tearing down the wall.
Solves the classpath hell problem entirely Classpath hell is the name for the condition where a large number of dependencies are “in scope” at once. It’s not unique to Java, or even the JVM. It is apparently called “DLL hell” on Windows environments, but I’m happy to report I know very little about that. I do know it can happen on platforms other than the JVM, however.
By isolating the influence of dependencies, modularization, especially the way OSGi does it, makes it an explicit process to import and export exactly what packages you wish to and from your modules. What the module uses in it’s private implementation under the hood is no longer relevant to the system at large.
What happens in the bundle stays in the bundle, in other words, with appropriate apologies to Las Vegas.
Management of dependencies is the cure for classpath hell, and modularization is key to this, as described in this excellent article.
Allows smaller pieces of the system to be versioned A key element in continuous releasability is the careful versioning of both individual modules and their dependency requirements. Modules give you the ability to release pieces of your system, not the whole system, better supporting the idea of continuous releasability and enhancing maintainability.
An article about software maintenance that a colleague recently sent me describes the problem perfectly – and independently versioned modules is a large part of the answer to this problem.
Re-Use Although re-use it touted often as a major benefit of modularity, I’d categorize it as the least important benefit, not the most. Although the idea of re-using well-designed modules in new projects is indeed very powerful, it’s beyond the technical and design capabilities of many teams, often causing them to reject the need for modularity because they see no reason to build for re-use.
In a way, they’re right: you don’t build for re-use. You build for just the specific case at hand, but when you separate concerns properly and adhere to modularity best practices, you end up with a module that may lend itself to later re-use much more easily. If such a re-use case emerges, well and good, then you can reap the benefit. It is a lot like a framework – you can’t design a good one in isolation from it’s use cases – a framework is an emergent artifact from the re-use of code on many different (although similar) projects. The same is true of re-usable modules: you don’t set out to build a module explicitly to be re-usable.
Allows dependencies to be isolated: e.g. all Weblogic-specific code in one module, or all XMLMill-specific code in one module, increasing ability to refactor safely
Suggested Reading: Agile Architecture Requires Modularity Modularity Patterns Runtime and Development time Modularity
In a later post we’ll look at some of the counter-arguments against modularity, and why I think they’re not especially valid or convincing in my experience.
On a recent project I’ve had the opportunity to help set up a new method of producing our software releases. We had a number of goals in setting up this process, and I think we’ve done pretty well in acheiving them. Some of these were:
In our environment we produce two applications at the moment, which are both part of the same suite of applications and communicate with each other when deployed. We have an automated integration test that verifies they communicate correctly.
One application is packaged as an EAR (Enterprise Application aRchive) file, and is deployed to a cluster of Weblogic servers. The other is packaged as a group of OSGi bundles, one of which is also a web application (WAR) file. We have automated the build/test/release process in the same way for both products, although their install process is of course quite different.
Let me walk through how our process works, starting from the point where a developer checks in some code that implements a new feature (or some other story given to us by the customer proxy).
For our OSGi-based product we use the exact same process, with the exception that the installer is Java/JMX based instead of a set of Jython scripts. In both cases, the user of the artifact to be installed just unzips the file and says “install.sh” (or install.bat).
We don’t use the Maven release plugin for this process at all, as it “tags” at the end of the build cycle, not the beginning. We don’t even necessarily need to use SNAPSHOT versions, as the version of the actual internal modules are never visible to the customer in any case, and we rachet to the target release number at the beginning of the release development cycle, not the end – in other words, if we’re heading towards releasing 3.2 of the product, the customer proxy changes (or asks us to change) the version of the aggregator POM to 3.2 immediately. Then we produce 3.2-r1234, 3.2-r1235, 3.2-r1236 and so forth until the customer proxy says “ok, it’s done”, at which point he (not we) sends it off to the customer, and bumps us to 3.3.
This is quite a different sequence than many teams use, although I have used it before (although not with TeamCity).
So far it’s working for us, we’ll report if we find things to optimize about it.
Comments and/or questions are most welcome!
If you work with Apache Maven at all, you’ve probably checked out the still-unreleased alpha’s of Maven 3. I’ve been working with the alpha-6 for a number of weeks, and I’ve found it solid and reliable, with no changes required to my pre-maven-3 pom file at all.
The only noticeable change is that it’s less chatty to the console, and it runs about 30% faster than before.
I did fall down one hole, but it was a short-lived problem: I was doing something that is arguably not too smart in the first place – that is, calling my “deploy” script on local workstations using a binding to a Maven lifecycle and using the Ant plugin. It turns out the point at which I’d bind to the lifecycle wasn’t well-defined, and Maven 3 had different behaviour in this area than Maven 2. The fix was easy: don’t do that . I now simply call my ant deploy script manually at the end of the build cycle, using the unix “&&” command, so it only runs if the build succeeded (as I don’t want to deploy the old code if the build of the new didn’t work).
On top of the speed boost I’ve seen with Maven 3, I also tried the very-alpha Maven Shell project, hosted at Sonatype.
This project basically gives you a shell, much like a unix/linux shell, but that is “Maven Aware”. You type your Maven commands just like before, e.g. “mvn clean install”, but the Maven Shell already has an instance of Maven loaded, so execution begins faster, and it caches POMs and such, so the runtime even beyond startup is faster still. I was able to take a 2+ minute build cycle (on an old, tired laptop) and take it down to 30 seconds, by using both Maven 3 and the Maven Shell.
Not bad, not bad at all.
We’ve got a number of high-priority tasks going on at work right now, so I’ve not had time to post about all the interesting tech we’re using. I have kept up on my reading, though, and I thought I’d share a few links to articles and sites I’ve found very helpful in the last while. We will soon return to our irregularly unscheduled postings
Dispelling Maven Myths Spring’s Roo Project: RAD for Standard Java Presentation on the Oracle/Sun Purchase and it’s impact on Java technologies Make-It-Easy: A tiny framework for writing Test Data Builders in Java The Tortoise and the Hare (Why there’s always time to do it right) Refactoring Large Software Systems Process-Centric vs. Information-Centric approach to SOA GitReady: Learning GIT, the easy way! A Primer on Cargo: Automated Maven Deployment A New Dawn for Java Oracle’s Strategy for Java Hudson, Best CI Tool! When “Given, When, Then” Doesn’t Fit
I owe the Vaadin framework an apology. When demoing some research I’ve been doing on Vaadin to a colleague today, he asked if Vaadin, like many Ajax-oriented frameworks, “breaks” the browser’s back button. What he meant was that when you’re working in a Vaadin app, pressing your normal “back” button in the browser takes you to whatever site you were visiting before you came to the Vaadin app, not to the last place you were inside the Vaadin app.
I told him yes – and hence the source of my apology. Strictly speaking, I was correct: if you take no extra care, then clicking the browser “back” will take you out of your Vaadin app altogether. However, this bothered me, and I went home to tinker with it a bit.
It turns out that Vaadin has full and easy support for “back”, bookmarking, and all of the other URL goodness you might expect in any other kind of web application or site, and it’s so easy.
By adding a component called a UriFragmentUtility to your first Vaadin window, you get a “hook” into the URL of your application, with the ability to both read and write the “fragment” portion of the URL, that is, anything after a # sign in the URL. For example, if your application’s URL is www.foo.com/myapp, then www.foo.com/myapp/#bar will have a “fragment” of “bar”.
I had already wired up my Vaadin to use Spring to dependency-inject it’s components, so I was able to map this fragment directly to the Spring ID’s of my components. So when my app says “www.foo.com/myapp/#bar”, I can have it automagically find the component “bar” and display it in the main area. Invalid names (if the user types a URL I don’t understand) can easily be handled with an appropriate message, or I can just send them back to the main menu.
The reverse is also true: if the user selects a new item from my menu, and the application loads the component, I can use the UriFragmentUtility to set the fragment seen in the URL box – so the user clicks a menu choice, and the URL changes to “www.foo.com/myapp/#baz”.
This means bookmarks work as well: if I bookmark “www.foo.com/myapp/#baz”, and come back tomorrow, it will work just as expected, taking me directly to the “baz” “page” of my app (I use “page” cautiously, as there is in fact no such concept as a “page” in a Vaadin app).
It’s equally easy to handle URL parameters, or to read a format of URL that doesn’t include the “#”, if you are bothered by it. So I can do things like “www.foo.com/myapp/someplace/1234″, and automatically load the component “someplace” and pass it the parameter 1234, or “www.foo.com/myapp/someplace?id=1234″ if you prefer.
All easy, all friendly to the browser back-button – which now does exactly what the user would expect – takes him/her to the last place in the Vaadin app he was.
So as far as I’m concerned, Vaadin doesn’t break the back button at all, and leverages it, bookmarking, and URLs and general very easily.
This also solves the issue of how to Search-Engine Optimize a Vaadin app – but that’s another post for a later day.
Thought I’d pass that along.
In my ongoing experiments with Apache ServiceMix, I recently installed the Felix Web Console project, and it was simple and helpful.
OSGi containers typically have a “console” for administering the bundles you’re deploying into our container via the command-line. ServiceMix has a nice “remote” feature, that allows me to get to the console of our ServiceMix instance from my local machine easily, without having to actually ssh to the ServiceMix box.
It looks like this:
You can list your bundles, install, uninstall, restart, etc, all from here.
What Felix Web Console brings to the table is a nice webapp front-end on all this functionality.
Installation is the easiest thing possible: you go to your existing console and type:
<dependency> smx@root:osgi> install http://mirror.switch.ch/mirror/apache/dist/felix/org.apache.felix.webconsole-2.0.2.jar
It reports that a new bundle is installed, and gives you the number (237 in my case).
Then I say:
start 237
And I can immediately surf to the following URL in my browser: http://servicemix.point2.com:8080/system/console
and see the following page:
Now I can browse my bundles, install, uninstall – and all the same good stuff as via the console, even view the output of the log service, except all from the comfort of my web browser.
Like many things in the OSGi world, it just works like it’s supposed to, and is much easier to do than it is to explain
In Part 1 of our OSGi adventures, we described how to build a nice Ajax-aware Vaadin UI app, and couple it to a generic back-end OSGi service we called the service dispatcher.
Now we’ll take the adventure to the next step, show how to deploy that webapp into our OSGi container and get it up and running, then go through the functionality of the Dispatch Service, and show how it routes requires through to the first of our domain-specific application services.
In our UI, we built a form that allowed the user to enter and save a “Dealer”, with some fields like name, phone number, etc., so the first service we’ll build for the service dispatcher to talk to will be our “dealer” service.
First, though, let’s see how to get our webapp into our OSGi container.
As we’re building our Vaadin app with Maven, we can easily add the small bits of additional configuration to turn our project into an OSGi-friendly WAR file.
OSGi deploys “bundles”, but a bundle is just a jar file (or a war, which is after all just a special kind of jar) with a bit of extra meta-data. The META-INF/MANIFEST.MF file is where the magic happens. We add the following to our POM:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <version>2.0</version> <configuration> <archive> org.springframework.osgi.web.context.support,\ org.springframework.web.servlet,\ org.springframework.web.servlet.handler,\ org.springframework.web.servlet.mvc,\ org.springframework.web.servlet.view,\ dwmj.domain,\ org.springframework.web.servlet.mvc.annotation,\ org.springframework.web.context <manifestEntries> <Bundle-ManifestVersion>2</Bundle-ManifestVersion> <Bundle-SymbolicName>com.point2.Admin</Bundle-SymbolicName> <Bundle-Name>Admin</Bundle-Name> <Bundle-Version>1.0.0</Bundle-Version> <Bundle-Activator>com.point2.ServiceClient</Bundle-Activator> <Import-Package>org.osgi.framework,com.point2.services.dispatch,javax.servlet;version="2.4.0", javax.servlet.http;version="2.4.0",org.osgi.service.http;version="1.2.0", org.osgi.util.tracker;version="1.3.2"</Import-Package> <Webapp-Context>admin</Webapp-Context> <Bundle-ClassPath>WEB-INF/classes, WEB-INF/lib/service-dispatch-api-1.0.0.jar, WEB-INF/lib/vaadin-6.1.3.jar</Bundle-ClassPath> </manifestEntries> </archive> </configuration> </plugin>
This bit of magic allows Maven to build us a MANIFEST.MF like this:
Manifest-Version: 1.0 Archiver-Version: Plexus Archiver Created-By: Apache Maven Built-By: mnash Build-Jdk: 1.6.0_15 Extension-Name: admin Implementation-Title: admin Implementation-Version: 1.0-SNAPSHOT Bundle-Activator: com.point2.ServiceClient Bundle-ClassPath: WEB-INF/classes, WEB-INF/lib/service-dispatch-api-1. 0.0.jar, WEB-INF/lib/vaadin-6.1.3.jar Bundle-ManifestVersion: 2 Bundle-Name: Admin Bundle-SymbolicName: com.point2.Admin Bundle-Version: 1.0.0 Import-Package: org.osgi.framework,com.point2.services.dispatch,javax. servlet;version="2.4.0",javax.servlet.http;version="2.4.0",org.osgi.s ervice.http;version="1.2.0",org.osgi.util.tracker;version="1.3.2" Webapp-Context: cadmin
Now we have our OSGi-friendly .war file, sitting in our target directory. We can then connect to the console of our OSGi container (in this case, ServiceMix), and say:
install war:file:/Users/mnash/experiments/admin/target/admin-1.0.0-SNAPSHOT.war
Our container honors the “Webapp-Context” tag in our manifest, so we can then surf to http://localhost:8181/admin/ to see our application. 8181 is the default port for the Jetty HTTP service in ServiceMix – it can easily be changed to another number as required, of course.
Dispatch Service Now that we can see how to get the wepapp into ServiceMix, let’s look at the Dispatch Service in detail.
Our Dispatch Service is going to be a regular OSGi bundle, so we have a separate Maven project that produces a .jar file in this case, not a WAR file.
Our MANIFEST.MF needs the following magic for this project:
Bundle-ManifestVersion: 2 Bundle-SymbolicName: com.point2.services.dispatch.DispatchService Bundle-Name: DispatchService Bundle-Version: 1.0.0 Bundle-Classpath: .,lib/commons-beanutils-1.8.0.jar,lib/commons-collections-3.2.1.jar Bundle-Activator: com.point2.services.dispatch.impl.DispatchServicePublisher Import-Package: org.osgi.framework Export-Package: com.point2.services.dispatch
You can see we’re again specifying a “Bundle-Activator” class, which, much like the ServiceClient class in our webapp, is called by the OSGi framework when the bundle containing this service is started.
One slight oddity: The dispatch service needs the two jars listed under “Bundle-Classpath:” to do it’s business – because we are building an OSGi bundle, we “embed” these jars in our bundle (which is, yes, another jar) by putting them in the src/main/resources/lib directory. We refer to them in that location in the POM, and Maven automatically includes them in the finished jar, where they’re available when our bundle needs them. The other alternative is to install the dependencies as their own bundles, but that’s a whole ‘nother post
In the case of DispatchService, we’ve got an activator like this:
public class DispatchServicePublisher implements BundleActivator { private ServiceRegistration registration; public void start(BundleContext context) throws Exception { registration = context.registerService(DispatchService.class.getName(), new DispatchServiceImpl(), null); DispatchServiceImpl.setBundleContext(context); System.out.println("Dispatch Service registered"); } public void stop(BundleContext context) throws Exception { registration.unregister(); System.out.println("Dispatch Service unregistered"); } }
Which simply takes a reference to the BundleContext on startup and passes it to the DispatchServiceImpl, which implements the DispatchService interface, like so:
public interface DispatchService { List<Map<String, Object>> call(String serviceName, String serviceOperation, Map<String, Object> parameters, String versionPattern, String securityToken) throws Exception; }
The big JuJu with OSGi is that only this interface is “exposed” from the bundle. No other service can see the innards of our service, unlike Jars on a classpath.
The implementation of the DispatchService is equally straightforward:
public class DispatchServiceImpl implements DispatchService { private static BundleContext context; public List<Map<String, Object>> call(String serviceName, String serviceOperation, Map<String, Object> parameters, String versionPattern, String securityToken) throws Exception { ServiceReference reference = getServiceNamed(serviceName); if (reference == null) throw new RuntimeException("There is no service with service-name " + serviceName); Object service = context.getService(reference); Adapter adapter = new Adapter(service); List<Map<String, Object>> response = adapter.callList(serviceName, serviceOperation, parameters); context.ungetService(reference); return response; } private ServiceReference getServiceNamed(String serviceName) throws Exception { ServiceReference[] references = context.getAllServiceReferences(null, null); System.out.println("There are " + references.length + " services available"); for (int i = 0; i < references.length; i++) { ServiceReference reference = references[i]; Object serviceNameValue = reference.getProperty("service-name"); if (serviceNameValue != null) { System.out.println("I see service with name " + serviceNameValue); if (serviceNameValue.toString().equalsIgnoreCase(serviceName)) return reference; } } throw new RuntimeException("There is no service available with service-name " + serviceName); } public static void setBundleContext(BundleContext newContext) { context = newContext; } }
The dispatch service implementation simply looks through the “published” services in the OSGi context, and looks at the “service-name” property of each service to find the one specified in the call. Assuming it finds an appropriate service, it then uses another class, Adapter, to do the type conversion of the generic Map of parameters to the specific types needed for the service being called, then uses Java reflection to actually make the call and return the response as a List of Maps, again converting the service-specific types to a generic Map in order to pass the response back to the user interface.
Why do we go through those gyrations, instead of just having access to the domain-specific beans in our UI? If we want a full decoupling, and the advantages that come with it, the type-independence of this approach gives us that. It also allows parallel development of the UI and the back-end services without an ever-increasing number of binary dependencies as well – for that matter, with the static data adapter in the Vaadin app allows us to develop on our UI entirely without the back-end services. That’s pretty decoupled.
The Dispatch Service by itself, though, doesn’t give us any functionality. It acts much like a router on a network, simply moving the request from the client to the proper service on the back-end, so let’s build such a service – in this case, a service for handling Dealers.
Dealer Service The bulk of our application-specific code will be prepared as independent OSGi services, just like this one. In later postings, I’ll describe how to set up dependencies between services (and how to use Spring DM to make doing that kind of thing far simpler).
First, let’s look at our MANIFEST.MF for our new service (again, we can use Maven to produce this for us, but the result is the same):
Bundle-ManifestVersion: 2 Bundle-SymbolicName: com.point2.services.dealer.DealerService Bundle-Name: DealerService Bundle-Version: 1.0.0 Bundle-Activator: com.point2.services.dealer.impl.DealerServicePublisher Import-Package: org.osgi.framework Export-Package: com.point2.services.dealer
The two key elements above are the Bundle-Activator, a class called DealerServicePublisher, and the Export-Package.
OSGi best practices dictate that the package that is exported should only contain the interface for the class (or classes) you wish to make available from your service. In our case, that comprises a bean class, called “Dealer”, and the interface to our actual service, DealerService.
The Dealer bean is very simple, just your basic property-holding JavaBean:
public class Dealer { private String name; private String phone; private String contactLast; private String contactFirst; private String address1; private String address2; public String getContactLast() { return contactLast; } public void setContactLast(String contactLast) { this.contactLast = contactLast; } public String getContactFirst() { return contactFirst; } .....
I’ve ommitted the rest of the getters and setters here for brevity (in case you’re wondering, yes, this is much easier to express in Scala, and yes, OSGi works just fine with Scala…)
The interface for our service is even simpler:
public interface DealerService { public void save(Dealer dealer); public Dealer findById(int id); }
Of course, a fully fleshed-out service might in fact have more methods, but again, you get the idea.
I won’t bore you with the actual implementation of the DealerServiceImpl class, but it’s important to note that it’s not in the same package as the Dealer bean and the DealerService implementation. They are the only two classes (well, one class, one interface) in that package, as the entire package is “exported” by OSGi, and therefore visible to other services and clients.
The DealerServicePublisher is the last piece to examine, and it’s pretty straightforward as well:
package com.point2.services.dealer.impl; import com.point2.services.dealer.DealerService; import org.osgi.framework.BundleActivator; import org.osgi.framework.BundleContext; import org.osgi.framework.ServiceRegistration; import java.util.Dictionary; import java.util.Hashtable; public class DealerServicePublisher implements BundleActivator { private ServiceRegistration registration; public void start(BundleContext context) throws Exception { System.out.println("Registering dealer service"); Dictionary dictionary = new Hashtable(); dictionary.put("service-name", "dealer"); registration = context.registerService(DealerService.class.getName(), new DealerServiceImpl(), dictionary); } public void stop(BundleContext context) throws Exception { System.out.println("Unregistering dealer service"); registration.unregister(); } }
Essentially all we do in this activator is “register” our service, making it visible to the OSGi container and other services or clients that need it. We add an extra property via the “Dictionary” object, which allows us to specify arbitrary properties to be associated with our service. Because we want to look up our services from the dispatcher by name, rather than by class or interface name, we use the string “dealer” and associate it with the key “service-name”. If you examine the code for our DispatchService, you’ll see that it uses this property to find services.
Now we can build our new service with a simple “mvn package” command, and install the resulting jar into our OSGi runtime with “install file:/Users/mnash/experiments/dealer/target/dealer-1.0.0.jar” (again, you can do an install directly from the Maven repository, or from a URL, as opposed to a file).
That’s it – our “dealer” service is now available, and our “dispatch” service is fired up and ready to locate it.
If we deploy our Vaadin application in our OSGi container (as described in the previous post), you’ll find that calls to the “call” method of the ServiceClient now return the “real” data from our service.
In the next few posts we’ll examine an even easier way to define new services, and explore the power of OSGi for updating and working with our decoupled services. Then we’ll look at how to hook services together, allowing services to call other services to do their jobs as required.
