In a project I was tinkering with lately I found the complexity getting to the point where I could use component dependency injection. There are in fact a number of techniques to do this in “plain” Scala, but they generally seem more painful to me than just dropping in a lightweight DI framework, so I went with Guice.

As it happens, Guice works with Scala very nicely, and is very unobtrusive. There’s a few minor things I haven’t quite figured out yet, but I’d like to share what I’ve discovered so far.

After adding this section to my project POM:

 
            com.google.inject
            guice
            2.0
        

I had all I needed of Guice embedded in my app. Now I can scratch up a series of tests that exercise all of the different ways use Guice from Scala. Let’s start with the simplest, and work our way up:

 @Test
  def basicDependencyInjection {
    class ScalaModule extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).to(classOf[SomeService])
      }
    }

    class ScalaModule2 extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).to(classOf[SomeOtherService])
      }
    }

    val injector = Guice.createInjector(new ScalaModule)
    val component = injector.getInstance(classOf[MyComponent])
    assertEquals("someService", component.callTheService)

    val injector2 = Guice.createInjector(new ScalaModule2)
    val component2 = injector2.getInstance(classOf[MyComponent])
    assertEquals("someOtherService", component2.callTheService)
  }

So let’s explore the test a bit. I’m using Scala to build a JUnit test, as shown by the annotation on line 1. One of the advantages of this is that it’s easy to use Scala’s namespace capabilities to build test classes that are only visible within the test itself, which is what I’m doing starting line 3. I create a class call ScalaModule with extends Guice’s base class AbstractModule. I override one method called “configure”, which sets up the bindings of interfaces to implementations for this Guice module. (This is all covered in detail in Guice’s doc).

On line 6 I bind the interface “AService” to an implementation of AService called “SomeService”. Here’s what they look like (these classes are defined later in the file – I don’t embed them inside the test class as Guice didn’t let me).

Here’s the trait for the service:

trait AService {
  def service: String
}

And a simple implementation:

class SomeService extends AService {
  def service(): String = "someService"
}

Now I have a component that uses that service, called MyComponent. It looks like this:

class MyComponent @Inject()(val service: AService) {
  def callTheService(): String = service.service
}

Note the annotation on the first line. This is how you put an annotation on the constructor in Scala, Annoyingly, the () is apparently required. This Inject annotation indicates that when this component is requested from Guice, it should have the appropriate implementation of Service injected.

So now to return to our test, on line 10 you’ll see we are building a second module, this time binding the AService interface (trait, actually) to the SomeOtherService implementation, which looks like this:

class SomeOtherService extends AService {
  def service(): String = "someOtherService"
}

Then we get into the meat of the test in line 17. First, we ask Guice for our injector, the “god object” for Guice, from which you get all the other object you’ll need. In the first case, we use our ScalaModule class. Then we ask this injector for our component, and it constructs the component for us, injecting the dependency as necessary. In this situation, we have not told Guice that MyComponent is a singleton (we’ll deal with that later – in several different ways), so it builds a new one for us every time.

On line 18, we make the assertion that proves that our MyComponent instance has in fact been built with the “SomeService” instance, as opposed to any other instance, by verifying our string that SomeService responds with.

Then we do the whole thing again, except this time asking for the ScalaModule2. ScalaModule2 is where we have the SomeOtherService implementation of AService wired up, so when we assert on line 23, we now see the “someOtherService” string returned instead, which means that we’re properly wired, as expected.

Now let’s give Guice a little harder work, exploring some scenarios we’ll see in more sophisticated applications.

In many situations, we’ll want to avoid creating a new instance of our service object every time we use it. There are a couple of ways to do this with Guice.

@Test
  def instanceBindingExample {
    // if you want to bind to a specific instance of a class...
    class InstanceExampleModule extends AbstractModule {
      @Override
      protected def configure() {
        val instance1 = new InstanceService("instance1")
        bind(classOf[AService]).toInstance(instance1)
      }
    }
    val injector = Guice.createInjector(new InstanceExampleModule)
    val service = injector.getInstance(classOf[AService])
    assertEquals("instance1", service.service)
  }

In the code above, we’re configuring our module, called InstanceExampleModule, with a binding that specifies our AService trait uses a specific instance of the class called InstanceService.

Here’s InstanceService:

class InstanceService(val value: String) extends AService {
  def service(): String = value
}

Back in our test, line 13 ensures that we’re actually talking to the right service, as we fed the service it’s return string (“instance1″) when it was created in the module.

Ok, so now we can inject services, and specify an instance of a service instead of a newly-created one every time. That’s a good start, but we want to do more.

What if we have to do some more work to instantiate our singleton instance? E.g. what happens if it takes a whole method to create the service?

Guice has that covered as well, like so:

Here you see a new module being built, but this time we don’t actually say anything in the “configure” method. Instead, we annotate another method in the module with the @Provides annotation – this means to Guice “whenever you need a thing of the type this method returns, call this method”. Then we verify that that’s in fact what’s happening by creating the injector and asking for the service.
Ok, so now we can create instances of services however we want, but we’ve still only created a single instance of a given trait. What if we’ve got a whole whack of services that implement our interface? Ok, that’s also easy:
@Test
  def bindingWithNameAnnotationExample {
    // what if I have two implementations of the same interface, how do I say which one I want?
    class ScalaModule extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).annotatedWith(Names.named("foo")).to(classOf[SomeServiceNamedFoo])
        bind(classOf[AService]).annotatedWith(Names.named("bar")).to(classOf[SomeServiceNamedBar])
      }
    }

    val injector = Guice.createInjector(new ScalaModule)
    val barComponent = injector.getInstance(classOf[MyBarComponent])
    val fooComponent = injector.getInstance(classOf[MyFooComponent])
    assertEquals("bar", barComponent.callTheService)
    assertEquals("foo", fooComponent.callTheService)
  }

Now we’ve got a couple of services to declare:
class SomeServiceNamedFoo extends AService {
  def service(): String = "foo"
}
class SomeServiceNamedBar extends AService {
  def service(): String = "bar"
}

The services themselves both implement our trait, of course, but now we have two different components, one which needs the Foo service, other other one needs the Bar:
class SomeServiceNamedFoo extends AService {
  def service(): String = "foo"
}
class SomeServiceNamedBar extends AService {
  def service(): String = "bar"
}

In our test, you can see we ask the injector for the two components. What’s very interesting to me is that the component themselves are not listed in the module. Why that’s important is that unlike Spring (to pick an example many people know), you don’t need any configuration provided at all for stuff that Guice can work out on it’s own. This is a good thing.
Anyway, you can see by our asserts that in this situation the Foo service gets the Foo implementation of the AService (SomeServiceNamedFoo), and the Bar service gets SomeServiceNamedBar instead.
Now that we can handle multiple implementations, let’s go back and consider the singleton vs. new instance issue from another angle: Scala has no concept of the idea of “static” classes, but it does have companion objects, which are, in all cases I’ve seen, better.
If we declare a service like so:
object SingletonService extends AService {
  def service(): String = "singleton"
}

and another (non-singleton) implementation, like so:
class NonSingletonService extends AService {
  def service(): String = "nonsingleton"
}

Now we can write a test that tries this out like so:
  @Test
  def singletonOrPrototype {
    class ScalaModuleSingleton extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).toInstance(SingletonService)
      }
    }

    class ScalaModuleNonSingleton extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).to(classOf[NonSingletonService])
      }
    }

    val singletonInjector = Guice.createInjector(new ScalaModuleSingleton)
    val singleton = singletonInjector.getInstance(classOf[AService])
    assertEquals("singleton", singleton.service)
    val secondInstance = singletonInjector.getInstance(classOf[AService])
    assertSame(singleton, secondInstance)

    val nonSingletonInjector = Guice.createInjector(new ScalaModuleNonSingleton)
    val nonsingleton = nonSingletonInjector.getInstance(classOf[AService])
    assertEquals("nonsingleton", nonsingleton.service)
    val secondNonSingletonInstance = nonSingletonInjector.getInstance(classOf[AService])
    assertNotSame(nonsingleton, secondNonSingletonInstance)
  }

The money here comes at the line that binds the classOf[AService] to a singleton – in this case, the SingletonService. This isn’t a class, so we don’t use to(classOf[SingletonService]), we say toInstance(SingletonService) instead.
As you can see from the test, this works very nicely, and gives us a much more Scala-canonical way to do singleton.
However, there are times when we want to declare a regular Scala class, and have Guice only instantiate one of em for us, and handle it as a singleton that way (some examples might be when we’re using a constructor to inject other dependencies – although we can use field injection for this, which we’ll look at later).
We can annotate our class explicitly to tell Guice it’s a singleton like so:
@com.google.inject.Singleton class SingletonClassService extends AService {
  def service(): String = "singletonClass"
}

Now we can write a test like so to prove that we get the same instance of our service, even if we ask the injector for it multiples times:
@Test
  def singletonClassExample {
    class ScalaModuleSingleton extends AbstractModule {
      @Override
      protected def configure() {
        bind(classOf[AService]).to(classOf[SingletonClassService])
      }
    }

    val singletonInjector = Guice.createInjector(new ScalaModuleSingleton)
    val singleton = singletonInjector.getInstance(classOf[AService])
    assertEquals("singletonClass", singleton.service)
    val secondInstance = singletonInjector.getInstance(classOf[AService])
    assertSame(singleton, secondInstance)
  }

The assertSame in JUnit asserts that we have the exact same instance of an object, not that they’re only equal.
One last scenario: What is the service isn’t the singleton, but the component is? E.g. what if we have an object instead of class that needs a service injected into it? Turns out Guice can handle that nicely as well. The only extra work we need is to define a trait for the binding of our component, like so, with the service injected into the object:
trait SingletonComponentInterface {
  def callTheService:String
}
object SingletonComponent extends SingletonComponentInterface {
  @Inject val service:AService = null
  def callTheService(): String = service.service
}

Now we can write a test to make sure this is doing what we want:
@Test
  def injectIntoSingletonExample {
    // if you want to inject into an object instead of a class
    class InstanceExampleModule extends AbstractModule {
      @Override
      protected def configure() {
        val instance1 = new InstanceService("instance1")
        bind(classOf[AService]).toInstance(instance1)
        bind(classOf[SingletonComponentInterface]).toInstance(SingletonComponent)
      }
    }
    val injector = Guice.createInjector(new InstanceExampleModule)
    val component = injector.getInstance(classOf[SingletonComponentInterface])
    assertEquals("instance1", component.callTheService)
  }

That’s it for my experiments so far. Here’s my conclusions, please comment with your experiences:
The Good, The Bad, and the Ugly
The Good

There was a lot I liked about Guice in Scala. First, no XML was injured in the making of these examples – everything was code, and fully type-safe (I mentioned it was Scala, right?) I always find myself writing a “smoke test” for my Spring apps to verify I haven’t made a typo in my “beans.xml” file – here I didn’t need to. Yes, I know you can do XML-free Spring as well, but it’s the norm with Guice, and it feels a lot lighter weight.
I don’t need to configure anything for classes that just get injected. This means that the configuration I do need to write is very short, and because it’s a class, it’s extensible. In my “real” project, for instance, I was able to write a module that created services that use static lists for their data (instead of reading and writing to a real database, for instance), then extend that class with an OSGi-aware version that used the “real” services from other modules. No repeated work, no wiring, no config – it just worked.
Auto-handling of singletons, both with companion objects and classes, as I prefer: It was nice to be able to write Scala in a very Scala-like way, and just have the DI framework slot right in whichever way I needed it to, as described above.
The Bad

I have to annotate my classes with Guice-specific annotations, although there is a JSR on the way to standardize these, which makes it a bit less objectionable. What happens if I need to inject to a class I don’t have source for (e.g. something in an existing Java library that I’m calling? I also haven’t tried this in a Servlet environment, although I understand there’s a Guice extension for that, so I don’t expect any trouble there.
The Ugly

I have to (it seems) use the @Inject() format, not just @Inject, and I have to insert it in what looks like an odd place in the declaration. Not a big deal, and I suspect I’ll get used to it.
            

I’ve had an opportunity recently to work on a product that needed an RIA web interface, and I chose my recent favorite tool for this, Vaadin. The services for this project needed to be highly scalable, and lent themselves well to functional techniques, so I selected Scala as my language of choice. I build my projects with Maven, for reasons I won’t go into right now, and I do much of my JVM-language work in Intellij’s excellent IDEA IDE.

Given these tools, I found a way to facilitate very rapid development of web UI’s, and I thought I’d pass it along.

Another technique I use, which I’ll expound on later, is creating “dummy” implementations of all of my backing services for my application. The “real” implementations are written as OSGi services, in separate modules from my UI. The UI is packaged as a war, but is also OSGi aware, with a bundle activator. This activator only gets called if the war is deployed into an OSGi container, and not otherwise. This allows the app to select which implementation of the services it uses – the “dummy” ones when it’s deployed outside of OSGi, and the “real” ones when they’re available.

This means I can use the handy Maven jetty plugin to quickly spin up my application and test it on my local workstation, without needing all of the dependencies (like a data store and such) of my real services. That’s good, in that I can get my “cycle time” down to a few seconds, where “cycle time” is the time between making a change and actually being able to test it in my browser.

We can do better, though.

I’m using Scala as my language of choice for building the UI as well, as it works just fine with Vaadin (and with everything else in the JVM ecosystem, for that matter, which is why I didn’t choose a non-JVM language – but that’s yet another rant).

I compile my Scala with the Maven scala plugin – here’s where the next handy bit comes into play. Turns out the Scala plugin has a goal called “cc”, for continuous compilation. Using this, I can fire up Maven with a “mvn scala:cc” command, and leave it running. Then I also use the “mvn jetty:run” command in another window to fire up the web application, and leave it running as well.

Here’s my configuration for the Scala plugin:

                org.scala-tools
                maven-scala-plugin
                2.9.1
                
                    
                        
                            compile
                            testCompile
                        
                    
                
                
                    ${scala.version}
                    
                        -target:jvm-1.5
                    
                
            

And for Jetty:

                org.mortbay.jetty
                maven-jetty-plugin
                6.1.9
                
                    10
                    src/main/webapp
                    jetty.xml
                    
                        
                            file
                            realm.properties
                        
                    
                    stop
                    8889
                
            

Now I go back to my IntelliJ and start coding. Every time IntelliJ saves (which it does automatically unless you tell it not to), the Scala plugin compiles the files. This generates a new .class file, which the Jetty plugin (well, technically, Jetty itself) detects, and in response, reloads the running classes for the web application.

Net effect is that I can make my change and by the time I switch back to the browser, my new code is running. I test my change, emit the appropriate profanity, and go back to editing, all within a second or two.

This has profound effects on how you develop a UI, which every dynamic language aficionado knows (e.g. like Ruby/Rails or Python/Django). You don’t hesitate to experiment, and you get to see the visual effect of your changes right away. The good news is that I get to do this with my language of choice, and with all the power of the JVM ecosystem to support it.

The technique is not perfect – I’ve found that if you edit some resources or webapp files (images and such), it’s possible the Jetty plugin doesn’t “see” the change. Of course, two things help with that considerably: first, it’s lightning-fast to just Control-C the jetty plugin task and re-launch it, and second a Vaadin app generally doesn’t use many resources, unlike JSP or many other frameworks that make extensive use of templates.

Once in a while I’ve found the scala:cc task will report that it’s lost it’s connection to the “fsc” (fast scala compiler) background process – again, quickly control-c-ing the task and starting it again solves the problem every time.

Overall, I can crank UI out pretty darn quick with this method, and given that I can TDD even my UI code using Vaadin, I find the overall combination very effective and efficient.

I’ve had the chance over a recent weekend to have a first crack at a web framework called Vaadin.

I was originally browsing for news about the latest release of Google’s GWT framework when I stumbled on a reference to Vaadin, and decided to go take a look. What I found intrigued me, and I decided to take it for a test drive, as I was down sick for a couple of days with a laptop nearby…. My back became annoyed with me, but it was worth it, I think.

First Look
First off, the practicalities: Vaadin is open source, and with a reasonable license, the Apache License. The essential bits of Vaadin are contained in a single JAR, and it’s both Ant and Maven friendly right out of the box.

The next thing that struck me about Vaadin was the documentation. The first unusual thing about it’s documentation was the fact of it’s existance, as open source projects are downright notorious for poor documentation. Vaadin is a pleasant exception, with tons of examples, a well-organized API doc, in the usual JavaDoc format, and even the “Book of Vaadin”, an entire PDF book (also available in hardcopy) that takes you through Vaadin in enough detail to be immediately productive.

Given that surprisingly pleasant start, I dug deeper, creating a little app of my own.

Just the Java, Ma’am
The main thing that kept me interested in Vaadin once I started digging further was that it’s pure Java. Many frameworks talk about writing your UI work in Java, such as Wicket, but there’s still a corresponding template and some wiring that happens to put the code and the template together. Not so with Vaadin.

When they say “just Java”, they mean it – your entire UI layer is coded in Java, plain and simple. No templates, no tag libraries, no Javascript, no ‘nuthin. It’s reminiscent of the Echo framework, except in Vaadin’s case the Javascript library that your code automatically produces is Google’s GWT, instead of Echo’s own Core.JS library.

Unlike GWT, though, the Vaadin approach doesn’t bind you to any specific source code though, it’s just a binary jar you put on your classpath.

The only thing in my sample app, other than 2 Java files, was a web.xml and a css stylesheet, both of which were only a few lines long. And this was no “Hello, World”, either, but a rich AJAX webapp with a tree menu, fancy non-modal “fading” notifications, images, complex layouts, and a form with build-in validation. And it took maybe 4 hours of total work to produce – and that was from a standing start, as I’d never heard of Vaadin before last Thursday. Not bad, not bad at all.

I found I was able to get a very capable little webapp up and running with no need to invent my own components, even though I had trees and sliders and menus and other assorted goodies on the page. It worked in every browser I was able to try it in, which is certainly not the case for my own hand-rolled JavaScript most of the time

I haven’t yet tried creating my own custom components, but it certainly looks straightforward enough.

I did try linking to external resources, and included non-Vaadin pages in my app, with no difficulties, so it appears that Vaadin plays well with others, and can be introduced into an existing project that uses, for instance, a whack of JSP’s that one might want to obsolete.

Webapps
I think Vaadin warrants more exploration, and I intend to put it further through its paces in the next few weeks. It appears extremely well-suited to web applications, as opposed to websites with a tiny bit of dynamic stuff in them.

It offers an interesting alternative to some of the patterns I’ve seen for advanced dynamic webapp development so far.

One approach I’ve seen a lot is to divide the duties of creating an app into the “back end” services and the “UI”. Generally the UI is written in either JavaScript, or uses Flex or some other semi-proprietary approach. The “back end” stuff is frequently written to expose it’s services as REST, then the two are bolted together. The pain point here happens when the two meet, as it’s common and easy to have minor (or major!) misunderstandings between the two teams. This usually results in a lot of to-and-fro to work out the differences before the app comes all the way to life.

The other approach, more common on smaller or resource-strapped teams, is to have the same group responsible for both UI and back-end services. This reduces the thrash in the joints a bit, but doesn’t eliminate it, because the two technologies on the two sides of the app aren’t the same. You can’t test JavaScript the same way you write Java, for instance, and they’re two different languages – one of which (Java) has far better tooling support than the other. IDE support, for instance, is superb for Java, and spotty at best for JavaScript.

With Vaadin, both of these approaches become unnecessary, as its the same technology all the way through (at least, what you write is – technically it’s still using JavaScript, but because that’s generated, I don’t count it).

You get to use all of the tools you know and love for the back-end services to write the code for the UI, which you can then unit and functional test to your heart’s content.

The temptation to mix concerns between UI code and back-end service code must still be resisted, of course, but at least that code isn’t buried somewhere in the middle of a JSP page, ready to leap out and bite you later.

Because you’re using dynamic layouts, the app always fits properly on the screen without any extra work, addressing a pet peeve of mine, the “skinny” webapp, restraining itself to the least common denominator of screen size, thus rendering impotent my nice wide monitors.

Scala
Just because Vaadin is a Java library doesn’t restrict you to using Java to drive it, however. I made another little webapp where the whole UI was defined in Scala, calling the Vaadin APIs, and it worked like a charm. In some ways, Scala is an even better fit for Vaadin than straight Java, I suspect. I haven’t tried any other JVM compatible language, but I see no reason they wouldn’t work equally well.

Deployment and Development Cycle
As I was building the app with Maven, I added a couple of lines to my POM and was able to say “mvn jetty:run” to get my Vaadin app up and running on my local box in a few seconds. My development cycle was only a few seconds between compile and interactive tests, as I was experimenting with the trial-and-error method.

TDD would be not only possible, but easy in this situation.

I successfully deployed my little Vaadin app to ServiceMix, my OSGi container of choice, without a hitch.

Performance appeared excellent overall, although I haven’t formally tested it with a load-testing tool (yet).

Summary
So far, I’m impressed with Vaadin. I’m more impressed with any web framework I’ve worked with in a number of years, in fact. I’m sure there are some warts in there somewhere, but for the benefits it brings to the table, I suspect they’re easily worth it. I think the advantages to teams that already speak fluent Java is hard to overstate, and the productivity to produce good-looking and functioning webapps is quite remarkable.

Apache’s ServiceMix 4 is a unique combination of OSGi container and JBI-compliant ESB, but in addition to this it’s also a capable web application container. More specifically, it can contain a webapp container, but the effect is the same…

My current favorite tool for cranking out MVC-style webapps is the Lift framework, so I thought I’d document here the process for deploying a Lift webapp into ServiceMix 4.

The good news is that it’s very straightforward.

First, build the .war for your Lift app using mvn package. This will result in a .war file in the target directory of your project, and is well-covered by the Lift documentation if you need more details. In my case, I started with a slightly modified version of the PocketChange example app for Lift.

Now start servicemix with

bin/servicemix

Done from the root directory of your ServiceMix install. You should see the ServiceMix banner after a few moments, followed by a prompt.

In the console, type the “osgi” command to enter OSGi mode.
then

install war:file:///tmp/admin-0.1.war?Webapp-Context=admin

You’ll see some startup messages as your webapp is digested by ServiceMix, turned automagically into a bundle, and processed by various intestinal organs such as the HTTP service.

Now surf to

http://localhost:8080/admin/

And up comes your application!!

Another update in our team’s journey to functional testing nirvana via Specs and Selenium.

We discovered an earth-shattering revelation (not): Internet Explorer Sucks. Unfortunately, it’s the “target” browser for a certain project we’re working on, therefore we’re stuck with it. Being stuck with it means finding a way to make it suck sufficiently less that it will actually run our lovely tests, which Firefox (and several other browsers) digest quite happily.

One way in which IE sucks is that it’s painfully slow – test that take a few seconds on Firefox timeout after several minutes in IE. If you keep cranking out the timeout they might eventually pass, but we want test results in our lifetimes, so we set about finding a way to get our feedback out of IE a bit faster.

It turns out one of the very slow things about IE is asking for an element via an xpath, and we were doing this a lot. We tried reducing the number of places we did this, and indeed that helped a bit, but there were some things on our existing app that were really hard to refer to without xpath. To ease the pain further, we took a slight detour. Instead of asking Selenium for a specific xpath, which then caused Selenium to ask the browser, we fetched the entire output of the browser (via the getHtmlSource method in Selenium).

What we’d like to do instead, where possible, is simply make assertions on the XML of the entire page – so we get it just one time, and so that we don’t depend on the Xpath implementation of the browser to find bits of our page for us.

Unfortunately, we still can’t ask anything xpath-ish of it, as although it bears a close resemblance to XML, it’s not really html, so any parser we could find choked on it.

Enter HtmlCleaner a handy little jar that can digest even the most disguisting HTML and deposit clean shiny XML in it’s place. Now we could take our XML version of what the browser spat out and finally make Xpath calls to it.

Now Scala itself has excellent support for XML (see the whole book here), but it’s support for Xpath is… interesting. Xpath is actually done via a series of methods and functions, meaning although it is very powerful, it’s not the same xpath as you might be used to, or, more importantly in our case, not the same as you could expect a browser to understand. This means we could either re-write all our Xpath queries (and make it harder to use Selenium IDE or Firebox to help us write new ones), or find another way…

Enter Jaxen, a handy Java lib with full Xpath support, without all the weight of other solutions. Of course, as Specs is written in Scala and Scala happily uses any existing Java lib, we could just drop a jar into our “lib” dir and work some magic.

We needed to convert the HTML produced by Selenium into XML, then turn it into a DOM document that Jaxen can process, so we can ask questions in xpath of the resulting Jaxen document.

This little incantation:

 val cleanerAndProps = initCleaner

  private def initCleaner = {
    var cleaner : HtmlCleaner = new HtmlCleaner();
    var props : CleanerProperties = cleaner.getProperties();

    props.setTranslateSpecialEntities(true)
    props.setRecognizeUnicodeChars(true)
    props.setOmitComments(true)
    (cleaner, props)
  }

Returns a tuple of the HtmlCleaner instance and it’s properties, which we can then use for our conversion somewhat like so:

def getHtmlText(element: String) = {
      var node : TagNode = cleanerAndProps._1.clean(selenium.getHtmlSource())
      val domSerializer: DomSerializer = new DomSerializer(cleanerAndProps._2)
      val document: org.w3c.dom.Document = domSerializer.createDOM(node)
      val xpath = new DOMXPath(element)
      xpath.stringValueOf(document)
  }

This method then takes a string (actually an xpath query), “cleans” the HTML from Selenium, then serializes it into a DOM (a regular org.w3c.dom.Document), allowing us to use Jaxen’s DOMXpath object to fish out the string value of whatever our Xpath query returns, allowing us to say things like:

manufacturerSelected = getHtmlText("//table[@class='DataList']//a")

In our tests.

Of course, we can dress this with other methods that get attributes and other things from the DOM tree, but you get the point.

The nice part about all this is that it doesn’t require a new round trip to the browser – we can assert as many things as we want and fish out as many values as we need directly from the resulting page in milliseconds now, instead of several seconds (or worse for IE) before.

So now we can run our tests much faster than before – but we’re writing a lot of tests, so it’s going to get slow again if we can’t scale better, even with the optimization with Jaxen and HtmlCleaner.

So we looked into Selenium Grid. Before, we had our Ant script that ran the tests (whether locally or on TeamCity), fire up a Selenium RC Server before the run, and shut it down again afterwards. This was time consuming, and problematic as we added more suites of tests. We wanted our tests to be able to run all at the same time, but each Selenium server (which does the communicating with the browser), needs to be on a different port for this to happen. Gah – this was getting complicated.

Selenium Grid, however, address this problem and several more at once.

It allows you to start a single “Hub” server, on one port (4444 by default). This hub then redirects load from tests to one of a number of actual Selenium RC servers that you fire up and leave running long-term.

In our case, we started 3 RC servers to talk to FireFox, and one for IE (it’s not a good idea to run more than one IE RC server on a single machine, as IE doesn’t play nice with other IE’s).

Then on other VM’s we can fire up even more RC servers – both IE and FireFox flavors. Selenium Grid gives you a nice control panel to see the servers and which ones are available, and you can add new servers (and remove them) while the grid is up.

The max number of test runs you can have going at once is now the total number of Selenium RC servers in your grid, allowing many short sets of tests to all run at once, giving much faster actual elapsed time before you get feedback when something breaks.

Of course, one of the benefits of this new approach with Selenium Grid is that I can now test if something works via IE on Windows – from my Mac! For that matter, any combination of browsers and operating systems that we’ve got an RC server for can be tested, all at once. Good stuff.

There are still plenty of frustrations in this process – if you kill a test suite before it finishes, for instance, it appears that the Selenium RC server it was talking to remains unavailable forever, and when you restart the hub, you have to restart all of the RC servers… but it’s a step up from where we were.

I recently had the chance to dive into a new project, this one with a rich web interface. In order to create acceptance test around the (large and mostly untested) existing code, we’ve started writing specs acceptance tests.

Once we have our specs written to express what the existing functionality is, we can refactor and work on the codebase in more safety, our tests acting as a “motion detector” to let us know if we’ve broken something, while we write more detailed low-level tests (unit tests) to allow easier refactoring of smaller pieces of the application.

What’s interesting about our latest batch of specs is that they are written to express behaviours as experienced through a web browser – e.g. “when a user goes to this link and clicks this button on the page, he sees something happen”. In order to make this work we’ve paired up specs with Selenium, a well-known web testing framework.

By abstracting out the connection to Selenium into a parent Scala object, we can build a DSL-ish testing language that lets us say things like this:

object AUserChangesLanguages extends BaseSpecification {

  "a public user who visits the site" should beAbleTo {
    "Change their language to French" in {
      open("/")
      select("languageSelect", "value=fr")
      waitForPage
      location must include("/fr/")
    }
    "Change their language to German" in {
      select("languageSelect", "value=de")
      waitForPage
      location must include("/de/")
    }
    "Change their language to Polish" in {
      select("languageSelect", "value=pl")
      waitForPage
      location must include("/pl/")
    }
  }
}

This code simply expresses that as a user selects a language from a drop-down of languages, the page should refresh (via some Javascript on the page) and redirect them to a new URL. The new URL contains the language code, so we can tell we’ve arrived at the right page by the “location must include…” line.

Simple and expressive, these tests can be run with any of your choice of browsers (e.g. Firefox, Safari, or, if you insist, Internet Explorer).

Of course, there’s lots more to testing web pages, and we’re fleshing out our DSL day by day as it needs to express more sophisticated interactions with the application.

We can get elements of the page (via Xpath), make assertions about their values, click on things, type things into fields and submit forms, basically all the operations a user might want to do with a web application.

There are some frustrations, of course. The Xpath implementation on different browsers works a bit differently – well, ok, to be fair, it works on all browsers except Internet Exploder, where it fails in various frustrating ways. We’re working on ways to overcome this that don’t involve having any “if browser == ” kind of logic.

It’s also necessary to start the Selenium RC server before running the specs, but a bit of Ant magic fixes this.

We’ve got these specs running on our TeamCity continuous integration server, using the TeamCity runner supplied with Specs, where we get nicely formatted reports as to what’s pending (e.g. not finished being written yet), what’s passing, and what’s failing.

The specs written with Selenium this way are also a bit slow, as they must actually wait in some cases for the browser (and the underlying app!) to catch up. When run with IE as the browser, they’re more than just a bit slow, in fact…

They are, however, gratifyingly black-box, as they don’t have any connection to the code of the running application at all. For that matter, the application under test can be written in any language at all, and in this case is a combination of J2EE/JSP and some .NET.

There’s a lot of promise in this type of testing, even with it’s occasional frustrations and limitations, and I suspect we’ll be doing a lot more of it.

A colleague the other day need to write a test that checked a string to ensure it was all digits, and happened to be using Scala for it.

An initial thought was to just iterate through the string character by character – but that was laborious and verbose…

A second thought was to just try to convert the string to an integer with toInt, then watch for the exception when it wasn’t, but even this seemed a bit heavy-handed, now that we’re getting used to Scala’s expressiveness.

After a quick experiment we came up with a cleaner solution: Use the built-in abilities to convert the string to a list, then use a partially applied function to filter the list, then take the length of the resulting all-digits string. If this length is not the same as the length of the original string, then there are non-digit characters in the string – otherwise, it’s all digits.

Here’s a transcript of a Scala command-line session trying this out:

scala> "foo".toList.filter(_.isDigit).length
res5: Int = 0

scala> "123".toList.filter(_.isDigit).length
res6: Int = 3

scala> "abc123".toList.filter(_.isDigit).length
res7: Int = 3
</pre>

assertEquals(originalString.length, originalString.toList.filter(_.isDigit).length)