I project I happened across a while ago is starting to intrigue me more and more: Specs

It’s a framework written in Scala designed to support Behavior-Driven design and testing. This style of testing encourages tests to describe the be “behavior” of the class or component under test, ideally in way that is easy to understand by both developers and non-developers alike.

How does this differ from other forms of testing? Mostly in the way functionality is described – a feature is expressed in terms of a series of interactions with the system under test, describing functionally in a narrative form. We don’t describe a functional technically, as in “passing 43 should result in a response of ‘ABC’”, but in terms of valuable feature interactions, such as “when a valid login is entered, the foo link becomes available to be clicked”, for example.

Behaviour-driven design and development make sure we’re developing code that implements the right features, as opposed to test-driven development, which focuses more on developing the code correctly and accurately. BDD and BDT is more about the “what” as opposed to the “how”, in other words.

If the subject-area expert for the system under development can understand the language of the test, then it becomes more of an executable specification than just a test – it may even form the acceptance criteria for a story or feature, especially when multiple such specifications are chained into a narrative.

Flowing from the oft-described “As a… I want… So that…” way of describing a story, a narrative describes a sequence of flow through an application, describing and verifying the behavior of the system at each step in the narrative.

Specs supports this very nicely, allowing you to write in a DSL (Domain-Specific Language) which is nonetheless a fully capable programming language at the same time (in this case, Scala).

The basic unit of a Specs test is the “specification”, implying, appropriately, that you literally write executable specifications – documentation that actually verifies that it properly describes the system being described!

Scala is, by design, an extremely extensible language, so it’s support for DSL’s such as specs is very good – it doesn’t burden you with a lot of unnecessary syntax or verbosity, so the resulting specification is highly readable and relatively english-like. So much so that it’s even possible for a domain expert non-developer to write, much less read, the executable specifications – especially if they have examples to go from.

Scala fits seamlessly into any JVM-based environment, plays nicely with IDEs, Ant, Junit, Maven, and all the tools you might already be using to build your systems, so it’s adoption doesn’t cause much of a ripple effect. Scala files simply live in a directory structure inside an otherwise all-Java project, for instance, and are compiled to .class files, just like Java.

Specs also leverages other powerful test frameworks, such as Mockito, JMock and Scalacheck, among several others.

My team is using Specs in two modes, so far: One is as a part of the primary build process, just like you’d run your JUnit or SpecsTest tests. The Scala tests in this case are used as either actual unit tests (although expressed in a behavior-driven way) or in-process functional tests (if they include several classes or a whole subsystem under test).

The other way we use Specs is as a set of stand-alone acceptance tests. In this mode we run Specs externally from the system under test. This is especially useful for our systems that rely heavily on REST web services. In these cases we write our narratives to assert various responses from the REST services found at various URLs, while launching Specs from Ant in it’s own VM on another box entirely. These are true “black box” tests, where the tests have no knowledge of or access to the code under test – they must interact with the running system in a very “real world” way. This provides an ideal environment for acceptance tests, especially given the readability of the Specs specificiations. Our user-proxies also like the fact that the test itself can’t be influencing the code “inside the box”, as it’s easy for programmers to write “happy path” tests

Of course, once you’ve used specs and Scala for your testing, you may be tempted to infiltrate it into your other development, but that’s another post

A technique I’ve seen a lot in recent years, and I believe for good reason, is using one language (programming language, that is!) to test code written in another.

There are some non-obvious advantages to this that I’d like to explore a bit below, as well as some obvious (but perhaps less important) disadvantages.

On the downside, of course, you need a developer or team conversant in at least two different languages. This is more rare than you might think, although I think more experienced developers have the edge in that they’ve likely been working in more than one language if they’ve been in the industry a while. Also, many developers like to tinker, and will often pick up one of the “low ceremony” languages to do that tinkering with, and end up gaining at least a passing familiarity with a new language as a result.

Another downside (and often the reason this isn’t done more) is a lack of common tool support between the two languages. If you’re working in an IDE that only speaks Java fluently, for instance, you’re less inclined to use another language for your tests, as then you’ve not only got to switch languages, but endure the “context switch” overhead of changing IDE’s midstream as well. If you’re doing real TDD, that’s a lot of context switches.

On the upside, though, there are a number of good reasons you might want to consider using a different language for your tests:

• Intentional context switch: The same reason some people avoid switching languages for testing can actually be an advantage. When you’re tests are in another language, you’ve got a built-in mental gear-change between the code and the tests
• Decoupling: If you’re changing language, you’re more inclined (maybe even required) to limit the touch-points between the tests and the actual code – e.g. you might isolate the interface to a single class that’s made visible in the test. This can be a good thing, as a too-tightly coupled test is a brittle test (and often not a unit test, but that’s a whole ‘nother topic)
• Better test framework: While xUnit-like frameworks abound in just about every language, some of the higher-level and BDD-type frameworks are best in one specific language. RSpec is an excellent example: It’s originally in Ruby (although I understand some attempts have been made to port it), so it might be that you’d consider using Ruby as a test language in order to get the goodness of RSpec. Ditto for easyb, which is written in Groovy (a very close relative of Java, in many ways).

  The reverse could be true: You could write JUnit unit tests for your Scala application, for instance, taking advantage of JUnit 4 and friends being on the Java platform.

• Better language suitability: Some languages are much “lower ceremony” than other – e.g. compare Ruby and Java. You need a lot of scaffolding and fuss around a large Java application, but Ruby is a bit more nimble and in some cases, more expressive. Expressiveness being a key quality of a good test, this might mean testing a Java app with Ruby (or perhaps JRuby, to be specific) is a good plan, especially with the existence of Test::Unit and RSpec and such
• Multi-Lingual project: It’s becoming more and more common to see a project use more than one development language in any case (e.g. for the ‘real’ code, not just the tests). In this case, there’s even less of a barrier to doing your tests in one of those languages – you’re only helping to maintain your fluency in both language as well, which can’t be a bad thing.

So, how about it? Anyone out there have good or bad stories about multi-lingual testing to share?