I project I happened across a while ago is starting to intrigue me more and more: Specs

It’s a framework written in Scala designed to support Behavior-Driven design and testing. This style of testing encourages tests to describe the be “behavior” of the class or component under test, ideally in way that is easy to understand by both developers and non-developers alike.

How does this differ from other forms of testing? Mostly in the way functionality is described – a feature is expressed in terms of a series of interactions with the system under test, describing functionally in a narrative form. We don’t describe a functional technically, as in “passing 43 should result in a response of ‘ABC’”, but in terms of valuable feature interactions, such as “when a valid login is entered, the foo link becomes available to be clicked”, for example.

Behaviour-driven design and development make sure we’re developing code that implements the right features, as opposed to test-driven development, which focuses more on developing the code correctly and accurately. BDD and BDT is more about the “what” as opposed to the “how”, in other words.

If the subject-area expert for the system under development can understand the language of the test, then it becomes more of an executable specification than just a test – it may even form the acceptance criteria for a story or feature, especially when multiple such specifications are chained into a narrative.

Flowing from the oft-described “As a… I want… So that…” way of describing a story, a narrative describes a sequence of flow through an application, describing and verifying the behavior of the system at each step in the narrative.

Specs supports this very nicely, allowing you to write in a DSL (Domain-Specific Language) which is nonetheless a fully capable programming language at the same time (in this case, Scala).

The basic unit of a Specs test is the “specification”, implying, appropriately, that you literally write executable specifications – documentation that actually verifies that it properly describes the system being described!

Scala is, by design, an extremely extensible language, so it’s support for DSL’s such as specs is very good – it doesn’t burden you with a lot of unnecessary syntax or verbosity, so the resulting specification is highly readable and relatively english-like. So much so that it’s even possible for a domain expert non-developer to write, much less read, the executable specifications – especially if they have examples to go from.

Scala fits seamlessly into any JVM-based environment, plays nicely with IDEs, Ant, Junit, Maven, and all the tools you might already be using to build your systems, so it’s adoption doesn’t cause much of a ripple effect. Scala files simply live in a directory structure inside an otherwise all-Java project, for instance, and are compiled to .class files, just like Java.

Specs also leverages other powerful test frameworks, such as Mockito, JMock and Scalacheck, among several others.

My team is using Specs in two modes, so far: One is as a part of the primary build process, just like you’d run your JUnit or SpecsTest tests. The Scala tests in this case are used as either actual unit tests (although expressed in a behavior-driven way) or in-process functional tests (if they include several classes or a whole subsystem under test).

The other way we use Specs is as a set of stand-alone acceptance tests. In this mode we run Specs externally from the system under test. This is especially useful for our systems that rely heavily on REST web services. In these cases we write our narratives to assert various responses from the REST services found at various URLs, while launching Specs from Ant in it’s own VM on another box entirely. These are true “black box” tests, where the tests have no knowledge of or access to the code under test – they must interact with the running system in a very “real world” way. This provides an ideal environment for acceptance tests, especially given the readability of the Specs specificiations. Our user-proxies also like the fact that the test itself can’t be influencing the code “inside the box”, as it’s easy for programmers to write “happy path” tests

Of course, once you’ve used specs and Scala for your testing, you may be tempted to infiltrate it into your other development, but that’s another post

If you ask most software teams, they’ll tell you that they would like to be producing high quality software. If you ask them if they are, they might hem and haw, but their answer will depend, more times than not, on subjective criteria – how they feel about the software their producing.

It’s important for good software teams to have pride in what they produce, no question, and most developers very much want to produce a quality product, but having the feeling that you’re producing quality is not enough. We need to actually establish that we are producing quality software. But how?

First we have to define what quality software is, as we can’t say yes or no if we don’t know what the question means. In this post I’m going to kick that definition around a bit, then in later posts we’ll talk about how that definition can be applied to our software to make objective measurements of quality.

Here’s where we run into our first bit of difficulty, as the definition of quality software is slightly different depending on who you ask. Everyone thinks they know what it is, but it’s surprisingly difficult to specify when you get asked.

Our best bet then, is to try to determine what the most common attributes of quality software are, the ones that most people would agree are part of a quality software system, or at least would agree have a significant impact on quality. Let’s list a few, keeping in mind that some of these factors are much more important to some situations that to others.

Requirement
The best applications as far as quality must be said to begin with high-quality requirements. A topic unto itself, high-quality requirements must be descriptive, not prescriptive – that is, describe what is required in sufficient detail that it can be clearly understood, yet not say or imply exactly how that functionality is to be implemented, in order to leave the programmer with enough flexibility. A good requirement should also include clear criteria on how to verify if the desired functionality has been achieved – e.g. a good and consistent definition of “done”. There are many more attributes in this area, which we’ll explore in a future post.

Quality of design
A design that fulfills the requirements and takes into account the other attributes that make up quality is often hard to evaluate. Most developers agree, however, that quality software must start with a quality design. Again, this is a large topic that we’ll explore in more detail at another time.

Quality of conformance to the design
Something a little easier to quantify is the actual conformance to the design of the system that implements it. Does the code do what the design said it ought to do?

Reliability
Software that continues to operate the same way despite the passage of time and the variation of environment shows the trait of reliability. If the software is deterministic, e.g. it responds the same way to the same inputs every time (unless of course it’s not supposed to – like a game with a random element), then it is showing some of the attributes of reliability. If it’s a service, then it’s uptime can be used as a gauge of reliability. Reliable software is considered to be of higher quality than unreliable software.

Maintainability
How difficult is it to maintain (e.g. fix and/or improve) the software? Various things about how the software is structured, what language it is written in, how it’s modularized, and many other factors can help give us an idea as to the software’s maintainability. Easier maintainability is a widely accepted attribute of quality software.

Completeness
Quality software is complete: it not only implements every part of the design, it includes the necessary pieces the allow it to stand either without a lot of supporting systems, or at least with a small well-defined set of such systems. Software that stops short of the complete design can’t be said to have this aspect of quality.

Verbosity
The verbosity, or rather the lack of it (e.g. the terseness) of the actual source code can be a contributing factor to quality, at the correct level. Too terse and we can have obscure and unreadable source, too verbose and we’ve got the same. Where the sweet spot is depends a lot on the programming language being used, but it’s often an aspect of quality that directly affects readability and maintainability.

Portability
Although not an attribute in some cases as far as portability from one environment to another, it’s generally considered a good thing if software is not overy dependent on it’s environment – and this is also an indicator of portability. Higher quality software will be able to be used in a variety of different environments more easily than lower quality software.

Consistency
An overall consistency of both design and implementation is certainly related to quality – if we have three pages in our UI design, for instance, they should share some common attributes.

Testability/verifiability
The ability to test software is a critical quality metric. If it’s easy to test, it’s almost certainly of higher quality than if it’s hard to test. Software that can be verified as correct is even better, although somewhat rare and generally requiring a functional programming environment.

Usability
Usability is an aspect of quality that is often the hardest to quantify, although certainly not impossible. It doesn’t only refer to software that’s intended to have a user-facing portion, but may actually apply to software designed as a web service or utility – the usability in this case is more a function of how easily other systems and programs can connect to it and consume it’s services.

Efficiency
Although related to performance to some degree, quality in efficiency is more concerned with making the best user of what resources are available, rather than the overall speed of processing. You might measure aspects of efficiency by looking at memory footprint, CPU utilization, and so forth. A higher degree of efficiency would usually translate into a lower percentage consumed of the available resources.

Security
Although the requirement for this type of quality can vary, it is generally true that software that takes into account security and is by default secure is considered of higher quality than software that is insecure. Witness OSV and Windows, for example (donning flame-proof suit)

Coupling
An aspect of quality closely related to usability in many cases is the degree of coupling, sometimes also called the degree of dependance. If a software application or service requires a lot of other services in order to do it’s job, it often is exhibiting a lower quality in terms of coupling compared to a system that has few dependencies, as the system with more dependencies can be harder to install, deploy and maintain. In a SOA environment, of course, a service can reasonably expect to depend on other services, but as few as possible is generally best.

How the coupling is handled in terms of packaging and distribution can also be an important quality aspect.

Composability
This aspect is best demonstrated by the traditional Unix command-line utilities, such as grep, sed, pwd, and so forth. They do one thing, and do it simply but well – but they are eminently composable, being designed to be strung together to handle more complex tasks. If a service or application exhibits high composability, it is easily able to be fit into a series or of such services to perform more complex tasks than any one of them can handle alone.

A high-quality component is often one that can stand alone and perform a useful task, but also be combined with other packages as required easily and quickly.

Ensuring that all of the criteria (or at least all the ones we care about in our situation) usually requires a certain level of quality and consistency to our process, as well, but that’s another whole discussion.

I’ve recently had occasion to start an open source project, and the correct tool for the job appears to be Scala.

So far the project is going well, but the pain has been around the build and IDE support for rapid and convenient development in Scala. Although all three of the major IDEs I’ve worked with recently (Eclipse, IntelliJ IDEA and Netbeans) have plugins for Scala, they are all early releases, and have various degrees of pain associated with them.

I ended up using Netbeans for editing and as a subversion client, then building with Maven when I wanted to compile and/or run tests. Calling Maven from within Netbeans to build a Scala project is still a bit creaky, so I was doing it from a terminal window directly.

This is very inconvenient, for a number of reasons. First, I’m working in a Behaviour-Driven Development mode, using specs as my BDD framework. This means I first write a specification in specs, see it fail, then write the code necessary to make it pass, then write (or extend) the next specification for the next behaviour I want, and so forth.

When I want to run a test, I had to flip to a command window, issue a Maven command to build and select the specified test to run, something like this:

mvn -Dtest=foo test

In order to make this work I had to declare my specs as JUnit tests (with the @Test annotation), even though they don’t use anything else from JUnit. This felt like a bit of a hack, albeit a useful one. Another pain point was the startup time for Maven (although I understand there’s a “console” plugin for Maven as well that can perhaps reduce this particular pain).

As I like to tinker with new stuff, I thought I’d make a departure from Maven and give sbt a try. Sbt is a build tool written in Scala that supports building both Scala and Java (and mixed) projects in a very simple way. Unlike Ant, there’s no up-front pain to write a build script, though, as sbt can make reasonable assumptions (which you can override) about where to find your classes and libraries, so you hit the ground running.

In literally seconds I was up and running after following the install instructions on the sbt site. After a bit of experimenting I found the “console” mode in sbt, where you launch sbt and leave it running.

Once in console mode you can either just type “test” every time you want to build and run all tests, or be more selective, and run only the tests that failed last time, or just a single specified test, if you’re working on just one feature. Any of these operations are fast – mostly because sbt is already loaded and running, but also because sbt does a bit less work then Maven does on every build.

Although sbt can be configured to work in conjunction with Ivy or Maven repositories, you can also just drop your dependency libs in to “lib” directory in your project. For open source this is rather nice, as it saves the user of the project the trouble of trying to find them. Even supplying a Maven pom that specifies the repositories from which to download your dependencies is not a guarantee, as repositories change over time. Many is the time I’ve gone to download a dependency (or rather, Maven has gone to do it for me), only to find it’s not where it used to be, is a different name or version, or some other problem causes my build to fail. Like Ant, sbt can avoid this problem by keeping dependencies locally. Unlike Ant, it can also go get the dependencies the first time for you from the same repositories Maven uses – perhaps giving you the best of both words in some situations.

Even more interesting was the command

~ test

Which runs all the tests, then waits for any source code to change (test or main code). When it sees a change, it runs all the tests again (after compiling the changes, of course). Poor mans continuous testing

Wait, it gets even awesomer! When you say

~ test SomeTest

sbt will wait for any changes, then run just the specified test. This is ideal when you know you’re only working on a specific set of functionality (and therefore affecting only a single test). When sbt is waiting, you can just hit any key to return to the interactive mode, so it’s easy to change it from one of these modes to another.

Other commands in sbt are also very familiar and quick, such as “compile”, which does exactly as you’d expect from the name. “Package” is another good one – it produces a jar artifact, just like the Maven command of the same name. I haven’t yet tried it’s deploy mechanisms properly, but early results look promising.

I also like the “console” command, which runs the Scala command-line console, but with your project on the classpath, along with all it’s dependencies. This lets you do ad-hoc statements quickly and easily, and see the results right away. When you’re not sure what’s going on with a failing spec, I’ve found this mode very helpful to experiment. Scala is such an expressive language, I can write a quick experiment in one or two lines of code, see the result (as the Scala console also evaluates expressions by default), and go back to coding and testing, all without re-starting sbt. Quite nice, and somewhat reminiscent of the similar functionality in Rails and “irb” (and JRuby’s equivilant, Jirb).

There are many other things I’ve found about sbt that I like so far, but those are topics for another post later on….

Java has always had the ability to develop concurrent applications – right from the start, the “Thread” class and its kin gave a single Java application the power to spawn off a separate process, while continuing with the original process. Like all concurrency, it was of course limited by the amount of cores or systems available to run processes, but the basics were there.

More recently, however, it’s become clear that higher levels of concurrency are required in order to make our applications as scalable as we’d like, especially in light of recent hardware trends that give us not as many huge leaps upward in terms of speed, but provide more and more cores for our applications to spread themselves into in each system.

Concurrency is more than running multiple processes on the one box, though – we must think of the problems we’re trying to solve in new ways, ways that lend themselves well to expressing groups of smaller tasks that can all be happening at once, with as little temporal dependency as possible – that is, trying to reduce the need for one task to wait for another task to complete before it can move ahead. This is not just a programming issue, it has to be taken into account at design-time as well, to a degree.

Java’s original concurrency support was oriented more towards richer threads – entire streams of execution that could exist for a long time if needed. Just the thing if you’re writing a web server, or something similar. These richer threads were expensive to create, however, so a developer didn’t want to just spin off thousands of them, or the hope-for increase in scalability would be lost to the overhead pretty quickly.

Other problems lend themselves better to a lighter-weight kind of concurrency, where the cost of breaking up into smaller concurrent tasks is lower, and the tasks generally shorter-lived and less complex than a full thread. This leads us to lightweight alternative methodologies such as Fibers, Processes, and Actors, such as those found in more functional-oriented languages, such as Haskell, Erlang and others.

In more recent incarnations of Java, libraries were added to the JDK to allow us to do quite a bit of lightweight concurrency right in good ole’ fashioned Java, however, and these bear closer inspection if you’re not familiar with them.

Let’s take a closer look at java.util.concurrent and it’s classes by way of a code example…

Let’s say we’ve got a class to do some useful piece of work, say a piece of mathematics, that takes some time to process. Let’s call it “UsefulWork” in a fit of creativity. It’s created with a parameter that gives it what it needs to do the useful thing it does – in our example, just an integer.

We have a lot of these classes to run in order to solve our overall problem, each with a different set of parameters. When we’re all done, we need to add up the results and present the final answer to our caller.

In the sequential world, we might defined our UsefulWork process as a simple method, like so:

 private long doUsefulWork(int param) {
    ... do the number crunching...
    return result;
 }

and we might call this method a bunch of time, once with each of our parameters, adding up the overall result as we go, like thus…

long finalResult = 0L;
for (int i = 0; i < numberOfJobs; i++)
   finalResult = finalResult + doUsefulWork(param[i]);
}

Each computation complete, we add up our answer, and life is good. But if we look at this problem in a different way, there’s nothing about each call to doUsefulWork that requires the answer from the previous call – each one does it’s job independently. So what’s stopping all the useful work from happening concurrently, rather than one piece at a time? Not a thing – let’s see how that would look.

Instead of a method, we now define a little class to do our useful computation for us, while implementing the Callable interface, like this:

    private class UsefulWork implements Callable<Long> {

       private int param = 0;

        public UsefulWork(int param) {
            this.param = param;
        }
        @Override
        public Long call() throws Exception {
            ... work out the result, using the param, however it is we do that....
            }
            return amount;
        }
    }

The “call()” method is defined in the Callable interface, and is a bit like the “run()” method from Thread, except it allows us to return a result.

Now we tee up a series of our UsefulWork classes, like so:

List<UsefulWork> usefulWorkList =
        newArrayList(new UsefulWork(100), new UsefulWork(50), new UsefulWork(200), new UsefulWork(10),
        new UsefulWork(500), new UsefulWork(125), new UsefulWork(80));

(we’re using a method from the invaluable Google Collections library to spin up a new ArrayList of the proper type here)

Now we fire up our new Callables like so:

ExecutorService executorService = Executors.newFixedThreadPool(POOL_SIZE);
List<Future<Long>> futures = executorService.invokeAll(usefulWorkList, 5, TimeUnit.SECONDS);
Long total = 0L;
        for (Future future: futures)
            total = total + (Long) future.get();

In the first line above, we create an ExecutorService, in this case, using a fixed-size “pool” of threads which will be used to run our jobs (this is the expensive bit, so it’s nice to do this in some kind of initialization only once – the same pool can be re-used).

Then we create a list of Future objects, each of which deals with type “Long”, by calling the executorService.invokeAll. The parameters are our list of Callables, timeout, and the unit in which the timeout is expressed – in this case, seconds. What we’ve said is “start doing all these things I’m giving you at once, but don’t allow any one of them to run more than 5 seconds”. We can also use unlimited timeouts, but then a hanging Callable can get us in trouble.

The rest of our code above is just collecting the results – once we’ve made the call to “invoke” our UsefulWork’s are all churning away (POOL_SIZE of them at a time, anyway), doing whatever they do to get their invdividual answer without waiting for the herd.

Our for loop at the end simply collects the fruit of their labours, retrieving the results of the calculations that have already been performed. It’s important to note that the call to “get()” is not actually doing the calculation at that moment – the actual calculation is already finished, ready to return. This is the power of the Future interface – the ability to do it’s job and then be available at a later time for retrieving the result.

This is of course an extremely simple example, and doesn’t take into account several things you might need to think about for real production code, but I think it gives a taste of what’s possible just with what comes with the standard JDK.

In a later post we’ll lean on java.util.concurrent a bit harder, and see what other things we can do.

Where to begin… where to begin… Firstly, Welcome. You’ve stumbled upon my newly rejuvenated blog, wherein I plan to rant and ramble on about a number of software-related themes, as I continue to plough through the wild and wonderful world of software development.

Stay tuned… I’m sure I’ll have something significant and interesting to say soon