JUnit 4.9b3 (beta-)released

Five months and a dozen regression errors later, JUnit 4.9b3 is now ready for your testing pleasure. This release's theme is Test-class and suite level Rules. Please read the release notes, download or slurp from Maven, and give feedback before the final release.

New project: junit.contrib

I'm planning to start a junit.contrib project.
In this post, I'll first review the reasons for such a project, and
secondly, I have a proposal for its structure, and I'd like feedback.

Background: reasons for junit.contrib

JUnit has always officially strived to be "the intersection of all
useful Java testing frameworks". Thus, the goals for code that gets
into the junit repository include that it should:

1. Minimize dependencies on external libraries or particular JDK
   versions beyond a widely-used minimum.
   
2. Be very unlikely to change API moving forward.
   
3. Emphasize opportunities for extension over the richness of core features.
   
4. Have potential usefulness to all Java developers, regardless of
   their application domain.
   
5. Match in style to the current JUnit codebase.
   

Many people have made JUnit extensions that would be useful to a healthy portion of Java developers, but do not meet all of the above criteria. For example, the extension may target a particular IDE, or data that is stored in a SQL database, or in XML, or it may be experimental, and likely to change API, or it may require an external dependency. Also, as the popularity of non-Java languages on the JVM has grown, some people have made extensions to JUnit that make it easier to use JUnit's core functionality in ways idiomatic to those other languages. It has always been an option for developers of these extensions to publish them on their own repositories. However, having a central clearing-house as an option for extension developers has some advantages, including discoverability, documentation, and dependency management. The goal is for junit.contrib (just started here) to be this clearing house. All Java classes in this project should be in packages prefixed with org.junit.contrib. Now, the

Proposal

The root structure of the project will contain:

/
 README
 docs/
 java/
 scala/ [?]
 clojure/ [?]

Thus, a folder for overall documentation, and then a folder per-language. I haven't thought about how non-java language extensions should be organized. Under java, there should be a folder per "subproject" (names chosen, with apologies, by my favorite naming tool):

java/
 brewmasters/
 callisto/
 hekler/
 norway/
 ...

Each subproject should be organized as its own maven project:

java/brewmasters/
 pom.xml
 src/main/java/org/junit/contrib/brewmasters/...
 src/test/java/org/junit/contrib/brewmasters/...

As potential users and contributors to such a repository, are there
pitfalls in this organization that I should be aware of in advance?

Very very simple testing in bash

Imagine I'm starting today on a software system that I expect to spend
at least a year on. Being me, somewhere in the top five questions I
ask myself is "How am I going to test this?" Frequent automated testing
gives me confidence that I won't find out next week that today I broke
something that was working last week. Writing tests is also an important way
to organize my thinking about what I'm doing. Once I've spent time specifying
where I'm going, it's easier to get there quickly.

Since I'm going to be spending a year on this, it's worth my time to invest time
in tools and frameworks that will make specifying and running these tests easy.
I might even write some of my own.

However, suppose that I'm working on a quick demo that I'm only going to spend
a couple weeks on. Or a data-processing script that I need to have finished
tomorrow. Or a sysadmin task that I expect will only take half an hour?

In these cases, I still want the benefits of testing, but I want to spend
as little time as possible getting started. Also, it's not always clear
what technologies I'll be using: I may start in Java, and then find that
a Python library will give me 80% of what I need.
So, I've started to write start projects with test suites that consist of
nothing more than a bash script with a standard footer I copy-and-paste
from project to project. The advantages:

• Fast startup: I don't need to download anything or
  run through any "new project" wizards.
• If my project lasts longer than I expected, it's easy to remember how
  to run the tests: just run the script.
• I can easily refactor to use a more heavyweight framework like JUnit by
  calling out to it from the script.
• It isn't tied to any underlying implementation choice.
• Perhaps best of all, it rewards a UNIX style of small stdin-to-stdout
  plug-and-play components.

With no further ado, here's an example script, this one drawn from
the JUnit build process:

JUnit 4.9b2 (beta-)released

This release's theme is Test-class and suite level Rules. Please read the release notes, download, and give feedback before the final release.

Running a JUnit 4 test from a Scala script

I'm using Scala now and then as a language for side projects. One of the nice things about Scala is that you can bootstrap with a single-file script, similar to Python or Ruby. Since I enjoy building things with test-driven development, it's not long after I write a script that I want to start writing tests for classes and functions. However, when I tried to do this in the straight-forward way, it doesn't quite work. Herein, the problem, and its solution.

As a first attempt, I expected this to work:



However, running this produces a failure in the infrastructure:



What's going on here is that when scala is run in single-file script mode, it implicitly wraps all of the declarations within the declaration of an anonymous singleton object (Main$$anon). When JUnit tries to reflectively create an ArithmeticTest object, it runs into the fact that, from the Java perspective, ArithmeticTest is a non-static inner class of Main$$anon. To get around this requires a bit of a dance, luckily made fairly short due to Scala's compact OO verbiage:



If I find myself doing this often, I imagine I'll be pulling this out into an importable library. Share and Enjoy.

JUnit 4.8.2

This is a small bugfix release. Download now.

Dotmesh: get free rich assertions from your existing methods

It's been over two years since JUnit 4.4 was released with support for the assertThat method initially conceived by Joe Walnes. A classic statement using assertThat is:



assertThat was based on the Matcher interface from hamcrest, and led to more readable code, more readable error messages, and custom, combinable assertions. However, after two years of use, I’ve seen several situations in which the Matcher-based assertThat API can fail developers:

1. 
   Custom matchers can be too much code for simple ideas. Ideally, custom assertions should just flow off the fingers, but I find it very telling that the JUnit self tests only have four custom matchers, all in the ResultMatchers class. Most of them are about 7 lines of mostly-boilerplate code, such as this:
   
   
   
2. 
   Matchers can be difficult to discover. They are generally returned by static factory methods--if your IDE isn’t set up to auto-complete these methods, it can be difficult to track down whether a matcher method exists for the assertion you want, or which of several available has the right meaning. Ideally, we could restrict the types of Matchers that could type-check for a given value, to give your IDE hints. Unfortunately...
   
3. 
   I have discovered a long, painful proof that it is impossible to provide strong types for matcher-based assertThat using:
   
   • Java generics...
   • as implemented in all of the compilers we currently support...
   • without breaking the use of Matchers in jMock
   
   This blog post is too narrow to contain it.
   
4. Similar to the above, introducing a dependency from JUnit on hamcrest, the matcher library, causes problems for any users using a different version of hamcrest for another use (for example, jMock)
   

Based on these difficulties, I propose a new API for assertions, which I call dotmesh*. The idea is to use imposterization to generate assertions based on every boolean method available on every class. The new form of the assertion above would be:



If this fails, the error message is:

Failed: <ERROR: blah blah blah>.contains(<OK>)

Here’s some more examples of dotmesh in action:



Advantages of the dotmesh approach over the Matcher approach:

1. 
   No additional code: assertions and error messages are automatically generated from any method that returns boolean.
   
2. 
   Super-easy discovery: know a boolean method? You know a dotmesh assertion.
   
3. 
   Strong typing for free: since we’re using the same methods and types, assertions only compile if they make sense.
   
4. 
   As a bonus, dotmesh provides a reward for interfaces that follow the “Suggestion of Demeter”, and provide accessors returning booleans or collections for essential information.
   

An ambivalent point:

• 
  Both dotmesh and hamcrest must use specially-named methods to “talk about” operations that would otherwise be more clearly stated with operators. For example, you can easily:
  assertTrue(x > 5)
  but in order to find out why this assertion fails, you must use:
  assertThat(x, greaterThan(5)) in hamcrest, or
  assertThat(x).greaterThan(5) in dotmesh.
  

Disadvantages of dotmesh:

1. 
   The standard imposterization tricks break down when the target being imposterized is a primitive type or final class. This isn’t too big a problem for primitive types and Strings: we can introduce a wrapper interface for each, and then be done. The more disconcerting issue is third-party final classes. We can’t generate new synthetic classes that replicate their interface, so this is going to fail at some point:
   
   
   
   My approach for the moment is to be very apologetic in the error message. Once I see where this problem tends to come up for users, one of several possible mitigation tricks may prove possible.
   
2. 
   The hamcrest assertThat allowed (to a certain extent) for combinations of matchers:
   
   
   
   dotmesh can’t match that directly, but there’s often useful workarounds:
   
   
   
3. 
   In order to pull off imposterization, I’m using objenesis and cglib, which might conceivably lead to the same versioning issues currently imposed by hamcrest. However, this dependency is completely encapsulated from the user, so we could use a solution like jarjar to avoid any conflicts.
   

If I’ve piqued your interest enough to try some truly bleeding-edge, no-guarantees code, check out:

http://github.com/dsaff/dotmesh








* Why "dotmesh"? Well, it creates assertions by "meshing" a new meaning into the "dot" that indicates a method invocation. There's another reason, which is a cryptic homage to hamcrest,..