Frog wars

From the woefully incomplete Wikipedia article, "It is generally the case that the second railway to arrive [at a junction] has to bear the cost of the special trackwork needed to cross the first. This includes the cost of any interlocking tower or signal box."

Thus, there was tremendous incentive to be the first railroad to reach a prime location, incurring extra costs on any latecomers.

This was called a "frog war", named after the "frog", the specialized rail piece that goes in the center of a junction and looks like a jumping frog.

Frog wars were dangerous.  In one case, after which is named the Grand Crossing area of Chicago, the second line was built secretly, to avoid the cost of the proper precautions, leading to a crash that killed 18.

I recommend "frog war" as also pertaining in software, where the second person to check in a change in a given location incurs the cost of merging with the first change.

I've lost a couple frog wars recently, but I'm hoping to win this next one...

A new role at Google

For the last four years, I've been working at Google, supporting the Google Books project as a Software Engineer in Test.  There's at least a blog post and a half to be written about how that role fits within Google's larger development practices, and the opportunities it provided to see what automated testing could do with Google-scale hardware and software behind it.  I enjoyed it, and loved the team; I've spent more time with the team of SET's behind Google Books than with any previous team in my career.

This project also served as a capstone of 10 years in which my primary professional focus has been building software that helps software builders build software: frameworks, code analysis, testing.

Starting today, I'm excited to be shifting my focus back toward end users, as I join the team behind the Books app for Android, working on feature development.  This also marks the first time in 3 years that I go back to being a full-time user of JUnit, which I'm quite happy about.

2012 should be exciting.  If there's something you've always wanted an eBook to do for you, let me know!

No new JUnit release announcements, period, here.

If by chance you've been following this blog in order to get announcements of new JUnit releases, whether beta or final, this is the last time they are likely to be posted here. You can follow along at junit.org, or the junit@yahoogroups.com mailing list. I'll be keeping this blog for whatever comes to mind that is more exciting that JUnit releases, which are becoming more predictable and mundane. By the way, JUnit 4.10 is released. Share and Enjoy!

No new beta-release announcements here

Hi, all. In order to reduce the noise until there's time to raise the signal here, I'm going to start only announcing JUnit beta releases on the main user's mailing list: http://tech.groups.yahoo.com/group/junit/ If you are a devoted beta-tester, and that's a problem, let me know. Thanks.

JUnit 4.9 released

Over a year in the making, we are happy to announce the release of JUnit 4.9.

This release's theme is Test-class and suite level Rules. The release notes are copied in part below, and can be read in full here.

Go ahead and download, or merely let maven do the work for you (as we are now directly releasing to sonatype's maven repository on the same day we release the jars).

Following this release, work on JUnit and related code will continue on three fronts:

• Bug fixes against existing functionality will be committed to the 4.9.1 branch.
  
• New core features will be committed on the 4.10 branch.
  
• We have created a new junit.contrib project to be a home to new experimental features, features that use additional dependencies, and features that are outside of JUnit's core mission. As a first example, Thomas Mueller has created assertthrows, a package that uses all kinds of JDK trickery to allow direct assertions like this:
  

  
    List<String> emptyList = new ArrayList<String>();
    assertThrows(emptyList).get(0);
  

We are excited that github allows extensive community participation and discussion in the ongoing development. Please pitch in! Remember, JUnit celebrates programmers testing their own software. Let's build something.

JUnit 4.9 release notes (highlights)

ClassRule

The ClassRule annotation extends the idea of method-level Rules, adding static fields that can affect the operation of a whole class. Any subclass of ParentRunner, including the standard BlockJUnit4ClassRunner and Suite classes, will support ClassRules.

For example, here is a test suite that connects to a server once before all the test classes run, and disconnects after they are finished:

@RunWith(Suite.class)
@SuiteClasses({A.class, B.class, C.class})
public class UsesExternalResource {
    public static Server myServer= new Server();

    @ClassRule
    public static ExternalResource resource= new ExternalResource() {
        @Override
        protected void before() throws Throwable {
            myServer.connect();
        };

        @Override
        protected void after() {
            myServer.disconnect();
        };
    };
}

TestRule

In JUnit 4.9, fields that can be annotated with either @Rule or @ClassRule should be of type TestRule. The old MethodRule type, which only made sense for method-level rules, will still work, but is deprecated.

Most built-in Rules have been moved to the new type already, in a way that should be transparent to most users. TestWatchman has been deprecated, and replaced by TestWatcher, which has the same functionality, but implements the new type.

Maven support

Maven bundles have, in the past, been uploaded by kind volunteers. Starting with this release, the JUnit team is attempting to perform this task ourselves.

LICENSE checked in

The Common Public License that JUnit is released under is now included in the source repository.

JUnit 4.9b4 (beta-)released

This release's theme is Test-class and suite level Rules. This beta release contains a number of community-authored fixes for regression errors in documentation and test class validation. Please read the release notes, download, and give feedback before the final release.

[Edit: fixed release notes link]

JUnit 4.9b3 (beta-)released

Five months and a dozen regression errors later, JUnit 4.9b3 is now ready for your testing pleasure. This release's theme is Test-class and suite level Rules. Please read the release notes, download or slurp from Maven, and give feedback before the final release.

New project: junit.contrib

I'm planning to start a junit.contrib project.
In this post, I'll first review the reasons for such a project, and
secondly, I have a proposal for its structure, and I'd like feedback.

Background: reasons for junit.contrib

JUnit has always officially strived to be "the intersection of all
useful Java testing frameworks". Thus, the goals for code that gets
into the junit repository include that it should:

1. Minimize dependencies on external libraries or particular JDK
   versions beyond a widely-used minimum.
   
2. Be very unlikely to change API moving forward.
   
3. Emphasize opportunities for extension over the richness of core features.
   
4. Have potential usefulness to all Java developers, regardless of
   their application domain.
   
5. Match in style to the current JUnit codebase.
   

Many people have made JUnit extensions that would be useful to a healthy portion of Java developers, but do not meet all of the above criteria. For example, the extension may target a particular IDE, or data that is stored in a SQL database, or in XML, or it may be experimental, and likely to change API, or it may require an external dependency. Also, as the popularity of non-Java languages on the JVM has grown, some people have made extensions to JUnit that make it easier to use JUnit's core functionality in ways idiomatic to those other languages. It has always been an option for developers of these extensions to publish them on their own repositories. However, having a central clearing-house as an option for extension developers has some advantages, including discoverability, documentation, and dependency management. The goal is for junit.contrib (just started here) to be this clearing house. All Java classes in this project should be in packages prefixed with org.junit.contrib. Now, the

Proposal

The root structure of the project will contain:

/
 README
 docs/
 java/
 scala/ [?]
 clojure/ [?]

Thus, a folder for overall documentation, and then a folder per-language. I haven't thought about how non-java language extensions should be organized. Under java, there should be a folder per "subproject" (names chosen, with apologies, by my favorite naming tool):

java/
 brewmasters/
 callisto/
 hekler/
 norway/
 ...

Each subproject should be organized as its own maven project:

java/brewmasters/
 pom.xml
 src/main/java/org/junit/contrib/brewmasters/...
 src/test/java/org/junit/contrib/brewmasters/...

As potential users and contributors to such a repository, are there
pitfalls in this organization that I should be aware of in advance?

Very very simple testing in bash

Imagine I'm starting today on a software system that I expect to spend
at least a year on. Being me, somewhere in the top five questions I
ask myself is "How am I going to test this?" Frequent automated testing
gives me confidence that I won't find out next week that today I broke
something that was working last week. Writing tests is also an important way
to organize my thinking about what I'm doing. Once I've spent time specifying
where I'm going, it's easier to get there quickly.

Since I'm going to be spending a year on this, it's worth my time to invest time
in tools and frameworks that will make specifying and running these tests easy.
I might even write some of my own.

However, suppose that I'm working on a quick demo that I'm only going to spend
a couple weeks on. Or a data-processing script that I need to have finished
tomorrow. Or a sysadmin task that I expect will only take half an hour?

In these cases, I still want the benefits of testing, but I want to spend
as little time as possible getting started. Also, it's not always clear
what technologies I'll be using: I may start in Java, and then find that
a Python library will give me 80% of what I need.
So, I've started to write start projects with test suites that consist of
nothing more than a bash script with a standard footer I copy-and-paste
from project to project. The advantages:

• Fast startup: I don't need to download anything or
  run through any "new project" wizards.
• If my project lasts longer than I expected, it's easy to remember how
  to run the tests: just run the script.
• I can easily refactor to use a more heavyweight framework like JUnit by
  calling out to it from the script.
• It isn't tied to any underlying implementation choice.
• Perhaps best of all, it rewards a UNIX style of small stdin-to-stdout
  plug-and-play components.

With no further ado, here's an example script, this one drawn from
the JUnit build process:

JUnit 4.9b2 (beta-)released

This release's theme is Test-class and suite level Rules. Please read the release notes, download, and give feedback before the final release.

Running a JUnit 4 test from a Scala script

I'm using Scala now and then as a language for side projects. One of the nice things about Scala is that you can bootstrap with a single-file script, similar to Python or Ruby. Since I enjoy building things with test-driven development, it's not long after I write a script that I want to start writing tests for classes and functions. However, when I tried to do this in the straight-forward way, it doesn't quite work. Herein, the problem, and its solution.

As a first attempt, I expected this to work:



However, running this produces a failure in the infrastructure:



What's going on here is that when scala is run in single-file script mode, it implicitly wraps all of the declarations within the declaration of an anonymous singleton object (Main$$anon). When JUnit tries to reflectively create an ArithmeticTest object, it runs into the fact that, from the Java perspective, ArithmeticTest is a non-static inner class of Main$$anon. To get around this requires a bit of a dance, luckily made fairly short due to Scala's compact OO verbiage:



If I find myself doing this often, I imagine I'll be pulling this out into an importable library. Share and Enjoy.

JUnit 4.8.2

This is a small bugfix release. Download now.

Dotmesh: get free rich assertions from your existing methods

It's been over two years since JUnit 4.4 was released with support for the assertThat method initially conceived by Joe Walnes. A classic statement using assertThat is:



assertThat was based on the Matcher interface from hamcrest, and led to more readable code, more readable error messages, and custom, combinable assertions. However, after two years of use, I’ve seen several situations in which the Matcher-based assertThat API can fail developers:

1. 
   Custom matchers can be too much code for simple ideas. Ideally, custom assertions should just flow off the fingers, but I find it very telling that the JUnit self tests only have four custom matchers, all in the ResultMatchers class. Most of them are about 7 lines of mostly-boilerplate code, such as this:
   
   
   
2. 
   Matchers can be difficult to discover. They are generally returned by static factory methods--if your IDE isn’t set up to auto-complete these methods, it can be difficult to track down whether a matcher method exists for the assertion you want, or which of several available has the right meaning. Ideally, we could restrict the types of Matchers that could type-check for a given value, to give your IDE hints. Unfortunately...
   
3. 
   I have discovered a long, painful proof that it is impossible to provide strong types for matcher-based assertThat using:
   
   • Java generics...
   • as implemented in all of the compilers we currently support...
   • without breaking the use of Matchers in jMock
   
   This blog post is too narrow to contain it.
   
4. Similar to the above, introducing a dependency from JUnit on hamcrest, the matcher library, causes problems for any users using a different version of hamcrest for another use (for example, jMock)
   

Based on these difficulties, I propose a new API for assertions, which I call dotmesh*. The idea is to use imposterization to generate assertions based on every boolean method available on every class. The new form of the assertion above would be:



If this fails, the error message is:

Failed: <ERROR: blah blah blah>.contains(<OK>)

Here’s some more examples of dotmesh in action:



Advantages of the dotmesh approach over the Matcher approach:

1. 
   No additional code: assertions and error messages are automatically generated from any method that returns boolean.
   
2. 
   Super-easy discovery: know a boolean method? You know a dotmesh assertion.
   
3. 
   Strong typing for free: since we’re using the same methods and types, assertions only compile if they make sense.
   
4. 
   As a bonus, dotmesh provides a reward for interfaces that follow the “Suggestion of Demeter”, and provide accessors returning booleans or collections for essential information.
   

An ambivalent point:

• 
  Both dotmesh and hamcrest must use specially-named methods to “talk about” operations that would otherwise be more clearly stated with operators. For example, you can easily:
  assertTrue(x > 5)
  but in order to find out why this assertion fails, you must use:
  assertThat(x, greaterThan(5)) in hamcrest, or
  assertThat(x).greaterThan(5) in dotmesh.
  

Disadvantages of dotmesh:

1. 
   The standard imposterization tricks break down when the target being imposterized is a primitive type or final class. This isn’t too big a problem for primitive types and Strings: we can introduce a wrapper interface for each, and then be done. The more disconcerting issue is third-party final classes. We can’t generate new synthetic classes that replicate their interface, so this is going to fail at some point:
   
   
   
   My approach for the moment is to be very apologetic in the error message. Once I see where this problem tends to come up for users, one of several possible mitigation tricks may prove possible.
   
2. 
   The hamcrest assertThat allowed (to a certain extent) for combinations of matchers:
   
   
   
   dotmesh can’t match that directly, but there’s often useful workarounds:
   
   
   
3. 
   In order to pull off imposterization, I’m using objenesis and cglib, which might conceivably lead to the same versioning issues currently imposed by hamcrest. However, this dependency is completely encapsulated from the user, so we could use a solution like jarjar to avoid any conflicts.
   

If I’ve piqued your interest enough to try some truly bleeding-edge, no-guarantees code, check out:

http://github.com/dsaff/dotmesh








* Why "dotmesh"? Well, it creates assertions by "meshing" a new meaning into the "dot" that indicates a method invocation. There's another reason, which is a cryptic homage to hamcrest,..

JUnit 4.8.1 is released

JUnit 4.8's category support had a glaring error in the javadoc (CategoryType was a quickly-deleted design decision that lived on in the documentation), and in the implementation (Class-level annotations were not trickled down to methods). 4.8.1 fixes this:


Release notes

Javadoc

Download

JUnit 4.8 is released

JUnit 4.8 has been released. The primary new feature is preliminary support for Categories. For more information, see the release notes and javadoc, and then download!

Categories in JUnit 4.8

This is from the tests for the upcoming JUnit 4.8 release.  It's the simplest-to-implement framework we could find for labelling tests with named categories, and running only tests with a given label.

 public interface SlowTests extends CategoryType {

 }

 public static class A {
  @Test
  public void a() {
   fail();
  }

  @Category(SlowTests.class)
  @Test
  public void b() {
  }
 }

 @Category(SlowTests.class)
 public static class B {
  @Test
  public void c() {

  }
 }

 public static class C {
  @Test
  public void d() {
   fail();
  }
 }

 @RunWith(Categories.class)
 @IncludeCategory(SlowTests.class)
 @SuiteClasses( { A.class, B.class, C.class })
 public static class SlowTestSuite {
 }