The Green Bar

A new role at Google

2012-01-17T08:22:00.000-08:00

For the last four years, I've been working at Google, supporting the Google Books project as a Software Engineer in Test. There's at least a blog post and a half to be written about how that role fits within Google's larger development practices, and the opportunities it provided to see what automated testing could do with Google-scale hardware and software behind it. I enjoyed it, and loved the team; I've spent more time with the team of SET's behind Google Books than with any previous team in my career.

This project also served as a capstone of 10 years in which my primary professional focus has been building software that helps software builders build software: frameworks, code analysis, testing.

Starting today, I'm excited to be shifting my focus back toward end users, as I join the team behind the Books app for Android, working on feature development. This also marks the first time in 3 years that I go back to being a full-time user of JUnit, which I'm quite happy about.

2012 should be exciting. If there's something you've always wanted an eBook to do for you, let me know!

No new JUnit release announcements, period, here.

2011-09-29T12:55:00.000-07:00

If by chance you've been following this blog in order to get announcements of new JUnit releases, whether beta or final, this is the last time they are likely to be posted here. You can follow along at junit.org, or the junit@yahoogroups.com mailing list.I'll be keeping this blog for whatever comes to mind that is more exciting that JUnit releases, which are becoming more predictable and mundane.By the way, JUnit 4.10 is released.Share and Enjoy!

No new beta-release announcements here

2011-09-22T11:17:00.000-07:00

Hi, all.In order to reduce the noise until there's time to raise the signal here, I'm going to start only announcing JUnit beta releases on the main user's mailing list:http://tech.groups.yahoo.com/group/junit/If you are a devoted beta-tester, and that's a problem, let me know. Thanks.

JUnit 4.9 released

2011-08-23T08:39:00.000-07:00

Over a year in the making, we are happy to announce the release of JUnit 4.9.

This release's theme is Test-class and suite level Rules. The release notes are copied in part below, and can be read in full here.

Go ahead and download, or merely let maven do the work for you (as we are now directly releasing to sonatype's maven repository on the same day we release the jars).

Following this release, work on JUnit and related code will continue on three fronts:

Bug fixes against existing functionality will be committed to the 4.9.1 branch.
New core features will be committed on the 4.10 branch.
We have created a new junit.contrib project to be a home to new experimental features, features that use additional dependencies, and features that are outside of JUnit's core mission. As a first example, Thomas Mueller has created assertthrows, a package that uses all kinds of JDK trickery to allow direct assertions like this:
```
  List<String> emptyList = new ArrayList<String>();

  assertThrows(emptyList).get(0);
```

We are excited that github allows extensive community participation and discussion in the ongoing development. Please pitch in! Remember, JUnit celebrates programmers testing their own software. Let's build something.

JUnit 4.9 release notes (highlights)

ClassRule

The ClassRule annotation extends the idea of method-level Rules, adding static fields that can affect the operation of a whole class. Any subclass of ParentRunner, including the standard BlockJUnit4ClassRunner and Suite classes, will support ClassRules.

For example, here is a test suite that connects to a server once before all the test classes run, and disconnects after they are finished:

@RunWith(Suite.class)

@SuiteClasses({A.class, B.class, C.class})

public class UsesExternalResource {

    public static Server myServer= new Server();



    @ClassRule

    public static ExternalResource resource= new ExternalResource() {

        @Override

        protected void before() throws Throwable {

            myServer.connect();

        };



        @Override

        protected void after() {

            myServer.disconnect();

        };

    };

}

TestRule

In JUnit 4.9, fields that can be annotated with either @Rule or @ClassRule should be of type TestRule. The old MethodRule type, which only made sense for method-level rules, will still work, but is deprecated.

Most built-in Rules have been moved to the new type already, in a way that should be transparent to most users. TestWatchman has been deprecated, and replaced by TestWatcher, which has the same functionality, but implements the new type.

Maven support

Maven bundles have, in the past, been uploaded by kind volunteers. Starting with this release, the JUnit team is attempting to perform this task ourselves.

LICENSE checked in

The Common Public License that JUnit is released under is now included in the source repository.

JUnit 4.9b4 (beta-)released

2011-08-12T08:29:00.000-07:00

This release's theme is Test-class and suite level Rules. This beta release contains a number of community-authored fixes for regression errors in documentation and test class validation. Please read the release notes, download, and give feedback before the final release.

[Edit: fixed release notes link]

JUnit 4.9b3 (beta-)released

2011-07-06T14:04:00.000-07:00

Five months and a dozen regression errors later, JUnit 4.9b3 is now ready for your testing pleasure. This release's theme is Test-class and suite level Rules. Please read the release notes, download or slurp from Maven, and give feedback before the final release.

New project: junit.contrib

2011-03-22T11:24:00.000-07:00

I'm planning to start a junit.contrib project.
In this post, I'll first review the reasons for such a project, and
secondly, I have a proposal for its structure, and I'd like feedback.

Background: reasons for junit.contrib

JUnit has always officially strived to be "the intersection of all
useful Java testing frameworks". Thus, the goals for code that gets
into the junit repository include that it should:

Minimize dependencies on external libraries or particular JDK
versions beyond a widely-used minimum.
Be very unlikely to change API moving forward.
Emphasize opportunities for extension over the richness of core features.
Have potential usefulness to all Java developers, regardless of
their application domain.
Match in style to the current JUnit codebase.

Many people have made JUnit extensions that would be useful to ahealthy portion of Java developers, but do not meet all of the abovecriteria. For example, the extension may target a particular IDE, ordata that is stored in a SQL database, or in XML, or it may beexperimental, and likely to change API, or it may require an externaldependency. Also, as the popularity of non-Java languages on the JVMhas grown, some people have made extensions to JUnit that make iteasier to use JUnit's core functionality in ways idiomatic to thoseother languages.It has always been an option for developers of these extensions topublish them on their own repositories. However, having a centralclearing-house as an option for extension developers has someadvantages, including discoverability, documentation, and dependencymanagement.The goal is for junit.contrib (just started here) to be this clearing house.All Java classes in this project should be in packages prefixed withorg.junit.contrib.Now, the

Proposal

The root structure of the project will contain:

/
 README
 docs/
 java/
 scala/ [?]
 clojure/ [?]

Thus, a folder for overall documentation, and then a folderper-language. I haven't thought about how non-java languageextensions should be organized.Under java, there should be a folder per "subproject" (names chosen,with apologies, by my favorite naming tool):

java/
 brewmasters/
 callisto/
 hekler/
 norway/
 ...

Each subproject should be organized as its own maven project:

java/brewmasters/
 pom.xml
 src/main/java/org/junit/contrib/brewmasters/...
 src/test/java/org/junit/contrib/brewmasters/...

As potential users and contributors to such a repository, are there
pitfalls in this organization that I should be aware of in advance?

Very very simple testing in bash

2011-01-27T12:29:00.001-08:00

Imagine I'm starting today on a software system that I expect to spend
at least a year on. Being me, somewhere in the top five questions I
ask myself is "How am I going to test this?" Frequent automated testing
gives me confidence that I won't find out next week that today I broke
something that was working last week. Writing tests is also an important way
to organize my thinking about what I'm doing. Once I've spent time specifying
where I'm going, it's easier to get there quickly.

Since I'm going to be spending a year on this, it's worth my time to invest time
in tools and frameworks that will make specifying and running these tests easy.
I might even write some of my own.

However, suppose that I'm working on a quick demo that I'm only going to spend
a couple weeks on. Or a data-processing script that I need to have finished
tomorrow. Or a sysadmin task that I expect will only take half an hour?

In these cases, I still want the benefits of testing, but I want to spend
as little time as possible getting started. Also, it's not always clear
what technologies I'll be using: I may start in Java, and then find that
a Python library will give me 80% of what I need.
So, I've started to write start projects with test suites that consist of
nothing more than a bash script with a standard footer I copy-and-paste
from project to project. The advantages:

Fast startup: I don't need to download anything or
run through any "new project" wizards.
If my project lasts longer than I expected, it's easy to remember how
to run the tests: just run the script.
I can easily refactor to use a more heavyweight framework like JUnit by
calling out to it from the script.
It isn't tied to any underlying implementation choice.
Perhaps best of all, it rewards a UNIX style of small stdin-to-stdout
plug-and-play components.

With no further ado, here's an example script, this one drawn from
the JUnit build process:

JUnit 4.9b2 (beta-)released

2011-01-21T16:51:00.000-08:00

This release's theme is Test-class and suite level Rules. Please read the release notes, download, and give feedback before the final release.

Running a JUnit 4 test from a Scala script

2011-01-04T12:08:00.000-08:00

I'm using Scala now and then as a language for side projects. One of the nice things about Scala is that you can bootstrap with a single-file script, similar to Python or Ruby. Since I enjoy building things with test-driven development, it's not long after I write a script that I want to start writing tests for classes and functions. However, when I tried to do this in the straight-forward way, it doesn't quite work. Herein, the problem, and its solution.

As a first attempt, I expected this to work:

However, running this produces a failure in the infrastructure:

What's going on here is that when scala is run in single-file script mode, it implicitly wraps all of the declarations within the declaration of an anonymous singleton object (Main$$anon). When JUnit tries to reflectively create an ArithmeticTest object, it runs into the fact that, from the Java perspective, ArithmeticTest is a non-static inner class of Main$$anon. To get around this requires a bit of a dance, luckily made fairly short due to Scala's compact OO verbiage:

If I find myself doing this often, I imagine I'll be pulling this out into an importable library. Share and Enjoy.

JUnit 4.8.2

2010-04-08T10:29:00.001-07:00

This is a small bugfix release. Download now.

Dotmesh: get free rich assertions from your existing methods

2010-03-15T16:06:00.000-07:00

It's been over two years since JUnit 4.4 was released with support for the assertThat method initially conceived by Joe Walnes. A classic statement using assertThat is:

assertThat was based on the Matcher interface from hamcrest, and led to more readable code, more readable error messages, and custom, combinable assertions. However, after two years of use, I’ve seen several situations in which the Matcher-based assertThat API can fail developers:

Custom matchers can be too much code for simple ideas. Ideally, custom assertions should just flow off the fingers, but I find it very telling that the JUnit self tests only have four custom matchers, all in the ResultMatchers class. Most of them are about 7 lines of mostly-boilerplate code, such as this:
Matchers can be difficult to discover. They are generally returned by static factory methods--if your IDE isn’t set up to auto-complete these methods, it can be difficult to track down whether a matcher method exists for the assertion you want, or which of several available has the right meaning. Ideally, we could restrict the types of Matchers that could type-check for a given value, to give your IDE hints. Unfortunately...
I have discovered a long, painful proof that it is impossible to provide strong types for matcher-based assertThat using:
- Java generics...
- as implemented in all of the compilers we currently support...
- without breaking the use of Matchers in jMock
This blog post is too narrow to contain it.
Similar to the above, introducing a dependency from JUnit on hamcrest, the matcher library, causes problems for any users using a different version of hamcrest for another use (for example, jMock)

Based on these difficulties, I propose a new API for assertions, which I call dotmesh*. The idea is to use imposterization to generate assertions based on every boolean method available on every class. The new form of the assertion above would be:

If this fails, the error message is:

Failed: <ERROR: blah blah blah>.contains(<OK>)

Here’s some more examples of dotmesh in action:

Advantages of the dotmesh approach over the Matcher approach:

No additional code: assertions and error messages are automatically generated from any method that returns boolean.
Super-easy discovery: know a boolean method? You know a dotmesh assertion.
Strong typing for free: since we’re using the same methods and types, assertions only compile if they make sense.
As a bonus, dotmesh provides a reward for interfaces that follow the “Suggestion of Demeter”, and provide accessors returning booleans or collections for essential information.

An ambivalent point:

Both dotmesh and hamcrest must use specially-named methods to “talk about” operations that would otherwise be more clearly stated with operators. For example, you can easily:
assertTrue(x > 5)
but in order to find out why this assertion fails, you must use:
assertThat(x, greaterThan(5)) in hamcrest, or
assertThat(x).greaterThan(5) in dotmesh.

Disadvantages of dotmesh:

The standard imposterization tricks break down when the target being imposterized is a primitive type or final class. This isn’t too big a problem for primitive types and Strings: we can introduce a wrapper interface for each, and then be done. The more disconcerting issue is third-party final classes. We can’t generate new synthetic classes that replicate their interface, so this is going to fail at some point:

My approach for the moment is to be very apologetic in the error message. Once I see where this problem tends to come up for users, one of several possible mitigation tricks may prove possible.
The hamcrest assertThat allowed (to a certain extent) for combinations of matchers:

dotmesh can’t match that directly, but there’s often useful workarounds:
In order to pull off imposterization, I’m using objenesis and cglib, which might conceivably lead to the same versioning issues currently imposed by hamcrest. However, this dependency is completely encapsulated from the user, so we could use a solution like jarjar to avoid any conflicts.

If I’ve piqued your interest enough to try some truly bleeding-edge, no-guarantees code, check out:

http://github.com/dsaff/dotmesh

* Why "dotmesh"? Well, it creates assertions by "meshing" a new meaning into the "dot" that indicates a method invocation. There's another reason, which is a cryptic homage to hamcrest,..

JUnit 4.8.1 is released

2009-12-08T07:31:00.000-08:00

JUnit 4.8's category support had a glaring error in the javadoc (CategoryType was a quickly-deleted design decision that lived on in the documentation), and in the implementation (Class-level annotations were not trickled down to methods). 4.8.1 fixes this:

Release notes

Javadoc

Download

JUnit 4.8 is released

2009-12-01T12:57:00.000-08:00

JUnit 4.8 has been released. The primary new feature is preliminary support for Categories. For more information, see the release notes and javadoc, and then download!

Categories in JUnit 4.8

2009-11-14T05:06:00.000-08:00

This is from the tests for the upcoming JUnit 4.8 release. It's the simplest-to-implement framework we could find for labelling tests with named categories, and running only tests with a given label.


 public interface SlowTests extends CategoryType {

 }

 public static class A {
  @Test
  public void a() {
   fail();
  }

  @Category(SlowTests.class)
  @Test
  public void b() {
  }
 }

 @Category(SlowTests.class)
 public static class B {
  @Test
  public void c() {

  }
 }

 public static class C {
  @Test
  public void d() {
   fail();
  }
 }

 @RunWith(Categories.class)
 @IncludeCategory(SlowTests.class)
 @SuiteClasses( { A.class, B.class, C.class })
 public static class SlowTestSuite {
 }