What autograding software do folks use for Java code?
For simple stuff I've used CodingBat. They've got an authoring system buried on the site. It's not terribly difficult to create your own questions, although the instructions are pretty wordy.

When you create a problem there are a set of fields to fill in. I'd attach a screenshot, but the network I'm on blocks imgur. I'll try to come back and add an image later.

doc 

This is the instructions that your students will read. 

hint 

If you want your students to have the hint button, add some text

cases 

List as many test cases as you want. Separate each parameter that you want to send into the method you're testing with a comma. The expected answer comes at the end. 

It looks like this for the method below. 123 goes into sumEvenDigits as n and then 2 is the expected answer. 

123,2
2006,8


code
Starter code for your students. 

public int sumEvenDigits(int n) {

}


tags
Let's you put questions into categories, specify language, and set a difficulty.