For the past couple of years, I’ve been using an automatic code evaluation system that I made for my AP class. The students write code according to a spec, and then I give them a precompiled class file that relies on that spec. It takes a bit of shuffling to get things in the right place, but JCreator will run the class file and I don’t have to give them the source. My goal was to cut down on my own workload, but I’ve also found that it speeds up the process by which students uncover their own errors and eventually learn to anticipate them well enough to write their own test cases.
Here’s an example of the output for a recent lab:
CLASSROOM LAB AUTO-GRADER 0 Checking for required classes... 1/1 1 Checking Classroom methods... 4/4 2 Testing add, remove, and size... 1/1 3 Testing createStudents... 2/2 4 Testing class average GPA methods... 2/2 TOTAL = 10/10
The grader uses the Java Reflection API to check for required classes, methods (including return type and parameter lists), and instance variables. It also captures the system output and input to allow me to simulate interaction with their code. This functionality is all in a base class. Then I write the specific test cases for a particular assignment in a subclass, as a series of scored tasks. Students get feedback via the scores, telling them which tasks succeed and which don’t. There’s also a facility for giving a hint when a task doesn’t get the maximum possible score.
I’ve hesitated to share the code, knowing that the style is weird (and not very Java-ish), that there are probably better automated code assessment tools out there, and that anyone else who thought of this would probably want to write it themselves. On the other hand, maybe this will inspire someone or be a starting point for something else. I’d love it if anyone has input on how to improve or generalize this.
You can grab the code, including an example, from github.com/benchun/autograder. As the kids say these days, please fork my repo — I’d be happy to discuss ideas and get your pull requests!