New Insights from AP Results
In 2009, I reached the conclusion that there was a strong correlation between the amount of optional work that students did in CodingBat and their exam scores. Last year, I started to suspect that this was not the case.
Now the most recent AP exam scores from my students confirm that negative result, despite controlling for some students who cheated on the CodingBat work.
What I do know is that my fall semester final is very strongly correlated (2010 r=0.93, 2011 r=0.85) with scores on the AP exam itself. One possible conclusion is that the first semester of this course matters quite a bit: students who gain mastery early keep it, while students who fall behind stay behind.
I had my complaints about the exam questions this year, but at the end of the day the test is the test and it doesn’t seem to give such different results than my own style of assessment. If you haven’t seen Chief Reader Jody Paul’s notes from the 2011 reading, there are some interesting charts in there, particularly an upward trend in the number of students taking this exam.
For 2012, my job is to keep the train on the tracks. Things are going okay. That’s an improvement from where I was a year ago. After seeing my 2010 numbers, my assistant principal wrote to me:
Your AP scores surprised me, sort of. I’m not surprised that they were so split — that was something you recognized throughout the year and I think lost some sleep over, or maybe just decided not to lose sleep over. Either way, you’re in an interesting position now. You know that you can teach some kids the content well enough that they’ll get a five. Now you have the challenge of figuring out what it will take to get the rest of the class at least to a passing level.
And I said, “yep. that involves breaking down the concepts into smaller chunks, offering more practice, more scaffolding. i’m working on that now. it’s really hard.” I built new lecture notes, designed new labs, and spent a lot of time sitting next to students and reading their code.
So in that context, I’m really proud of the numbers from 2011. I moved the pass rate (3 or above) from 50% up to 60% and the average score from 2.61 up to 2.97 with a much larger class (30, up from 18).
I also cut the rate of 1’s from 50% down to 17%. There was a cost in terms of 5’s but I think in some ways the number of 5’s in a given year is predetermined as long as I at least cover the material. It’s the number of 4’s and 1’s that I work to change.
This still trails the national average (and some of my cross-town competition), but we’re in the game now. I just have to figure out what to do about CodingBat.