Hard Questions

When I started teaching geometry six years ago, I remember being frustrated that there didn’t seem to be a good textbook. They were all either too formal or too simple, with either too many words or too many color photos. I thought: The subject hasn’t changed for 3000 years. Why haven’t we gotten this right by now?

One of the most frustrating things about education is that, despite all the resources, time, and energy that have been put into this challenge, we haven’t been able to fix it. In some ways, schools set us up to misunderstand the problem. The questions we ask students in our math classes have a right answer. The essays we assign in English classes have a given structure. At the end of an exam, there’s a grade. So it feels like there should be a solution to discover, some system by which we teachers can get all our students to learn everything.

But technology, demographics, and culture are changing constantly. We wouldn’t expect a business to be able to find a formula for profitability and then operate without regard to changes in the market. So why think schools can do it? We can all benefit from good curriculum design, but good is what works — by definition, a moving target.

A wise math teacher, Henri Piccoto, quotes Tofu Roshi: “There is no one way.” I’m reviewing this line of thought because I just finished grading the final exams for my AP Computer Science class. Last year, I had strong indications that students who did more independent programming practice did better on my exam and subsequently on the AP test.

This year, that same correlation really isn’t there. Many things are different, including my sharing the previous data with this year’s students and some changes to the exam.  Most significantly, these are just different humans, each coming in with their own mental models and skill sets.

Last year, the correlation with CodingBat completion was r = 0.71. This year, r = 0.18. So maybe doing some amount of those exercises matters for some students. Maybe not. Last year, when I looked at the correlation with CST Math scores, I found r = 0.49. This year, r = 0.64.

I think looking at this kind of data is important, and I like exploring it. I’m also pretty sure that answers to teaching problems won’t pop out of these numbers. The value for me is in reflecting on the learning process, thinking about who has been successful and who hasn’t, and looking for ways to help more people find their way on to the path. Looking at the past three years I see some signs of improvement, and also a lot more room to improve.

I’ve managed to shift the mean up a bit while keeping the standard deviation in check. The bipartite distribution is better this year, but it’s still there.

Some improvement might come via long-term initiatives, like my new plan to use BYOB with the 10th grade class this coming semester. In another two years, we’ll see how that plays out in AP performance. On a slightly shorter time scale, perhaps we’ll see some improvement via new content delivery ideas, like the lecture notes I started publishing this year or student-produced videos explaining key concepts (a project idea for when we get back from the break).

But there are also lots of opportunities for improvement in the smaller picture. Like, how about this exam I’ve written? I decided this year to swap out a question about Car objects for one about TeaCup objects:

Given a class TeaCup with a method getFlavor() that will print out the type of tea currently in the cup, and given an array of TeaCup objects called tray, write both the regular and enhanced for loop that will visit every TeaCup object in the array tray and call its getFlavor() method.  Your answer should have two loops.

Of the 30 students in the class, a couple were quite lost and had no idea what to do. That’s not great. The majority knew what I was asking and while they may have made some syntax errors, at least understood the question. But then there were 6 who thought I was asking for them to write the TeaCup class or to write the getFlavor() method. That’s one in five students — too many to be confused in this way.

There doesn’t seem to be any demographic determiner of this confusion. I also counted 9 students who thought their answer should include System.out.println. What’s up with that?

My goal for this question was to test their knowledge of both the regular and enhanced for loop syntax. I think I know how to rewrite the question to better check that knowledge. At the same time, I feel like the question, as written, was reasonably clear and ought to be adequate. It wasn’t perfect. It’s a very artificial test-question sort of question. But in a world of tests, in a class supposedly preparing students for a specific test, is that wrong?

I can dodge that issue for now by telling myself that this isn’t how AP questions are structured, and so I should rewrite it. The larger issue of connecting with what my students think and how they learn remains an open question: Why haven’t we gotten this right by now? I truly believe that everyone can understand these concepts. What systems can I build in my classroom to make sure that happens? I truly believe everyone can learn to decipher test questions and show their knowledge in that artificial context. What skills do I need to teach in order to help solve that problem?