Mistake Patterns

A professor at the computer science colloquium today was talking about her research project. She is trying to build a kind of software that identifies and categorizes knowledge points that a student does not understand, by looking at the mistake pattern they display on their problem sets.

Perhaps it was the way she presented it, but I felt deeply offended by the idea. I find people who would happily rely on this kind of software quite arrogant in their approach to education. The problem sets themselves, for one, would have to be really well-designed, so as to clearly delineate lapses in the student’s understanding. But however well-designed they are, one could not avoid mis-categorizing, or over-generalizing—a student chooses A instead of C, so obviously they are confused about knowledge X. This might not be the case, and the student could very possibly understand X perfectly, but does not understand X prime. To think that an imperfect set of statistics, filtered through a machine, could characterize the infinitely complex and irrational human cognitive faculties—I find this assumption difficult to agree with. This is no better than reducing human beings to data points, and make sweeping judgements on amortized behavior.

It is horrible that educators should think that this is the right approach. Educators should understand students individually, in all their peculiarities and divergences from the human average. This is what makes education interesting, and a learning experience. Otherwise, we would only fall into the predefined mistake patterns, not because we cannot avoid repeating history, nor because we have cracked the code to human cognition, but because according to statistical analysis there are no other possible mistake patterns for us to fall into. Humans succumb to machines not by making machines too powerful, but by becoming mindless machines themselves.

February 2, 2018