Knowledge Inference or Latent Knowledge Estimation

Ryan Baker, in week four of the Big Data in Education Course, provides various examples of methodologies all focused on "measuring what a student knows at a specific time," or knowledge inference.  This is "often operationalized as measuring what relevant knowledge components a student knows at a specific time."  He then describes what a knowledge component is.  It is, "anything a student can know, that's meaningful to the current learning situation, which might include skills, knowledge of facts, knowledge of concepts, knowledge of principles, knowledge of schemas."  In short, he says, "anything a student can know, can be a knowledge component."

Baker suggests at least three reasons why it is useful to measure what a student knows:

• Because education's primary goal is enhancing student knowledge, by measuring it, "you know whether you're making it better."
• In addition, if you measure what a student knows, you can inform instructors and other stakeholders about it
• Finally, "if you can measure it, you can make automated pedagogical decisions."

He adds the caution measuring  what a student knows is different than measuring performance.  Specifically he notes, "inferring if a student's performance right now is associated with successfully demonstrating a skill, is not the same as knowing whether the student has this latent skill."  A student could guess and perform well.  Or a student could slip up when they actual know the skill.  Baker suggests one difference is not just looking at performance "at one specific moment," but looking at performance over time to observe patterns in performance.

Observing patterns in performance is at the center of the research I am conducting in the context of online spreadsheet learning.  An individual's performance over time does show signs of misconceptions of knowledge.  By identifying the underlying misconceptions, in terms of knowledge components, intervention efforts can be focused to specific knowledge components instead of the traditional response - please try again.

One methodology that seems especially relevant to this research is Learning Factors Analysis.  I will write a followup post about this particular methodology.