"The highest accuracy model that predicts the test scores of users based on just seven mock questions test scores of users based on just six mock questions"
The score prediction for a particular test requires the user’s actual test score data (label), which can be collected based on the user’s active participation. However, the amount of data is often scarce given a lack of active user participation.
Riiid proposes a deep learning Transformer-based assessment model to overcome this limitation and improve predictive accuracy with sparse data.
A model is developed that pre-trains a user’s probability in making correct/incorrect answers and in-time problem solving. The model is then fine-tuned to match score predictions based on small amounts of score data.
While all existing score prediction algorithms perform score prediction after preliminary learning of problem content, Riiid suggests the first model that learns from a user’s problem solving data.
Riiid achieves a 28% higher predicted accuracy than 'QuesNet,’ which has the highest predictive performance among pre-learned models.