GMAT Scoring Algorithm Replicated with Python
For the plain-language explanation before the code, see how the GMAT’s adaptive algorithm works. Based on this research, I tried to recreate the GMAT’s adaptive algorithm in Python, and I managed to replicate the score trends.
The Core Mechanics of GMAT Adaptive Scoring
Simply put, the GMAT scoring mechanism is primarily based on these theories and steps:
- Item Response Theory (IRT): The GMAT uses a three-parameter logistic (3PL) model. Each question in the item bank has three assigned parameters:
- a (Discrimination): How well the question separates high-ability students from low-ability ones.
- b (Difficulty): The inherent difficulty of the question.
- c (Guessing Parameter): The chance a student guesses the correct answer.
- Together, these three parameters create a unique “Item Characteristic Curve” (ICC) for every question.
- Maximum Likelihood Estimation (MLE): After you answer each question, the system uses MLE to estimate your current ability level (Theta, θ) based on your response.
- Maximizing Information: Once your current ability level is estimated, the system scans the item bank to select the question that will provide the “maximum information” statistically to gauge your ability more precisely. The goal here is to pinpoint your ability level as accurately as possible.
- Score Conversion: These steps repeat until the test ends. Your final θ value is then converted into the familiar 60-90 Scale Score, with a percentile rank based on the last five years of test-taker data.
Based on this, I developed a complete algorithm in Python. By inputting the question numbers you got wrong from your official score report, it can replicate the trend of score changes and the relative difficulty of those questions.
Theory vs. Reality: 7 Key Differences Between My Algorithm and the Actual Exam
不確定自己現在幾分？
12 題免費測出分數區間 →
- Unscored Pretest Items: The actual GMAT includes unscored pretest questions. We don’t know how many or where they appear, and my algorithm doesn’t account for them.
- Starting Ability Level (θ): Your starting ability level (θ) for each section on the real exam might not be a neutral midpoint. It could adjust based on your performance in previous sections. The administrator keeps the specific adjustment algorithm confidential.
- Item Bank Size and Structure: We don’t know the actual size of the item bank (500 is just an example). More importantly, it’s stratified by question type and skills to ensure content validity. It’s unclear if question selection is random, follows a fixed proportion, or uses a more complex method to balance efficiency and information gain. Also, frequently seen questions might be temporarily “retired.”
- Item Parameters (a, b, c): The exact a, b, and c parameters for each question are proprietary, derived from analyzing responses from countless test-takers. We can never know the exact values. My algorithm generates these parameters randomly within a reasonable range.
- Impact of Response Time: While the administrator says response time “will not directly determine the score,” they also state that “test-takers with abnormal response times will be detected.” It’s unclear if there’s a “hidden exclusion mechanism” where abnormal response times might cause certain θ estimates to be excluded or weighted less.
- Theta to Scale Score Conversion Formula: The exact formula for converting θ to the 60-90 Scale Score is unknown, and we don’t know if it’s linear or non-linear.
Despite Limitations, Why is This Research Still Important?
More importantly, it lets us see the potential relative difficulty of each question you missed. This can help you reflect on your test performance and, ultimately, develop a more precise and strategic study and test-taking plan.
Follow Dustin and join the Terminator course to participate in data-driven, customized GMAT preparation planning.
