Re: some questions about our stats on student results (of our tests), I asked our algorithm guy to address this. Here's what he said [with some additions from me for clarification purposes]:
"Our tests are calibrated so that the mean is equal to the expected GMAT mean—and so that regardless of ability, our test is an unbiased estimator of GMAT performance. Some do better, some do worse, but the expected difference is zero.
I review data about once every 2 months to ensure this – data from practice tests coupled with actual, reported GMAT scores. I do so on an overall score basis and a Verbal and Quant percentile basis. I occasionally tweak the scoring if we get evidence that the difference is far different from zero. Usually, the average difference is between +5 and -5 points on an OVERALL scoring basis.
On the standard error issue:
(1) The 50-point std deviation is a WITHIN-STUDENT st dev. In other words, if one student takes the test 100 times [without significant additional prep in between], the st deviation of his scores will be about 50 points.
The st dev across students is a different story and not relevant (it is about 100, just like the GMAT supposedly is).
(2) The 50-point std deviation is also the st dev of the DIFFERENCE BETWEEN MGMAT (latest practice test) scores and GMAT scores. Some call this a standard error, but effectively it’s the same. The STANDARD ERROR OF MEASUREMENT is mathematically equivalent to the STANDARD DEVIATION OF the error of measurement.
Those two numbers are not [exactly] equal, but both approximately 50."
Also just wanted to add - note that we calculate from the latest practice test to the official test. Most of our students take the last practice test within a week of the real test - so there isn't a significant amount of prep / improvement between the two tests.
We are, of course, comparing apples to oranges (that is, we're comparing two different tests), so that complicates things - but, even so, we can get some useful data.
GMATPrep is still the best general indicator because it is from the same people who make the official test (with all that that entails). The test prep company practice tests have the advantage, though, that you can get explanations and analysis of your performance - so there are good reasons to use both GMATPrep and test prep company practice tests.
Please note: I do not use the Private Messaging system! I will not see any PMs that you send to me!!
Stacey Koprince
GMAT Instructor
Director of Online Community
Manhattan GMAT
Contributor to Beat The GMAT!
Learn more about me