I'll just comment on our tests (ManhattanGMAT).
- Our tests have a standard deviation of 50 points, compared to a standard deviation of 30 points on the official test. So, close, but not as good as the real thing (obviously it would be impossible to be the same as or better than the real thing!).
- Our algorithm mimics the algorithm used by the official test, though we do not have experimental question capability built in yet (though we expect to be the first prep company to have this, hopefully next year sometime).
- We run statistical tests of every single question to determine the question's validity, predictability, difficulty level, and about 5 other parameters. Any that fail even one of those tests gets pulled from the database and either fixed or discarded. As far as I'm aware, no other company continually checks and tweaks its question pool in this way (and between me and my colleagues, we have worked in the past for all of the major and many of the minor competitors out there). In fact, this can only be done for tests that are given online, so that the database can be changed dynamically. Tests taken from CDs obviously can't do this.
- We also ask our students to report their official results to us and we run backward tests to check the accuracy of their scores on our tests vs. what they got on the official test. We use that to tweak our scoring algorithm; this is what allows us to get our standard deviation so close to that of the official test.
Please note: I do not use the Private Messaging system! I will not see any PMs that you send to me!!
Stacey Koprince
GMAT Instructor
Director of Online Community
Manhattan GMAT
Contributor to Beat The GMAT!
Learn more about me