GMAT Prep vs MGMAT scores

Stacey Koprince wrote:We actually know this data already.

For the official test, GMAC has published a "within student" standard deviation of 30 points, and GMATPrep is supposed to be about the same. (Though there are intangibles - such as the fact that we know when taking GMATPrep that it's not the real test, which could impact nerves, how we take it, etc.) A "within student" SD means that if the student takes the test a bunch of times in a row (without studying / learning more in between), the standard deviation of that student's scores will be about 30.

We (MGMAT) have done extensive analysis of our large pool of students and their practice and official test scores.

- The "within student" SD is about 50 points on our tests. The "last MGMAT test to official test" SD is also about 50 points.

- Every 2 months, we calibrate our scoring algorithm to ensure that our test, on average (across all students now) is an unbiased estimator of GMAT performance. What this means: some students do better on the official test and some students do worse (compared to last MGMAT test). When we look at performance across all of our students, we attempt to ensure that the average difference balances out to zero. In practice, we ensure that the average difference in last MGMAT test score vs. official test score is between +5 and -5 points across all students. What this means: our tests do not consistently under- or over-estimate scores.

I took 5 Manhattan Cats and score an average of 650. With a S/D of 50 points I could get 600 or 700 in the real thing. Correct me if I am wrong, tks!

I am really thankful to Manhattan Team for their MGMATs..The assessment report for MGMATs is something which should be THE MOST IMPORTANT PREP TOOL for any serious GMAT Student. It's Scientific. Another good thing about MGMATs is that these tell the level of difficulty of each question..This helps in assessment & then fine tuning of the preparation.

MGMAT SC Guide is also a great book (I actually doubt that MGMAT have some hidden association with GMAC!!!! -- How do Manhattan guys know so much??)


Now, I'll come to the main topic ''MGMAT VS GMATPrep (The equivalent of actual GMAT)''

I have observed the following problem with MGMATs (Only My Opinion)
(This opinion is based upon my analysis of 6 MGMATs and 4 GMAT Preps)


If you do decently well i.e. maybe get around 15 questions correct from initial 25-27 Qs, and then seriously work on last 10-12 Qs, while getting all of them (or most of them) correct one by one, the MGMAT will LENIENTLY and continuously keep giving you a higher difficulty question even during the last stages of the test (A Lenient Question progression), and thus the percentile score will reach to 97-99% by last question, resulting in a high Final Score.
On the Other hand, on GMAT prep if you happen to get 10 / first 27 incorrect (homogeneously distributed), the test will make a decent judgment of your level (around 80-87 percentile) from your performance in this first 2/3rd of section, And now even if you get all of last 10 questions right, the test will keep RESTRICTING you to lower level difficulty question, Restricting you to 48 Max. (Q50 gives 20 or 30 more points on final score over Q48)

Somehow in MGMATs I have seen is that initial 2/3rd of a section is decently important, but what matters most is how you perform during last 1/3rd of your section.
For GMAT Preps it's opposite, the more important part is first 2/3rd of your sections, & performance in last 1/3rd of section will only fine tune the score.

MGMAT Quant questions are tougher than MGMAT Prep's, but MGMATs seem to make the adjustment by giving a slightly better score. This is specially true for Verbal Section, where difficulty level is almost same but the Question progression again is lenient & scoring is also lenient.

I followed the MGMAT question progression thing and started scoring 720 to 750 on MGMATs but when applied it back on GMAT Preps to see that the whole thing crashed, I scored 680 & 700.


Crux:
MGMAT Quant is tougher but GMAT Prep's marking is a bit tougher (But generally, people getting 46-49 in MGMAT get around 49-50 in GMAT Prep)
MGMAT Verbal is of same level (if not easier) but Scoring is MUCH lenient.

Generally:
for someone who scores Q50 in both, the relationship goes: MGMAT Score of X means GMAT Prep Score of X - 20
for someone who scores <Q48 in MGMAT: GMAT prep gives a better Q score; So, the relationship goes: MGMAT Score of X means GMAT prep Score of X +/- 30 (in GMAT Preps: Better Q score gets compensated by tougher V marking)

Having said that, I'll point out that if we compare all prep tests available, MGMATs come closest to GMAT prep (or Real GMAT)

BUT beware of 1 major unpredictability factor which comes with GMAT Prep and Actual GMAT: 9-12 unmarked questions, which can make your scores really swing...

@MGMAT Gurus: Do MGMATs also have some unmarked questions?

I had actually gone too far in applying MGMAT algorithm logic on GMAT, and yes the result was a disaster. Had scored 750 on MGMAT and then 680 on GMAT.

I have been reading your posts across websites and have noticed that you think a lot like a GMAT Student should, thus making your replies the most satisfying ones..I'll take this opportunity to thank you for your help not only on this forum but on other queries too.

Stacey Koprince wrote:Although you see a score assigned only in a 100-point range, the questions are actually assigned difficulties in much smaller increments - 10 points. So you could get a 740 wrong and then get a 720 level question and get that one wrong and then get a 700... and they'd all say "700 - 800" on your score report.

It's also the case, by the way, that adaptive tests factor in many other variables besides just the difficulty level. Different question types and sub-types, different content areas, etc.

Finally, although the general rule is "get something right, get a harder question; get something wrong, get an easier question," that is not necessarily the linear progression through the test. There are a lot of other things going on in there.

Disagree.

I have completed GMAT Prep, and in the Quant section I ended up with 16 incorrect answers with the first one being incorrect and the next 5 in row correct. Yet I have limited CAT test taking experience. The GMAT Prep scaled score for my Quant was 43.

I have completed MGMAT free online CAT, and in the Quant section I ended up with 25 incorrect answers and 12 correct answers - out of which 700-800 level 5 answers correct, 600-700 level 3 answers correct, the remaining 4 answers were 500-600 level. It was silly, as I made mistakes on 300-400 level questions, because I had to guess and/or move upstream for the test's finish time.

In my view, MGMAT was not representative of my GMAT Quant score at all. The Quant scaled score was 17!! I have documented my score from the actual GMAT without any preparation which I took last year with 8 more points, i.e. Quant scaled score 25.

p.s. I am not an occasional runner on the GMAT, I have finished exactly MGMAT guides and solved the problems from MGMAT materials, section end problems taking 2-6 minutes thinking. I am also done with OG 11 PS and DS review questions.

Stacey Koprince wrote:If you make a lot of careless errors on questions that you would ordinarily get right (which you said you did), that's going to change your score quite a bit. If you had timing problems (which you seem to say you did) and you get a lot of questions wrong in a row (especially at the end), that can drastically change your score (depending upon how many in a row you get wrong). In the roughly 35th to 85th percentile range, people answer approximately 60% of the questions correctly (give or take). On your GMATPrep test, you answered 57% of questions correctly. On your MGMAT test, you answered 33% of your questions correctly.

For instance, hypothetically, if two absolutely identical tests were given with the exact same performance, and the only difference was that the person ran out of time with 5 questions to go and left the last five blank, the difference in the two scores would be 55th percentile vs. 70th percentile. That's an extreme case (leaving 5 questions blank), but take a look at the huge difference in the two scores - there can be wide swings based on various kinds of "penalties," including getting too many lower-level questions wrong or getting many questions wrong in a row. (Those numbers are not something we made up; those numbers come directly from the Chief Psychometrician at GMAC, the person with primary responsibility for constructing the algorithm on the real test. I have seen him present those numbers myself - twice, actually.)

There are a LOT of factors that go into your test performance on any given day; the scores can vary widely depending on various conditions. My best guess is that you ran out of time towards the end and had a very large proportion of incorrect answers, and possibly a large number of incorrect answers all in a row. I can't know this for sure, of course, without more data (feel free to share, if you would like!).

The good news is that this happened on a practice test. Learn why it happened and what to do in order to prevent it from happening again on the real test. This article might help you to figure that out:
https://www.beatthegmat.com/a/2009/10/26 ... went-wrong

Dear Stacey, thanks for your reply. My point was that MGMAT CAT did not seem to be similar to my GMAT Prep experience. I am being thrown 700+ questions which I answer correctly and more such questions pop up on screen, at some point I grab myself and start rushing with the remaining questions. I know exactly that in order to succeed in MGMAT CAT one must make best guess on time consuming questions by half solving or leaning towards decision rule for 700+ hard questions - for solving them all during test is suicide for MGMAT CAT. The GMAT prep is relatively flexible for counting these issues. If in a row, I get 8 or even 9 questions answered incorrectly, GMAT Prep somehow considers either timing issue or some other element and I still end up with high scaled score. That's not the case with MGMAT CAT. Two different approaches - guess on hard questions and move on in MGMAT CAT, and solve each question and move on GMAT Prep. This seems like two ends for CAT approach. I decided not to do any more MGMAT CATs to save my GMAT Prep approach for the actual exam.