Is GMAT software changing?

This topic has expert replies
Junior | Next Rank: 30 Posts
Posts: 27
Joined: Tue Apr 03, 2007 2:53 pm
Thanked: 1 times

Is GMAT software changing?

by snuman » Thu Jan 10, 2008 1:47 pm
I took GMAT 2nd time on December 27, 2007 and got a score of 460 (Q 35, V 18), which is way below my scores in GMATprep. In both the GMATprep test my score was more than 600. This score of 460 is shocking for me because when i took GMAT first time in December 2006 my score was 540 (Q 43, V 21). At that time i was not satisfied with my preparation level, and the confidence was very shaky, especially in Quantitative. This time round i prepared a lot for both verbal and quantitative, and practiced a lot; hence my confidence was way up. But i am really shocked to see this low score of 46, which does not reflect my understanding of the subject matter tested in the GMAT; if GMAT tested this, not the understanding of the software!!

One thing that i noticed this time was that the actual test started with very easy questions, both in quant and verbal and gradually increased the difficulty level. This is quite contradictory to what i have heard and noticed in the GMATprep. You people must have heard that GMAT starts with average difficulty question and then adjust you score as the test moves along. And almost certainly if you do well in the initial questions and get the last 5-6 wrong in the end, your score would not come down drastically. But i think this has changed.

Now i think GMAT officials have changed the software for the actual test. It gives you easy questions in the beginning (maybe it starts with 200/300 score instead of average 500/550) and increases the difficulty level and adjusts your score as you move along. In this situation, you get difficult questions in the end carrying most weight in the score. And if you do not manage your time, you are bound to do bad on the last questions - the result is low score.

I would like to add further that one of my friends who gave the GMAT one week before me in December 2007 also faced the similar situation. We saw very easy questions in the beginning. In Verbal GMAT started with initial 3-4 SC to both of us. And these SC questions were not difficult either. We both felt that the level of difficulty increased as we moved along in the test.

One of the most interesting things is that my same friend got 95% in GRE Quant few months before his GMAT,and in GMAT he got 60% in Quant. He was consistently getting 90% in his GMATprep.

One more interesting thing: Just recently Princeton Review has issued GRE alert to all its centers that test takers who are just doin random guessing at the last few question for want of time in the end are getting lowest of the scores.

Any experience/suggestions for my 3rd attempt?

GMAT/MBA Expert

User avatar
Site Admin
Posts: 6774
Joined: Mon Feb 13, 2006 8:30 am
Location: Los Angeles, CA
Thanked: 1249 times
Followed by:994 members

by beatthegmat » Fri Jan 11, 2008 10:21 pm
I have not heard any news/evidence that the GMAT has changed since early 2006. Perhaps you had run into some experimental questions early on that threw you off on the algorithm?
Beat The GMAT | The MBA Social Network
Community Management Team

Research Top GMAT Prep Courses:
https://www.beatthegmat.com/gmat-prep-courses

Research The World's Top MBA Programs:
https://www.beatthegmat.com/mba/school

Junior | Next Rank: 30 Posts
Posts: 27
Joined: Tue Apr 03, 2007 2:53 pm
Thanked: 1 times

by snuman » Sat Jan 12, 2008 11:45 am
Thanks for the reply. But this low score of 460 has really jolted my confidence in GMAT especially 2nd time i felt that i prepared well and did well on the exam as compared to my first attempt when i got 540. I'm practicing for the 3rd attempt from LSAT and GRE for RC and CR and here my correction rate is above 70% in timed conditions. And same is the case with my math preparation. I am really baffled what will be the result 3rd time especially when I am scoring above 600 consistently in GMATPrep. I still think they have changed the software somehow. My friend experienced the same low score (he got 470) week before me. He was consistently scoring 47/48 in Quant and 30 in Verbal.

GMAT/MBA Expert

User avatar
Site Admin
Posts: 6774
Joined: Mon Feb 13, 2006 8:30 am
Location: Los Angeles, CA
Thanked: 1249 times
Followed by:994 members

by beatthegmat » Sat Jan 12, 2008 5:12 pm
This definitely sounds suspicious. If you feel fairly confident that you were performing well on your test, I would contact your GMAT test center and see whether they could connect you to the right people to get a review.

From your account of what happened, I agree--the story doesn't add up!
Beat The GMAT | The MBA Social Network
Community Management Team

Research Top GMAT Prep Courses:
https://www.beatthegmat.com/gmat-prep-courses

Research The World's Top MBA Programs:
https://www.beatthegmat.com/mba/school

User avatar
Legendary Member
Posts: 2469
Joined: Thu Apr 20, 2006 12:09 pm
Location: BtG Underground
Thanked: 85 times
Followed by:14 members

by aim-wsc » Sun Jan 13, 2008 4:53 am
quote="snuman"]Any experience/suggestions for my 3rd attempt[/quote]
STRANGE! I'm sorry to hear this. But I'll definitely investigate, find some trends/signs and get back to you.
You've presented two cases so I believe there must be something.

As all of us know GMAC algorithm is "black box" and nobody knows about it. And they like to keep it that way. Still we can see others testimony (like the one you shared with us) and know the trends.
beatthegmat wrote:I have not heard any news/evidence that the GMAT has changed since early 2006. Perhaps you had run into some experimental questions early on that threw you off on the algorithm?
I concur. yet... :?

Junior | Next Rank: 30 Posts
Posts: 27
Joined: Tue Apr 03, 2007 2:53 pm
Thanked: 1 times

by snuman » Sun Jan 13, 2008 5:48 am
Thank you guys.

I have already sent my complaint to the Pearson office. They have escalated it to a higher level for further investigation. Let's see what they come up with.

Nauman

User avatar
Legendary Member
Posts: 2469
Joined: Thu Apr 20, 2006 12:09 pm
Location: BtG Underground
Thanked: 85 times
Followed by:14 members

by aim-wsc » Sun Jan 13, 2008 8:02 am
snuman wrote:Thank you guys.

I have already sent my complaint to the Pearson office. They have escalated it to a higher level for further investigation. Let's see what they come up with.

Nauman
Do let us know about it.
& You can always PM us.

GMAT/MBA Expert

User avatar
Site Admin
Posts: 6774
Joined: Mon Feb 13, 2006 8:30 am
Location: Los Angeles, CA
Thanked: 1249 times
Followed by:994 members

by beatthegmat » Sun Jan 13, 2008 7:54 pm
Yes, please let us know what happens. And I'm really sorry to hear about your experience!
Beat The GMAT | The MBA Social Network
Community Management Team

Research Top GMAT Prep Courses:
https://www.beatthegmat.com/gmat-prep-courses

Research The World's Top MBA Programs:
https://www.beatthegmat.com/mba/school

Junior | Next Rank: 30 Posts
Posts: 28
Joined: Mon Jul 16, 2007 7:29 am
Thanked: 8 times
Followed by:1 members

by Leonard C » Tue Jan 15, 2008 7:53 pm
Guys,

I just want to add my two cents to this. I sat for the GMAT yesterday and scored a 670. This is way below my scores in the GMATPrep (750 and 770) and MGMAT CATS (started at 680, then consistently 720-750). In my quant practice CATS, I would typically get between 8-12 questions wrong, but still score 48 to 50 (does this seem right to you? or is my software not working correctly?)

I am quite baffled as to why my actual score was so much lower than my practice score, and I really think that there is a change in the system. I am not someone who is overconfident about things in general (actually, I am usually very underconfident) but I came out of the quant section thinking that I had done quite well. In my practice exams, I would typically score between the 85th and 95th percentile in quant. On the actual day, I only managed to get to the 73rd percentile.

Also, in verbal I was only given three RCs - I was expecting four.

Snuman - do you want to discuss our experiences? I am also thinking of investigating it further.

GMAT/MBA Expert

User avatar
GMAT Instructor
Posts: 2228
Joined: Wed Dec 27, 2006 3:28 pm
Location: Montreal, Canada
Thanked: 639 times
Followed by:694 members
GMAT Score:780

by Stacey Koprince » Wed Jan 16, 2008 1:37 pm
They haven't changed the algorithm recently in this way (although a lot of it is "black box," they do also tell the test prep companies a number of things about the way it works). I was at a GMAC conference in October and both watched a presentation from and spoke privately with the guy who's in charge of the algorithm. The only substantive change they've made recently is that there are smaller "jumps" in difficulty level when you get a question right or wrong (in terms of what difficulty level the test gives you next).

Also, snuman said "And almost certainly if you do well in the initial questions and get the last 5-6 wrong in the end, your score would not come down drastically."

This is absolutely a myth. Your score WILL come down drastically if you get 5-6 wrong in a row at the end. Each wrong answer reduces your score by about 2-2.5 percentile points. And if you simply run out of time and leave some questions blank, each blank question reduces your score by 3 percentile points. That's PERCENTILE - not just percentage - the penalty is severe. And these numbers are straight from the mouth of the guy who's in charge of the algorithm for the official test.

It is incredibly difficult to gauge difficulty levels just by looking at a question. The difficulty level is based upon the skills across the entire test-taking population. Your skills are not the same as the average set of skills across the entire population - you have strengths and weaknesses that do not mirror the population's. So you will think some things are easier than they are actually rated and you will think other things are harder than they are actually rated, simply because of your personal strengths and weaknesses. Add to that the fact that the experimental questions you get could be at any level, and you really can't gauge this effectively in real time while taking the test.

The most common reason people have a big drop in scores is timing - they spend too much time on earlier questions and run out of time towards the end. This results in a string of questions wrong in a row at some point, which is the biggest thing that can kill your score. For the above posters who experienced score drops: did you have to guess on any questions at the end, or leave any blank? If so, how many? And at what point did you feel like you had to start working more quickly than you were comfortable - say, answering every question in 60-90 seconds instead of 90-120 seconds?

(Oh, and finally, though the most common situation is to get 4 RC passages, the second most common situation is to get 3.)
Please note: I do not use the Private Messaging system! I will not see any PMs that you send to me!!

Stacey Koprince
GMAT Instructor
Director of Online Community
Manhattan GMAT

Contributor to Beat The GMAT!

Learn more about me

Junior | Next Rank: 30 Posts
Posts: 27
Joined: Tue Apr 03, 2007 2:53 pm
Thanked: 1 times

by snuman » Thu Jan 17, 2008 3:11 am
Yeah i had to guess randomly last 3-4 questions in both Verbal and Quant. But that was the case with all my GMATPrep scores. And i never got less than 34 in Verbal GAMTPrep, whereas in my actual i got just 18. Almost same happened with Quant; my lowest score in GMATPrep Quant was 40 with random guessing of last 4-5 questions, whereas in actual i got 35. I had hit 43 in Quant in the actual test a year back, while at that time i was not fully prepared in Quant and was feeling very shaky.

Nauman

Junior | Next Rank: 30 Posts
Posts: 27
Joined: Tue Apr 03, 2007 2:53 pm
Thanked: 1 times

by snuman » Thu Jan 17, 2008 8:20 am
Further, as you asked, i started feeling short of time after around 20th questions in both verbal and quantitative. I never left any questions unanswered.

So Stacy what I got from your advice is that one should never spend extra time on hard questions or questions that are beyond one’s capacity – just give it a try and move on without wasting extra time. This way you can maximize your score, which would be more reflective of your level. After all skipping hard questions with educated guess won’t diminish your actual level and capacity to achieve a score that would be more reflective of your level. But still, is GMAT checking our understanding of the software or understand of the subject matter i.e. Quant and verbal skills? It means timing is the key after you have achieved certain level of preparation.

I have another question, why the GMATPrep results seem inconsistent with the actual test results? First time when I got 540 I felt its more or less what I was getting in GMATPrep, but this time my score of 460 is not convincing neither too me nor to others.

Nauman

GMAT/MBA Expert

User avatar
GMAT Instructor
Posts: 2228
Joined: Wed Dec 27, 2006 3:28 pm
Location: Montreal, Canada
Thanked: 639 times
Followed by:694 members
GMAT Score:780

by Stacey Koprince » Thu Jan 17, 2008 11:10 am
Yes, you're right - never spend more time than the "normal" amount for anything that is really beyond your ability (and you will have things beyond your ability - that's just the way it works).

And, yes, you're right that timing is very important and can factor significantly into your score - which does seem odd, I agree, but I think of it this way: you have to take a test. The people who would be most suited for going back to school are the people who will make sure they learn what they need to learn to take that test. Those people will be prepared for both the content and the timing. People who don't prepare for all aspects of the test are not truly well prepared. It's the same thing as knowing, for a history teacher I had in high school, that he liked giving essay tests and he loved when we quoted people - that always got high grades. So I did what he wanted. For any test you take, know how to take the test.

As to why GMATPrep seems inconsistent with the real test, there are lots of possible explanations. First, there's just a basic level of standard deviation (about 30 points for the real test), so nothing is absolutely precise. You also know the real test counts and you know GMATPrep doesn't count - that can make a big difference. You could also happen to get a larger proportion of things you're good at or bad at on one test. The time of day you take a test can make a big difference - what you've eaten, your energy levels, etc. There's no real way to second-guess.

If you feel something did go wrong with the actual test, please do have them investigate - but know that there are a lot of variables here that can affect your performance.
Please note: I do not use the Private Messaging system! I will not see any PMs that you send to me!!

Stacey Koprince
GMAT Instructor
Director of Online Community
Manhattan GMAT

Contributor to Beat The GMAT!

Learn more about me

User avatar
GMAT Instructor
Posts: 3225
Joined: Tue Jan 08, 2008 2:40 pm
Location: Toronto
Thanked: 1710 times
Followed by:614 members
GMAT Score:800

by Stuart@KaplanGMAT » Thu Jan 17, 2008 11:41 am
A quick thought on how to best use your time on test day:

It's too extreme to say "never spend extra time on hard questions". For every question you answer on Test Day, you need to ask yourself "do I get a greater reward using time on this question or on other questions"?

To answer that question accurately, one needs to be intimately aware of one's own strengths and weaknesses. Near-perfect self knowledge is one of the keys to success on the GMAT and is one of the reasons why practice is so important. Only by doing a lot of GMAT questions in advance of test day can one make the most efficient decisions on test day.

When you review your practice tests, pay very close attention to how you spent your time. Most people who struggle with timing find that, if they had just guessed at the 2 questions in each section on which they spent the most time, they would have had time to finish the test. Knowing when to guess is at least as important as knowing how to guess.

So, figure out which types of questions you consistently get wrong and/or on which you waste a lot of time. Guessing on a "bad" question after 20 seconds is vastly preferable to guessing on that question after 3 minutes and 20 seconds. If you can quickly identify and guess on the "stinkers" you can bank that extra time for the questions that you could do if you just had a bit of extra time to spare.

Also, be very aware of timing during the test. Don't fixate on the clock to the point at which you lose focus on the test, but set benchmarks for yourself. For example, "after question 10 I should have 55 minutes left on the clock... after question 20 I shoudl have 35 minutes left on the clock" and so on.

If you set benchmarks, you can adjust your timing mid-test depending on if you're ahead of or behind the clock.
Image

Stuart Kovinsky | Kaplan GMAT Faculty | Toronto

Kaplan Exclusive: The Official Test Day Experience | Ready to Take a Free Practice Test? | Kaplan/Beat the GMAT Member Discount
BTG100 for $100 off a full course

GMAT/MBA Expert

User avatar
GMAT Instructor
Posts: 2228
Joined: Wed Dec 27, 2006 3:28 pm
Location: Montreal, Canada
Thanked: 639 times
Followed by:694 members
GMAT Score:780

by Stacey Koprince » Thu Jan 17, 2008 11:43 am
Technically - I didn't say "never spend extra time on hard questions"

I said never spend extra time on questions that are truly beyond your ability - that's not the same thing as a merely hard question. People will hit a few questions that are well beyond their ability - and, for those, you don't want to blow *extra* time on them b/c you can't do them in 4 or 5 minutes, let alone 2.

:)
Please note: I do not use the Private Messaging system! I will not see any PMs that you send to me!!

Stacey Koprince
GMAT Instructor
Director of Online Community
Manhattan GMAT

Contributor to Beat The GMAT!

Learn more about me