I was wondering about how the scoring algorithm determines the difficulty of a problem. From all the posts I've read on these forums, it seems like the idea is to get as many correct in a row as possible so that the algorithm gives you progressively harder questions. If this is the case, would getting 1 question wrong then 9 right be a higher difficulty level than getting 4 right then 1 wrong then 5 right?
I'm asking because my last practice CAT has better less incorrect problems and had more sections of consecutive answers, but yet my score only improved very marginally (3 raw points).
In any case, my study plan will remain the same - I'm just bewildered by what I thought was a much better performance, but apparently was not
Here are the printouts from two PR CATs I have taken:
5-Day Free Trial
5-day free, full-access trial TTP
Available with Beat the GMAT members only code
MORE DETAILS
consecutive correct answers and how that effects difficulty
This topic has expert replies
-
mayonnai5e
- Legendary Member
- Posts: 1018
- Joined: Tue Dec 12, 2006 7:19 pm
- Thanked: 86 times
- Followed by:6 members
- Attachments
-
- PR1 where I had more errors, fewer consecutive correct answers and a large number of errors at the end of the exam (I ran out of time and guessed on the last 15 or so)
-
- PR 2 where I have few errors, more consecutive correct answers and few errors at the end of the exam (my timing improved a lot between these two CATs)
GMAT/MBA Expert
-
Stacey Koprince
- GMAT Instructor
- Posts: 2228
- Joined: Wed Dec 27, 2006 3:28 pm
- Location: Montreal, Canada
- Thanked: 639 times
- Followed by:694 members
- GMAT Score:780
I can't comment on PR's algorithm because I don't know how they have constructed it - I've seen some data in the past that's made me think they don't base it on IRT (the algorithmic theory behind the official GMAT) or at least that it isn't very tightly based on IRT, but I don't know that for certain.
On the real test, you will find it incredibly difficult to get 9 in a row right. The test is constantly trying to find your upper and lower limits - if you're getting things right, it's going to try to "break" you by giving you things it expects you to get wrong.
It would be nice if we could get as many correct in a row as possible (how about 37 and 41?
) but there are only 2 scenarios here:
1) you can get many in a row correct without going over on time (in which case, yay, come work for us), OR
2) (the most common scenario by far) you spend extra time trying to do this but you can't actually sustain it because the test will start giving you impossible questions - and then you will also have to rush at the end or run out of time, which means you get questions wrong which you should have gotten right and your score nosedives.
Think of it not as what you want to do but want you want to avoid: you want to avoid getting many questions in a row wrong. The vast majority of people cannot maintain streaks (of right or wrong questions) of more than a few in a row - not because they haven't studied enough or they aren't smart enough or anything like that, but because this is how the test is constructed. You can't outsmart it (at least, not in this respect).
On the real test, you will find it incredibly difficult to get 9 in a row right. The test is constantly trying to find your upper and lower limits - if you're getting things right, it's going to try to "break" you by giving you things it expects you to get wrong.
It would be nice if we could get as many correct in a row as possible (how about 37 and 41?
) but there are only 2 scenarios here:1) you can get many in a row correct without going over on time (in which case, yay, come work for us), OR
2) (the most common scenario by far) you spend extra time trying to do this but you can't actually sustain it because the test will start giving you impossible questions - and then you will also have to rush at the end or run out of time, which means you get questions wrong which you should have gotten right and your score nosedives.
Think of it not as what you want to do but want you want to avoid: you want to avoid getting many questions in a row wrong. The vast majority of people cannot maintain streaks (of right or wrong questions) of more than a few in a row - not because they haven't studied enough or they aren't smart enough or anything like that, but because this is how the test is constructed. You can't outsmart it (at least, not in this respect).
Please note: I do not use the Private Messaging system! I will not see any PMs that you send to me!!
Stacey Koprince
GMAT Instructor
Director of Online Community
Manhattan GMAT
Contributor to Beat The GMAT!
Learn more about me
Stacey Koprince
GMAT Instructor
Director of Online Community
Manhattan GMAT
Contributor to Beat The GMAT!
Learn more about me
-
mayonnai5e
- Legendary Member
- Posts: 1018
- Joined: Tue Dec 12, 2006 7:19 pm
- Thanked: 86 times
- Followed by:6 members
I agree that PR's algorithm seems very iffy. My GMATPrep score had far fewer consecutive problems in a row and my raw score was 45. The data from my two PR CATs just does not make sense to me according to the algorithmic rules as they are explained on these forums.Stacey Koprince wrote:I can't comment on PR's algorithm because I don't know how they have constructed it - I've seen some data in the past that's made me think they don't base it on IRT (the algorithmic theory behind the official GMAT) or at least that it isn't very tightly based on IRT, but I don't know that for certain.
On the real test, you will find it incredibly difficult to get 9 in a row right. The test is constantly trying to find your upper and lower limits - if you're getting things right, it's going to try to "break" you by giving you things it expects you to get wrong.
It would be nice if we could get as many correct in a row as possible (how about 37 and 41?
1) you can get many in a row correct without going over on time (in which case, yay, come work for us), OR
2) (the most common scenario by far) you spend extra time trying to do this but you can't actually sustain it because the test will start giving you impossible questions - and then you will also have to rush at the end or run out of time, which means you get questions wrong which you should have gotten right and your score nosedives.
Think of it not as what you want to do but want you want to avoid: you want to avoid getting many questions in a row wrong. The vast majority of people cannot maintain streaks (of right or wrong questions) of more than a few in a row - not because they haven't studied enough or they aren't smart enough or anything like that, but because this is how the test is constructed. You can't outsmart it (at least, not in this respect).
You're definitely right in saying it's not about how many you get right in a row, but how many you get wrong in a row.
Thanks for the reply.
