consecutive correct answers and how that effects difficulty

[mayonnai5e]

I was wondering about how the scoring algorithm determines the difficulty of a problem. From all the posts I've read on these forums, it seems like the idea is to get as many correct in a row as possible so that the algorithm gives you progressively harder questions. If this is the case, would getting 1 question wrong then 9 right be a higher difficulty level than getting 4 right then 1 wrong then 5 right?

I'm asking because my last practice CAT has better less incorrect problems and had more sections of consecutive answers, but yet my score only improved very marginally (3 raw points).

In any case, my study plan will remain the same - I'm just bewildered by what I thought was a much better performance, but apparently was not

Here are the printouts from two PR CATs I have taken:

[Stacey Koprince]

I can't comment on PR's algorithm because I don't know how they have constructed it - I've seen some data in the past that's made me think they don't base it on IRT (the algorithmic theory behind the official GMAT) or at least that it isn't very tightly based on IRT, but I don't know that for certain.

On the real test, you will find it incredibly difficult to get 9 in a row right. The test is constantly trying to find your upper and lower limits - if you're getting things right, it's going to try to "break" you by giving you things it expects you to get wrong.

It would be nice if we could get as many correct in a row as possible (how about 37 and 41? ) but there are only 2 scenarios here:
1) you can get many in a row correct without going over on time (in which case, yay, come work for us), OR
2) (the most common scenario by far) you spend extra time trying to do this but you can't actually sustain it because the test will start giving you impossible questions - and then you will also have to rush at the end or run out of time, which means you get questions wrong which you should have gotten right and your score nosedives.

Think of it not as what you want to do but want you want to avoid: you want to avoid getting many questions in a row wrong. The vast majority of people cannot maintain streaks (of right or wrong questions) of more than a few in a row - not because they haven't studied enough or they aren't smart enough or anything like that, but because this is how the test is constructed. You can't outsmart it (at least, not in this respect).

[mayonnai5e]

I can't comment on PR's algorithm because I don't know how they have constructed it - I've seen some data in the past that's made me think they don't base it on IRT (the algorithmic theory behind the official GMAT) or at least that it isn't very tightly based on IRT, but I don't know that for certain.

On the real test, you will find it incredibly difficult to get 9 in a row right. The test is constantly trying to find your upper and lower limits - if you're getting things right, it's going to try to "break" you by giving you things it expects you to get wrong.

It would be nice if we could get as many correct in a row as possible (how about 37 and 41? ) but there are only 2 scenarios here:
1) you can get many in a row correct without going over on time (in which case, yay, come work for us), OR
2) (the most common scenario by far) you spend extra time trying to do this but you can't actually sustain it because the test will start giving you impossible questions - and then you will also have to rush at the end or run out of time, which means you get questions wrong which you should have gotten right and your score nosedives.

Think of it not as what you want to do but want you want to avoid: you want to avoid getting many questions in a row wrong. The vast majority of people cannot maintain streaks (of right or wrong questions) of more than a few in a row - not because they haven't studied enough or they aren't smart enough or anything like that, but because this is how the test is constructed. You can't outsmart it (at least, not in this respect).

I agree that PR's algorithm seems very iffy. My GMATPrep score had far fewer consecutive problems in a row and my raw score was 45. The data from my two PR CATs just does not make sense to me according to the algorithmic rules as they are explained on these forums.

You're definitely right in saying it's not about how many you get right in a row, but how many you get wrong in a row.

Thanks for the reply.