GMAT Strategy

I just saw this thread for the first time. I hate to be negative about the studies performed above, but there's no chance you'll be able to reverse-engineer the GMAT scoring algorithm from response patterns in GMATPrep. A few things about the scoring algorithm:

* despite what you might read in books, the algorithm is not a secret. It's based on a field of test theory called Item Response Theory that you can learn about on wikipedia;

* that said, the mathematics behind the algorithm is horrendously complicated. Unless you have an undergraduate degree in probability theory or statistics, you shouldn't even look at it;

* there are two components to the algorithm that work hand-in-hand: the scoring algorithm, and the question selection algorithm. All of the research above is inherently ignoring the importance of the question selection algorithm, and without understanding question selection, you can't understand how GMAT scoring works.

Oversimplifying the math, the algorithm basically does this: at the end of your test, the algorithm looks back over all the questions you answered and tries to determine what difficulty level of question you can handle comfortably, and what level is too hard for you. So if the test looks back over your answers and sees that you get most of your sub-600 level questions right, but that you don't get very many questions right that are above the 600-level, the algorithm will conclude that you're most likely a 600-level test taker. When the algorithm does this, it has no idea when you saw each question. If you get a 500-level question wrong early in your test, that hurts you just as much as getting a 500-level question wrong at the end of your test. The algorithm is completely blind to question 'position'.

One consequence of this is that it hurts you considerably more to get a 250-level question wrong than to get an 800-level question wrong. If you answer an 800-level question wrong, that is only evidence that you might not be an 800-level test taker, but you still might be a 750-level test taker. On the other hand, getting that 250-level question wrong is evidence that you can't handle even the lowest level questions, and you'll need to do some work to persuade the algorithm that your wrong answer there was an anomaly. It's for this reason that your number of correct answers is only weakly correlated with your score. If you were to take a test consisting of only 200-level questions, you'd need to get almost all of them right to get a respectable score. If instead your test consisted only of 800-level questions, you could get a ton of them wrong and still get a good score. For example, the Diagnostic Test at the beginning of OG12, mentioned in the post above, is full of hard questions, so you can get a lot of them wrong and still get a good score according to their score table.

All of that said, your answers to early questions play a role in determining what questions you see later in your test. As you proceed through the GMAT, the test is constantly updating its estimate of your ability, and it attempts to give you questions around your estimated ability level. This is why it is better to get the first 27 questions right, then the last 10 wrong, than to get the first 10 wrong and the last 27 right. If you get the first 10 questions wrong on your test, that's worse than a chimpanzee would normally do just entering random answers - after all, you'd expect, guessing randomly, to get 20% of your questions right. So if you get your first 10 questions wrong, the algorithm will think that you're completely hopeless. From that point forward, it will be giving you very easy questions, and you'll need to get a lot of those right just to persuade the algorithm that you can handle 200 or 300-level questions. If, on the other hand, you get your first 10 questions right, the algorithm will be persuaded that you're a very high-level test taker. You'll then be getting mostly 750 to 800-level questions, and even if you get a lot of those wrong, the algorithm can still think of you as a 700-level test taker, at least up to a point - if you get too many of these wrong, the test will keep lowering its estimate of your ability.

So in one sense, early questions are more important than later ones, since answering them correctly gives you the opportunity to see harder questions later on. But if you're a genuine 600-level test taker, then even if you answer your first 10 questions correctly (something, I should add, that a 600-level test taker will almost never be able to do) and start seeing a lot of 700-level questions, eventually you'll start getting most of your questions wrong (if you didn't, you wouldn't be a 600-level test taker, you'd be a 700-level test taker). You won't be able to 'outsmart' the algorithm by some exceptional response pattern early in the test.

There are several further complicating factors:

* when the test selects your next question, its estimate of your ability is only one factor that determines which question you see. The test also has content restrictions to ensure you see the same number of, say, word problems as any other test taker. There is also a security requirement; the test does not want any question to be given to too many test takers, just in case some questions are (illegally) publicized - then knowing about that question in advance would give a test taker an unfair advantage if he or she was highly likely to see that question on the test. Because the question pool is not infinite, that means there is usually only a relatively small number of questions the test has available at each point. So even if you're a 700-level test taker, you'll see some questions which count which are either well below or well above your level late in your test; the test simply doesn't have an infinite supply of questions precisely at your level that it can give you. This is why all of the research above can't possibly reach any exact conclusions: on any test you take there will be a different question pool, and you won't be able to predict the difficulty level of questions at each point in the test, even if you answer using the same response pattern. It is also why it can hurt you to guess at the last few questions; some of those questions might be well below your level, and it will hurt you to get those wrong. Long story short: a wrong answer early in your test might not hurt you much at all because the question might be way above your level, and a wrong answer late in the test can hurt you a lot because the question might be way below your level. Unless you have a ton of information about the underlying question database, you can't determine how often this will happen in a given test, which is why it's impossible to look at GMATPrep results and reach any reliable conclusions about what will happen on a real GMAT.

* further complicating things is the presence of experimental questions. Some of your questions simply don't count, and you won't have any way to know which those questions are (where they occur on your test is determined completely at random). If GMATPrep models the presence of experimental questions correctly, then it will be the case that some wrong answers have no influence at all on your score.

* the observation above that wrong answers in RC hurt more than in SC or CR is likely due to the fact that RC is the least 'adaptive' question type. RC passages are always accompanied by a group of questions, and while on average these questions might be hard or easy, each question individually will have its own difficulty level - even in a batch of RC questions which is hard overall, one or two of the questions might be easy. So if you answer several RC questions incorrectly, you're more likely to be answering easy questions incorrectly, which hurts you more than answering difficult questions incorrectly.