1. I don't have to read any 'literature' to know that a mathematical theory works in practice only when its assumptions are met. The assumptions in the 'adaptive scoring' are that errors are indicative of the test-taker level and that there is a pristine correlation between a candidate abilities in different math sub-areas.

As a practicing tutor I know that BOTH assumptions are completely wrong: (1) smart test takers do make dumb mistakes quite often because GMAT is a rushed nerve wrecking exam and (2) ofthen there is no correlation between abilities in different math areas tested on GMAT.

So much about the 'nice theory of item response'. I guess the other tests like GRE, SAT, ACT have not read the item response theory too, cause they keep the old non-adaptive format.

2. You missed the point about the test with 92%. According to GMAT, 1 in 12 people will get a better score than that particular test I simmulated. I am aware what percentile means (thanks for the lecture though). I question that percentile truthfulness as a skill measure due to the way the score was obtained. My claim was that the test taker was penalized just because he made 2+2 errors in the begining, something that happens pretty often in real life, and was not given second chance later, while some of the 1 in 12 'better' test takers most probably made about 6-7 errors scattered around the middle and end of the test.

So my claim is, due to the skewed adaptive algorithm, test takers that tend to make errors earlier in the test are underestimated, in comparison to ones that don't. Now how probable is that to happen? From my experience, everybody underperforms in the begining and end of the test. How probable is that to happen in two questions in a row? As probable as two questions not in a row LOL

In that light, the old 'myth' that a test taker should be especially careful with the first 10 problems is actualy reallity.

In a real truthful test, covering all topics uniformly, with problems from lowest to highest difficulty in each topic, such problem would not arise. But who am I to argue with the 'item response theory', which is so much better for scoring because it is based on probabilistic assumptions that nobody has ever bothered to check. Yeah lets use likelihood instead of a simple old fashioned test, it's so much fancier LOL

3. I hope you aren't trying to tell me that YOUR posts are a useful yardsticks for a professional tone? Because to me they sound like a white noise and don't offer an actual scientific discussion of the facts presented but simply your condescending brain-washed opinion.

tutor -

Please remember why the GMAT is here in the first place. It is not a test of math! In fact the Quant section only tests math because it needs some kind of subject. At a meeting with the officials at GMAC, the leading companies were basically told that the reason that math and sentence correction and so forth are even on the test is that they need something to be the subject. But math is the canvas it is not the painting. The GMAT is a test of REASONING. It is a test that puts people under pressure and tries to cause them to make these avoidable errors!

Graduates of business school can hire math geniuses if they need them. The one thing that all MBAs need is the ability to make decisions. Please start to see the test through this lens. I, and many others, who have not spent as much time in pure math still outperform many mathematicians on the Quantitative Reasoning section. This is not unfair, it is what the GMAT is designed for; it is what business schools wanted. People who can reason, not a class full of "math jocks."

Now with that understood, it would be great if you could take a step back and take a look at what is being said on this thread.

We are all telling you that some of what you are saying is true, but that it is not a big deal. Yes, missing questions that are very low in difficulty early in the test does impact your score. Of course, one of the worst things that you can do is to miss several easy questions out of the first ten. We all understand that and we all agree with you. But that does not mean that the algorithm is flawed. If you had a tutoring student and you asked them to try 5 lower level difficulty problems from the Official Guide and if the student missed several of those you would probably be skeptical of that students ability as well.

You mention the "First 10 questions" myth and say that it has some truth in it. Of course it does, most myths do. Here is the truth of it...The first 10 questions cannot make your score, even if you get all of them right, but they can lower your potential to score if you get lots of them wrong. We have been telling students this for years, because it is true.

You talk about the fact that silly mistakes lower a student's score on the test as if this were a revelation of some dark secret. Of course not. Much of the time that I spend with students is to help them to avoid these avoidable errors. This is a huge part of the test. As I said above every part of the GMAT is a test of decision-making and not a test where the mathematician is automatically given the higher score on the Quant section.

tutor -

Now to the point of the tone. If you look at the experts who have posted on this thread, you can see that both Ron and Ian have been since 2008, I am the newest of the three as I have only been here for 2.5 years. I have made over 1100 posts and been thanked nearly 600 times while Ron and Ian have made more than 2000 posts each and been thanked hundreds of times each. Ironically for this discussion, Ian and Ron are known for understanding the scoring algorithm as well as anyone outside of GMAC and for their postings on quantitative topics.

The entire reason that Ian and Ron even joined this discussion was to share their knowledge with you. Knowledge that they have gained over many, many years of looking deeply into the questions that you are exploring.

In other words, they, who are in a way are your competitors, since they teach the GMAT - as do I - have taken time out of their lives to help you to understand this subject a little better. I for one would appreciate it if you would adopt a slightly different tone. A more professional tone if you like. No one is attacking you. We have been trying to share information with you. So please look at it in this manner and do not get defensive.

My tone, here has been very professional, so if you respond to me in the same way that you have to Ron and Ian we will know that you do not want to learn things so that you can help your students, but rather that you just want to pick a fight with the test itself and with those who teach it.

Tutors who bang their heads against the wall figuring out a method to "game the game" instead of actually assisting students with quantitative and verbal strategies and fundamentals immediately lose all credibility in my eyes.

For some of us, the GMAT is actually a tool to help us get into our dream business schools, not some Rubik's cube for nerds to obsessively dissect as a hobby.

Since experts are here, let me ask a question (stemmed from Ian's post saying "If the questions on which you guess are all very hard questions, that won't hurt your score much. If many of them are easy questions, that will hurt your score a lot. ")

For a student who typically score around 750, would the score be more negatively affected if he/she missed a easy question (say 500-600 level) or a hard question (say 700-800 level), assuming the question is not experimental?

My guess: if the student's overall performance is indicative of a 750 level, the CAT will treat an occasional mistake at a low-level question as an outlier, which does not count much. so missing a "on-level question" (a 700-800 level question) will be more severe. Just guessing....
If the answer is the opposite - missing easier questions weights more, then the implication may be that it's more important for a 750-level student to avoid silly mistakes on easier question than to instead take that extra proof-reading time to tackle hard questions.

I believe you experts have personal experience on this issue. Your insights are greatly appreciated.

Thulsy - the folks at GMAC have actually answered this question just recently in their "ask the test maker" forum.

Here is what they said:

"Yes incorrectly answering easier questions will lower your score. The raw IRT score is a value that takes into consideration the item difficulties and the responses to all the questions that were administered. This IRT score is then transformed to the historic GMAT scaled score with a proportional deduction for questions not answered. This is not the same as a penalty for guessing. If a question is very hard (say a 750 question and you are really a 650 test taker) and you guess incorrectly, the question will not count much. If a question is very easy (say a 550 question and you are a 650 test taker) and you answer it incorrectly, again, the question will not have much weight. If however, you miss many 550 questions, then this set of questions will count more when all the responses and difficulties are evaluated.

I added the bold type to their posting as this is what really addresses your question.

Here is what I said in response -- sort of an interpretation of this GMAC wording.

"This confirms that one of the most important things that a test taker can do is to focus on correctly answering the questions that she can/should get right. A single false positive or false negative does not matter much. A string of false positives (a student correctly answering a question far above her level) is very unlikely since that would mean that she is indeed capable at that level. It is a number of false negatives that can skew the score, since it is certainly possible for a student to make a number of errors and miss questions that are within her level of ability. For example, if a test-taker scores 100 points lower than expected then one explanation could be that the test-taker made several avoidable errors on questions below her level of difficulty."

And your implication is right, the first thing a test taker can do is to limit silly mistakes. You can only be hurt so much by missing several 95th percentile questions. After all a 51 is 98th percentile so missing questions at this level may take you down to a 50 scaled score (92nd percentile) or so.

But essentially in the (GMAC example above) if you miss lots of 550-level (50th percentile questions) when you are otherwise a 650-level (80th percentile) scorer - as the GMAC quote tells us - "this set of questions will count more when all the responses and difficulties are evaluated."

Does that help to answer the question?

Completely agree with Ron!

This will be my final word on the subject. I never have my students think about the difficulty level of any question. I only speak about these things in the context of making sure that you avoid "silly mistakes" on test day. Some people mistakenly hurry through questions that they think are "easy" in the search for "harder" questions --as if they can recognize those. My point is simply that students should take care to avoid silly mistakes that cause them to miss questions that they would otherwise get right.

Wow GMAC said it itself:

This confirms that one of the most important things that a test taker can do is to focus on correctly answering the questions that she can/should get right. A single false positive or false negative does not matter much. A string of false positives (a student correctly answering a question far above her level) is very unlikely since that would mean that she is indeed capable at that level. It is a number of false negatives that can skew the score, since it is certainly possible for a student to make a number of errors and miss questions that are within her level of ability. For example, if a test-taker scores 100 points lower than expected then one explanation could be that the test-taker made several avoidable errors on questions below her level of difficulty.

That was exactly what I was saying, despite the fact that some of the experts here were claiming that a string of false negatives is 'not very likely'. Notice GMAC did not say a string of false negatives is 'unlikely' like they said about a string of false positives.

Thulsy,

you should not worry about question difficulty on the exam BUT your should worry about it in your preparation. For some students, improving their score boils down to improving their accuracy (the ability to solve correctly a question at your level or below). Analyze your mistakes on GMATPret tests and see if you make silly errors on questions below your level and train strategies to avoid those.

Another thing you should know, that is not in the above quote, is that the scoring algorithm is more sensitive to a string of false negatives in the begining of the test when it is 'making its mind' about your level. That suggests that you should train to be especially careful and focused with the initial 10-15 questions on the test. That doesn't mean that you can do whatever you want at the end of the test, but be especially careful in the begining. A string of mistakes below your level in the begining will swing the scoring algorithm badly towards giving you lower level questions later and you won't be able to earn your true level score even if you answer those correctly.

I've shown that effect with GMATPrep tests, despite the fact that the experts here were claiming that this is not true and that it doesn't really occur in real life. Many people taking GMAT get scores quite lower than their practice scores with GMATPrep exactly because they have higher anxiety on the test day and they swing the scoring algorithm in the begining of each section towards lower scores with a string of careless errors below their true level.

The quote that you cited above is actually from me. The other quote in my earlier posting is from the GMAC psychometric team.