News from the GMAT Summit
Last week, I attended the annual GMAT Summit, held by the fine folks at GMAC (who own / make the GMAT), and I have some interesting tidbits to share with you.
It really is a myth
You know what Im going to say already, dont you? The first 7 (or 10, or 5) questions are not worth more than the questions later in the exam. Ive written about this topic before but Im going to mention it once again because of something that happened at the conference.
Fanmin Guo, Ph. D., Vice President of Psychometric Research at GMAC, was answering questions after a presentation on the test algorithm. A couple of people were peppering him with questions about this myth and apparently just didnt seem to believe that it could possibly be true that the early questions arent worth more. One of the questioners also made a pretty significant faulty assumption in his argumentsand now Im worried that an article is going to pop up trying to revive this debate. I dont want any of my students led astray on this topic.
First, to understand why the early questions actually arent worth any more than the later ones, see the article I linked a couple of paragraphs back.
Second: here was the faulty assumption that I heard:
You said that the earlier questions arent worth any more than the later ones. So youre telling us that students should spend the same amount of time on every question.
Dr. Guo was saying the first part: that the location of a question on the test doesnt impact its weighting in the overall score. He and the other GMAC folks werent saying anything, though, about how you should take the test.
In fact, it would be silly to spend exactly the same amount of time on every question. Some questions are harder than others. In addition, you have various strengths and weaknesses in terms of both accuracy and speed. There are, in fact, very good reasons not to spend the same amount of time on each question. All Dr. Guo was saying was that the location of the problem in the section is not one of those reasons.
So, if you read something that says that you should spend more time on the earlier questions, roll your eyes and click away. Alternatively, if you read something that concludes that you should spend the same amount of time on every question, drop that source as well. Take a look at the data in my other article to see that GMAC actually does know what its doing and the GMAT is not just a test of how you perform on the first 7 or 10 questions.
GMATPrep offers more data
GMAC has been building more score reporting functionality into GMATPrep to give us a better idea of how we do when we take the official practice CATs. In fact, this capability has already launched! I need to go download the newest version of GMATPrep to see exactly whats offered (and Ill report back to you once Ive done so), but theyve started to offer data for sub-categories such as question type and content area.
They are also beta-testing a new score report for the official test; Im very excited about this. This score report would offer various data points about your real test performance, beyond the basic scores. In other words, you could find out more detailed information about your strengths and weaknesses on the real testvery useful if you need to take it again! They haven't yet finalized exactly what data they'll give, but they're planning to release this early next year sometime, I believe. (They're also planning to charge $25 for the service. In written feedback, I recommended that they avoid any fee, as the only people who will be taking advantage of the service are those who don't like their score...so they're about to have to spend another $250 to take the test again. But I'm assuming that decision probably isn't going to change.)
IR continues to gain traction
While most schools still are not using IR during this years admissions season, some schools are using it and we expect that number to continue to increase. Eileen Talento-Miller, Ph.D., Senior Psychometrician at GMAC, presented the most recent validity studies for IR.
Validity studies measure whether the test is telling us valuable information about the test-taker. The IR section has been out long enough now that Dr. Talento-Miller was able to run a study of IR scores against business school grades to see whether there were any correlations.
Turns out, IR is doing a good job of predicting grades in business school. The IR section is adding to the overall validity of the full GMAT scoresthat is, the new section is helping business schools to better predict who is likely to do well in business school. GMAC also gets requests from some employers who want to know candidates GMAT scores, and some companies are expressing interest in the IR score as well.
Comparisons within cohorts
Dr. Talento-Miller also ran various group comparisons to see when the various test scores (quant, verbal, IR) were especially useful in helping to predict graduate school grades. For students with STEM (science, tech, engineering, and math) undergrad degrees, the verbal section score was the most important predictor of b-school grades. This isnt surprising; these folks are already strong in quant, so the real differentiator for them is the verbal score. By contrast, for those who studied humanities or social sciences in undergrad, all three sections of the test in conjunction were important.
For students in the US or Canada, the quant section is showing up as the most useful indicator of b-school performance. For those coming from Asia and the Pacific, the verbal and IR scores are almost tied as the best predictors, while the quant score doesnt predict much of anything. For those coming from Europe and Latin America, the three sections are roughly equally indicative of b-school performance.
This data may lead you to speculate that business schools may start to compare applicants within cohorts, by undergraduate degree, geography, or other factors. In fact, they already have. In September, GMAC released a tool to business schools that allows the schools to sort and compare applicants by certain factors. (Note: the WSJ article takes a bit of a different slant, but I read the news of this tool release in a whole new way once I heard Dr. Talento-Millers presentation on the way in which different sections of the test have varying predictive ability depending on the candidates background.)
TL;DR? Sure. In short, IR is slowly becoming more important, so Im going to officially up my go for this score recommendation. For next years application season, aim for a 5 or higher, 6 if you want to apply to a top school. And Im expecting that number to go up again in a year. Also, if youre coming from a traditionally high quant group, then your verbal and IR scores are more important factors in determining your chances for admission.
GMAT vs. GRE
Finally, yours truly (yes, me!) presented at the conference, alongside the General Manager of Manhattan Prep, my colleague and friend Rey Fernandez. We recently completed a study that, in-house, we called GMAT vs. GRE: The Smackdown. (No, we didnt really. I just made that up. Its important to have some fun with your job. :))
Our colleague Michael Bilow, data guru extraordinaire, designed this study to answer the question How are you deciding which test to take? Those choosing the GMAT were most influenced by the fact that some schools that do take both tests nevertheless prefer the GMAT over the GREHaas (UC-Berkeley), and Anderson (UCLA) fall into this category. Typically, when a school does prefer the GMAT, it will say so right on its website (that is, the schools are not trying to hide that information).
Respondents who were still undecided were concerned about that fact as well but they were also concerned about the quant portion of the test; they werent as confident that their quant performance would be strong enough on the GMAT. One of the attendees at the conference (not someone from GMAC) pointed out that youre taking the GRE up against candidates who are planning to apply to science and engineering Masters programs, so while the GRE math may feel easier to some, you may not necessarily be able to lift your percentile ranking appreciably. If so, then you might as well take the GMAT, particularly in case you decide to apply to one of the schools that does prefer that test.
Whats the big takeaway?
Im going to leave you with one quote (well, a paraphrase) from Dr. Guo of GMAC: because the GMAT is an adaptive test, your study time is best spent studying questions that are just a bit too hard for your current levelmaybe they take you too long to do or maybe you can sometimes do them but you sometimes make mistakes. Dont waste your time studying only the hardest questions, as you wont even see those questions on the test unless you can answer the lower-level questions correctly (and efficiently!) to lift your score up to the stratosphere.
Happy Studying!