Wow, this question is no joke - quite challenging for the GMAT.
Here's my reasoning:
There are (10 choose 3) = 120 ways of choosing three books, with no restrictions.
It's easier to find ways to VIOLATE the conditions than to observe them, so let's work backwards.
Our ILLEGAL groups of three fall into two types:
1) All three books are adjacent (a row of three)
2) Two of the books are adjacent and the third is not adjacent to either of these two
There are 8 ways to choose three books, all of which are adjacent. (Essentially the middle book of the three adjacent books can be any of the non-red books below.)
B B B B B B B B B
B
So that gives us 8 illegal groups of three.
Now let's find the number of ways to pick two adjacent books, with the third book a book that ISN'T adjacent to either of the first two.
If I pick the two adjacent books at the end, I could have either
B B B B B B B B B B
or
B B B B B B B
B B B
In either case, as long as I pick one of the 7 non-red books for my third book, I have a two-adjacent book selection. This gives 2 * 7 = 14 more INVALID arrangements.
If I pick two adjacent books, neither of which is a book on the end, there are 7 such pairs. Here is one of them:
B B B
B B B B B B B
As long as my third book is one of the 6 non-red books, I have a two-adjacent selection. This gives 7 * 6 = 42 more INVALID arrangements.
My total number of invalid arrangements is thus 8 + 14 + 42 = 64.
Total - Invalid = Valid, so 120 - 64 = 56
A word of caution, Sana: you're asking a lot of combinatorics questions, many of which are quite hard by GMAT standards. Combinatorics questions are
rare on the GMAT, especially involved ones like this, so I'm not sure this is a productive way to study. (But of course it's up to you
)
