Hi rakaisraka,
This question is based heavily on algebra patterns. If you can spot the patterns involved, then you can save some time; even if you can't spot it though, a bit of 'brute force' math will still get you the solution.
We're given the terms (X+Y), (X+5Y), (X-Y) and (5X-Y). We're asked for the probability that multiplying any randomly chose pair will give a result that is written in the format: X^2 - (BY)^2.
Since there are only 4 terms, and we're MULTIPLYING, there are only 6 possible outcomes. From the prompt, you should notice that the 'first part' of the result MUST be X^2....and that there should be NO 'middle term'....which limits what the first 'term' can be in each of the parentheses....
By brute-forcing the 6 possibilities, you would have...
(X+Y)(X+5Y) = X^2 + 6XY + 5Y^2
(X+Y)(X-Y) = X^2 - Y^2
(X+Y)(5X-Y) = X^2 + 4XY - Y^2
(X+5Y)(X-Y) = X^2 + 4XY - 5Y^2
(X+5Y)(5X-Y)= 5X^2 +24XY - 5Y^2
(X-Y)(5X-Y) = 5X^2 -6XY + Y^2
Only the second option is in the proper format, so we have one option out of six total options.
Final Answer: E
GMAT assassins aren't born, they're made,
Rich