Well, that largely comes from practice. So, I'd suggest you to solve a lot of quadratic equations.
Anyway, for this problem, let me try to explain my thought-process.
a^2 + 12*a - 540 = 0
I knew that I had to split 12*a into two parts such that they multiply to -540*a^2. Since -540*a^2 is a negative number, one of the two parts should be positive and one should be negative. Then I went on looking for two numbers that multiply to 540 and whose difference is 12.
I tried out these pairs which multiply to 540:
20 and 17, doesn't work because the difference is 3.
30 and 18, Voila! It works! So, the two parts must be 30*a - 12*a.
It might look like a long procedure, but when you've practiced hundreds of quadratic expressions, it is a 5-second thing. So, Practice!