Hi archu.mohan,
Many questions on the GMAT can be approached in a variety of ways, so it's beneficial to you if you know more than one way to approach a question. In simple terms, you have two goals for any question that you're going to attempt to answer:
1) Get the question correct
2) Do so as quickly as possible
You can certainly use a "math" approach to answer most Quant questions, but in many cases, THAT way of thinking takes too much time to implement. So you should keep an open mind to all of the various tactics that are available to you. Sufficient practice (including re-attempting past questions using different tactics) should help you to gain familiarity with all the options (and help you to figure out the right approach for any given question).
You'll find that many posts in these forums will showcase a variety of approaches, so you have a great reference here to see the options.
GMAT assassins aren't born, they're made,
Rich