You say that you have problems knowing how to set up the formulas. That is the first problem. The GMAT test is not a math test. Doing well on the test is not a matter of knowing how to generate formulas.

Every problem should be approached as though it were the first time you had seen a problem like this. To a beginner, many solutions are possible. To an expert, there are few options available and there is one, set, standard way to approach the problem. Learning default solution paths will help you reach the mid 600s, but it will never give you mastery over the test.

We start with two empty pools. Accordingly, the initial ratio of filled to empty volumes starts at 0:8. Then, we fill the smaller pool. So the filled part goes up to 3. What happens to the empty part? Does it go down or stay the same? I think we can see that it goes down. So our ratio ends up at 3:5. What happens to the ratio when the water is moved from the small pool to the large pool? Nothing. It stays 3:5.

If you're doing a lot of math, chances are you're doing something wrong.