Plugging in numbers is a more reliable approach for remainders questions than for questions on pretty much any other subject - there's a whole area of math that you don't need to know about called modular arithmetic that guarantees remainders follow certain patterns, so what's true for a couple of sets of numbers is usually true for other numbers.
One fact from modular arithmetic that can be useful to know: you can add remainders, as long as you're dividing by a fixed number. That is (to take an example), if the remainder is 2 when you divide x by 7, and the remainder is 3 when you divide y by 7, the remainder will be 5 when you divide x+y by 7. You could use that here (along with the first part of the explanation below) to see that the remainder should be 1 using Statement 2.
We can prove everything in the abstract, though it's a bit tedious to do:
if p is odd, and p is the sum of two squares, it must be the sum of an odd square and an even square - that is, it must be the sum of the square of an even number and the square of an odd number. So we know p = x^2 + y^2, where x is even and y is odd.
If you take an even number x, we can write x as 2k, for some integer k. Then x^2 = 4*k^2. That is, every even square is a multiple of 4.
If you take an odd number y, we can write y as 2m + 1, for some integer m. Then y^2 = (2m + 1)^2 = 4m^2 + 4m + 1 = 4(m^2 + m) + 1. So any odd square is 1 more than a multiple of 4 -- that is, it has a remainder of 1 when you divide by 4.
If we now add: p = x^2 + y^2 = 4k^2 + 4m^2 + 4m + 1 = 4(k^2 + m^2 + m) + 1, we see that p is one more than a multiple of 4, so the remainder must be 1 when we divide p by 4.
For online GMAT math tutoring, or to buy my higher-level Quant books and problem sets, contact me at ianstewartgmat at gmail.com
ianstewartgmat.com