In a quadratic equation, if the roots are say p and q, then the negative of their sum which is -(p+q) is given by the coefficient of x and the product of the roots which is pq is given by the constant term.
For the quadratic equation given above, roots are a and b and coefficient of x is -1. Also constant term is 1.
So -(a+b) = -1 and ab = 1
Or a+b = 1 and ab = 1.
Now a^2+b^2 = (a+b)^2 - 2a*b = 1^2 - 2*1 = 1 - 2 = -1.

So algebraically, we are getting -1 as the answer.
Don't look for actual values because only imaginary numbers will satisfy the condition a^2+b^2= -1, because in real numbers squares are always positive and so their sum can never be negative.