Is it 6? To maximize the volume (pi*r^2*h), we need to maximize first r and then h, take radius of 12 and height of 10.

IMO radius should be 5

The cylinder can stand on any of the six faces, creating 3 cases

1) 10 * 8 base and 12 height
In this case Volume would be (Pi)r2H = 4 * 4 * 12 * (pi)
Since on 10*8 base, the cylinder can have a max radius of 4

2) 10 * 12 base and 8 height
Volume = 5 * 5 * 8 * (pi)

3) 12 * 8 base and 10 height
Volume = 4 * 4 * 10 * (pi)

Out of this 2nd case gives max volume i.e. 200(pi), for radius 5.

IMO 5

if it is 6 it wont fit in:D

OA is 5.

Can someone explain how the radius was chosen in each of the 3 cases Alara mentioned ?

Thanks

You need to picture the tank inside the box. suppose the tank was inside a box, then the outer rim of the tank will be the radius and diameter. The radius and diameter falls along the z axis of the box.