You are trying to maximize the volume which is the area of the circle times the height of the cylinder.

The easiest thing to do is to draw 2 two-dimensional boxes.

The first box will be 10 by 8, since these are the biggest dimensions. The height of your cylinder will then be 6. Since the cylinder is limited by the dimension 8 [if it is 10 in diameter it won't fit in the box], its radius will be 4.

4^2pi * 6 = 96pi

The seconf box will be 6 by 8 and the height of the cylinder will be 10. Since the limiting dimension is 6 [if the diameter is 8, the cylinder will not fit in the box], the cylinder's radius will be 3.

3^2pi * 10 = 90pi

I know that the last box will be even more limiting to our maximization of the cylinder's volume, but I'll show it to you anyhow.

The box will be 10 by 6 and the height will be 8. Again, 6 is the limiting dimension with radius 3.

Volume = 3^2pi * 8 = 72pi.

You now see that the most volume maximizing radius is 4 when the box is 10 by 8 with a height of 6.

still not fully clear to me . Can u please elaborate it more how u take as radius of cylinder

There's a full discussion of this problem elsewhere on the forums, if you can find it.

In brief: height is a linear function and area is a square function. So, in general, maximizing the dimension related to area will have a bigger affect on volume.

Since the dimension that affects area is the radius of the circle, that's what we want to maximize.

So, let's lay the circle down in the 8*10 side of the box.

To make a perfect circle, we need a uniform radius. The biggest posible diameter we can uniformly make on an 8*10 surface is 8 - if we try to make it bigger, then it's not going to fit along the 8 dimension of the box. In other words, if we choose a diameter of 10, it's not going to fit inside the 8*10 surface anymore.

So, if our diameter is 8, that gives us a radius of 4 and a circle of area 16pi.

With our final dimension of 6 equalling the height of the box, we get a cylinder of volume = 16pi(6) = 96pi.

Though the cylinder is three-dimensional, it still consists of two circles (the base and the top) with its height (volume) in-between. For the cylinder to fit into the box, the diameter of its circular top must correspond to the width or lenth of the box with a height that matches the box (once again, in order to fit in).

The the box measures 8 width by 10 lenth by 6 height, your cylinder's top cannot have a diameter (measure across) more than 8. You might think that it can measure 10, but it cannot because it won't fit the width of the box, which is 8. So, you are always limited by the width of the box. Here the width/diameter is 8. Half of the diameter is radius, which is 4. the formula for area of a circle is radius squared multiplied by pi. And to get the volume of the cylinder, you multiply the area of the circular base or the top by the height of the cylinder.