Here is a example when you use numbers:

New Jersey's is one of the five highest number of reported cases of Lyme disease in the United States.
(A) New Jersey's is one of the five highest number of reported
(B) New Jersey's is one of the five highest numbers in reporting
(C) New Jersey has a report of one of the five highest numbers of
(D) New Jersey has one of the five highest numbers of reported
(E) New Jersey reports one of the five highest number of

You use numbers because there is a set of numbers and not just one number. (D) is the right answer. But I think you can't use "the numbers of". There should be a modifying adjective such as "five highest" or "varying".

For example varying numbers:
This month in the journal Nature, Justin Halberda and Lisa Feigenson of Johns Hopkins University and Michele Mazzocco of the Kennedy Krieger Institute in Baltimore described their study of 64 14-year-olds who were tested at length on the discriminating power of their approximate number sense. The teenagers sat at a computer as a series of slides with varying numbers of yellow and blue dots flashed on a screen for 200 milliseconds each - barely as long as an eye blink. After each slide, the students pressed a button indicating whether they thought there had been more yellow dots or blue.

Source: NYT

If this post helped you than give me a thanks. =)
I don't understand the second part of your question. Can you give an example?

when you use a 'article' before number then 'numbers' is wrong
otherwise 'numbers' can be fine.

A number of students [ correct] : -> numbers wrong

The number of students [ Correct ] -> numbers wrong