Infinitives are verbal forms, but unlike tensed verbs, they are not used with subjects. So you're right - they're not really verbs! In fact, they function as nouns or modifiers. For example:
noun: I like to dance. "to dance" is the object.
modifier: I went to dance class to learn the samba. Here, "to learn" is modifying why I went to dance class.
It's unclear what you mean by "we should disregard them," though. Every piece of an SC sentence is important. If your first step in analyzing a sentence is to identify the main subject and verb, then infinitives will almost never be a factor. But that doesn't mean they're not important!
Ceilidh Erickson
EdM in Mind, Brain, and Education
Harvard Graduate School of Education