Are there any jobs with babies in a hospital that doesn't require state licensing?

No. To work directly with babies you must have some sort of licensing in order to be allowed in the same room as infants. You would need to be a nurse or CNA or LPN. But typically I think only nurses and doctors are allowed in the room where infants are kept.

Try to become a LPN and then later go for nursing if you like working with patients. Nurses typically have to have college training for years to have a specialty like a labor and delivery nurse.

You might consider becoming a Unit Secretary or Unit Coordinator. These are secretarial type positions that help make sure orders are sent to the Pharmacy, everything is entered in the chart, etc. etc. They usually do not have direct patient care but you could work in the Nursery