Can I cancel my health insurance at any time?

To elaborate I just enrolled in a new marketplace plan through healthcare.gov since my old policy's rate is tripling this January. My old policy states the 15th is my last day to make changes. However, I do not want to cancel until I get a confirmation letter from the new insurer that I am actually enrolled in my new marketplace policy. Due to last years reports I read where people thought they were enrolled then actually weren't insured.

As I am not familiar with the thousands of pages of the Affordable Care Act. Do I still have the right to cancel my old policy at any time without penalty?