Muslims, why is Mecca and not Medina not the holiest city in Islam?

Hello. I am admittedly not a Muslim, but I am interested in your religion's history and tradition as I find it overall fascinating. I'm just curious, if the Prophet Muhammad settled in Medina (Yathrib at the time) to flee persecution, was accepted, and started his own formidable Islamic community, and was buried there, then why isn't Medina the holy center of Islam? I know that Yathrib was even renamed to Madīnat an-Nabī, which means "City of the Prophet", so why isn't this so? 

Thanks in advance for your answers.