When did North Americans really start to have an accent different from the Brits?

Probably within a generation or two. 

Due to the lack of quick world travel and the fact that many moved to America and never went back, the language would have evolved quickly. 

However, the accent in the 19th century was not the accent of today—further proving this is a relatively rapidly changing phenomenon.  The late 19th century had the advantages of the first recording devices. For changes like the Great Vowel Shift in the Middle Ages.. they rely on writing and specifically poetry to see which words rhyme with each, etc. 

Accents can also be artificially created. The mid-Atlantic accent is a perfect example. FDR and Katherine Hepburn had them. It’s sound sort of British and sort of American (hence the term mid-Atlantic) and was designed by prep schools to be acceptable in both countries. 

But it fell out of favour after WWII and only exists in movies and media today when you want to give something a sort of old-timey feel. 

And when I talk about accents, it’s in the broad sense. Both Britain and America have numerous regional accents. A non-rhotic New England accent (cah = car fah = far) sounds a lot more “British” than your typical midwestern twang. 

Note that 'the Brits' have many different accents across a far smaller area than North America, or even the eastern seaboard of North America, which is where British people ended up if they did not end up in the Caribbean.

Note also differences in vocabulary, in some cases taken direct from Spanish: canyon; sierra; mesa, Las Vegas, etc.

About 2 years after arriving in the USA you speak what you hear