Do USA (or other country) have non-voting territories, part of the country without voting rights?

Puerto Rico is non voting in the Presidential election.

Yes

Those that reside in US territories can vote in presidential primaries but not the general election

They are exempt from federal income taxes but do pay into SS and Medicare

The inability of the Puerto Rico residents to vote is contentious considering they were granted US citizenship in 1917

Puerto Rico AND Guam.

IIRC residents of DC cannot vote in federal elections either.

Only residents of US states can vote in federal elections. Residents of US territories can vote in local elections, but they do not pay federal taxes and that is why they can't vote in federal elections.