Why exactly was the concept of 'Social Darwinism' abandoned by cultural anthropologists?

Because it was a misreading of the theory of evolution and, as such, had no evidence base

Primarily the advent of civil rights. Social Darwinism existed prior to even Darwin. The Spanish thought that they enjoyed a higher status than the inhabitants of South America because they were superior to the indigenous people there. Not oddly, it was the Pope who told them otherwise, but they didn't want to listen.

Because the fruits of that philosophy came very quickly. Bad fruit comes from a bad tree.

It's a slippery slope to fascism and eugenics.