What are the dangers of using a foreign banks savings account?

There are countries outside of Europe who all offer incredibly high interest rate returns on your deposits. Although some economies are unstable, and therefore very risky, there are other stable countries who offer very good rates on you deposits, and in many cases Tax Free returns.

As British citizen or European, why would you not use a foreign savings account?