Should Answers get rid of the reporting system?

No, they should not. Yes, anyone CAN report your answer, but the status of the reporter plays a part in the weight given to the report. Usually two or more reports are required before deletion. You have the right to appeal deletion. The system is a mix of systems and humans. Personally I would like, as always, for there to be more humans involved and for the system to act more quickly, but that all costs. Overall I think the system works well. If you have given an honest and straightforward answer no-one should feel the need to report it, and if they do you have the opportunity to set it right.

Because without the reporting this site would be filled with spam, rants and personal attacks. Most of my appeals have been approved. Maybe you need to review what you consider to be a straightforward answer? Or maybe you are targeting just a couple of people and constantly banging on their questions and they're sick of hearing from you? Or you post the same thing on every question in a group because you're trying to convince everyone you're right? Those seem to be the most common issues. I've been reported several time, like I said, had most appeals approved and never been suspended. So, in some way, your postings are out of control.

The reporting system may have flaws, but it's the best thing we have to remove the real troll activity and violation content. If you take it away, you may as well start calling this site Yahoo Trolling. Remember that it takes at least 2 users to report the answer to have it removed. The lower the 'trust level' of the user, the more people it takes to report to remove it. Believe it or not, it does work pretty well. Most of the people that complain their answer or question was removed actually do deserve to have it removed.

There is nothing wrong with the reporting system. What is wrong is allowing users to post content without ensuring they have read the TOS and the Community guidelines. If you insist on posting content in violation then you run the risk of being reported. Rule is a Question must seek knowledge or advice beneficial to all. Check your questions and see if they are seeking knowledge or posted just to start a debate.