Why do we have a rating system for movies, video games and TV but not books?

Exactly what J-Dawn said.

Furthermore, movies, videos and games are visual in the sense you can actually see what's happening. In books you are reading words and are left to your own imagination.

All above are true and there is a classification system, it is not as universal as the others.

Books stores can pull books that do not meet local standards and restrict sales as they wish.

Because unlike your average moviegoer or video game player, people who read books are intelligent enough not to need such a stupid system.

Does seem unfair, but you have to be good enough at reading to appreciate the book, which is a sort of inbuilt restriction. Not that it is a very good book.

The pearl-clutching f#ckwits who invented the ratings systems for media don't read books. Thank the stars.

Because books are categorized differently. You can imagine that children's books are appropriate for children, teen fiction and young adult books are appropriate for teens to early 20s, and adult books appropriate for adults.

Because books are just descriptions, not graphic reenactments.