Do you get annoyed when someone comes over and starts talking to you when you are eating your lunch?

I would hate it too. Bring your phone and put ear buds in, even if you don't have music on. Or bring a book. Or even just tell him you have a headache and need some quiet. I would do whatever I had to lol. You can't enjoy your lunch when you're nodding and replying to dumb stuff between bites.

Patrick, first, I would like to note this is the Etiquette section and veiled profanity is not only offensive, it is a sign of a weak vocabulary.

That said, to your direct question: Yes. However, I choose to go out to lunch, so I avoid the familiarity of the type of coworker you describe. Even to go outside, to your car, the lobby - all of those are options. Some of these folk you can just ignore and they will take the hint; with others, you have to come right out and inform them, "I'm trying to take a few moments to relax and enjoy my meal. Would you please excuse me?"