Red x over internet connection randomly?

I've been having a strange internet issue at my apartment for college. Sometimes I will be browsing the Internet at the other end of the apartment when I'll randomly get a red X over the Internet connection icon on the taskbar, even though the Wi-Fi on my laptop is on. Even after flipping the switch and looking through the Network and Sharing Center, I cannot resolve the red X and have to restart the system. Once I do, I get my internet back. Sometimes, it doesn't happen at all. Other times, like yesterday, it happens throughout the entire day every thirty minutes or so. It seems to vary with how well my cable is coming through. (We get both internet and cable through Comcast).

I've only observed this issue in my bedroom, which is on the second floor at the opposite end of the apartment, probably the furthest point away from the router and modem. However, if I go to the living room where they are located, I don't ever recall losing my connection in such a manner. Whats more, back home, where I have a wireless internet connection, I have no such issues with my laptop, no matter where I am in the house. This seems to suggest that it isn't the laptop causing the problem.

So I'm wondering, is it possible that the router and/or modem are causing the issue? And if so, what can I do to resolve this issue? As much as I don't mind going to the living room to use the internet, I would like to be able to have some kind of privacy when it comes to doing school work.