I have a guy that keeps staring at me in class?

Feels weird to be posting but I figured I'd ask and see what people say. I'd say I'm considered unattractive to most people so this is why I'm turning here instead of talking to the dude instead, lack of self confidence I guess. I have noticed in my class that this guy will glance at me and look away, but I had not really thought much about it until last week when he glanced at me 5-6 times in our class and every time he'd quickly avert his gaze. It took me aback when he did it because I didn't realize he was staring at me, I figured he was looking out the window until we made eye contact a few times.

He's a relatively confident guy from what I have seen, opening answering questions in class and we have briefly talked before about nothing important, it was like a sentence both times when I made jokes about something that happened in class and he happened to be there. I would figure if he was interested in me he would openly ask me out? Not sure, figured I'd get some guys and girls opinions and figure out where to go from there.