How do I tell him I want to go on a date?

Do you really though? The way he asked you out was immature and so is the way he is behaving. You were ready to accept upfront and then he backpedaled and sabotaged himself. He is the one who made all of this awkward, you didn't do anything wrong. It's a red flag that he isn't really ready to date at all. But if you want to go out with him anyway, then the only solution is to be assertive and take control. Tell him bluntly you have decided to accept and give him the time and date you want to go out. If you aren't comfortable with that, then take it as a sign you aren't compatible and move on.

Just ask HIM to coffee.

If dating is fine and any upsets between the two of you on a personal level isn't going to interfere with your professional lives, then why not call him, invite him out for coffee and tell him about your two week holiday.  If the coffee session goes well and gets you back to where you were before the holiday - suggest a date with him the next time you have coffee together.  However, be wary.  This guy has gone all silent and stand-offish because he thinks you've rejected him.  That's not very adult behaviour from him is it.  Try a couple of coffee meets before you invite him on an outside-work-date.  If you just do nothing, you'll get nothing and more of the silent treatment won't you.

Just walk up to him and say you have an answer to his question and that you would like to get coffee if he still wants to 

People go to work to work, not play the dating game.

Guys don't like games so be blunt and just ask him. It'll show him confidence and that's an attractive feature.