Is this the proper way to say this in Japanese?

"Nanji ni modorimasu ka?"

or

"Nanji ni kaette kimasu ka?"

seem more proper to me because you are asking when the person is coming BACK.

If you say "nanji ni kaerimasu ka?", depending on where the speaker is, it is also like asking:

"what time are you going home?"

"what time are you coming home?"

"what time are you leaving?"

Isn't jikan for lyk, counting hours. Like, 3 jikan Tokyo kara sapporo made kakarimasu.

I think ur wrong. Nan ji ni kaerimasu ka.

You don't smush the verb and ka together.

I'm gonna ask me a Japanese about this. Trust me, my town is flooded with Asians and Mexicans. I'll find the answer one way or another.

I would suggest the following:

"nanji ni kaerimasu ka".

（何時に帰りますか）

You are correct when you say that "jikan"（時間） does mean "time", but it mean time in a general sense. I'm pretty sure that "hour" in Japanese is just "ji"（時）.

If you want to say "what time?" then the commonly used expression is "nanji"（何時）. I think that "nanjikan"（何時間） is more like "how much time?" or "how long?" as in the "How long was your flight?".

PS:

cb400 is right. It all depends on where or how you perceive where the speaker is. If the speaker is asking the listener "what time will you be back (to where the speaker is)?" then "nanji ni kaette kimasu ka".

But, if the location of the speaker is not relevent or important then "nanji ni kaerimasu ka" is probably ok.

Since you used the word "back", cb400's answer is probably closest to what you're looking for.

Regardless, it's should be "nanji ni" and not "nanjikan ni"