Should I worry that my cat won't use the litter box? And, what can I do about it?

Over the last week we got our first snow fall. It was nothing big -- just an inch or two -- but it, and the cold was enough to keep my cats inside. Up until that point, they were spending a big part of the day, and night, outside.

One cat was a feral kitten, the other a stray, and both are very fussy about territory. Back when I kept them inside (different times), each was very fussy about using the litter box -- fussing to bury any leavings, etc. The cats have been out and about for a year now, and a very particular about where they do their business outside -- in the good weather they would travel at least a few gardens away to do anything, and would certainly never do anything in my garden. Now both are stuck inside, and they are together, and they are having a problem reverting back to using their litter boxes.

The female is not so bad -- she will use the litter box -- but, still will only go when she absolutely has to. The tom just will not use the litter box. He will wait several, even 12, hours before forcing himself outside to brave the wind and snow to find a place to pee or poop.

Should I be concerned? And, what, if anything, can I do to ease these guys back into using the litter box(es)?

Btw, I have two litter boxes, side by side, in a room dedicated to the cats. Both cats will eat the food and drink the water in this room, and will use the climbing tree, but, when it comes to the litter boxes, the tom just will not use them. Also, it is just not practical to put a litter box anywhere else, and I do not think it would help RE the tom -- if he is not going to use the existing setup, he will not use a litter box, period.

It is going to get a lot colder and snowier over the next few months. Will my tom break down and just do it, or could I be looking at an eventual health crisis caused by his holding on too long?