Do the sailors in the US Navy wear their caps with their working uniforms when inside, or is this not appropriate?

Yes you wear your cover inside or indoors when you are armed and are watchstanding; also for certain ceremonial duties. Outside yes as it is part of the uniform and also flight deck and flight line personnel wear helmets.

covers are only worn in very specific circumstances: when under Arms, during certain official functions held indoors( Pass in Review, for instance)

We always take our covers off when inside a building, unless instructed otherwise.

you never wear hats inside in any service, unless you are under arms

Yes. Those who do are on duty. Those who don't are not.