Is there a wasp infestation in my house?

I can see the bottom line is you must search every inch of every room at question, looking for any cracks or holes where they could get in from outside. An outside inspection also makes sense. If you find openings you can caulk it with a gun. (Unless some holes may need a little carpentry.)

If your research shows no openings then all you need to do is spray the room, leaving the doors and windows shut and leave it alone while the family goes elsewjhere. Then, you come back and open all the windows and doors to air it out. Then the family moves back in.

I( am not fasmiliar with paper wasps but I am very familiar with ordinary wasps, which make mud nests under the eves. For these, modern sicence has developed the long range spray so you can spray the nests, seemingly innocent. While sneaky wasps may come up from behind and bite you you will not feel it--at first. Then you will feel it bigtime and be scratching for a week. I wouldn't wish that on anyone. Somehow I am thinking paper wasps don't sting. Maybe someday, someone will look something up on the internet?

I did look it up and these are what I call plain wasps. Based on my experience fighting them and getting stung, I would say the are much too hostile to move in. Unless a very rural place was vacant for a long time. You are going to have to deal with it especially for your wife. Or move.

It is very possible you have a nest in your attic and they are getting into your room through a very small hole. Because of the allergy I would recommend calling a pest control company to spray around the eaves and in the attic - especially if he finds a nest,