How can I get rid of the wasps living in my small birdhouses?

The wasps will die off in winter or go an hibernate in various narrow spaces. Most likely you have paper wasps, and all but fertilized queens will die off.

So all you have to do is wait until it has been decently cold, then you take the birdhouses down and clean out the nests and whatever the wasps have left behind. In the case you have yellowjackets (where more workers can survive through winter) just take the nest down on a nice cold day. The cold anaestetizes wasps and they can't move and even less fly. There is no need to kill them with chemicals, for if you expose wasps to freezing cold they will die. I would think it's not very likely that you have yellowjackets, since they want bigger spaces.

Even if you would have had wren nesting in them, you need to clean out the bird houses every winter anyway.

You are saying that the birds made their nests in the bushes. My experience tells me that birds do not like bird houses, but you may know differently. As for wasps, there are sprays. Other than that, if you could get close enough without getting stung, you could tape the entrances shut.

You need to smoke them out, with pure heavy smoke, then put a box of Arm and Hammer Baking Soda in the houses, cover the boxes with a film of Saran Wrap, to keep the birds out of it, they will not eat it, this is what my neighbor did this summer, and they still have not come back.

Wasps and Bees hate smoke, it also makes them sluggish so you can get them all out, and not be stung, but wear gloves to be safe.

Good luck

You can rub bar soap on the inside of the bird house and it will keep wasps from building. First you need to evict them...good luck with that.

Call an exterminator... now! I love all creatures, but wasps are dangerous if they're too close to your house.

go to any hardware store and buy a can od spray to kill them. you can use it from 20 ft away. follow the directions and spray at dusk when they are in the nest.