How to tell your supervisor to do their job?

My supervisor always wants everyone to do good even when they bring the whole team down. In the 3 years I've been employed no one has ever been written up for everything and we have people who are chronically late, sleep during their shift, make a mess, etc, etc. I have the pleasure of working the day shift with said supervisor and since April his performance has been really slacking due to him buying a house and issues at home. He sits all day at his desk looking this up for his new house or on the phone with this person or that person and I feel like I'm running the entire show while he collects his near 6 figures to play around all day while I bust my ***. I don't mind the extra workload sometimes if it's not a regular occurrence but I'm becoming increasingly burnt out from the double or even triple workloads for what is now months on end. Is there any way to talk to my supervisor and tell him to step up his game or should I just go right to our Manager and express my concerns there?