Low sugar fudge recipe hints?

I need to make some fudge for the base of a peanut butter pie (it's like a giant Reese's cup). I made it last week, and it was good, except the fudge was too sweet. I used a recipe for simple fudge - it was just confectioners sugar, butter, milk and baking chocolate powder. I reduced the sugar slightly, but not too much because I was afraid it would alter the consistency. Is there something I can do to reduce the sugar, like maybe use something in it's place? I've only made fudge that one time, so I really have no idea.

I tried to google recipes, but all I could find were recipes using sugar substitues. I don't want a substitute, I just want it to be less sweet while still having the consistency of fudge.

Does anyone have any ideas for me?