In the UK, are you allowed to accelerate when a car is trying to overtake you?

So I was following this driver up a single carriageway road which was a 40 mph zone. He was driving very slow, about 25-30 mph even though it was very light traffic. After about 1/2 mile I saw that there was no oncoming traffic and I determined it was safe to pass him. I indicated to pass him, pulled out and easily pulled alongside him, at which point he seemed to be excessively accelerating himself, at which point I got scared because I didn't have the power to get past him so I blew my horn to make sure he knew I was there, but he just ignored me and I had to brake and fall in behind him! So this moron happened to be going to the same destination as me, and he got out of his car and started shouting about what was my problem. I told him I was trying to overtake and he was accelerating, creating a dangerous situation. His response was "It's a 40mph zone!". I'm not sure what he meant by that. Was it that he thought I was breaking the speed limit? I'm not sure if I was or not tbh at that point because I was panicking and just trying to get into a safe position in front of him and I wasn't looking at my speed; but in that case he's a hypocrite because if I couldn't get past him he was obviously doing the same speed as me! Or if he meant that he was allowed to speed up to 40mph even though I was overtaking him, which I pretty certain is wrong because my understanding is you're supposed to maintain speed and allow the person behind to overtake. Who was in the wrong?