My ex employer gave me an amazing letter of recommendation and I found that he was giving me a bad reference on the phone, help!?

This reference must be used because he is my supervisor from the only contracted teaching job I have.

I was told he is telling people he "had concerns" while I was there and "wasn't surprised of my leaving"

My letter of recommendation is extremely positive and shows I was up to the challenge of starting late in the year, I took full charge of myself students caseload and assistants and organized my duties successfully and I am a very positive employee.

There was one meeting that we had he went over concerns he had with me, he told me I had to sign not because I agree with the document but because I saw the document. I asked on that bc I didn't agree with some things on the list brought up. 2 weeks later he asked for my resignation.

Since then I cannot get a job.