My 3 yr old daughter's dad wont let me have my child back!?

My daughter's dad (previous relationship) has only been a parent when it is convenient. He has previously spent no more than 4 days a month with my daughter. His parents were always the ones to pick her up and 50% of the time dropped her off. When my daughter would see her father, he'd also go out and get drunk all night because his parents would care for her. He was not allowed to take her to his own place because of his room mates (maybe they don't like children?). Regardless I still let his parents take her on holidays, no matter what holiday it is because I have always wanted them to be a part of her life. i have never once turned down them seeing their grand child or him seeing his daughter. Not even 1 time.

After giving birth to my youngest baby, my husband and I had decided to go to California where we ended up moving (from MN) A big part of that was that my husband needed a different environment. Where we lived there had been shootings and robberies. We have family out west.

My daughters dad suddenly was very upset because his 2 days a month were no longer at his finger tips. (he thought he'd legally be able to claim her seeing her 2 days a month). Finally the two of us came to an agreement that she'd visit him for 2 months and she'd be with me for two months. And I was very lenient on this because I wanted her to be able to see her family.

He came to pick her up with his mom on November 3rd 2012. He handed his notarized copy of our agreement to me. I had mailed my about 5-7 days later. I informed him that id be getting my daughter on the 11th of January because pricing was too high on the 3rd but that he could still get her back as planned. I booked the flight to go there and the one back with my daughter and I... (2 seats).

I go to my moms in Wisconsin. and a police officer knocks and serves me papers. Saying my daughters dad is going for sole custody. I called him and he told me that I can not pick her up as planned because that's what his lawyer advised. He also would not allow for me to go see her and say hi,

I didn't even get to see my baby :(

He has her in day care 5 days a week and I am a stay at home mom so she is with me allll day when she is here. She has her own room and tons of toys here and they all just sit her collecting dust. Her sisters are very very sad. They cry for hours every day.

Help me! What do I do? I was all for joint custody and equal rights but now, I just want sole custody. I do not trust this person at all.

I can't afford a lawyer just yet. I may be able to afford on in February.