Is it possible for child's grandparents/family to stop an adoption?

My husband has a 6 year old little girl that is absolutely the light of my life. She is the little girl I always wanted but never thought I'd have. I met them when she was attending a pre-school that my niece also went to (same age) and I'd see him when I picked her (my niece) up & dropped her off. My step-daughter and niece became good friends and so I would take them on play dates and they went to each others birthday parties & things like that.

Her mother died when she was 2 and her dad has raised her since then, he and her mother were never married and were not together when she died. We married last year. We want for me to be able to adopt her legally and are starting the process for it. He tries to stay on good terms with her mothers family for her but they have made things difficult, when is ex died his ex's sister applied for custody of his daughter without his knowledge and she was given custody (how I don't know) so he actually had to fight for custody of her. He told her grandparents & aunt last week that we were going to start the process for me to adopt her but that wouldn't change anything for them, they would still see her as much as they always have. But they appear to want to stop it and mentioned getting a lawyer.

I've always been on good terms with my daughter's family from her mother. I'm usually who makes sure she has time with them (they have no court ordered visitation, we live in a state with grandparent rights laws). I make sure she remembers her mom, we go to her grave on her birthday (mom's birthday), mothers day, christmas and we keep pictures of her in her room for her. I don't want her to forget her because of me. She also has a half sister from her mother, and we make sure that she sees her often too, that sisters father and us are also on good terms, her dad will let us take her for a few days here & there so they get time together.

I'm respectful of her mom's memory and her mom's family. Around them I refer to her as my step-daughter, anywhere else I just call her my daughter. Right now she just calls me by my name but she has asked a few times if she should call me mom or by my name, right now I've told her to call me what she is comfortable with.

I'm very good to her. I always make sure she has everything she needs, I have the means to support her financially. The medical insurance where I work is better than her dad's so she's on mine. I do things with her all the time things like the zoo, park, dress up I go on some of her field trips with her (and with my niece, they are in the same class).

My niece doesn't live with me. My brother works out of town everyday and her mom works late hours so I help them by picking her up and taking her to her activities. I have a flexible schedule at work, and I'm off 1-2 days during the week days.