In two minds about having another baby?

I'm 24 and I have a four year old child who I share the care of with her father, and whilst I love her, I find her very difficult. I work a lot in a job that I love (it would be flexible for family life as well). I have two years left of my second (very demanding) uni degree. I also have some travelling that I want to do (although I have done a bit, I have a few more destinations in mind)

My experience of parenting her was negative when she was a baby, as I was a young single mother. I find that I am easily stressed by parenting.

My now partner is desperate for a baby. It is very important to him as a person to be a father, and it is important in his culture to be a father. He has asked me to give him a baby as he is 30 now doesn't want to wait for a long time. He loves children and he is great with my daughter. I know that he'll find joy in it, and he will help me.

Can I do the above things with a child?

I'm worried that it will negatively impact my existing child, as she may be confused as to why she doesn't live with me all the time and feel hurt by this...

I'm worried that I will get negative responses from my very conservative/judgemental family.

What do you think? Please give me some logical advice.