Please help me my bf lied about seein his ex?

ok well heres the thing i have been dating my boyfriend for 3 years we have been having problems with his ex that he was with for about 3years has recently been calling and I try to let it go recently i went for a ride with my bf sister and it slipped out that his ex along with a guy friend that my bfs a sister was talking to stopped by the house and wen she realize that she slipped up she tried covering it up saying he didnt know blah blah and i called my bf and he said that he knew about her coming over and he said he went out side and supposidly saw her and walked back in the house this happened about 3weeks ago and he never told me. So i texted him cause he was at work nd while i was doint that his sis was textim apoloigizin to him that she slipped with it and he began to say that he didnt want me to kno cause it wasnt that big of a deal well than if it wasnt than y lie he would kill me if i every even said hi to my ex and feel disrespected what the hell should i do this is the second time hes lied to me about talkin to her or seein her? Is this **** worth it??