I am just so afraid to lose!!!!?

You only seem to thinking about your lovelife and not anybody else, he's abusive and a horrible person to you yet you still go back to him? I would not trust this man around my kids, I would say you were being a bit selfish, you could do a lot better and you don't need this man, I would just get rid, good luck.