some of the comments on here are ridiculous. sounds like he is seriously harassing you and your family, which nobody should have to accept, especially if you have kids. speak to the council and involve the police if necessary so they take it seriously. if he's in council accommodation he needs to behave properly or they can evict him.