Personnel Energy and other people

Has anyone ever noticed that there always seems to be certain people you encounter who just drain you physically and emotionally. Why do we allow this to happen. You would think that we would either eliminate these people from our lives, or protect ourselves from this if we have to be around them. How much do we have to get drained from us before we learn. There are some things we do because we have too, but there is no reason to put ourselves through this. They just make us feel so exhausted, and possibly do things that we would not normally do. So this is basically a note for myself, If I know that I am going to be around someone who I know is going to make me exhausted, I will make sure that I protect myself before being exposed to them.