Why is it acceptable to call a Arab/Muslim a terrorist?

Today my boyfriend's Jewish Friend's Family publicly called me a terrorist simply because I am Palestinian. Whenever something like this happens, I am called anti-Semitic, because I am speaking up for my self. Why is it acceptable to accuse muslims/Arabs of something, but when we stand up for ourselves, we are shot down and called anti-Semitic?