I completely fail to see what's supposed to be pejorative about the word. It accurately describes the condition of those seeking refuge from the destruction of their homes and cities. It boggles me that use of the word is considered racist. To me, it's an absolutely color-blind word.
On a lighter note, during a meeting at work where we were arguing this: "Only among editors can you have an argument over who has the bigger dictionary..."