I mean think about it.
Evolutionarily speaking, one would naturally want to be wary of people from other caves/fields/villages/forests etc.

One would want to defend one's own women, children, food, territory, weapons, animals etc. against people not in your own social community/structure.

So if one needed to be so xenophobic to survive, why is it such a bad thing when one acts racist or xenophobic in certain situations ?
Is one not just falling back on basic instincts ?