Dare I inquire about unonscious hatred and mistrust?

One particularly interesting feature of prosopagnosia is that it suggests both a conscious and unconscious aspect to face recognition. Experiments have shown that when presented with a mixture of familiar and unfamiliar faces, people with prosopagnosia may be unable to successfully identify the people in the pictures, or even make a simple familiarity judgement ("this person seems familiar / unfamiliar"). However, when a measure of emotional response is taken (typically a measure of skin conductance) there tends to be an emotional response to familiar people even though no conscious recognition takes place3.

This suggests emotion plays a significant role in face recognition, perhaps unsurprising when basic survival (particularly security) relies on identifying the people around you.
-http://en.wikipedia.org/wiki/Prosopagnosia