I think intolerance generally is subject most powerfully to three things:
1) Individual personality, including genetics - I know some members of the same families who are at about every point on the tolerance scale, so it's not just upbringing.
2) Upbringing - Visiting the sins of the fathers upon the heads of the children to three or four generations is a practical statement, imo, of how long upbringing generally can and does affect children.
3) Cultural domination and/or isolation - When both are combined, it is very easy to slip into an us vs. them mentality - ESPECIALLY when there actually is a real history of persecution and reviling.
Mormons really are "hated and/or condemned" broadly in some places. Some of it is brought on by our own beliefs and actions; some of it is fueled purely by bigotry; some is taught and inculcated from the earliest ages.
I'm saddened by that, but I'm more saddened when Mormons hate and/or condemn others - and I've seen that, as well.
Finally, defensiveness almost never helps. A good sense of humor has worked best for me.