For those who are in congregations with leadership that generally is patient, humble and not into unrighteous dominion, there is a vibrancy that draws and keeps and empowers people. However, in areas where the leadership is controlling, overly-demanding and/or prideful (and even coercive), there is an apathy (and even discord) that repels and oppresses people.
When people are raised in or converted into the first situation (or, at least, have lived in such an area), they tend to be more positive about "The Church" - but when people are raised in or converted into the second scenario (or, at least, have lived in such an area), they tend to be more negative about "The Church". Also, since human psychology says that one bad experience will be valued the same as six or seven good experiences, a bad leader can unravel all the good done by multiple good leaders - past and present.
I've said many times in many places that I believe one manifestation of the "genius of Mormonism" is its reliance on lay members leading at the local level - but I also have said that same administrative structure is the biggest weakness, as well.