"Perfect" is defined in Matthew 5:48 as "complete, whole, fully developed".
Based on that definition, the Church isn't perfect, since it isn't "complete, whole, fully developed". At the most fundamental level, our Articles of Faith say as much ("He will yet reveal MANY great and important things pertaining to the kingdom of God.") - and the organizational development continues to evolve.
To me that's so basic and obvious that I just have to laugh inside when I hear someone say that the Church is perfect but the members aren't. I don't dispute it much publicly, but I do address it whenever the situation makes it possible to be productive and not seen as threatening.