It's an interesting question, and I think it's an important one to ask as a launching pad for any discussion about dress standards and why they tend to fall more heavily on women than on men. The following response is nowhere close to exhaustive, and it is a bit generalized, but I want to include it in this post to make a broader, more important point about modesty and how we talk about it - and ought to talk about it differently - in the Church:
It's evolutionary (a survival of the fittest instinct to attract men and ensure protection), cultural (look at the fashion and celebrity industries, where "innovation" and "attention" are paramount), biological (there is a physiological difference between women's breasts and men's chests), power-political (most communal leaders throughout history have been men, and they think more about women's bodies than about men's bodies), etc. It's a complex, fully human issue, and it is influenced by just about every aspect of communal life.
It's not a simple issue that can be fixed easily, but I like the concept of leadership teaching the correct principle, without specifics (modesty meaning moderation in all things, not just those related to sex and how we clothe our bodies), and governing ourselves. In that way, this topic is no different than tithing (individuals determine how to pay, based on a general principle), Word of Wisdom (individuals determine how to be spiritually and physically healthy without unnecessary addiction), church attendance (individuals determine how much time they can spend in church-related meetings while maintaining a proper balance with family, job, community, personal health, etc.), and on and on.
True modesty allows us to do and be more than we can at any extreme - and that's interesting to consider when our ultimate goal is to be "perfect" - meaning "complete, whole, fully developed". Extremism inhibits that type of perfection, while modesty allows it.