This is due to the changes in enzyme expression and steroid production caused by 5AR antagonists like finasteride. Because the effects are not localized, it causes a downregulation of aromatase along with 5AR. I don't recommend you do this at all (especially since I've already done it countless times), but the way one could prove this would be to take high dose finasteride (5-10 mg daily) for a few weeks. What will happen is that your 5AR will be bound faster than your body will be able to react to the changes and your aromatase expression will still be high like it would be if you were still producing normal amounts of DHT. This will leave your hair and skin with high concentrations of both testosterone (and/or androstenedione) and estrogens that result from the aromatization thereof.
To answer your question about why men's skin starts to age as they age.. this is because serum estrogens increase with age causing a decrease in aromatase expression. So, what you end up with high estrogen concentration in the blood is that the body will adjust to lower this... and the only way it can do this is to globally decrease aromatase expression. Thus, you end up with a more acceptable serum concentration of estrogens (or the best your body can do with its adjustment) and no longer a high enough aromatase expression in the skin to produce enough estrogens locally for healthy, supple skin. There can be a multitude of reasons for this, but I'm not really sure why it happens to begin with. For some people, being overweight could cause this (excess fatty tissue will aromatize androgens and increase serum estrogen concentration), but I think a more likely explanation is that an enzyme that deactivates estradiol and estrone may be missing or insufficient. So, basically, when the body is finished using the estrogen locally, it never gets metabolized and eliminated from the body. This stagnates estrogen synthesis... you end up with a bunch of junk estrogens in serum that should be eliminated, but can't be for some reason. So, the body adjusts to lower estrogen synthesis in all tissues to manage the serum estrogens. I actually feel like this is likely the genesis of hair loss. It's what causes 5AR expression to increase and aromatase to lower... in order to offset the effects of excessive serum estrogens.