I have the same concerns as you and spoke to one of the Australian docs and they have seen numerous clients conceive on finasteride.
Bradley Cooper's highly likely on finasteride for years and he had kids a year or two back. You can stop for two months if you're worried and get back on after you conceive is the consensus. Not my personal experience, of course; but based on listening to many others.
