The point at which you have acquired enough career capital to get meaningful control over your working life is exactly the point when you've become valuable enough to your current employer that they will try to prevent you from making the change.