Thinking of delaying downsizing? It could cost you more than you think