If you've had Late Blight, the spores can fall from the plant to the soil but the spores don't live on in the soil once you've removed and destroyed any affected plants (and potatoes). Leaving infected plants or potatoes lying around can result in reinfection.
If Late Blight strikes again next season it will be a new infection. The spores arrive through the air, travelling on the breeze. If you want to be doubly tripley sure, you can remove and replace the top couple of inches of soil.