The party would bring in a levy on additional homes based on the property value, it was announced as the Labour Conference started.
Money raised from the tax would help tackle homelessness, Labour said.
The policy would target properties used as holiday homes, as opposed to those rented out by landlords.
It’s estimated the tax would average £3,200 a property and raise around £560m.
Shadow housing secretary John Healey said: “As part of the next Labour government’s plan to rebuild Britain, we will introduce a levy on second homes used as holiday homes to help homeless families.
“Labour will act to put a brake on the growing gap between Britain’s housing ‘haves and have nots’.”
Polly Neate, chief executive at housing shelter Shelter welcomed the proposal.
She said: “Our housing crisis has reached such depths that tens of thousands of families are being tipped into homelessness and forced to live in temporary accommodation.
“This scandal must be addressed, so it’s good to see this proposal to fund vital homelessness support services.
“At the same time, we must tackle the root causes of homelessness head on, including the devastating drought of social homes.”