In a survey carried out by the Building Societies Association, 49% objected to developments of 300 properties in their area while 53% were against projects of 100 to 299 new build homes.
The majority of Britons were only willing to support developments of one to 10 properties being built in their local area.
But despite the closed-minded attitude towards building on any large scale, people were more imaginative when it came to living in different types of homes.
One in five people said they would be open to buying a shared ownership property, living in an off-the-shelf kit home, living in a converted retail or office space and even renting long term.
Paul Broadhead, head of mortgage policy at the BSA, said: “These results illustrate the major barrier that the government has to overcome when it comes to boosting housing supply in the UK.
“People are open to new developments and even different types of housing and tenure, but the message is clear ‘not in my backyard’.”
Broadhead wants to see the position of housing minister elevated to a full cabinet role to give the housing strategy the strong leadership it needs from the within the government.
He added: “We also need strong leadership to ensure that we have more of the types of properties that a growing portion of the country is crying out for like custom build, kit houses and shared ownership.
“A bigger and more diverse property market that provides a range of housing options for the UK’s 64 million inhabitants is the shortest route to remedying our current housing crisis.”