While there remains interest in buy-to-let borrowing from landlords and aspiring investors, making the sums add up is “virtually impossible” in many cases, according to mortgage brokers.