Mortgage brokerages have said they are finding it harder to recruit new adviser talent, which is at least partly down to the turmoil seen in recent weeks.