According to a report from Sky News, this leaves it likely the struggling lender will seek additional funding from hedge funds backing it.
Virgin Money is the latest institution to rule itself out of bidding for Co-op bank after TSB pulled out earlier this year.
As Mortgage Solutions reported in March, the bank posted a pre-tax loss of £477.1m in 2016, against £610.6m in 2015.
The results showed it was relying on performance in its mortgage division to help turn around its fortunes.
The retail mortgage book grew by 5% to £14.1bn from £13.4bn in 2015, representing a 1.1% market share with £3.1bn of gross lending last year and is targeting annual increases of £1bn through its broker mortgage lender.
The Sky News report noted that there was still the possibility of resurrecting talks with Virgin Money, but if no new investors for the bank could be found then regulators would be forced to wind-down its operations.
This could involve similar sized banks taking on its four million customers and others targeting loan books.