David Cameron confirmed the appointment on his Twitter account.
It has previously been confirmed that Iain Duncan Smith will continue as work and pensions secretary at the head of the Department for Work and Pensions.
The Prime Minister earlier confirmed Priti Patel will be minister of state for employment at the DWP.
She had previously been tipped to be in the running for the pensions minister post.
The Prime Minister is still yet to confirm whether Brandon Lewis MP will continue as housing minister.