James Cleverly would not be drawn into revealing details of more than 4,000 asylum seekers said to have gone missing after being earmarked for removal to Rwanda.

Labour accused the Home Secretary of appearing to “have lost thousands of people” and failing to get a grip of the issue.

Shadow Home Office minister Stephen Kinnock said the Home Office has lost contact with 85% of the 5,000 people identified for removal to Rwanda.

Mr Cleverly said MPs on his side of the House of Commons “are absolutely united in our desire” to tackle immigration.

Mr Kinnock told the Commons on Monday: “The shambolic incompetence of this Government across every aspect of its disgraceful mismanagement of our country’s asylum system knows no bounds, but today I will highlight a particularly egregious example.

“We already knew that removal of asylum seekers whose claims have been rejected have collapsed by 50% since Labour left office in 2010, but over the weekend it emerged that the Home Office has lost contact with an astonishing 85% of the 5,000 people who have been identified for removal to Rwanda.

“Can I ask the Home Secretary where on earth are these 4,250 asylum seekers who have gone missing?”

Mr Cleverly said: “The mask has slipped with regard to the Labour Party… a quote from the Labour Party that even if the Rwanda scheme were to be successful they would not keep it. That shows what the party opposite really thinks about this.

“They have no plan, they have no commitment, and they even said that if something is working, they would scrap it.”

Later in the debate, shadow home secretary Yvette Cooper pushed Mr Cleverly on the issue.

Speaking during Home Office questions on Monday, Ms Cooper said: “We need the Home Office to have a grip. The Home Secretary gave no answer earlier on the 4,000 people he has lost from the Rwanda list. Can he tell us if he has also lost the 35,000 he has removed from the asylum backlog? How many of them are still in the country?”

She added: “Returns have dropped 50% since the last Labour government, and he’s still not telling us where these missing people are. He appears to have lost thousands of people who may have no right to be in the country and lost any grip at all.”

Mr Cleverly said: “We are driving down the numbers of people in the backlog, we are processing applications more quickly, we are ensuring that decisions are made so that those who should not be in this country can be removed either to their own country or a safer country.”

He added: “The members of Parliament on my side of the House are absolutely united in our desire to get a grip of this issue.”