Unions have reacted with anger after Government Efficiency Minister Jacob Rees-Mogg issued a call for the “rapid return” of civil servants to their Whitehall desks.

In a letter to Cabinet ministers, Mr Rees-Mogg said they needed to issue a “clear message” to their departments that with the end of Covid restrictions in England, officials should be back in the office.

He argued that ending working from home would bring the benefits of “face-to-face, collaborative working” as well as delivering wider benefits for the economy.

But the move was branded “vindictive” by Dave Penman, the general secretary of the FDA representing senior civil servants, who said ministers were out of step with practice in the private sector.

With up to three-quarters of staff still reportedly working from home, Mr Rees-Mogg accompanied his letter with a league table showing how many staff in each Government department were attending the office on an average day.

“Now that we are learning to live with Covid and have lifted all legal restrictions in England, we must continue to accelerate the return of civil servants to office buildings to realise the benefits of face-to-face, collaborative working and the wider benefits for the economy.

“To deliver this, the Chancellor of the Duchy of Lancaster, Steve Barclay, and I, urge you to issue a clear message to civil servants in your department to ensure a rapid return to the office.”

A Whitehall source quoted by The Daily Telegraph said there was a minority of hard-working officials who were in the office “all the time” while “a silent majority of people aren’t pulling their weight”.

The comments infuriated Mr Penman who accused the ministers of behaving like “Luddites” while the private sector embraced the benefits of flexible working.

“There is no rationale for this. Ministers can’t point to productivity losses, which is why it’s always anonymous sources making the insulting accusations,” he said.

“Ministers’ obsession with ending flexible working and micro-managing the Civil Service increasingly just looks vindictive.

“Jacob Rees-Mogg is the Minister for Government Efficiency, yet that isn’t even a consideration in his correspondence with ministers.

“The private sector has embraced hybrid working, recognising the efficiencies it delivers and competitive edge it gives to employers in a tight labour market.

“Meanwhile, the Luddites in Cabinet insist on micro-managing the Civil Service, which will only deter good people from joining while simultaneously demotivating those already there.”

However Mr Rees-Mogg was backed by the Northern Ireland Secretary Brandon Lewis who said it was “a very good thing” for people to be back in the workplace.

“There’s something different about that face-to-face interaction,” he told Times Radio.

“I think anybody who’s been missing that recognises that, but there’s also the issue, particularly for new staff and younger staff, about being able to become part of a team and get to know their colleagues.

“Actually, that doesn’t work in a virtual world.

“You need to be in an office to be able to learn from others to work with others and get to know each other, so I think people being back in the workplace is a very good thing.”

A Government spokesman said: “Ministers have been clear that departments should make maximum use of office space and progress is being monitored.”