Rishi Sunak cannot be held “personally responsible” for the Tories’ dire poll ratings, and the party must now stick with him as leader until the election, former Cabinet minister Sir Jacob Rees-Mogg said.

The Prime Minister has faced speculation about a possible Tory revolt against his leadership, with plotters lining up Penny Mordaunt as a potential replacement.

But Sir Jacob said it was “inconceivable” for Commons Leader Ms Mordaunt to take the job, and the Tories “need to just get on with it” with Mr Sunak in post.

Labour has an average poll lead of around 20 points, fuelling Tory unease in a general election year.

But Sir Jacob said: “In defence of Rishi Sunak, it is quite hard for a leader at this stage… to be significantly more popular than his party.

“The Conservative Party’s popularity fell before Rishi Sunak’s did, so I don’t think we can hold him personally responsible.”

Former defence secretary Ben Wallace has said it is “too late” to replace Mr Sunak and Tory candidates at the election just had to “march towards the sound of the guns”.

Sir Jacob echoed that view: “In an election year you need to present to the voters a leader who can be the prime minister for the next parliament. We need to just get on with it, and so I think Ben Wallace’s advice is wise and we should listen to it.”

He told Times Radio the Tories needed to “deal with the current situation and not what might have been”, adding: “I think Rishi Sunak is an honest, decent and intelligent man who is doing the job effectively.

“Times at the moment seem to be against him but things have changed before.”

Cabinet minister Lucy Frazer said Mr Sunak was “the right man to lead the country”.

Asked whether Ms Mordaunt would be a good leader, Ms Frazer told BBC Radio 5 Live: “We’ve got lots of excellent talent on our benches but the Prime Minister, who I worked with when he was chancellor and obviously I’m in Cabinet with now, I think does an outstanding job.

“He is full of integrity. He has a plan which will deliver, which is already delivering, and has huge knowledge about the economy…

“So I think he’s absolutely the right man to lead the country and he has shown he can deliver across the board, but importantly on that key issue, which is the economy.”

Ms Mordaunt told reporters she was “getting on with my job” when asked if she supported the Prime Minister on Tuesday morning.

Reports at the weekend suggested she was being lined up as a unity candidate acceptable to both moderates and the Tory right if Mr Sunak faced a no confidence vote.