Theresa May has come under further fire from her own side as one of her MPs said the "window is closing" on her leadership and her former chief of staff laid into the government's lack of goal.
She said it was "absolutely" the prime minister's prerogative to carry out a reshuffle, and that she had turned down the move because she felt "passionately about social mobility". "I know that there are always disappointed individuals but they're going to have to live with disappointment".
"Dominic Johnson (party treasurer) stands up and says: 'I love Theresa May, who could possibly want to replace her?' What he didn't expect was about a quarter of the room to say 'yes (we do)'".
"I think she is doing an important job for our country".
Mr Timothy, who resigned from his Downing Street post in the wake of the Tories' disastrous general election showing, said: "It feels increasingly clear that the country is exhausted of austerity, and that public services are starting to feel the strain a little bit".
A new poll suggested there is growing public support for another referendum on Brexit, but the possibility has been shot down by both major parties.
"I think the party needs to set up processes that will enable it to debate its future goal and policies because if that is left until 2020/21, that is going to feel too late in the electoral cycle".
Ms Villiers, who campaigned for a Leave vote in the European Union referendum, said she had long "made the case for compromise and moderation" in the government's approach to negotiating Brexit.
"That means, I think wrongly, to be perfectly frank, that young people from more disadvantaged poorer backgrounds are coming out like for like on the same course with more debt than their better off peers", she said.
The former education secretary said she had been against a review of student finance because "the danger of a review is that you just kick things into the long grass".
The Universities minister said that while tuition fees were an issue "so are living costs".
Mr Walker told the BBC Radio 4's World at One programme: "It seems to happen every two weeks, we have these spasms within the party where there is this rocking of the boat, there is this off the record briefing of the media, and I simply can't understand, really, what is motivating this small minority of colleagues to behave in this way".