"It is with great sadness and remorse that I feel that I have no choice but to submit my resignation as Minister for Children and Families because I have accepted and undertaken these assignments in good faith," Quince wrote on Twitter.
On Tuesday night, British Finance Minister Rishi Sunak and Health Minister Sajid Javid Boris Johnson left the government in protest of the British Prime Minister's leadership.
Seven parliamentarians also left their posts on Tuesday evening.
Ministers published their resignation letters addressed to Boris Johnson on Twitter.
The defection comes after the latest scandal in which the high-ranking member of parliament Chris Pincher was forced to resign after accusations that he had drunk paws on two men.
The Pincher scandal comes in the wake of the scandal that in the British press is called "partygate", and a subsequent vote of no confidence in the Tories where Boris Johnson managed by a small margin.
Laura Trott, who also announced her resignation on Wednesday, writes in a post on Facebook:
"Trust is - and must always be - of the utmost importance.
But unfortunately, it has been lost in recent months ".