The environment secretary hit out at Sadiq Khan, who broke ranks from official Labour policy to call for another vote on leaving the EU.
Mr Gove called the suggestion “interesting and troubling” and warned seeing it through would be a “profound mistake”.
“It was interesting and troubling – that he wants essentially to frustrate the vote that we had two and a half years ago,” Mr Gove told the BBC’s Andrew Marr Show.
“People voted clearly – 17.4 million people voted to leave the EU.
“Sadiq is essentially saying: ‘Stop, let’s delay that whole process, let’s throw it in to chaos.’
“And I think that would be a profound mistake.”
He added that “the Chequers approach is the right one for now” but that “a future prime minister could always choose to alter the relationship between Britain and the EU”.
Mayor of London Mr Khan broke his silence about the possibility of a second vote on Saturday, ramping up pressure on the Labour leadership.
He proposed a three-option referendum, with people being able to choose between remaining in the EU, accepting the government’s deal with Brussels, or leaving with no deal.
But it was swiftly rejected by senior frontbencher Barry Gardiner.
The shadow international trade secretary told Sky News holding another referendum “would be to throw this government a lifeline”.
He added that if the government could not get its Brexit deal through parliament, a general election should be held instead.
First Minister of Wales Carywn Jones also admitted to Sky News there “might be a time” when he will call for a national vote on the EU exit plan – if a general election is called and it produces another hung parliament.
The idea has been categorically ruled out by Downing Street, with Theresa May calling it a “great betrayal” of democracy.
Mr Corbyn has also said it is not something Labour is “calling for” at the moment, and that the “ship has sailed” on remaining in the EU.
But calls have been mounting from over the last few months, including from footballing legend Gary Lineker and Conservative backbenchers.