Tweet by a Labour MP
So I found an MP, David Lammy, who tweeted the following:
Theresa May's £120 million "Festival of Brexit Britain" is historically illiterate.
The Labour government's 1951 Festival of Britain marked a new era of growth and international cooperation. The opposite of where this Tory government is taking us.
The tweet ended with a link to an article by The New European.
Article by The New European
That article starts with the quote below, but it does not mention who spoke or wrote the words in quotation marks:
The prime minister has announced plans for a “festival of Brexit Britain” in a bid to quieten discontent from MPs at the start of Tory party conference.
Other than that, the article does not present any quotes by the government directly linking the festival to Brexit. It does, however, end with the tweet at the beginning of this answer. Well, that's just going in circles.(?)
Luckily, the article gives us another lead:
The idea was originally put forward by leading Brexiteer Jacob Rees-Mogg, who said earlier this year that it would be a “huge celebration”.
The Express quoting Jacob Rees-Mogg
There is this article by the Express, dated January 21st 2018 (whereas the other articles are all dated September 2018 or later) and it contains the following quotes on Jacob Rees-Mogg, who is a senior Conservative MP and vocal Brexiteer but not in the government (emphasis is mine):
North East Somerset Tory MP Mr Rees-Mogg said: “A Festival of Brexit would be excellent. There should be a huge celebration and in the spirit of friendship of our European neighbours, upon leaving we should drink lots of champagne to say that though we may be leaving the European Union, we don’t dislike Europe.”