TLDR; Brexit is a large task while it has ostensibly gained support of a vast majority for politicians it is still a large political and administrative task and the civil service is the easiest scapegoat for the apparent lack of tangible progress.
I appreciate this questions is about the opinion of the civil service however, I suspect the the quote is more about political issues and the civil service is being used as a smoke screen.
Firstly it should be noted that pro-brexit politicians and some pro-brexit general public have been peddling the idea that the UK can simply get up and go when it pleases with no problem. This makes the argument that the civil service is messing about very easy because we're still here but yet politicians have agreed to leave. Also there are some people who feel that the result of the vote should have meant instant departure.
Secondly, the brexit department may be headed up by Britain's most staunch brexiteers (or not) with the sole objective of leaving the EU but it can't just leave on its own. Leaving requires all departments to do something, even if the civil service is performing optimally this still requires ministers to take action including those ministers that openly wish to remain and those that are still waiting for the dust to settle before fully committing themselves either way.
Thirdly the civil service is a massive bureaucracy and in general unless there is some kind of major emergency parliament is also rather slow. This is in contrast to brexiteers who want to get things done while they still have some momentum and before people have lost too much interest. Also the quicker they can get brexit to happen the less bad news it can generate before it happens and therefore the less chance of a reversal or second referendum.
As you can see irrespective of civil servants personal feelings there are a lot of issue that have to be overcome both in political and administrative terms. The civil service are simply an easy scapegoat to avoid having to face larger issues