At the level of abstraction you're describing it, it doesn't sound odious.
Even if you're personally honest, if you accept public office, you're going to risk being accused of misconduct -- for example, by political opponents who disagree with your policy choices and hope to get rid of you by alleging personal faults. There's a good argument that if the state didn't have a policy of defending its officials against such accusations, it would be difficult to get honest, competent administrators to agree to hold office.
Thus, even though such a policy might benefit a crook from time to time, having it in the first place would still serve the public interest, and hence a be justifiable use of public money.
(Providing a solid defense becomes more important the easier it is for accusations of official misconduct to be made -- but this ease is itself a public good).