According to this article in a Russian source he did regret the decision:
Yeltsin regrets that he gave power to Putin
Less than a year later, former President Yeltsin regretted handing
over the reins of government to Putin. And, oddly enough, the reason
for this was not some kind of foreign policy or domestic political
mistake, but ... dissimilarity of musical tastes. According to the
first president, "such things are not joked about." In an interview
with Komsomolskaya Pravda, Yeltsin said that he regretted that he
shifted responsibility for solving a number of issues to his successor
Putin. In particular, Yeltsin was referring to the adoption of a new
state symbol of Russia, as well as questions about the burial of
Lenin's body and the prohibition of the Communist Party.
In vain the coast, it turns out, it would have been better to do
everything himself then," Yeltsin laments now. But what confuses him
the most is the activities of the current president, Putin. According
to him, Putin is becoming a symbol of the Communists. And Boris
Nikolaevich is most upset because he "unnecessarily shifted
responsibility to the future president." It would be better to appoint
Chubais, Yeltsin believes, because he always speaks very correctly.
For example, regarding the anthem, Chubais "well said that the
president of the country should not blindly follow the mood of the
people."
By dissimilarity of musical tastes is meaned the return of the Soviet anthem, which was banned by Yeltsin as a relic of the Stalin era.