Between January 20th and January 23rd (inclusive)
The Democratic majority in the Senate will only be cemented once Kamala Harris becomes VP, her replacement in the Senate (she was serving as one of California's Senators) is sworn in and seated, and the two new Senators from Georgia are sworn in and seated. If either of the three hasn't occurred, the Republicans retain the majority. So we just need to know when is the latest those events can occur.
Per the constitution, Kamala Harris will be sworn in at noon on January 20th (before Biden is sworn in as President, by tradition), so the transition cannot possibly occur before then.
Exactly when her replacement will be sworn in is a little hard to pin down ahead of time. Traditionally Senators who are going to become VP or President will resign their seat several days or more ahead of time, and then the US Constitution allows their state governor to appoint a replacement as soon as the vacancy arises. President Obama resigned his Senate seat several months in advance of his first Presidency, while his then-to-be-VP Biden waited until several days before Jan 20th. In this case California's Governor, Newsom, has had plenty of time to consider the matter and settled on his choice of replacement, Alex Padilla, some time ago. As of the time I actually was reminded by Phoog in the comments to consider this part of the sequence, the precise timing of things had been settled. In this case, Harris formally resigned and vacated her seat on Monday the 18th. California's governor, Newsom, quickly appointed Alex Padilla to her seat. According to this news report, Harris will swear in Padilla to the Senate after she is sworn in as the VP. So Harris's replacement will be sworn in and seated sometime in the afternoon of the 20th.
Finally, by Georgia state law, all counties have until the 15th to finalize their vote tallies and submit them to Secretary of State Brad Raffensperger. Raffensperger then has until the 22nd to certify. Once certified, Warnock and Ossoff may be sworn in and take their seats as Senators. Currently I do not know if Raffensperger must complete the certification by a particular time on the 22nd. If it can be dragged out late into the day, the new Senators may have to wait for the next day; even though they might still be able to be officially sworn in right away, the practical effects of this, where the Senate can assert a new majority and the new Senators may cast votes on anything, wouldn't occur until the next day. However, in any case, Raffensperger can also complete the certification before the 22nd, in which case the Senators may be sworn in and first participate in Senate business anytime between the 15th and 23rd, inclusive.
Thus we get that majority control will transfer no earlier than the 20th (at noon), and no later than the 23rd (whenever the Senate convenes that day and procedurally establishes its new majority).
Update (how it's actually going down)
Brad Raffensperger certified Warnock and Ossoff's victories on Tuesday the 19th, and are expected to be sworn in along with Alex Padilla on the afternoon of the 20th. So the Democrats will have established a majority at that point, and Biden will essentially assume the Presidency with (slim) day 1 majorities.
