TL;DR: If both Senators of a state are sworn in on the same day, the Senator who is sworn in first will become the senior Senator. However, there are several factors that decide who gets to be sworn in first.
From the Senate website, the tie-breaking factors (in order of precedence) are:
- previous Senate service
- service as the vice president
- previous House service
- service in the Cabinet
- service as a state governor
- population of their state at the time of swearing in
If all of the above factors are tied and both Senators are from the same state (rendering #6 moot), it appears that the Senator who is elected to the full term will become the senior Senator. From a report by the Parliamentarian of the House of Representatives:
Of the two Senators from a State serving at the same time, the one who was elected first--or if both were elected at the same time, the one elected for a full term--is referred to as the "senior" Senator from that State. The other is referred to as the "junior" Senator.
Some, including Congresswoman Nikema Williams, however, noted that the Senators will be sworn in by alphabetical order of their first names. I could not find any official sources regarding this.
Nevertheless, in Ossoff / Warnock's case, Ossoff will be the senior Senator regardless of which rule above is being followed.
It's also worth noting that these tie-breakers are not hard and fast rules. Sometimes, discretion can be exercised on which Senator is to be sworn in first (and, consequently, hold the title of "senior Senator").
The last time such a dilemma had occurred (that I could recall) was in Arizona. Kyrsten Sinema was elected in the 2018 (scheduled) election while Martha McSally was appointed to the other seat (previously held by John McCain) in Dec 2018. Some had speculated that McSally could be sworn in first since her seat was already vacant, unlike Sinema who had to wait until the start of the new Congress (Jan 3, 2019).
Ultimately, Gov. Ducey made clear that McSally's swearing in would be delayed until after Sinema was sworn in, the reason being Sinema was elected.
"I'm also going to respect the will of the voters. Senator-elect Sinema was elected to the office and she's going to be sworn in first," Ducey said.