Section 1 of the 20th Amendment to the United States Constitution reads as follows:
The terms of the President and the Vice President shall end at noon on the 20th day of January, and the terms of Senators and Representatives at noon on the 3d day of January, of the years in which such terms would have ended if this article had not been ratified; and the terms of their successors shall then begin.
This is completely unambiguous language. The change of power must occur at noon on January 20th of the appropriate year.
There is absolutely nothing that can be done to prevent or delay this except for another constitutional amendment to be passed to modify the 20th amendment.
Of course, if there were extremely serious concerns about the person to be sworn in, the Congress could prepare Articles of Impeachment and have them ratified by 12:01 on January 20th, effectively blocking that person from assuming the office of the Presidency.