what happens if the President of the United States is deemed a risk to U.S. national security?
By whom? That will make a large part of the difference as to how it is handled.
If it is by the appropriate majority in Congress, (two thirds in the senate), a president may be impeached, resulting only in his removal from office. Another answer has already addressed impeachment, which is largely independent of any potential criminal prosecution.
All are accountable to report to the public and to the appropriate authorities such facts as they themselves are witnesses to, and all persons holding any office are responsible to perform the functions of that office.
A recent, well-documented example of "what happens":
In his book "Dereliction of Duty", Lt. Col. Robert "Buzz" Patterson -- Bill Clinton's military aide for several years who held the control satchel infamously called the "nuclear football" -- publicized his eyewitness account of how the president actively dismantled national security efforts and created numerous breaches that literally risked complete and one-sided nuclear war not in our favor. He recounts how the president misplaced the nuclear launch codes during his affair with Monica Lewinsky, exposing the U.S. to non-retaliable nuclear attack for as long as several days. He details how the president and his entourage also stole outright hundreds of dollars worth of valuables while visiting a foreign embassy--not exactly a prudent foreign relations move. He is also documented to have regularly sexually assaulted female staff on Airforce One and aboard other aircraft.
This and many similar, non-isolated incidents led Patterson to conclude "the greatest security risk to the United States [is] the president himself."
The appendices to his book also include other witnesses including Caspar Weinberger's reports that the Clintons empowered and emboldened Saddam Hussein and North Korean dictators, and reports that the Clintons donated hundreds of millions of dollars worth of export-controlled technologies to the China, to perfect their intercontinental ballistic missile guidance systems, among many other things.
Numerous such other accounts have surfaced by those most knowledgeable. In the epilogue,
To my knowledge, no official action was ever taken against Clinton for these acts of severe national security breach.
Nonetheless, Patterson introduces his book in these solemn tones:
"I hope it is also a warning to the American people that we must never
allow the purveyors of such dangerous military policy and
irresponsible foreign policy claim the power of presidency again."