There are several time frames involved here, and the answer is different in each.
Between the election and the voting of electors:
If the electors in a given state have not yet met and voted, then they will need to vote for someone who is eligible rather than the person they had pledged to vote for, since they would be Constitutionally prohibited from voting for their person. (And/or their vote would just be ignored, like if they abstained, since it isn't a valid vote.)
Between the voting of electors and certification by Congress:
This may go one of two ways, because the Twelfth Amendment doesn't explicitly give Congress the ability to avoid certifying whoever got the majority of the elector's votes as President. One possibility is that they will say they're obligated to certify what they were given, which will just pass the buck to the next timeframe. Another is that they will count all votes for other candidates, determine that no one who is eligible has a majority, and fall back to the "no majority" rules (House decides President, voting by states).
Between the certification by Congress and the swearing in:
The Twentieth Amendment specifies that:
if the President elect shall have failed to qualify, then the Vice President elect shall act as President until a President shall have qualified;
Thus the VP would be sworn in as VP and then become acting president.
After taking office:
This is where there's no clear framework for what would happen. Assuming the candidate was unaware of their ineligibility, it would be hard to justify impeaching them for "Treason, Bribery, or other high Crimes and Misdemeanors". But there's no other way to force a President out of office. Ideally, they would resign as soon as this was discovered. But in the case where they don't, we'd be in Constitutional Crisis territory. Maybe Congress will try to declare the position vacant. Maybe the Supreme Court will get involved. Maybe the VP will pull the equivalent of a coup on Constitutional grounds (which is very roughly what's happening in Venezuela right now). As with all Constitutional crises, the personality of the people involved and exact circumstances will inform whatever outcome ends up sticking, and none of that can be predicted in advance.