For example, if the Republican party wanted to hold a new primary in 2020 and pick someone other than Trump, is there any specific reason they could or could not do so?
Absolutely, it happens all the time. An incumbent losing to a challenger in their own party's primary election is sometimes called "getting primaried".
Search for "getting primaried" or "primaried out", you can find lots of articles and books:
- Getting Primaried, The Changing Politics of Congressional Primary Challenges by Robert G. Boatright
- We unlucky few: A look at the incumbents who lost their primaries, 1994-2012
- New progressive PACs warn Democrats to oppose Trump — or get primaried
It has happened to several presidential incumbents as well, according to NPR's When Has A President Been Denied His Party's Nomination?
- John Tyler, Whig, 1844
- Millard Fillmore, Whig, 1852
- Franklin Pierce, Democrat, 1856
- Andrew Johnson, Democrat, 1868
- Chester Arthur, Republican, 1884
Of these 5, only Pierce was elected prior to his primary loss. The other 4 took their office in other circumstances (death of the prior president).