U.S. elections are complex, especially presidential ones, which have several layers of indirection. Because of the complexity of the question, I'm going to focus on the case of presidential elections. (There are important differences between presidential elections and all other kinds of elections.)
I also want to note that although insider selection of candidates was formerly common, the fact that it is possible in theory does not mean that parties could easily transition back to this method of selection. Also, attempts to disregard the results of state primary elections and thereby reduce the level of democracy (as indirect as it is) in the process could conceivably be subject to judicial review or regulation by state or federal legislation, although this would probably raise constitutional questions about the scope of power of different government branches over the nomination process.
It seems that in formal terms, the key moment when party "insiders" would be able to exert influence is when the rules for the national convention of party delegates are adopted. In the case of the 2016 Republican National Convention, the rules committee included delegates from each state, territory, and Washington, D.C.; but I assume the makeup of equivalent bodies and the method for selecting the delegates could vary depending on the party or perhaps from state to state.
Hopefully someone else can provide further details.
Links to some related prior questions:
My current attempt to answer the numbered questions is as follows:
- Who are these insiders? Are they elected representatives, national party chairs, or maybe someone else?
- It seems like it would be whoever sets the rules for the national party conventions; but I don't know details of who these delegates are or can be in general. So I can't say who specifically in the parties would have the power to change who is nominated.
- Can these party insiders exercise the same sort of control over their parties as many parliamentary governments can? (e.g., total insider selection of candidates, the ability to kick elected representatives out of their party.) [...]
- No; the role of political parties in the presidential nomination process in the United States is not comparable to party control in parliamentary governments. The parties play a role in the election of the president, but I believe parties have no legal influence over an elected president, only political influence. Within a legislative body, such as the houses of the United States Congress, parties have a "caucus" that can play a more similar role to parliamentary party discipline.
- How does this all work for state-level governments?
- Nominees to state-level positions are not selected by national party conventions. I'm not sure what the main opportunities for party insiders to influence such elections would be: in some cases, it seems parties are responsible for submitting the names of candidates for primary elections.
General election ballot access for presidential elections
The President of the United States is elected by the electoral college, and the Constitution gives state legislatures the power to direct how a state's electors are appointed. So after the electors are appointed, I am not aware of any way that parties could play an official role in US presidential elections. Affiliation with a party is not technically a prerequisite to become president, so even if there were a way for a party to officially expel a member (I'm honestly not sure if there is), it wouldn't have any effect in and of itself on that person's ability to be elected president (although, like anything else, it might conceivably influence the decision of the Electoral College).
Per Wikipedia, "In practice, the state legislatures have generally chosen to select electors through an indirect popular vote, since the 1820s."
Therefore, the ultimate point of a party nominating a presidential candidate is to get the states to appoint electors that will vote for that candidate, which requires getting the candidate on the ballot in each of the state-level general elections.
The rules for who appears on a ballot in elections are called "ballot access". Ballot access laws are made by states, subject to judicial review by the Supreme Court. This includes rules for which candidate appears on the ballot as the nominee for each political party. (There are also rules for independent and write-in candidates, which are not relevant to your question.)
The National Association of Secretaries of State has the following summary for how general election ballot access works: SUMMARY: STATE LAWS REGARDING PRESIDENTIAL BALLOT ACCESS FOR THE GENERAL ELECTION From skimming, it appears to me that most states have chosen to give a large role to political parties here.
As I said, I'm focusing in this answer only on presidential elections, but it seems worth noting here that states are under no obligation to carry out general elections that follow the format of having at most one candidate nominated per party: for example, as Anonymous Penguin pointed out, California has a "jungle primary" system for most positions, including U.S. Senator, where the top two candidates in each primary, regardless of party, advance to the general election.
Here is a PDF laying out the general election ballot access rules for the state of Alabama that applied in 2012. The requirements for political party nominations were as follows:
- Political Party
Any political party’s national convention or other like party assembly having state ballot access for the general election may nominate. To qualify for statewide ballot access, a political party must:
Receive more than twenty percent (20%) of the vote cast in the state at the last general election. (By this method, only the Alabama Democratic and Alabama Republican parties qualify for statewide ballot access in the 2012 election.) [§17-13-40]
File a petition by March 13, 2012*, with the secretary of state containing at least 44,828 signatures of qualified electors. The number of petition signatures for statewide ballot access must equal or exceed three percent (3%) of the electors who cast ballots for the office of governor in the last general election. [§17-6-22(a)(1)]
Parties submitting a petition must also:
- File with the secretary of state an original copy and a copy
in digital format or other electronic format acceptable by the
Secretary of State of the party’s emblem by September 7, 2012.
- Hold a mass meeting, beat meeting, or other like assembly
to nominate candidates for public office or delegates for a convention at which candidates would be nominated. A public notice of the time and place must be filed at least five days prior to the meeting date with the probate judge in the county where the meeting will be held and published in a newspaper of general circulation in the county at the expense of the political party. [§17-13-50(a)-(b)]
The certificate of nomination issued by a political party convention must be signed by the presiding officer and secretary of the convention and by the chairman of the state executive or central committee of the political party making the nomination.[§17-14-31(b)]
So for Alabama in 2012, it seems like the signatures of specific officials from the convention and from the state executive or central committee of the political party are required.
For comparison, here is a form I found titled "CERTIFICATE OF NOMINATION PRESIDENTIAL ELECTORS To be used by political party established in North Dakota":
I, [chair of state convention or state party chair] do certify that I am the  chair of the state convention  state party chair of the [party name] Political Party of the State of North Dakota and that on [date of nomination], 20__, the following individuals were duly nominated as Presidential Electors by the [party name] Political Party, duly convened in accordance with the bylaws of [party name] Political Party and the laws of this state
It's completely unclear to me whether "duly convened in accordance with the bylaws of [party name] Political Party and the laws of this state" is enforceable in any meaningful sense. I'm also not sure if the linked form is the one used by the Republican and Democratic parties, or if it might only be for use by smaller parties.
I'm not going to try to find comparable information for all other 48 states.
The Democratic and Republican national conventions seem to be the "last stop" before the presidential selection process moves out of the hands of the parties
The nomination process followed at the national conventions is set by party rules, not state laws, but both parties' rules assign an important role to pledged delegates who are expected or in some cases legally bound to vote for the candidate who won the primary election or was selected by caucus in their state.
There seems to be quite a bit of detail at Wikipedia on the Democratic National Convention and Republican National Convention.
Parties clearly have quite a bit of freedom to determine the rules for their nomination process, but there seems to be some uncertainty about whether this process is completely exempt from regulation by the U.S. Congress or by state legislatures.
Opponents of congressional action generally point to two Court cases that support the primacy of political parties in setting their own nomination rules. In Ray v. Blair (1952) the Supreme Court ruled that it was constitutional for a state to allow a state party to make potential electors to the electoral college sign a pledge of loyalty to that party. In Cousins v. Wigoda (1974), the Court ruled that political parties have the authority to set their own rules for their nomination process, including the rules that determine how delegates are seated at the national convention, and that such party rules trump state laws.
("Why is the presidential nominating system such a mess?", Elaine C. Kamarck, pp. 5-6)
There's some information about the efforts to avoid nominating Trump in the Wikipedia article 2016 Republican National Convention:
The Rules Committee, which sets the rules of the convention and the standing rules that govern the party until the next convention, met on July 14. The rules it passes must be adopted by the full convention to take effect. This committee is regarded as the most powerful. It consists of 112 members, including one male delegate and one female delegate from each state, territory and Washington, D.C. Members of this committee are elected at state conventions. The Rules Committee was chaired by Enid Mickelsen of Utah and Ron Kaufman of Massachusetts.
In June 2016, activists Eric O'Keefe and Dane Waters formed a group called Delegates Unbound, which CNN described as "an effort to convince delegates that they have the authority and the ability to vote for whomever they want." Republican delegate Kendal Unruh led an effort among other Republican delegates to change the convention rules "to include a 'conscience clause' that would allow delegates bound to Trump to vote against him, even on the first ballot at the July convention." Following a "marathon 15-hour meeting" on July 14, 2016, the Rules Committee voted down, by a vote of 84–21, a move to send a "minority report" to the floor allowing the unbinding of delegates, thereby guaranteeing Trump's nomination. The committee then made the opposite move, voting 87–12 to include rules language specifically stating that delegates were required to vote based on their states' primary and caucus results.
From what I can see, it seems to have been a desperate measure that had little or no realistic chance of success. However, I don't have a detailed technical analysis of this scheme. I think it's likely that any astonishing maneuvers would be challenged in court, but I don't know whether such a challenge would succeed. The Wikipedia article on Delegates Unbound says
According to the group, "There is no language supporting binding in the temporary rules of the convention, which are the only rules that matter" and "barring any rules changes at the convention, delegates can vote their conscience on the first ballot." The group's co-founder, Eric O'Keefe, said, "Our goal is simple, to ensure the delegates are not misled to believe they must follow orders or rules set by others." O'Keefe added that he believes state laws and state party rules purporting to bind delegates to a particular candidate based on primary or caucus results are unenforceable and would lose a court challenge.
Another useful summary is laid out here: The U.S. Presidential Nominating Process (Council on Foreign Relations)
Primary elections are an optional preceding step before the nomination of a candidate for a general election; whether to have a primary is up to the party, but parties do not have total freedom to dictate how a primary is carried out
Currently, many states have primary elections, which create some blurring between what can be viewed as public and private actors.
Primary elections are carried out at the state level and under state law. Parties may choose not to participate in a primary election in a particular state; this choice applies at the state level and the details of how it works may therefore vary by state. (It might be as simple as the party just not submitting any candidate names to the state: e.g. it seems that Florida leaves that entirely up to the parties.) A specific example of a state where the Republican Party canceled its primary for the 2020 presidential election is South Carolina. Here's a news article that mentions some other states.
However, when a primary is held, parties do not have unlimited rights to restrict who participates in "their" state primary election: in Smith v. Allwright (1944) the U.S. Supreme Court held that it is not allowed for a party to use racial criteria to exclude voters from a state primary election. On the other hand, the Supreme Court has struck down state laws that required parties to allow non-party members to participate in selecting the party's nominee (California Democratic Party v. Jones, discussed further by Magarian below). Another recent case that seems relevant is the lawsuit by the Republican Party over Utah's Elections Amendments Act of 2014 ("SB54"), which affected the primary process; it reached the United States Court of Appeals for the Tenth Circuit which affirmed the act, and the U.S. Supreme Court declined to hear the appeal.
Kamarck (cited above) discusses some of the development of the current process of selecting presidential candidates, how it is subject to judicial oversight, and how it may be regulated by states or potentially even by Congress:
For much of American history, political parties and their activities were treated as private, not public, matters. Indeed, prior to the widespread use of binding presidential primaries—which didn’t happen until the 1970s—the entire pre- reform-era nomination system was essentially a semi-private, if not entirely private, enterprise. As state legislatures introduced primaries at the turn of the century, spurred on by the Progressive reform movement, the result was a certain degree of regulation through statutes creating the primaries. But by and large the federal government stayed out of the business of party nominations at all levels.
Then, in 1944, the Supreme Court ruled in Smith v. Allwright that political parties are state actors and cannot discriminate. At the time, many southern Democratic state parties restricted participation to whites, effectively disenfranchising African Americans. By prohibiting them from voting in primary elections, the state parties, for all practical purposes, prevented African Americans from having any say in the electoral process, given the Democrats’ uncontested control of the South. The Supreme Court saw a compelling interest in protecting the right to vote and intervened to prohibit all-white primaries. The Court has also intervened in the political process in several other important cases. In Burroughs v.United States, decided in 1934, the Court found the Federal Corrupt Practices Act constitutional on the grounds that Congress had the authority to protect the election process from corruption. Similarly in Buckley v. Valeo (1974) the Court ruled that parts of the new campaign finance law were constitutional because they empowered Congress to prevent corruption in the nominating system. And in Oregon v. Mitchell (1970) the Court ruled that states had the power to extend the right to vote to those who were 18 years old.
Some Supreme Court cases regarding rights of association for the parties in primaries are discussed in this article: "Regulating Political Parties under a "Public Rights" First Amendment",
Gregory P. Magarian (April 2003; William & Mary Law Review Volume 44 (2002-2003) Issue 5, Article 2).