In the originally-envisioned system, Presidential electors are appointed by the states; the states decide how those electors are chosen. Accordingly, states could hold an election, but in most the state legislature chose the electors. Since the Civil War, all states have chosen electors according to popular vote, but ballot access rules (for the general election) are still determined by the states.
At present, Presidential primaries are theoretically run by the parties, who can make their own ballot-access rules. (But they may be subject to restrictions under state law.) Effectively, major-party candidates must follow their party's procedure to gain primary ballot access; third-party or independent candidates (or major-party candidates that lost their primary but want to continue their campaign) must follow the state procedure.
Typical requirements include submitting some number of registered voters' signatures and paying a filing fee; alternatively a candidate nominated by a party that received some percentage of the vote in the preceding election often automatically gets ballot access. The intent is to ensure that only serious candidates get on the ballot. Allowing large numbers of frivolous candidates would confuse voters, and increase the chance that the first-past-the-post system elects an undesired candidate (by splitting the vote between similar candidates). (Consider comedian Stephen Colbert's candidacy in 2008; he was denied primary ballot access because his candidacy was deemed to be a joke.)
A candidate/party that lacks ballot access can still be elected by write-in votes, but many states require write-in candidates to submit forms first.
A bizarre case where the ballot-access rules went wrong was a Tennessee State Senate election in 1998, in which candidate Byron Looper removed his only opponent from the ballot by assassinating him (Tennessee law denied ballot access to deceased people).
It is worth noting that some states even provide ballot access to constitutionally ineligible candidates that meet the state requirements. For example, for the 2012 election, the Party for Socialism and Liberation nominated a 28-year-old and a foreign-born citizen for President and Vice President, but the ticket nevertheless obtained ballot access in nine states.