I'm wondering about both the separation of federal prisons and state prisons, and the separation of the former prisons to county jails.
Are there political factors that influence how that separation is used, and that it exists in the first place?
This is mostly a function of who pays for the very high cost of keeping someone incarcerated (typically $25,000-$75,000 per person per year depending on the type of facility involved).
State Prisons v. Federal Prisons
Existing Practice
People sentenced to felony prison sentences by state courts for state crimes are incarcerated in state prisons at state expense, while people sentences to felony prison sentences by federal courts for federal crimes are incarcerated in federal prisons at federal expense.
This is a function of the jurisdiction of each government under a dual sovereignty system as discussed in the answer by @hszmv
The Motivation Behind Existing Practice
The separation of federal and state prisons prevents the federal government with burdening states with the cost of paying to incarcerate people incarcerated as a result of federal laws, and vice versa.
If the federal government wants long sentences for drug offenders, it is going to have to foot the bill for paying for keeping those drug offenders in prison for a long period of time.
Jails v. State Prisons
Existing Practice
County jails are used to incarcerate people pending trial in courts usually organized on a county by county basis for bureaucratic and budgetary purposes, and also for people convicted of misdemeanors (traditionally punishable by up to one year in jail, although local practice varies and most states have multiple grades of misdemeanors with different maximum sentences).
(The federal government often contracts with county government to hold federal criminal defendants awaiting trial or sentenced to misdemeanors in county jails on a fairly generous per diem basis because it doesn't have a high enough volume of such defendants to justify having its own jail system in most places - about 99% of criminal prosecutions are handled by state and local governments, and federal criminal defendants are often white collar criminals who can post bail pending trial and then face felony sentences if convicted.)
State prisons hold people convicted of state crimes called felonies that are punished by more than one year of incarceration (conventionally).
The division between state prisons and county jails is more arbitrary and a matter of tradition. State law (or at least an easy change to a state constitution) could change it.
The constitutional allocation of responsibility for criminal justice matters between the national government and state governments in Germany is closer to the allocation of responsibility for criminal justice matters between state and local governments in the United States than it is to the allocation of responsibility for criminal justice matters between the federal government in the United States and the state governments in the United States. German states routinely administer criminal laws enacted by the German federal government. But, in the United States, it is very rare and controversial for U.S. states to administer criminal laws enacted by the U.S. federal government. In contrast, U.S. counties routinely administer criminal laws enacted by state governments.
The Motivation Behind Existing Practice
Putting the economic burden of pre-trial detention on counties makes sense because it discourages counties from underfunding local courts and prosecuting attorneys' offices that determine how long it takes to get trials scheduled.
A related factor is that local law enforcement typically varies much more in how they exercise their discretion in prosecuting misdemeanor offenses than they do in how they exercise their discretion in prosecuting felony offenses.
For example, there would be a great deal of variation in how often local law enforcement officers and prosecutors prosecute homeless people for trespassing, but there would be very little difference with how often local law enforcement officers and prosecutors investigate or prosecute a report of an armed robbery or a murder. Keeping local fiscal responsibility for costs that involve more local discretion, while keeping state fiscal responsibility for costs that involve less local discretion makes sense.
While felonies are typically punished near the middle of the allowed range of sentencing discretion, misdemeanors are predominantly punished near the bottom of the allowed range of sentencing discretion insofar as incarceration is concerned, often less than a month, even for the most serious misdemeanor offenses punishable by up to a year or more in jail.
The notion behind having misdemeanor sentences following a conviction of a misdemeanor offense served locally is that the bureaucratic bother of transferring a convicted misdemeanor defendant who often has only a modest number of days of incarceration left after time served awaiting trial (or simply because misdemeanor sentences are often much shorter than the permitted maximum sentence) isn't worth the trouble in the vast majority of case. If the defendants were sent to prison, they'd be processed into the system and then immediately start the bureaucratic process of getting them out, with lots of travel expense each way in prisoner secure buses.
Some states, notably California, have tinkered with how the burden of incarceration costs are handled in their unified court system, which has changed the balance of how prosecutors choose to charge felonies v. misdemeanors. Prior to reform there was an incentive to charge someone with a minor felony rather than a serious misdemeanor, when possible, in order to shift incarceration costs from the local government to whom they reported, to the state.
