Not sure this counts as a full answer, but here goes.
The Afghanistan quagmire was probably largely unintentional, just like the Iraqi one. If you read up on the level of group thinking and hubris under the Bush administration circa 2001-03, it is pretty amazing in its naivety. I read a number of these books, mostly about Iraq though. These people truly thought that, once the natives were gifted with the insights of US democracy and free marketry they would see the light and things would magically fall into place.
After the Iraq invasion, the US tried, quite hard, to get the UN to carry the bag for them. The UN was reluctant, and when their envoy was killed by a car bomb, they entirely demurred, leaving the US in charge.
Once it was clear no one else was going to do their work for them, the US then had to try to put in place a government suitable for their interests. That didn't really happen in Iraq, but at least they left a government of sorts in charge in 2008, even though Western countries had to bail Iraq out from ISIS 6 years later.
In Afghanistan no stable government has been achieved to date, despite years of, competent or not, attempts to do so. Not unlike South Vietnam (at a political rather than military level) the US-backed governments never found much legitimacy, were frequently accused of corruption and commanded only limited performance from its own military.
Pretty sure the US would have loved to leave the locals in charge, had they managed to have a viable government that suited them handy earlier on. (that there is no oil in Afghanistan only strengthens my gut feeling about this).
FWIW, Biden's take in 2008/9, reported in Obama's Wars by Woodward was to leave only a strong anti-terrorist force capable of rooting out anti-US terror groups if needed, not controlling the country. Note that this is 2-3 years before OBL gets killed. OBL's last real undisputable presence in Afghanistan was during the Tora Bora battle in late 2001. The context was also that the US had just semi-stabilized Iraq with the "surge" under Petraeus and the Pentagon felt they could do it again in Afghanistan, given 40K more troops.
What do I mean by "suitable"?
- Not Taliban - they were after all the guys the US booted in 2001.
- Not likely to host international terrorists groups
- Nice and cuddly to women, minorities and everyone, thus justifying Pax Americana and the hundreds of billions of $ spent.
Apparently, the US has resigned itself that if #1 and #3 can't be achieved, then perhaps the Taliban can be cajoled into #2. Which isn't necessarily a bad call: the Taliban know that they managed to oppress Afghanis just fine despite international outcry, until OBL pulled off 9/11. So they will probably be wary of being threatening enough to motivate another regime change.
To quote another user's answer, concerning the US and Iran this time (my bolding):
Invasions are commitments of the utmost gravity. You're not just committing materiel for the opening, kinetic phase of the war, but also a permanent force to occupy and govern. This was the key strategic error the United States made in the second Iraq war, they conquered Iraq easily but could not then govern or even effectively and swiftly prop up a new government. The costs for invading even a nation that offers no credible military threat to you are far in excess of what most observers understand.
In summary, the US stayed more by accident, and yes, by initial hubris, than by design. Later, they desperately did not want to lose face, even as they realized their "plan" to kickstart Afghanistan into the 21st century wasn't working.
p.s. "Pivot theory"?
Well, if the USA was so forward-seeing and competent, why did they not do their best to gain influence in Afghanistan in 1992, once Najibullah got kicked out, and before 1994 when the Taliban kicked out the warlords? Had they cared the Afghans would have looked a lot more kindly on the people who had helped booting the Russians. Instead, helping with reconstruction and nation-building was proposed by some in Congress and quickly ditched as policy. This was most unkind to the Afghans (who deserve massive credit for helping rid the world of the USSR), set the world up for the events of 9/11 and, coincidentally, also totally against Pivot Theory. So, despite Pivot Theory being 100 years old it was totally ignored from 1992-94 but nefariously revived in 2001??? India, more calculating, but also with more immediate reasons to (Pakistan), has tried to support and gain influence with the US-backed governments.