Writing partially from memory reading War and Decision: Inside the Pentagon at the Dawn of the War on Terrorism. Note that this answer is more geared towards early Bush policy, as per the question:
There was no great intent to nation-build. Feith is unapologetic about it and makes his case that Rumsfeld, et al., felt that a large occupation force and presence would trigger Afghan resentment at an alien presence.
as candidate, Bush was on the record criticizing open-ended US commitments to nation-building capacity under Clinton, a notion shared by his team.
MR. BUSH -- Somalia. It started off as a humanitarian mission then changed into a nation-building mission and that's where the mission went wrong. The mission was changed. And as a result, our nation paid a price, and so I don't think our troops ought to be used for what's called nation building. I think our troops ought to be used to fight and win war. I think our troops ought to be used to help overthrow a dictator when it's in our best interests. But in this case, it was a nation-building exercise. And same with Haiti. I wouldn't have supported either.
It is also useful to remember, that unlike the much larger US and NATO forces later stationed in Afghanistan, or indeed the Iraq invasion in 2003, the initial overthrow of the Taliban was done on the cheap, with a thousand or so Special Forces/CIA collaborating with Afghan anti-Taliban forces and directing airstrikes. There just wasn't much to be building with, at the start.
Quoting Washington Post - Built to Fail:
Eventually, however, officials interviewed for the Lessons Learned project said, the Bush administration recognized it had a duty to help Afghanistan build a new economy from scratch. Although Afghanistan had scant experience with free markets, the United States pressured the Afghans to adopt American-style capitalism.
Now, this is WP stating this and I wasn't following Afghanistan all that closely at the time. However, this disastrous playbook is exactly the one that the US pursued in late 2003 in Iraq, under Bremer. Feith BTW makes Bremer a scapegoat for everything that went wrong in Iraq despite him being put in place after sidelining the State Department at Defence Department insistence.
Later on, claims that aid money was being diverted into Taliban coffers, as "protection money" became more common and credible and the nation-building opportunity had probably passed by later years, if it ever existed.
It is also been said that, unlike other Western powers, the USA had little institutional experience of administrating remote, poorer, cultures (i.e. colonialism) and was easily taken in by "sympathetic" locals with a gift for manipulating them. In Afghanistan that was Karzai, in Iraq that was, almost, Chalabi.
Further reading: Council of Foreign Relations
April 17, 2002
Reconstructing Afghanistan
President George W. Bush calls for the reconstruction of Afghanistan in a speech at the Virginia Military Institute. “By helping to build an Afghanistan that is free from this evil and is a better place in which to live, we are working in the best traditions of George Marshall,” he says, evoking the post-World War II Marshall Plan that revived Western Europe. But the United States and the international community do not come close to Marshall Plan-like reconstruction spending for Afghanistan. The U.S. Congress appropriates over $38 billion in humanitarian and reconstruction assistance to Afghanistan from 2001 to 2009.
November 2002
Establishing a Reconstruction Model
The U.S. military creates a civil affairs framework to coordinate redevelopment with UN and nongovernmental organizations and to expand the authority of the Kabul government. These so-called provincial reconstruction teams, or PRTs, are stood up first in Gardez in November, followed by Bamiyan, Kunduz, Mazar-e-Sharif, Kandahar, and Herat. Command for individual PRTs is eventually handed over to NATO states. While credited with improving security for aid agencies, the model is not universally praised. Concern mounts that the PRT system lacks central controlling authority, is disorganized, and creates what a U.S. Institute of Peace report calls “an ad hoc approach” to security and development. Such criticism grows beyond the PRT program and becomes a common theme in the NATO war effort, as a maze of ìnational caveatsî restricts the activities of member forces. Critics contend this limits the coalition’s effectiveness.
Let's see the actual spending, as reported by Office of the Special Inspector General for Afghanistan Reconstruction (SIGAR). (Note:
SIGAR was created only in 2008).
(Blue is defense spending, gray is reconstruction. Note the low early year numbers. Note: I assumed that military-led projects, common for US forces, fell under the blue umbrella, but apparently not as DoD also gets itemized under Reconstruction, 76.6B$).
(image copied)

Last, indirectly related, in Obama's Wars Woodward says that Biden had the role of doubter in the Obama-Pentagon tussle about how many soldiers to send to Afghanistan, preferring to minimize US presence as well.