I can imagine a few reasons.
Not everything that is Open Source is worthy of funding just because it is Open Source
Lots of things are open source. Not all of it is useful for governments, or even useful for a general audience. In addition, there are lots of other things that compete for government funding that are arguably more socially desirable than open source software. Like roads or the healthcare system or taking care of poor people. Governments spend lots of money on helping poor people, and it gets looked at funny when it tries to spend money on helping mid-to-upper middle class people, which most software developers are.
Open source projects that are broadly useful often already have adequate funding
As you say, the ones that are useful to business are typically funded by businesses. You've mentioned forming businesses on top of OSS projects to monetize them, but this is not the most common model; for very useful projects (like, the Linux kernel) multiple companies, including those who are competitors, will contribute money and effort because it reduces their costs.
Software that is useful to governments is often not produced under open source licenses because the application is too narrow to support an open source community.
I work at a company that makes biometric software. Our customers are usually very large corporations or government agencies that want to run fingerprint background checks. There's not going to be an open source solution for that kind of thing because it's not actually possible for your average OSS developer to make at home, let alone something average users of computers want to use, because average people don't have a need, desire, or ability to run fingerprint background checks.
Governments do lots of things like that, so they're always going to need some highly customized software that to meet those requirements. The only reliable way that software gets made is under a proprietary business model in most cases.
It is not always easy to give money to open source projects because they don't always have the ability to easily accept payments from governments
Getting money from the government is actually really difficult if you aren't recieving it as part of an existing entitlement program or through the tax code. Governments have lots of rules about who to give money to and how to give money to them in exchange for goods and services. You usually have to put in a competitive bid to get paid, and that's a complicated process that requires very skilled and knowledgeable sales people who can work on that for a year or more. If you win that process, then you usually don't get money, but a purchase order that takes an even longer time before it becomes actual money that you can use.
Most open source projects, by contrast, are run by a small community of people who have a PayPal account or something else intended to accept small dollar donations from individual people. Governments typically can't just dump money in there the way a person can, and OSS projects can't afford to jump through all of the complex hoops a larger organization can to get money that other way.