(I get to leverage my experience working in the MSW (Municipal Solid Waste) sector as a recycling educator here, neat!)
First off, we have to acknowledge that "recycling" is an extremely broad umbrella - at least as broad as the term 'infrastructure,' and just as fraught with political motivations for labeling something as being under the umbrella. Consider, the chasing-arrow logo with the numbers inside, manufacturers fight tooth and nail to earn the legal right to put those on their products, whether or not the product itself is even remotely recyclable with current technology.
You say:
If nothing else, it should fall neatly into the category of public utilities...
But it's fall therein is anything but neat.
Consider the end-of-life phase of a consumer product's lifespan, assuming it is a conventionally recyclable item, we'll go with a soda bottle.
- The consumer puts the item in a curbside recycling bin (The bin is produced by a private company, and sometimes distributed by a private company as well).
- A truck comes by to pick it up. (The truck is owned by a private contractor, almost always - in the US anyway - a large, for-profit, solid waste company.)
- The material is brought to a sorting facility, the most modern of which are automated beasts called Material Reclamation Facilities (MRFs). (These are also privately owned, for-profit enterprises.)
- The bottle goes through a series of sorts and filters until it is finally deposited in a large bunker with other bottles made of the same material. It is compressed and strapped into a bale (yes, bale, like with hay). It is now a small part of a larger unit of commodity good which will be sold to a buyer (the buyer is a private, for-profit company).
- The bottle will be shredded, the resulting cullet will be chemically washed several times, and the cullet will then be melted and re-injection molded into a new bottle if the cullet was of good quality, but more likely into some lower grade resin-based product like a flower pot. This product will now be trucked out and sold as any other consumer product. (This is all done by for-profit manufacturers.)
As you can see, at no stage of it's path is it being handled by anything that one can readily consider a public utility. The 'public infrastructure' that is being interacted with here are roads and sometimes railways.
All of this comes from the fact that, in the United States, recycling is viewed as the responsibility of the consumer primarily, and the producer secondarily, not the government's problem. Is this how it should be? Not if we want to see meaningful recycling rates (as it stands, warehouses full of bales of low-grade plastic waste and other 'recyclables' are going unbought because consumers don't know how to differentiate between recyclate and contaminant), but it's what we've got.
Private enterprise is almost definitionally not 'public infrastructure.' What recycling does happen in the United States is a form of value extraction from what would otherwise be waste streams. It's driven by the fact that solid waste disposal is also a private, for-profit affair. There it's more frequently viewed as a municipal service but the 'infrastructure' part of it is just roads and bridges again. The market prices for disposal services is part of the value proposition for recycling.
tl;dr - Recycling in the United States is an entirely private-sector, for-profit affair, rather than a publicly provided-for system.