So let's say all roads are private. This is fine. Road use is allocated by tolls, and the marginal cost of building and maintaining a road can be set equal to the marginal willingness to pay for the use of that road. Congestion pricing even becomes more feasible with lots of private tolls. All wonderful stuff.
But is it?
This is only optimal if we wave our hands when we get to the topic of ability to pay. It's very plausible that there are lots of people of lower means out there whose benefits would exceed the toll but who really couldn't pay it five days a week to get to their job, particularly if it means maintaining a car on top of it. It's plausible that their welfare gains from having a car and using it on a toll road would be considerably higher than someone of more substantial means, so that the only constraint is ability to pay.
Remedying that with substantial redistribution introduces its own problems, because then you're just putting your thumb on the scale of another market. That's no good, but at the same time it's not clear why optimality in a goods market (like the market for toll road use) should be conditional on optimality in the labor market. Why should toll road users have their welfare constrained by ability to pay? Why shouldn't employers have their profits constrained by ability to pay?
I don't think there is an easy answer here.
But I'm certainly reluctant to privilege entirely private toll roads, even if we could internalize all the externalities (which is probably unlikely since there are network externalities involved in infrastructure) as being "optimal" relative to a good deal of open access public roads as long as there's this big wedge between willingness to pay and ability to pay. In some markets, obviously, the wedge is bigger than in other markets.