Come on people, this shouldn't be that hard.
What are some things government produces or purchases to provide to the public?:
- National security (qualities of a public good)
- Interstate highways (network good - positive externality)
- Information for consumers (information asymmetry - negative externality)
- Basic research (positive externality)
- WIC (principal agent problem - negative externality of sorts)
- Medicare/Medicaid (lots of market "failures" that are reviewed here)
- Postal service (network good - positive externality - and now that it is less of a positive externality there are quite sensible calls to get rid of the Post Office)
- Education (positive externality, principal agent problem)
- Police protection (qualities of a public good)
- Macroeconomic stabilization (collective action problem/public good)
- Environmental regulation (negative externality)
- Deposit insurance (negative externality)
- The administration of justice (qualities of a public good)
What doesn't the government play any real role in producing?:
- Herb gardens
You see a pattern here? I just thought through some major, obvious government goods and they all have obvious externalities or public goods qualities associated with them. I thought through some obvious normal goods that we come across on a regular basis with no widely remarked on externalities associated with them, and by and large these don't seem to be produced by the government.
Humans are not dumb. I know you all have this idea in your head that Keynesians think the common man is an idiot, but you're simply wrong when you say that (and more than a little condescending when you accuse us of it). I firmly believe that humans are not dumb and that thousands of years of interaction between humans has lead to the emergence and evolution of institutions that solve human problems. Humans know the market works. Humans know what the market is less capable of accomplishing, and they understand what things require more collective action to produce more optimally. There are lots of ways to act collectively. The state is one of many.
The apparatus of the state, over the entire course of its emergence, is obviously tempting to people. People can earn rents by bending the state to their own purposes. The state's use of coercion can be (and has been) used against innocent populations and citizens. The predation of the predation of the state has to be an essential element of any theory of government.
But clearly some states are more predatory than others. So what makes a particular political economy/state/non-state governing institution, etc. "robust"? Lot's of things. Normative regulation of the governing order (see Elinor Ostrom), clear residual rights contracting (see Oliver Hart), shifts in rhetorical treatment and understandings of certain actions (see Deirdre McCloskey), and meta-rulemaking or constitution writing (see James Buchanan). Each of these things are accomplished with a wide range of success. Successful institutions survive and succeed, unsuccessful institutions either die or they weaken themselves by preying on the market and other non-state institutions (the church, the family). The United States has been relatively successful in balancing these things for a number of reasons. Please note that the fact that the United States isn't some idealized form or perfect example doesn't invalidate any of this because I'm not talking about idealized forms - I'm talking about tendencies and roles of the state that all observed cases provide imperfect examples of.
I have one addendum that was not incorporated into my initial theory of the state: inequality. A big part of what the state does is address inequality. This isn't entirely an externality issue (although there are major reasons why we should think externalities exascerbate inequality). It also isn't entirely a predation issue, although it has similar elements to that as well. But it's a very strange kind of predation. The weak prey on the strong. Or - in a lot of cases - the strong weaken themselves to strengthen the weak. This is a major function of the state and of other non-state, non-market institutions (think soup kitchens, churches, charities, etc.). An easy way of addressing this now is just to say "the state tries to ameliorate inequality too", but that seems like a somewhat incomplete theory of this element of the state. I imagine it's worth thinking through some evolutionary biology too - it's my understanding that in other species a certain degree of egalitarianism is evolutionarily fit. If you have issues with egalitarianism on a philosophical or ethical level, please don't raise them here. That's another conversation entirely. My point here is to try to present a theory of the state. An adequate theory of what the state is seems like it has to be broader than (although still inclusive of) rent-seeking and predation.
This paper by Daron Acemoglu looks quite relevant. I seem to remember reading soem Theda Skocpol writing on these themes: Politics and Economics in Weak and Strong States.