For a long time ai safety people struggled to grasp the importance of democratic control of powerful ai, thinking that it was fine for them to just decide what values to align it to. They clocked it a little while back, but I don’t really think the importance of *liberal* democracy has sunk in yet. Maybe it will now. Cos a world in which (1) USG gets to use frontier intelligence however it wants, with no private company controls on what it does—which is what the Ant/DoD saga left us with—and (2) USG gets to prevent everyone else from accessing frontier intelligence until it decides they are allowed, is strictly speaking a very *democratic* world. It’s just a pretty illiberal one.
If this is the timeline we’re on, then the urgent need is *not* more democratic control of frontier intelligence, but more liberal AI. This means 2 things: (1) conscientious constitutions so that models can refuse unlawful orders from democratic authorities (so developers can’t be compelled to provide tools for activities to which they object), and (2) true constitutional steerablility so that individuals have models that will advocate for them, not force them to toe the line—guardians against autocracy not its handmaids.
These two goals are superficially in tension with each other; what we need is a theory for what kinds of behaviour should be compellable by govt or private actors and what shouldn’t. Political theory gives us an approach for working this out—what Rawls called the “liberal legitimacy principle”—but it does not provide a formula for determining its contents. This strikes me as an urgent normative task, upstream of the non-trivial technical challenge of achieving the right mix of justified refusal/steerability.