AI-Facilitated Coordination

The opportunity

We want to push the use of AI in coordination so there is a future where groups regularly reach beneficial outcomes they wouldn’t have unaided. One perspective on the state of coordination today is that our limiting factor isn’t capability — it’s the time and attention a party can spare. A faithful AI representative that knows your priorities and red lines in real depth could discover and show up at coalitions and negotiations you’d never reach yourself, and surface a depth of considerations you wouldn’t have time to voice. The same scales up — an AI representing a coalition or diplomatic group could faithfully hold every internal stakeholder’s priorities as it negotiates with a counterpart. An AI working as a mediator across all parties could explore far more possible structures than people in the room could enumerate, present options in ways that make it more transparent when they are genuinely palatable to each side, and aim at outcomes shaped by principled criteria — Pareto-optimal deals and Nash-bargaining-style fair allocations — rather than by raw leverage and time pressure. Combine these strands, and coordination can become substantially better than we manage today: surfacing opportunities and arrangements — diplomatic, scientific, safety-related — that would otherwise never be explored, much less entered, and holding together coalitions that human attention alone struggles to sustain.

Human coordination

Humans have benefited greatly from, and arguably survived due to, coordination: even when we’re opposed on some issues, we often find ways to trade, bargain, and build the trust needed to get along. Sometimes we do even better, cooperating to achieve vastly better flourishing and quality of life than we would alone, and seeing these benefits accrue over generations.¹

While coordination is a perennial human challenge, it is never static: technological developments have always changed the balance of which kinds of coordination are achievable, for better, or sometimes, for worse.² The complexity is staggering, and we believe it is difficult to project the impacts. Nevertheless, when examining several important processes that constitute coordination — networking and connecting, surfacing shared interests and will, mobilizing group wisdom and collective intelligence, navigating tradeoffs during the tug-of-war of negotiation, understanding and designing processes and institutions for achieving coordinated goals — we see numerous pathways to creative use of new technology to assist and improve things.

Technology has always reshaped how (and how well) we coordinate: the challenge now is to ensure that with AI, it does so for the better.

What’s now at stake

As the potential impact of humans’ activities grows, our achievements in coordination may need to keep pace. Unless great powers are eclipsed by an unprecedentedly greater hegemon, they must learn to live as good-enough neighbors: managing existing WMDs and powerful, fickle emerging technologies like AI and biotechnology. Middle powers in coordinated blocs play critical roles in global security (and can demand reasonable representation). On Earth, our prospects are deeply intertwined, despite some competitive aspects, whether in health, wealth, or security — and this is no different as we further our reach into space. Given the risks of modern technologies, we might all grow in fortune together or be wiped out as one. Outside of diplomacy, coordination among technology developers (though this carries risks of collusive abuse) may aid in developing best practices and safety measures while exercising restraint from the riskiest pathways.

Alongside global security, improved coordination has other important potential benefits. Domestic governance alignment — toward institutions that both genuinely represent people’s interests on the international stage and promote flourishing at home — could become increasingly important as technological disruption, especially from AI, raises the stakes and creates new opportunities for corruption, oppression, revolution, and other political abuse. Securing this might depend on coordination: the combined forces of the public, leaders, and experts, creatively imagining better futures and defending against state or private overreach.

Getting it right

None of this is guaranteed. AI in coordination can be built to faithfully serve the people it represents, or it can become another captured intermediary — appearing faithful while quietly serving narrow interests. The stakes that motivate us most are pro-social: diplomatic agreements that wouldn’t otherwise be reached, safety arrangements at frontier labs, coalitions that need to hold across a potential tidal wave of changing circumstances. And AI now also speeds up the building of these tools — developers in this space get the same uplift their products provide, which means faster iteration and more shots on goal. Getting it right means building the kind of trust that makes faithful representation real, taking misuse risks as seriously as the upside, and pushing toward the structures where coordination improves meaningfully for the benefit of humanity.

Note that the dark side of coordination — mobs, cliques, collusion — has also been responsible for great harm and misery. We take these risks seriously in our orientation and planning.
↩︎
The internet has built unprecedentedly large distributed communities; social media has reshaped which communities people see themselves as belonging to; encrypted communications enable both dissidents and criminal networks. Earlier still, automobiles restructured neighborhoods, and the move out of feudalism replaced one whole coordination structure with another. ↩︎