“What's really strikes me about ACA is the potential to shape the next generation of leaders on arms control and nuclear policy. This is something I witnessed firsthand as someone who was introduced to the field through ACA.”
Anticipatory Arms Control for the AI Age
July/August 2026
By Douglas B. Shaw, Stephen Herzog, and William C. Potter
“All models are wrong, but some are useful,” observed statistician George E.P. Box.1 Indeed, today’s large language models and other artificial intelligence systems are far from perfect. Yet, as components of larger social and technical systems, these tools may be reaching a threshold at which their design could shape the future of world order as much as the choices of their users. Frontier AI laboratories have consequently become powerful actors, capable of altering global security faster than states and international organizations can design regulations.2 Decisions about whether and how to release advanced models can directly affect risks to humanity posed by weapons of mass destruction (WMD).
In response, the James Martin Center for Nonproliferation Studies launched the “Asilomar process for AI and global security” April 8-9 with a conference titled “Silicon, Swords, and Ploughshares: The Perils and Promise of AI in the Nuclear and Biological Domains.”3 The meeting in Pacific Grove, California, brought together more than 100 experts from universities, think tanks, governments, U.S. national laboratories, and the AI industry. The objective was to begin imagining a new chapter of anticipatory arms control necessitated by the collision of powerful AI tools with catastrophic dangers such as nuclear war, WMD proliferation, and pandemics.
The conference addressed a gaping disconnect between two ways of thinking about AI risks. One, often called AI “doomerism,” views AI as a possible source of human extinction or loss of control over our collective future. The other, a narrower misuse paradigm, asks whether low-capability actors could use AI to build a nuclear weapon, precipitate a meltdown of a nuclear power plant, or engineer a pathogen capable of causing a pandemic. But much practical work lies between these camps. Governments, AI firms, security experts, and the global epistemic community concerned with human survival must jointly shape technology before today’s choices become tomorrow’s constraints.
Why AI Needs Anticipatory Arms Control
Anticipatory arms control seeks restraint before a technology arrives in forms that are already militarized or difficult to restrict. AI fits poorly with the legacy of treaty-based constraints on capabilities that states can define, limit, and monitor. Its most important effects may appear through systems it changes rather than objects that can be counted or banned. In the nuclear and biological domains, AI could reshape proliferation pathways and elements of strategic stability.
This WMD-relevant challenge also arrives as arms control institutions are perhaps least able to absorb a fast-moving technological shock. The New Strategic Arms Reduction Treaty expired February 5 without a replacement. The 2026 nuclear Nonproliferation Treaty (NPT) Review Conference failed to achieve a consensus final document, and negotiators removed all language pertaining to AI from the draft text.4 Meanwhile, the Biological Weapons Convention (BWC) still lacks an effective verification mechanism.5
Changes in military innovation compound these difficulties.6 AI’s most consequential capabilities are emerging from frontier laboratories and startups, often before regulators even understand the problems and opportunities that they pose. The technology rewards speed, offering market incentives to labs and strategic advantages to states.7 That uncertainty and private sector acceleration make the current limitations of AI models potentially misleading.8 Complacency here would be ill advised, given the technology’s rapid trajectory.
Jane Vaynman and Tristan Volpe’s work on “dual-use deception” helps explain why formal AI arms control may be contentious.9 Restraint becomes harder when military and civilian applications are difficult to distinguish and a technology is deeply integrated into military systems and the civilian economy. AI has this character in spades. Threats posed by AI depend far less on the model itself than on its user, what information it can access, and what decisions it is allowed to shape.
The Asilomar process aims to encourage anticipatory arms control before the machinery of governance has caught up. It asks whether laboratories building frontier systems can sit with governments and weapons experts to shape choices that may become facts on the ground. Once model capabilities are released, adopted by nuclear planners, or woven into biological infrastructure, the space for restraint narrows considerably.
Principles for a Missing Forum
The process deliberately invokes a tradition of scientific self-governance. The 1975 Asilomar Conference on Recombinant DNA in Pacific Grove, California, addressed an emerging biological technology whose risks were still being defined.10 Later gatherings at the same site helped make “Asilomar” synonymous with scientists confronting the social consequences of their work.
The April 2026 meeting tested whether the right communities could reason together before a crisis. The process begins with the premise that AI developers, governments, and security experts each see only part of the picture. There is a missing standing forum where those partial perspectives can be bridged early enough to improve AI governance.
After further deliberation, the conference secretariat released seven principles for governing AI applications in nuclear and biological security, available in full elsewhere:11
- AI must protect human survival.
- Nuclear weapons use decisions must remain under meaningful human control.
- AI governance must strengthen nonproliferation and strategic stability.
- AI developers must contribute to anticipatory risk governance.
- AI-enhanced monitoring and verification must be responsible and ethical.
- AI governance must be globally inclusive.
- AI must not enable disinformation or attacks on nuclear and biological facilities.
Importantly, the principles cover both model-level choices inside AI laboratories and the responsibilities of states, international organizations, and the arms control community. They are meant to give these actors a common starting point to focus on restraint before emergent capabilities harden into crises.
Lessons for Nuclear and Biological Security
The gathering clarified that most AI-related nuclear and biological risks may lie between the poles of extinction speculation and the “bomb in a cave.” The latter phrase captures the fear that a terrorist group or technically weak state could use AI to leap over material and organizational barriers to building a weapon. That concern is legitimate, especially in synthetic biology, but it should not define the field of vision.
Material constraints to building and maintaining nuclear weapons are considerable: fissile materials, precision manufacturing, delivery systems, testing, and command and control. Recent research suggests, however, that AI could eventually help technically advanced would-be proliferators navigate bottlenecks to weaponization.12 Likewise, purpose-built AI might assist nuclear-armed states in designing more sophisticated warheads. AI integration into command-and-control or decision-support systems may also shape how states in deterrence relationships interpret warnings or assess adversarial behavior.13 The late Daniel Ellsberg described multiple large nuclear arsenals ready for prompt use as a single, globe-spanning “doomsday machine.”14 It might be triggered by any of several governments, but none can prevent this independently. AI could deepen that danger if it makes leaders believe they have found a fleeting advantage or a usable window for escalation. AI also might enable nonstate actors to identify vulnerabilities in civilian nuclear facilities, which could be exploited to unleash nuclear violence.15
Biological risks, made vivid by COVID-19, raise a different problem. Advanced AI systems may make biological design feel less like specialized knowledge and more like an assisted workflow, especially when paired with cloud laboratories that let remote users run experiments.16 No one should pretend that engineering a pandemic would be easy. Yet, those responsible for monitoring and verifying such events may increasingly not know where to look until risky work is already well underway.17
AI also could strengthen tools for evaluating compliance with agreements and norms without replacing human judgment.18 Machine learning applied to satellite imagery may help analysts notice changes at sensitive facilities.19 Data fusion across platforms could assist interpretation of noisy indicators, from procurement patterns and orders for biological materials or services to remote laboratory activity. Digital twins of sensitive facilities could help inspectors from the International Atomic Energy Agency and the Organisation for the Prohibition of Chemical Weapons prepare better questions before they arrive at sites.
Although not every catastrophic scenario involving advanced technology belongs in the same category, the lesson for nuclear and biological security is that AI may profoundly alter both landscapes long before the usual guardians of restraint know how the world has changed. The Asilomar process endeavors to look beyond weak actors seeking shortcuts and ask how powerful tools may shape decisions about WMD before new patterns become normalized.
Where Diplomacy Has Not Reached
The process is meant to strengthen, not sidestep, existing forums tasked with ensuring nuclear and biological restraint. The NPT and BWC remain indispensable, but neither is well-positioned to absorb the AI question quickly. Consensus diplomacy can discourage bold and timely action regarding emerging technologies, while the BWC’s absence of verification will become far more consequential as AI touches the practical work of synthetic biology. Concerningly, AI could potentially enable deadly biological capabilities to emerge before anyone can confidently say that a weapons program exists.
AI diplomacy has fortunately moved faster in some spheres than arms control diplomacy, although at a higher level of generality. The AI safety summits at Bletchley Park in 2023, Seoul in 2024, and Paris in 2025 drew international attention to frontier AI risk.20 The “responsible AI in the military domain” process—with summits in The Hague in 2023, Seoul in 2024, and A Coruña in 2026—has also done important work.21 These gatherings have a considerable international reach, but their breadth is also limiting. Nuclear and biological arms control can disappear inside larger conversations about AI safety, innovation, military autonomy, and responsible uses.
The Asilomar process is intended to fill that vital space. It focuses on catastrophic nuclear and biological weapons dangers, can move faster than treaty review processes, and aims to bring sometimes disparate expert communities into conversation.
AI’s ‘Pugwash’ Moment?
The April 2026 meeting was only the beginning. The process is an ongoing effort to support new standards of anticipatory arms control among frontier AI laboratories, governments, and security experts. But the next phase must expand the circle beyond participants already convinced that AI will shape global security. Many firms, investors, and research communities driving humanity’s AI future do not yet see nuclear and biological restraint as their responsibility. Many also harbor heroic expectations about the capacity of governments to regulate exquisite tools still being imagined inside the private sector. The time to develop shared AI standards and thresholds in the nuclear and biological domains has already arrived, while oversight is dramatically lagging.
To their credit, some forward-leaning frontier labs have begun hiring staff to think seriously about security consequences of their most powerful models, but the pool of qualified talent is shallow and there is great competition for those individuals. Some firms also have experimented with more restrictive model release practices.22 As a result, several leading AI companies already have endorsed the need for an ongoing Asilomar process, as have multiple states. They share a recognition that the primacy of state governance on issues involving dual-use disruptive technologies has diminished and is unlikely to change trajectory. As such, early engagement between industry and government offers the best prospects for ensuring the development of technologically sound rules and practices that serve both commercial and national security interests.
Even with deeper industry engagement, the participant base must include intergenerational voices from around the world, a point underscored by India’s hosting of the 2026 AI Impact Summit in New Delhi.23 A process shaped predominantly by powerful Western states and their best-capitalized companies would lack legitimacy and likely reproduce inequalities that have long undercut nuclear and biological governance. Those who bear the consequences of new rules should have a role in shaping them.
A second Asilomar conference is planned for April 2027—by the James Martin Center in partnership with the UN Institute for Disarmament Research and the Berkeley Risk and Security Lab—to deepen and widen this discussion. The process takes inspiration from the Pugwash Conferences on Science and World Affairs, founded in 1957 and later awarded the Nobel Peace Prize for helping scientists pull the world back from the nuclear brink.24 There is reason for cautious optimism that this process can awaken similar awareness and collaboration among those who shape transformational AI technologies.
ENDNOTES
1. George E.P. Box, “Robustness in the Strategy of Scientific Model Building,” in Robustness in Statistics: Proceedings of a Workshop, eds. Robert L. Launer and Graham N. Wilkinson (New York: Academic Press, 1979), p. 202.
2. On this “pacing problem,” see Gary E. Marchant, Braden R. Allenby, and Joseph R. Herkert, eds., The Growing Gap Between Emerging Technologies and Legal-Ethical Oversight: The Pacing Problem (Dordrecht, Netherlands: Springer, 2011); Justin Key Canfil, “Convergent Flexibility: How International Law Keeps Pace with Technological Change,” International Organization, Vol. 80, No. 2 (Spring 2026): pp. 237-280.
3. James Martin Center for Nonproliferation Studies, “CNS Organizes Iconic Asilomar Conference on AI and WMD,” April 14, 2026.
4. See “Implementation of the Treaty on the Non-Proliferation of Nuclear Weapons for its Twelfth Review Cycle,” 2026 Review Conference of the Parties to the Treaty on the Non-Proliferation of Nuclear Weapons, NPT/CONF.2026/CRP.4, May 21, 2026.
5. Matthew P. Shearer, Christina Potter, Rachel A. Vahey, Nancy D. Connell, and Gigi Kwik Gronvall, “BWC assurance: increasing certainty in BWC compliance,” The Nonproliferation Review, Vol. 29, Nos. 1-3 (February-June 2022): pp. 47-75.
6. Stephen Herzog and Dominika Kunertova, “NATO and Emerging Technologies—The Alliance’s Shifting Approach to Military Innovation,” Naval War College Review, Vol. 77, No. 2 (Spring 2024): pp. 47-69.
7. Michael C. Horowitz, “Artificial Intelligence, International Competition, and the Balance of Power,” Texas National Security Review, Vol. 1, No. 3 (May 2018): pp. 36-57.
8. David M. Allison, “Integrating Artificial Intelligence with Nuclear Weapons: A New Paradigm of Irreversible Transformation,” The Nonproliferation Review, Vol. 33, Nos. 1-3 (2026): forthcoming.
9. Jane Vaynman and Tristan A. Volpe, “Dual Use Deception: How Technology Shapes Cooperation in International Relations,” International Organization, Vol. 77, No. 3 (Summer 2023): pp. 599-632.
10. Paul Berg, David Baltimore, Sydney Brenner, Richard O. Roblin III, and Maxine F. Singer, “Summary Statement of the Asilomar Conference on Recombinant DNA Molecules,” Proceedings of the National Academy of Sciences, Vol. 72, No. 6 (June 1975): pp. 1,981-1,984.
11. Stephen Herzog, Allison Berke, Yanliang Pan, William C. Potter, and Douglas B. Shaw, “AI is changing biological and nuclear risks; governance must change accordingly,” Bulletin of the Atomic Scientists, June 18, 2026.
12. David M. Allison and Stephen Herzog, “Artificial Intelligence and Nuclear Weapons Proliferation: The Technological Arms Race for (In)visibility,” Risk Analysis, Vol. 45, No. 11 (November 2025): pp. 3,839-3,859.
13. Herbert S. Lin, “Artificial Intelligence and Nuclear Weapons: A Commonsense Approach to Understanding Costs and Benefits,” Texas National Security Review, Vol. 8, No. 3 (Summer 2025): pp. 98-109; Douglas B. Shaw, Isabelle Williams, Patricia Jaworek, Kevin Park, and Pravin Rajan, “Nuclear Security Implications of AI and Emerging Technologies: A FutureSafe Analysis of Risks and Opportunities,” Nuclear Threat Initiative, November 13, 2025; and Joshua A. Schwartz and Michael C. Horowitz, “Delegating Destruction: Coercive Threats and Automated Nuclear Systems,” International Organization, Vol. 80, No. 1 (Winter 2026): pp. 179-204.
14. Daniel Ellsberg, The Doomsday Machine: Confessions of a Nuclear War Planner (New York: Bloomsbury, 2017).
15. Sarah Case Lackner and Zaheed Kara, “Artificial Intelligence and Nuclear Security Governance: Addressing the Risks of Frontier AI,” Vienna Center for Disarmament and Non-Proliferation, December 2025; Muhammed Ali Alkış, Zoha Naser, and Sarah Tzinieris, “A Fifth Face? The Evolving Threat of Nuclear Terrorism in the Age of Artificial Intelligence,” The Nonproliferation Review, Vol. 33, Nos. 1-3 (2026): forthcoming.
16. Jeffrey Lee, Bria Persaud, Barbara Del Castello, Allison Berke, and Gustavs Zilgalvis, “Documenting Cloud Labs and Examining How Remotely Operated Automated Laboratories Could Enable Bad Actors,” RAND Corporation, PE-A3851-1, April 2025.
17. Doni Bloomfield et al., “AI and biosecurity: The need for governance,” Science, Vol. 385, No. 6,711 (August 22, 2024): pp. 831-833.
18. Jane Vaynman, “Better Monitoring and Better Spying: The Implications of Emerging Technology for Arms Control,” Texas National Security Review, Vol. 4, No. 4 (Fall 2021): pp. 33-56; Francisco Parada, Nathan Martindale, Alisa Reasor, Scott Stewart, and Lindsey Ukishima, “AI for Nuclear Safeguards Verification,” Oak Ridge National Laboratory, ORNL/LTR-2024/3674, October 2024; OPCW temporary working group on AI, “Artificial Intelligence: Report of the Scientific Advisory Board’s Temporary Working Group,” Organisation for the Prohibition of Chemical Weapons, SAB/REP/1/26, March 2026.
19. Yanliang Pan, “Next-Generation OSINT: Machine Learning for Tracking Nuclear and Missile Proliferation,” The Nonproliferation Review, Vol. 33, Nos. 1-3 (2026): forthcoming.
20. Tony Oweke, “The World Is Trying to Govern AI. The UN Wants In.” Council on Foreign Relations, May 29, 2026.
21. Giacomo Persi Paoli and Yasmin Afina, “AI in the Military Domain: A briefing note for States,” United Nations Institute for Disarmament Research, March 2025; Jules Palayer and Laura Bruun, “Artificial intelligence and international peace and security,” in SIPRI Yearbook 2025: Armaments, Disarmament and International Security (Oxford University Press, 2025), pp. 329-346.
22. See, for example, Anthropic Frontier Red Team, “Assessing Claude Mythos Preview’s cybersecurity capabilities,” Anthropic, April 7, 2026.
23. Indian Ministry of Electronics and Information Technology, “India AI Impact Summit 2026,” February 16-20, 2026.
24. John P. Holdren, “Arms Limitation and Peace Building in the Post-Cold-War World,” Nobel lecture on behalf of the Pugwash Conferences on Science and World Affairs, Oslo, December 10, 1995.
Douglas B. Shaw is a nonresident scholar at the James Martin Center for Nonproliferation Studies at the Middlebury Institute of International Studies at Monterey. Stephen Herzog is the James Martin Center’s professor of the practice and an associate of the Project on Managing the Atom at Harvard’s Belfer Center for Science and International Affairs. William C. Potter is the director of the James Martin Center and its Sam Nunn and Richard Lugar professor of nonproliferation studies.