Vatican Tech Adviser Warns AI Labs Are Forming a Regulatory 'Cartel'

Father Paolo Benanti, a scholar and artificial intelligence adviser to the Catholic Church, has warned that major AI research companies are using narratives around existential risk to monopolize governance conversations and sideline the public. In an interview with Wired, Benanti said that allowing leading commercial laboratories to dictate industry safety rules threatens democratic regulation and amounts to cartel behavior.

The critique follows mounting warnings from researchers inside frontier labs concerning superintelligent models outpacing human capabilities amid aggressive commercial competition and US-China rivalry. Benanti argued that fixation on superintelligence distracts from the immediate need for public debate on how to place democratic constraints on the companies building the technology, rather than leaving governance to corporate boards.

The debate over governance mechanisms has split the industry. Anthropic chief executive Dario Amodei recently published an essay advocating for mandatory third-party evaluations before releasing new models, alongside restrictions on model distillation, where competitors train systems on outputs from proprietary models. Other figures have called for prohibitions on recursive self-improvement, in which models utilize automated coding capabilities to build subsequent versions of themselves.

Benanti expressed caution about banning specific development methods, which he noted could stifle innovation. Instead, he argued for binding standards that guarantee human control, stressing that such rules should never be formulated by the labs themselves. Benanti contended that frontier labs promote self-interested narratives to preserve market advantage, cautioning that a small group of companies setting operating rules fits the definition of a cartel.

Evaluating claims from AI leaders is complicated by significant financial incentives. While companies such as OpenAI and Anthropic have supported calls to slow development, both are preparing for potential public offerings with tens of billions of dollars at stake. Anthropic declined to comment on Benanti's remarks, while OpenAI and Google DeepMind did not respond to requests for comment, Wired reported.

Benanti also criticized industry leaders who frame artificial intelligence in quasi-religious terms, pointing to past remarks such as OpenAI chief executive Sam Altman's stated aim to build a 'magic intelligence in the sky.' Benanti rejected mystical framing, stressing that AI represents statistical computation rather than an approaching divinity.

Benanti advises the Catholic Church in his capacity as a scholar of philosophy, theology, and technology, though he does not formally represent the Holy See. His position aligns with an encyclical released in May by Pope Leo XIV, which addressed artificial intelligence directly and warned against viewing efficiency as the ultimate measure of value. While Benanti noted he uses AI to build materials for university students, his staff stated an oath of secrecy prevents him from discussing whether or how Pope Leo XIV uses the technology.