The Church's AI Watchdog Warns of Tech Cartels

AI's Hidden Frontier

Recently, the global AI community has been gripped by warnings from researchers inside the world's largest artificial intelligence labs about the rapid pace of development. These fears center on the idea that, propelled by fierce competition and geopolitical rivalry between the US and China, labs are racing toward superintelligent models whose abilities could far surpass those of humans. Yet, rather than panic, we should be examining who controls this emerging frontier.

I've been watching closely how these powerful players—OpenAI, Anthropic, DeepMind—frame their AI ambitions. They often present themselves as the guardians of humanity's future, but what's missing is a more inclusive conversation about the ethical and regulatory frameworks that must guide these technologies. Enter Father Paolo Benanti, a priest and AI advisor to the Catholic Church who challenges this narrative with a calm yet pointed voice.

"Ethics cannot be something that belongs to the board of trustees of a company," says Benanti. "It's something that has to be balanced, mitigated in debate in the public square."

This isn't just about morality—it's about governance. As AI becomes more entrenched in society, we must ensure it remains accountable to all, not just those who control its development.

Cartels of Innovation?

Benanti makes a compelling point: when a few powerful companies dictate the rules under which an entire industry operates, that's not innovation—it's cartel behavior. And if those cartels are shaping policy and regulation behind closed doors, the consequences could be profound for public trust and democratic accountability.

There's been a lot of noise about how AI systems might evolve beyond human control, especially around recursive self-improvement—a process where AI models enhance themselves using their own code. Critics argue that we need to regulate such behaviors, but Benanti pushes back on top-down restrictions that may stifle creativity. Instead, he advocates for transparency and a clear set of shared principles.

He warns that these companies often frame their work with a kind of reverence typically reserved for the divine. When AI is spoken of as a "magic intelligence in the sky," it becomes harder to have rational discourse about its risks and benefits. That's not just problematic—it's dangerous, because it shifts focus away from the real decisions humans must make.

The Vatican's Call for Collaboration

In May, Pope Leo XIV issued his first-ever encyclical on artificial intelligence—a document that reflects the Church's evolving role in the digital age. In it, he condemned what he called the technocratic logic of the AI industry—where efficiency is prized above all else and human beings are reduced to data points in a system designed for optimization.

"The danger of humanity becoming a victim of its own achievements," the Pope warned, echoing concerns raised by Benanti. This isn't about fear-mongering—it's about ensuring that technological progress serves the common good rather than enriching a select few. The encyclical calls for international cooperation between rival powers and companies, urging them to set aside geopolitical ambitions in favor of ethical development.

What's particularly striking is how this message intersects with Benanti's own concerns. While he doesn't speak officially for the Church, his role as a scholar of technology allows him to offer a unique lens into how these developments impact not just economics or science, but also society and faith. He emphasizes that the real challenge lies not in creating smarter machines, but in maintaining human agency in their design and deployment.

Who Gets to Decide?

The question of who governs AI is becoming increasingly urgent. Currently, decisions about safety standards, data usage, and ethical boundaries are made primarily by large tech firms with enormous financial stakes in the outcome. This is not a sustainable model—especially when those same firms are preparing for public offerings and competing fiercely to dominate global markets.

Benanti sees this dynamic as both economically and morally unsustainable. "The narrative of the frontier labs is self-interested, to gain or maintain a position in the market," he says. "This has a really clear name: a cartel."

I've seen similar patterns play out in other industries, particularly during periods of rapid innovation—when private interests begin to overshadow public accountability. We've seen this with pharmaceuticals, financial services, and even social media. If we don't take steps now to prevent AI from following the same path, we risk a future where the benefits of innovation are concentrated among a small elite.

Building the Right Brakes

One of Benanti's most important insights is his emphasis on control—not control by companies or governments, but control by humanity itself. "We need to have brakes and a wheel," he argues. That means ensuring that even as AI grows more capable, it doesn't outpace our ability to direct it toward beneficial ends.

This is about balance—between speed and safety, innovation and ethics, ambition and accountability. It's also about transparency. We must not allow companies like OpenAI or Anthropic to define the limits of what's possible without meaningful input from citizens, ethicists, and public regulators.

In an age where artificial intelligence shapes everything from healthcare to finance, it's time we ask: who is steering the ship? And more importantly, are we all on board?

A New Role for Faith

Benanti believes that AI's rise presents a unique opportunity for religions to play a meaningful role in public debate. It's not about religion dictating policy—it's about providing moral clarity in an increasingly complex world.

The Catholic Church, through figures like Benanti, is positioning itself as a voice of reason amid the hype and fear surrounding AI. And while it may seem unconventional for a religious leader to weigh in on tech policy, there's wisdom in that perspective. After all, when machines become more intelligent than us, we still need to be clear about what we value.

What's next? We're likely to see continued tension between the push for faster development and the call for broader societal input. If we're going to avoid the mistakes of previous technological revolutions, we must embrace this dialogue—not just among scientists, but across all sectors of society.