Check out the latest episode of That Nerdy Catholic!

The era of AGI? Ask the Sci-fi prophets.

3 minutes to read

The “AGI era” is less about the technology and more about our attitudes around the technology. As I was reading the news (and hype) about Astra this morning, my copy of Pope Leo XIV’s Magnifica Humanitas sitting here next to me seemed like one of the most important documents written in my lifetime. At the same time, Brian Herbert’s (son of Frank Herbert) Dune: The Butlerian Jihad suddenly seemed prophetic. I think the question is less “are we ready” for AGI, and more about whether-or-not we’re ready for the global disruption to the idea of “work” that’s unfolding before our eyes. It always strikes me as odd that the conversation around A.I. centers almost solely around how close we are getting to be freed from work. How will we make a living in the post-AGI era? How will we feel a fulfilling sense of accomplishment, growth, and learning in our work without the annoyances and frustrations that go along with it? Why is it that we think, in our pursuit of freedom from work, that we’re not becoming slaves to the machines?

This From the Wall Street Journal’s CIO Newsletter this morning:

There’s no standard definition of AGI, and that makes it hard to assess when it has arrived. Venture-capital investor Vinod Khosla told me almost two years ago that he “describes it as the point at which AI can perform 80% of the work involved in 80% of the world’s economically valuable jobs.” While we may not clear that bar for some time, it feels more and more inevitable that we will before too long, and that’s why I think we’re beginning to enter the era of AGI.

Fun fact: The Butlerian Jihad is somewhat of a tribute to the Victorian Era author, Samuel Butler. His letter, Darwin Among the Machines (1863), an ironic and satyrical read about “machinery” being almost superior to us humans, but not because of their intelligence, per se, but because of their efficiency: they don’t succumb to our moral shortfalls:

Inferior in power, inferior in that moral quality of self-control, we shall look up to them as the acme of all that the best and wisest man can ever dare to aim at. No evil passions, no jealousy, no avarice, no impure desires will disturb the serene might of those glorious creatures. Sin, shame, and sorrow will have no place among them. Their minds will be in a state of perpetual calm, the contentment of a spirit that knows no wants, is disturbed by no regrets. Ambition will never torture them. Ingratitude will never cause them the uneasiness of a moment. The guilty conscience, the hope deferred, the pains of exile, the insolence of office, and the spurns that patient merit of the unworthy takes—these will be entirely unknown to them.

Of course, that’s the irony. A created thing always takes on the nature of its creator. This is the reason why the Orange Catholic Bible (in the Dune universe) exists - humanity needed to realign with a collective moral foundation after the Crusade against the machines and their inventors…From Dune:

“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."— Reverend Mother Gaius Helen Mohiam (quoting the Orange Catholic Bible)

From Pope Leo’s Magnifica Humanitas:

Work remains a fundamental dimension of the human experience, for not only is it a means of sustenance, but it is also a context for expression, relationships and contributing to the community.” … “In this context, without bold decisions, the prospect of greater poverty and inequality looms large, which would leave many individuals marginalized, stranded and surrounded by the machines and automated systems that have replaced them.

A group of people in white robes wielding primitive weapons appear to be fighting or attacking robotic figures under a large yellow moon inside a futuristic setting.