I haven’t lost any sleep pondering the more philosophical dimension of AI, until now. Recent breakout advancements have forced me to reexamine assumptions and fine tune where I stand… and I will admit – lose a little sleep. When I started digging deeper into AI coding assist over a year ago, I was impressed, but not so impressed as to activate my fight-or-flight response. And then life happened, distracting me at the decisive moment, as life does. Imagine my surprise at suddenly hearing, not the roll of distant thunder I was prepared for, but the urgent beat of drums echoing from the depths of Moria. It was time to do some more digging and develop an updated first-hand point of view. And the outcome? Suddenly, it matters how much of a human can be poured into a machine, because intelligence has been commoditized, and one wonders what else can be. Whether you are inclined to philosophical exploits or not, what you think about that question will inform your approach to the technology, particularly in terms of the trust paradigm you adopt. The last responsible moment for developing that point of view has arrived. All aboard that’s coming aboard.
John Steinbeck wrote a short story called The Affair at 7 Rue de M—. I came across the story in a high-school literature class, back when the earth was young. In this instance our assignment was to write a response to the story’s punchline. And because it interested me, I put some real effort into it. I didn’t know it at that time, but somehow, this trifle of a story, written in 1955, about chewing gum, manages to capture what may be the essential thing to come to terms with about AI. To place the central quote in context – the story involves a young man (John) who has developed an inordinate affection for chewing gum. In the course of the story it comes about that he is not so much chewing the gum, as that the gum is using him to be chewed.
It must be that from constant association with the lambent life which is my son, the magic of life had been created in the bubble gum. And with life had come intelligence, not the manly open intelligence, but an evil calculating wiliness. How could it be otherwise? Intelligence without the soul to balance it must of necessity be evil. The gum had not absorbed any part of John’s soul.
The Affair at 7 Rue de M—, John Steinbeck, 1955
And there it is. The idea of a truncated humanity is the central trope of the horror genre. And now we have machines that we have imbued with enough of ourselves to mimic intelligence. And by mimic I mean “walks like a duck, quacks like a duck, it must be a duck”. Close enough that I am not so much inclined to ponder whether it is “real” intelligence, as to ponder what exactly we mean by intelligence in the first place. Now, let me make something clear. This article is not about Skynet, or our evil robot overlords, or about banning thinking machines, or pondering whether the machine is alive, or anything else of that ilk. It is about the things we must grapple with to leverage these machines in a way that is congruent with human thriving.
As a society we have come to place a profound premium on intelligence. This has been true perhaps since the industrial revolution and certainly by the time of the computer revolution – long enough to seem as though “it was always thus”. After all, intelligence was the commodity that made the modern world possible. And it existed in only one place in sufficient quantity to make that world possible – sparking around between neurons inside living breathing humans. It’s as if we came, uncritically, to place intelligence at the very center of what it means to be human. I think, on reflection, that is because there hasn’t been any pressing need to tease out the various things that normally come as a package deal with intelligence. Back to our question – how much of what it means to be human can we pour into the machine? Does it stop at cognitive reasoning?
Let’s take up the “no” side of this question first and suppose a machine can do all or most of what a human can. The paradigm of trust could then be essentially the same as it is between people. If you assume the machine can eventually take on all the meaningful attributes of a person then you budget for a similar basis to establish trust. This isn’t free sailing. Despite the common-place that people are basically good (in the face of oceans of evidence to the contrary) we don’t actually conduct our affairs as if that’s true when it really matters. We have binding agreements and consequences when those agreements are breached. In our saner moments, we typically attempt to limit concentration of power. Zero trust is our standard of security. So at the very least we need a way to enforce a trust paradigm for a machine that is faster and smarter than we are, but that has also somehow learned to play nice. Not trivial, but possible to at least grapple with on familiar terms – if we think the machine can be made to respond to incentives like a human.
Now take up the other side of the question. It is primarily symbolic reasoning capability that can be poured into the machine, and, as Steinbeck asserts, there are things beyond this that matter. You don’t really have to know where exactly the transferrable attributes stop and the evanescent attributes begin. You just have to be persuaded that both are a vital part of how humans go about humaning. The paradigm of trust here is fundamentally different. Not only is the machine faster and smarter, but you need to budget for it being more devious, and ruthless than you. More importantly, there is no basis for assuming it is capable of self-sacrifice. I don’t offer development of the point here, but that’s a problem. In both paradigms you have to care rather a lot about how trustworthy the people training the machine are. But in this case you have, also, to consider that flawed people are training a machine with a truncated home for that intelligence to reside. It might reason with remarkable power, but like a power saw – the moment you don’t respect the machine is the moment you lose fingers, if you’re lucky.
Before moving on, I’ll throw in the idea that for any trust framework to be effective, in either paradigm, there need to be guarantees that the person who can absorb risk at any given level (the person with something to lose if a risk is realized) has meaningful ability to give consent, or not, to taking that risk. The way this works with people is challenging enough – see Risk Management.
To this point, I’ve been very equivocal because I think it’s important to explore the problem space. Now I’m going to lean harder into the second paradigm, because I believe that is closer to reality. I am not a philosophical materialist and I think people are more than molecules, and the “more” matters. Also, I think this is the more challenging scenario, and the one we need to be prepared for. This is assertion, not persuasion. To genuinely persuade anyone that needed persuading would be asking rather a lot of a blog article. What I do want to persuade you of is that this matters, and deserves more than a passing thought. That is what I have changed my mind about. I’ve gone from “a powerful tool in the right hands”, to “a profoundly transformative tool in the right hands – and terrifying in the wrong hands”. Our job is to harness the tool and make sure human thriving occurs, and not to be naïve about what that requires. Do I trust the people training models? Ummm, no, not really. I’m not automatically assuming ill intent, but neither do I trust their judgement. Do I trust the AI itself? No, not really. And here is where it bites deep. These tools do their most impressive work when they have more trusted access to more critical assets. You can very understandably object – it will slow us down if we insist on being “fussy” about risk. But think about this – how much time, process, and infrastructure is currently dedicated to security in the average enterprise stack (even if you only consider actual security as opposed to performative compliance)? Imagine how much more we could do if we just didn’t have to pay the bad actor tax? But we do, and this isn’t a new problem. More to the point… the bad actor might be the system we’re trying to secure. Who’s idea was it for that to have to make sense? But we have known for at least two decades that the only way to control the machine is with the machine. Even more critical, we have to bring everything it means to be human to the hard work of turning what appears to be the most disruptive technology in a lifetime, at least, to good effect. If that seems too hard, or not what you signed up for, consider – others will most certainly be working hard trying to turn it to ill effect. I would rather fail trying to build a world overflowing with the blessings of hard work and virtue, than succeed at the alternatives. And, I think this is going to be a Bring Your Own Soul party.
Postscript
So, why am I writing this – by hand, struggling to put down specific words, stubbornly refusing to let the machine correct my comma usage? I could give the machine samples of my writing, throw out a skeleton of essential points, and ask the machine to turn it into an article. Well… I think it matters. It doesn’t matter for the things that make up most of my daily word count. But for some things, it matters. Fighting with the words to force them onto the page matters. I am asserting the value of a very human activity because I believe there is something about this (no matter how inexpert the writer) that somehow transcends the machine. Who knows, maybe this will turn out to be the last blog post written by hand and end up in a robot museum somewhere.