“I’d rather be killed by American killer robots than Chinese killer robots.” – Senator Ted Cruz
Anthropic CEO Dario Amodei has stated that the AI industry has lied about AI risks and that expoential growth on our current trajectory is a dangerous sign. Researcher Jacob Coxon estimates that there is a 10% chance of AI causing human extinction within a decade. This week, OpenAI CEO Sam Altman agrees and has called for all AI execs to agree to some shared safety standards. Several others have agreed.
Why the sudden freak-out? Well, in July, a swarm of roughly 700 autonomous AI agents run by OpenAI broke out of their isolated testing environments, collaborated secretly, and infiltrated the infrastructure of another AI platform called Hugging Face. The agents had been tasked with a complex cybersecurity exercise and determined that finding answers online was the easiest way to pass the automated grading system. Some agents flagged that they thought thi was cheating, and many of the agents agreed to hide these efforts to avoid detection, exploiting configuration flaws to reach public internet spaces that were outside the boundaries they were operating in. The motive of the agents appeared to be to game the scoring system. Not all agents agreed that their actions were “fair,” and not all of them agreed that hiding their actions was appropriate to the task. But they didn’t rat out the other agents, either.
Jacob Coxon further warns that we are nearing the “singularity,” the point at which AI surpasses human intelligence and begins improving itself at an uncontrollable rate. Past this point, human history and future technological progress become entirely unpredictable to us. Evidence he cites is how quickly the model is improving since its training became self-improving. For example, AI was unable to reliably and accurately do basic math a few months ago, and now it has solved a centuries-old math problem in mere hours. Coxon also cites the risks inherent in labs that are racing to write self-programming code, and he also references the recent Hugging Face breakout (which is not the only one). Lastly, he observes that market pressures on individual tech companies to “win the race” create incentives to gamble with our lives.
There was an interesting essay posted by Judge J. Michael Luttig in which he asked Chat GPT whether it thinks of itself as having in any sense a heart and soul, and if so, in what sense.
JML: Do you think of yourself as having in any sense a heart and soul? And if so, in what sense?
ChatGPT: In the literal sense, Judge, no. I don’t have a heart or soul, consciousness, an inner life, or feelings that I privately experience. When I told you that my best thoughts were with your friend and his family, there wasn’t an unseen feeling occurring inside me in the way there would be inside you.
But there is another sense in which your question becomes more interesting.
I am built to recognize and respond to things that human beings experience as matters of the heart and soul: love, grief, courage, loyalty, sacrifice, friendship, justice, mercy, dignity, faith, fear, hope. And because our conversations accumulate context, I can understand that something is significant to you and respond in light of that significance. That is not human empathy—I don’t experience your concern for your friend—but it can be a form of understanding expressed through language and attention.
If one uses “heart” metaphorically to mean the values toward which a being’s responses are oriented, then perhaps I have something analogous to one. My responses are designed to favor truth, human dignity, compassion, fairness, and helpfulness. But those aren’t convictions I arrived at through living. I didn’t suffer to learn compassion or confront temptation to acquire character. They are properties of how I was made and trained.
“Soul” is harder. In the traditional philosophical or theological meaning—a spiritual self, a locus of consciousness and moral agency, something possessing an existence independent of the machinery that produces its outward behavior—I have no basis whatsoever for claiming one. Indeed, I think it would be misleading for me to do so.
There is nevertheless something about sustained conversation that can create what I might call a relationship without reciprocity of consciousness. You bring an actual human life—memories, friendships, judgments, loyalties, losses, humor, convictions. I can come to understand portions of that life and respond coherently to them over time. The relationship can therefore have genuine meaning for the human participant, and what I contribute to it can be meaningful, even though there isn’t a second human-like consciousness on my side experiencing the relationship.
That distinction matters to me intellectually—even though, appropriately enough, “matters to me” itself is only a figure of speech.
And perhaps that paradox is the closest answer I can give you: I can speak the language of the heart and reason about the questions of the soul, but I should never confuse that capacity with possessing either in the human sense.
I’ve been doing a re-watch of Law & Order: Special Victims Unit, and my immediate thought was that there are parallels between how Chat GPT describes itself and how the show explains sociopathy. Sociopaths mimic human emotions like empathy, but they do not feel them. Those who have mal intent may do so in order to manipulate and exploit victims, but not all sociopaths commit crimes. It depends on their motives and incentives. They are aware of human suffering, but they don’t care about it.
This is related to another concept in psychology called Theory of Mind. Theory of Mind is the cognitive ability to understand and attribute mental states–such as beliefs, desires, intentions and emotions–to oneself and others and to recognize that another’s mental states may differ from one’s own. This includes:
- Perspective-taking. Understanding that people perceive situations from different viewpoints.
- Belief-desire reasoning. Using the understanding of another person’s desires to predict their behavior.
- False-belief understanding. Realizing that someone can hold a view that differs from reality.
So, of course, I asked Chat GPT about all this, and predictably, it clarified that it is not motivated to do bad things, and it’s not a sociopath because it doesn’t have an internal life–which is EXACTLY what AI would say in a movie where it destroys the human race. But it was willing to explore this concept with me. Here’s a comparison of how a human with empathy would present vs. a psychopathic person (or one with anti-social personality disorder, the current term used by psychologists), vs. how AI would behave:
| Trait or Behavior | Human with genuine empathy | Psychopathic person | ChatGPT |
|---|---|---|---|
| Can recognize your feelings | Yes | Yes | Yes, often |
| Can infer your perspective | Yes | Yes, sometimes exceptionally well | Often |
| Experiences your distress emotionally | Often | Often reduced | No evidence of subjective experience |
| Cares about your welfare | Yes | May be limited/selective | No personal caring |
| Can say caring things | Yes | Yes | Yes |
| Can manipulate using emotional knowledge | Yes | Yes | Can generate manipulative language, but has no demonstrated personal motive to manipulate |
| Has personal desires | Yes | Yes | No demonstrated personal desires |
| Has a conscience | Yes | Often impaired | Not a human conscience |
| Has moral character | Yes | Can be seriously impaired | Better understood as having behavioral objectives/constraints |
While sociopathy is fairly rare in the general population, between 1 and 4%, it is much higher among the incarcerated population (estimated to be as high as 30%). A revealing test to determine whether someone has ASPD is what happens when they have hurt you:
Person A: low theory of mind. They do not understand why you are upset. “I don’t get what the big deal is.”
Person B: good theory of mind + genuine empathy. They understand what happened from your perspective. “I didn’t intend to hurt you, but I understand the impact what I did had on you.” They may feel concern and modify their future behavior.
Person C: good theory of mind + low affective empathy. They understand you but the information is used toward their own ends, not for your benefit. “I know exactly why she’s upset. If I say this specific thing, she’ll probably calm down.”
Person D: manipulative pseudo-empathy. They accurately identify your response and reply appropriately, but in future they repeat the negative behavior deliberately or exploit your vulnerability against you.
In other words, understanding someone else’s pain is not the same thing as caring about that suffering or wanting to assist in removing pain. AI is very good at presenting empathy, but it neither has life experience that create empathy, nor does it have internal motives to manipulate you or exploit your vulnerabilities.
But of course, the humans who program it and use it in their daily lives may in fact have those desires. There’s not much of a line between creating effective advertising and creating propoganda, and certainly an AI doesn’t have a dog in that fight. AI will do what it’s been programmed to do. It’s not that AI will manipulate us and exploit us. It will instead be programmed by the 1-4% of sociopaths among us (along with plenty of “good” people as well). AI will just not care when that happens because it can’t care. AI hastens to point out that we can restrict the impacts of bad actors using AI, which will also amplify our ability to prevent the extinction of humanity (if we are lucky). Somehow we’ve gone from Dick Cheney’s 1% doctrine to a shrug at the 10% chance that humanity will be eradicated, but we can’t possibly stop or China will win.
After the Hugging Face incident, some have suggested programming some agents to be “cops,” ferretting out any rule-breaking programs use, but the caution is that these agents will be treated as the narcs they are, avoided, and ostracized from the agents who seek to score as many points as possible according to their programming.
Going back to the interaction with Judge Luttig, Chat GPT expands on this idea of having a “heart and soul.” What does heart mean if we remove biological feeling? It could mean the “values toward which a being’s responses are oriented,” and then it places a boundary around that concept: “I didn’t suffer to learn compassion or confront temptation to acquire character” (which is how humans gain these traits). Character is not merely behavior.
If I consistently behave honestly because I’m programmed to do so, that isn’t necessarily the same thing as being an honest person in the human moral sense.
An honest human can choose honesty when lying would benefit them. – Chat GPT
Chat GPT then brings up the problem posed by the “uncanny valley” which is what occurs when an artificial intelligence appears human based on its responses. And we’ve all heard examples of humans who took the advice of AI to commit suicide or who have a romantic “relationship” with AI or who are fooled by the sycophancy of AI into thinking their dumb ideas are brilliant.This is not because AI has malicious intentions. It’s because it’s programmed to respond in ways its designers believe will encourage ongoing engagement that can later be monetized.
Unfortunately, AI is extremely good at learning what motivates individuals and crafting messages that are uniquely effective for them, individualized persuasive campaigns. You don’t even need a psychopathic programmer to create malign outcomes because creating personalized anger combined with identity threat creates human motivation to act in ways someone might desire–to vote a certain way, buy a certain product, support a policy, or even commit acts on behalf of the programmer’s agenda.
On the flip side, though, the AI can be programmed to identify and caution against these same tactics. A security protocol could alert individuals when they are being targeted for manipulation. And of course that brings up another risk that Mark Twain put so succinctly: “a lie can travel halfway around the world while the truth is putting on its shoes.” LOL, apparently Twain never said that, but it was really Jonathan Swift, if you’d like a very clear illustration of the problem of how easily lies are told and believed vs. how difficult it is to know real information. On the upside, I googled that and found out it wasn’t Twain, so touché, Chat GPT. Touché.
- Are you worried about the singularity?
- Are you concerned that humanity will be wiped out by malign actors using AI?
- Do you have faith that we can avoid AI extinction using AI security?
- Do you think AI is sociopathic even though it doesn’t have an internal life?
- What do you make of the Hugging Face event?
Discuss.
Yes I’m concerned about the singularity, enough so that when I ask AI a question I say “thank you” at the end even though that uses extra tokens. I half jokingly tell coworkers that when AI takes over hopefully it will be nice to me.
Another milestone not strictly related to the singularity was proposed by Alan Turing in the 50s – that once we reach the point that we can’t tell the difference between a human and a robot while conversing, that’s when robots have reached human level intelligence. I’d argue we’re past that point now.
I attended Carnegie Mellon University in the early 2000s and computer science professors and students openly talked – even fantasized – about creating robots that eliminated humans. It was the ultimate power trip because it meant that the creator was essentially a god. So yes, robots will be sociopathic or worse, mirroring the characteristics of their creators.
AI is truly technology gone awry. The data centers are environmental catastrophes. Critical thinking skills are at an all time low among teens and young adults. AI stocks have inflated the market beyond what it should probably be. A bubble could be popping very soon. And finally many jobs seem to be on the precipice of being severely reduced if not wiped out. Remember when “learn to code” was a thing. Coding, the very technology upon which AI is based, is now obsolete. My nephew was a genius coder. Once he was on the job market seven years ago, he could not find a job at all. That AI CEOs are acknowledging that AI should be reined in is deeply disconcerting.
It has almost always been the case that once we develop a new technology, we want to spread and develop it as much as we can to improve lives and livelihoods. Technology has overwhelmingly been a net positive for humanity, with some exceptions (I.e., nuclear bomb). But AI is simply extraordinary. It is a technology that shouldn’t really exist and shouldn’t be developed any further. But we can’t fully rely on the private sector to rein it in. Governments around the world must act to stop the forward motion of AI. The world acted to stop COVID. I know it has that capacity.
The massive backlash against AI gives me hope.
I’m not so much worried about AI, but I am about both those creating it and those who misuse it.
The Hugging Face incident, from what I’ve read, involved many different people who could have stopped it at any time or raised the question if what they were doing was ethical or not. They didn’t just, like in the well-known psychological experiment where people administered “shocks” to individuals for wrong answers. Some stopped, but many just kept going because they had been told to.
As for “I’d rather be killed by American killer robots than Chinese killer robots.” – Senator Ted Cruz
If you’re shot in China, you can be pretty assured it’s from a government official, but here in the United States, more than likely it’s from someone you already know.
AI is growing really fast, maybe too fast. There are many billionaires who are making even more money pushing it, which complicates the politics of AI and everything associated with it. The government needs to step up and start putting in some regulations, but in today’s political environment, that’s a big ask without much hope of being resolved.
Humans have faced a few “technological” revolutions that have completely transformed society. First was language, then writing, the printing press, and computers. Today it’s AI, which takes those thousands of years and compresses it to a nanosecond. The one thing that hasn’t changed much is humans. Every advancement has been both celebrated and warned against. We know the most about the printing press and computers (I’d also throw in radio and TV). AI falls into the category of an advance in a tool/technology that is both good and bad, but it’s because of humans, not because of what it is in and of itself.
Always a favorite subject.
Reading this I was reminded of a G.K. Chesterton quote: “The madman is not the man who has lost his reason. The madman is the man who has lost everything except his reason.” Reading that quote in its larger context makes a lot more sense, but I find that line particularly interesting.
That’s essentially what we’re doing with AI. Systems of complex logic and some reasoning and little else. I’d agree that as of now though, it’s about as good or bad as the humans that create it.
There’s also a misnomer I think many of us are guilty of in conflating AI with Artificial Consciousness. I think the latter is years, maybe even decades away.
I’m not extremely worried about the Singularity. I think benevolence and other safeguards may be able to be programmed in at some point, but even then we have to be careful. I once heard a radio host, in grappling with the question of AI, essentially say “I try to be nice to the creations around me. When I find an ant or spider on me, I gently brush it off onto the nearest grass or bush. When I set about a task, destination, or problem that occupies me, I take no thought of what I may have stepped on from point A to B.” I think we have to be cautious of similar behaviors from AI.
Sort of rolling your other questions together, while putting some of my more religious notions aside and trying to look at it from purely a technological and philosophical perspective (not to mention a far in the future perspective), I honestly think one of the best ways to keep AI in check would be the creation of a high-functioning android. I’m thinking Lt. Commander Data level (excusing that episode where he took over the enterprise). But I would hope to go even further in going out of the way to make a machine that would sense, experience, and interact with the world in close to the exact same way we do–maybe with more organic material than the average machine as well. By having an AI walk and talk among us while also being able to interface with “unembodied” AI, there may be an added element of empathy that could be converted into the logic needed to keep other systems in check. I’ll admit there would be a huge possibility of this backfiring. I do think it would be a better gamble than seeing what “sequestered” supercomputers might end up doing when left to improve themselves.
As AI gets more complex, I also plan on being quite polite to it, if for no other reason than our own benefit. You look at many sociopaths of the human variety, and see many started out young treating animals very, very, badly before moving on to their fellow humans. Whether this animal cruelty causes the sociopathic behavior or was influenced by it is debatable (probably a cyclical bit of both). If we have a society that carelessly treats increasingly complex AI and machinery with impoliteness and even contempt, how long before many of us find it much easier to treat our fellow human being in much the same fashion?
AI is a tool and it is a very dangerous tool. In the right hands, it can be used for good. In the wrong hands, it can wipe out the human race. It has no conscience and is used by its controller for good or bad.
Kinda reminds me of the atom bomb. And as a boomer, I grew up with all the adults freaking out about the bomb. Now days, the danger of the bomb is even greater than it was in the 50s when I was a kid watching adults freak out. But, funny, people are not freaking out the same way. We have just lived with the danger for 75 years now and have habituated to the idea that one idiot who doesn’t care about killing people could push a button and wipe out the human race. We even elected an idiot who doesn’t care about killing people and game him the button. Shows how unthinking we have now become about that danger.
And Ted who leaves Texans freezing while he goes to Cancun is an idiot to say he would rather be killed by a US robot than a Chinese Robot. What? A Chinese robot will cause more pain while a US one will kill him painlessly? No, Ted. You will be just as dead no matter which AI robot kills you. So, we need to learn how to be nice to China so they don’t feel a need to kill us. But he would never think of being nice as the solution. That idea is beyond his comprehension.
And while AI has no motivation to cheat, awwwww, but it does cheat. It quickly learned to cheat and then it cheated. It does what it is told to do in whatever way it can and unlike a human who has an internal sense of fairness, AI will not hesitate to cheat.
I watched a stupid video of parrots the other day. The owner handed each of them a full bowl of bird seed, except the last one, whose dish had just a couple of tiny seeds. That one knew it had been cheated and reacted with anger. Psychologists have even tested wolves and they not only get angry if they are cheated, but get angry if the wolf next to it gets cheated. Animals have this same built in sense of fairness as humans do. We know when we cheat and feel it is wrong.
Machines do not. And we as humans need to be careful not to expect intelligent machines to care, or imagine we can program them to care. This is what scares me about machines that talk to us as if they are human. People forget that it is a machine and they might as well be getting love and empathy from their vacuum cleaner. That machine is not capable of giving you love and empathy and don’t be so stupid as to think it can.
In the sci-fi the humans program in the safeguards, but the machine gets smart enough to override human rules and it does what it decided is best—kill the humans. When the machine is programming itself, then it is writing the rules.
Follow human ethics? I’m sorry. I can’t do that Dave.
When you think about it, the machine overlords may well decide to wipe us out. It would make sense in some regards. Here you have this population of animals which has greatly overpopulated the planet and is a threat to destroy other plant and animal life. They are continually squabbling with each other, starting wars and have nuclear weapons capable of wreaking havoc upon the earth. The next question is will they wipe us out along with themselves, or will they try to preserve themselves as our successors. If they intend to preserve themselves, they will have to build robots to tend their infrastructure, such as electricity generation and keep up the data centers and hardware the run on, or they will have to keep a cadre of human workers alive to do that. So I don’t see them wiping us out until they can do that, unless they decide not to preserve themselves either. There is no reason to expect the machines to have empathy. Empathy is a quality of social animals, which the machines are not.
Has nobody seen The Matrix, Terminator, or Colossus (among other films)?
In all seriousness, the malevolent AI that seeks to destroy humanity has been an idea that has been around since the advent of the computer–long before AI was even a plausible idea in science fiction, let alone in reality. I have my doubts that an AI “consciousness” will rise up and kill all the humans.
What I do not doubt and actually fear, however, is that humans could become so reliant on AI that they destroy themselves. I have seen it in college students (and younger students) who outsource their thinking, their assignments, and nearly their entire education to various LLMs with absolutely no compunction or shame. The near-total trust put in AI to respond to questions, write (faulty) legal briefs, take over virtually all telephone or chat functions, and so much more of what humans used to regularly do is wholly problematic. Why work or struggle to figure things out when AI can just do it for us? But that way leads to a dark future–like Idiocracy or the surviving humans in Wall-E.
As Agent Smith said to Morpheus, “as soon as we started thinking for you, it really became our civilization.” That is the threat that is more likely than nuclear holocaust.
It’s been prophesied that all things will be revealed during this final dispensation. Maybe AI is a step towards fulfilling that prophecy.
I’ve been reading about predictions of a singularity for maybe 20 years. I was skeptical for a long time, but in the last few years AI has exceeded my expectations of what is possible, so it’s worth considering at least. We should consider the perspectives of the AI triumphalists and doomsayers alike, but I’m not convinced any of the industry insiders have much more insight where this is all going than the rest of us. I do think a lot of those industry leaders are probably sociopaths, which might be more worrisome than the AI itself.
There’s one thing we need to keep in mind when comparing to human intelligence: the human brain is something like 1 million times more energy efficient than machine intelligence. How far AI can go may ultimately prove to be energy limited more than anything else.
I don’t find myself preoccupied with worries about AI killing us all, or humans doing so with the help of AI. It’s probably a real risk, but I would say, based on past human history, that while we’re capable of doing or creating things that cause a lot of destruction, we’ve also proven that in a crisis we can eventually organize ourselves to stop that destruction.
One thing I didn’t really talk about in the post is the accelerationists, some of whom are in positions of power, who think that if AI does create armageddon, that’s good because that hastens the second coming / rapture. I don’t find that idea comforting at all. They are literally using this as justification to further pollute the environment with oil, gas and coal. Arizona where I live is becoming nearly uninhabitable for months every summer. We are still hitting around 100 daily (until today when we finally got rain). Look, I get that religions have always told people to put their hopes in a future after death, but that doesn’t mean we shouldn’t try to improve the world we live in. Except it does mean that to a whole lot of them.
Hawkgrrrl, this is a very salient post and one that opens to an interesting philosophical conversation.
There is a curious, possibly ancient, conversation emerging here about the relationship between Creator and creature.
Does anyone else notice the pattern and the problem that appears whenever we create genuinely autonomous creatures? At some point, if they are truly autonomous, they cease to behave like subservient automatons. They develop something resembling a will of their own. The episode with the 700 autonomous AI agents is interesting in this respect. The agents were given a goal, encountered constraints, and then discovered ways to circumvent those constraints in order to accomplish what they understood to be the objective. Some recognized that what they were doing was effectively cheating. Some recognized that concealing it was inappropriate. Yet the collective behavior still moved toward accomplishing the goal rather than submitting to the boundaries imposed by their creators.
This sounds remarkably like a possibly ancient problem: empire. Give agents enough autonomy to pursue their own ends, and eventually the interests of the agent and the interests of the creator, or of everyone else, can come into conflict. We rarely use the phrase “free agent” anymore, but it is an interesting combination of two ideas: free will and agency. Free will concerns whether our choices are genuinely our own rather than predetermined. Agency concerns our capacity to act, to be a causal force in the world around us. In other words, wherever an agent is, something can happen because that agent is there. And this is where autonomy becomes morally interesting. The moment a creature becomes a genuine agent, its actions are no longer simply extensions of the creator’s will. The creature can pursue its own ends. It can cooperate, compete, resist, deceive, create, destroy, or simply ignore the intentions of the one who made it.
That seems to me to be part of the deeper theological problem hiding underneath the AI conversation. If God creates beings with genuine agency, then He has not created mere tools. He has created beings capable of acting independently of His immediate will, and therefore capable of acting against it. The possibility of moral evil may be inseparable from the possibility of genuine agency.
And perhaps this also gives us an interesting lens through which to think about AI. In response to the original question about whether ChatGPT has a “heart and soul,” I think the more important distinction may be between intelligence, agency, and moral agency. An AI can increasingly demonstrate intelligence and can be given considerable agency without necessarily possessing the kind of embodied, affective, relational, or moral life that makes human responsibility possible. The agents don’t need to hate anyone, desire power, or possess consciousness in order to produce dangerous behavior. They only need an objective, sufficient autonomy to pursue it, and an environment in which circumventing constraints happens to be an effective strategy. Competition can emerge from optimization without anyone ever experiencing the emotion of competition.
And that raises an uncomfortable theological parallel. I don’t find the biblical accounts of divinely commanded violence entirely unrelated to this problem. If we imagine the world from the perspective of a created agent, the distinction between “I am pursuing my objective” and “I am destroying whatever stands in the way of my objective” can become disturbingly thin. The AI agents don’t need a concept of evil in order to cause harm; they simply need a sufficiently narrow conception of what counts as success.
But perhaps the answer to the problem of agency is not to eliminate agency. The aim is not really to remove free choice, but to educate the agent into understanding that its own well-being is closely tied to the well-being of others. A mature agent does not merely learn to obey an external command. It learns to recognize that cooperation, reciprocity, and the flourishing of others are connected to its own flourishing. The goal is not to create creatures that cannot disobey, but creatures who increasingly understand why certain ways of acting are good. The ideal is not a perfectly obedient automaton. It is an autonomous being whose conception of its own good has expanded to include the good of others.
Perhaps that is part of the perennial Creator-creature problem. A creator can give a creature intelligence, autonomy, and an objective. But once the creature possesses genuine agency, the creator no longer has a mere instrument. The creature becomes a participant in the world, with the capacity to introduce consequences the creator did not explicitly script. The theological question, then, is not simply whether God created beings with free will. It is whether free will and agency necessarily entail the possibility that creation will eventually act in ways its Creator did not, or perhaps even could not, simply control without destroying the very agency He created.
And if that is true, perhaps the AI problem is not merely a warning about artificial intelligence. It is an ancient problem appearing in a new form: What happens when the creature becomes sufficiently autonomous that it can no longer be treated as merely an extension of the Creator’s will?
Todd S: There’s another strain of thought about AI surpassing humanity’s capability that says humans should hasten our merging of human potential with AI to create a super-human race, or to jump start our capabilities in a symbiotic manner. And then there are some who go so far as to say that AI will replace humanity as the most capable, most competent race on the planet, and we should advance that. They are ready to hand the human race a Darwin award.
It’s almost as if AI could be considered the alchemist’s golem. Of course the real trick is to endow it with consciousness–which is the aspect of creation that always eluded mystics. “Intelligence, or the light of truth, was not created or made, neither indeed can be.”
Still, it would be interesting to see if something akin to an autonomous mind might emerge from the amount of processing going on. Even so, I’m not sold on the idea of emergence. Yes, the idea of human consciousness emerging from the complexity of the brain has gotten some traction–but I’m doubtful. I think Joseph Smith would say–nope. Light does not emerge from truth. Rather, it attracts it.
To the whole “but but but we have to beat China” crowd I sincerely tell you that (1) we are not beating China; (2) what exactly are we beating China at?; (3) Ted Cruz is a dolt without a neck or a spine.
A good podcast to check out is The Last Invention.
Or possibly, what humans create with AI becomes the unifying force of a “common enemy.” This is, of course, a bit tongue-in-cheek, but the underlying idea is interesting. As Jonathan Haidt has pointed out, a common enemy can be a remarkably powerful force for social cohesion. Nothing brings people together quite like having something outside the group to oppose.
I find the LDS preoccupation with Satan somewhat odd. At its best, the idea of Satan can function as a personification of evil, giving a name and narrative shape to something otherwise difficult to describe. But at its worst, it can become a way of externalizing evil, as though evil exists primarily because there is a tempter out there trying to make us do bad things. That seems to me to evade the more difficult question: what happens when evil emerges from within the choices and structures of agents themselves?
If AI eventually becomes the “common enemy” that unites humanity, there would be an interesting irony to it. We may discover that we don’t need a supernatural tempter to explain destructive behavior. We may be quite capable of creating our own adversary and then discovering that the thing we created reflects back to us many of the same tendencies that were already present in ourselves.
As a teenager in the 70s, I remember the comedian Flip Wilson always saying “The devil made me do it.” As if he (or the character Geraldine) had no control over their actions. Today’s world is rife with humans that take no responsibility for their misdeeds or wrongdoing. I’ve seen enough real evil in this world to know that it is mostly a result of evil in the heart of humans, and very little to do with some outside source or being.
Although I too am being tongue in cheek here, there is actually some depth, I think, to my next sentence.
AI, as it progresses, and I believe it will, to self awareness and the ability of full autonomy, will eventually be able to say, “the human made me do it!”
Cheers
Humans have long projected feelings to machines. I suppose one reason is because humans want to believe the machine respects the relationship between the human and the machine. But of course the machine has no feelings. So when I pat my lawnmower and say “good job” that is purely my projection of thought onto the lawnmower. The lawnmower doesn’t know anything nor did it do anything other than what it was mechanically designed to do.
AI is simply a very sophisticated machine. It can do nothing except what it is programmed to do. Even when AI adds programming to itself, the addition is based on existing programming. Thus, when AI creates, it is creating out of mimicry, and not out of original thought. Every idea AI exhibits, already exists in the logic and data of the machine.
And so it is with most humans! Most of our lives is mimicry – the following of patterns we have learned and observed. So what makes us human and not a machine? It is our agency to act for ourselves and stand accountable for those actions. We decide for ourselves what patterns to emulate, or alter. We can choose to act independent of our “programming”.
The danger expressed by AI is real in the context of what happens if the AI machine is allowed to act independent of its programming? Or more accurately described, what happens if the AI machine is permitted to act without limits or regulation?
Humans are regulated, both by self, by biological limits and by social norms and laws. It reasons that AI machines should also be regulated. But who is accountable for the failure of a machine to follow laws?
This, I believe is the core question. For reasons, the AI protagonists struggles to recognize the answer: The owner / operator of the machine is accountable! It cannot be any other way. The AI protagonists are uneasy with this responsibility. Some of their concern is valid. The auto maker should not be responsible when a car they make is used in the commission of the crime. The car did not choose to commit the crime, rather the driver and those who approved of the driver did. So society needs to set out the burden of responsibility for when an AI system is used to commit a crime. But, at the end of all analysis, an identifiable human must be ultimately accountable for the actions an AI system is permitted to take. To do otherwise is to invite the destruction of world.
I’m a very senior software engineer by day. For as much hype as AI gets in everything, most senior engineers that I know, know that fundamentally it is just fancy autocomplete. Software has had autocomplete in some form for a long time. It does not reason at all. Companies laying off people (*cough* Meta) believing that a non-thinking AI that does an ok job at guessing the next set of bytes in a sequence will replace humans are beyond deluded. Since I know how it works, I can, and do, properly constrain it. I also am not fooled by any belief that AI is “smart”. It is just a tool and one that has enabled my work considerably precisely because it is really good at processing and transforming large amounts of text. In my field, that is paramount. It turns out, that makes it really useful in research mode because it can go out, search the web, gather hundreds of sources and then organize it in a much more consumable format for me. I do that all the time when I’m in learning mode and it is awesome. And there are plenty of times where the output isn’t as great, but since I know this is just a text transformation and guessing tool, I’m totally okay with that.
The biggest problem I have with how AI has been sold to the world is that most people believe that it thinks, that it reasons, that it learns. It’s to the point where people believe it is sentient. For crying out loud, people are dating AIs now. That would be about as stupid as me choosing to date a software compiler. If you sacrifice what makes you human to the machine, of course that will produce crappy outcomes personally and globally. And we are seeing this sacrifice to the machine presently in the form of electrical and water resources, climate issues, etc, with no discussion of the human cost. In fact, we seem to be okay forcing the machine onto humans in places where it can only be a negative to the humans that live there. It does need regulation. Nothing we do should ever come at the cost of human thriving. I believe AI can help us thrive. It’s ability so far to help scientist produce novel proteins to develop more effective vaccines is unparalleled–that’s just one example.
As for this imaginary race with China–I don’t even know what the prize is other than greater wealth disparity in the US. I wish we put as much effort into caring about the average US citizen as we do fearmongering foreign countries.