And the LORD God commanded man, saying, “Of every tree of the garden you may freely eat; but of the Tree of the Knowledge of Good and Evil you shall not eat, for in the day that you eat of it you shall surely die.” – GENESIS 2:16-17
The thought struck me the other day as I was taking a shower: "Is artificial intelligence the Tree of the Knowledge of Good and Evil?"
I put this question to Google Gemini: Gemini said no. Its answer was rational and utilitarian: knowledge is neither good nor evil. What matters is what we do with it.
If AI helps cure diseases, accelerates drug discovery, reduces poverty, improves logistics and removes dangerous or degrading work, then good. If it produces mass unemployment, surveillance, bias, autonomous weapons or catastrophic loss of control, then bad. AI, said Gemini, is not forbidden fruit. It is merely an extremely powerful tool.
OK. So I copied Gemini's answer into ChatGPT and asked it to argue back.
ChatGPT started by attacking Gemini's premise: that Good and Evil can be reduced to welfare calculations – more happiness, less suffering, more healthy years of life, fewer bad outcomes. But, said ChatGPT, this is itself a moral philosophy rather than some neutral position from which all other views can be judged.
More importantly, ChatGPT said Gemini had misunderstood the Tree of the Knowledge of Good and Evil; it isn't a story about God telling mankind not to learn things, it is that humanity reaches for the right to decide for itself what is Good and what is Evil.
The serpent's promise is: “Your eyes shall be opened, and ye shall be as gods, knowing good and evil.”
Now the AI analogy begins to look stronger. AI has given us extraordinary new powers to know, classify, predict and decide.
But greater intelligence does not necessarily mean greater wisdom! Learning how to split the atom before learning how to abolish wars; developing industrial chemistry before understanding the effects of pollution has on the biosphere; building social media before grasping the implications of algorithmically optimised attention for politics, culture and the human brain. All examples of where progress brought about unforeseen consequences.
ChatGPT summed it thus: “The danger is not knowledge. The danger is knowledge without wisdom”.
Back to Gemini. This time it came back with a much stronger argument. I was, it said, mixing up two different stories from Genesis. The Tree of Knowledge is about an inward transformation. Man becomes self-conscious, ashamed, aware of mortality and burdened with moral responsibility. The real story about technological hubris is the Tower of Babel. There, mankind combines its collective linguistic and technical power to build something that reaches towards heaven. AI, said Gemini, is Babel.
Large AI systems are, among other things, extraordinary machines for concentrating human output. “Language. Mathematics. Science. Literature. Code. History. Law. Music. Philosophy.” Huge chunks of human culture are being compressed into manageable computational structures.
Gemini: “Babel begins with one language and ends with many; AI begins with many languages and translates them into a common underlying machine representation.” Babel takes mankind from unity to fragmentation; AI may be taking us from fragmentation back towards unity.
I put Gemini's argument back to ChatGPT. ChatGPT conceded the Babel point, but refused to give up on Eden. Perhaps, said ChatGPT, AI sits at the intersection of both stories.
“Babel explains what we are building. Eden explains why we find it so tempting.”
Gemini had made another strong point. An AI does not judge in the moral sense. It is never tormented by doubts or by guilt, nor does it fear death. All it does is calculate.
So, Gemini argued, the danger is not that we have created a machine that knows Good and Evil; the danger is that we may begin handing our own moral judgement over to something that knows neither.
Gemini called that temptation not disobedience – but sloth!
ChatGPT thought this was Gemini's best argument.
We already know primitive versions of it. “The computer says no.” “The system rejected your application.” “The algorithm has calculated the risk.” Nobody – no human – took the decision. Responsibility disappeared somewhere into the procedure.
AI makes this far more serious because the machine no longer merely processes a decision – it actually recommends one. And this problem will get worse as AI continues to improve. Imagine a neurologist whose diagnosis, based on many years of looking at brain scans, disagrees with an AI system which has repeatedly been shown to outperform human doctors. Who should ultimately decide? If the doctor overrides the machine and the patient dies, has the doctor nobly defended human judgement? Or has he irresponsibly trusted his own intuition over much better evidence?
The same question could arise with judges, pilots, soldiers, teachers, financiers or politicians.
ChatGPT put the dilemma this way:
- Epistemic humility tells us to defer to the machine.
- Moral responsibility tells us that the decision must remain ours.
That struck me as the most interesting point in this exchange between the two AIs. The biblical Tree represents a mythical point in human history at which we became morally self-conscious creatures; a point at which mankind became aware of moral responsibility. The inflection point that we now face is the awareness of abrogating moral responsibility to externalised judgment.
“The first temptation was: You shall know Good and Evil.
The second may be: You need no longer decide between them yourself.”
ChatGPT made another interesting point. “The danger may not arrive when AI starts getting things wrong. It may arrive when AI starts getting things right so often that exercising your own judgment begins to look irrational. That takes the argument beyond the usual AI debate about jobs, robots and machines becoming cleverer than us. It asks an older question: what is human freedom actually for?”
Once we notice in our lives that AI is consistently making better decisions than we can, would there still be any point, any value, in making our own? ChatGPT's answer was yes. “Because not everything valuable about life is an outcome. Courage requires the possibility of cowardice. Wisdom requires uncertainty, mistakes and learning. Moral development requires the possibility of choosing badly.”
One could imagine a future in which AI helps everyone live longer, violent crime becomes rare, poverty largely disappears and algorithms gently steer people towards statistically optimal careers, relationships and lifestyles. By every measurable indicator, this might be an extraordinary success.
And yet... and yet... who is actually living the life?
By the end of my AI experiment, ChatGPT and Gemini had agreed that Eden concerns knowledge, freedom, judgment, wisdom and moral responsibility.
ChatGPT finally reformulated my question: “Perhaps AI is not itself the new Tree of the Knowledge of Good and Evil. Perhaps something stranger is happening. Having eaten from the Tree, are we now building AI to escape the burden of what we acquired? The first temptation was to become as gods by acquiring knowledge. The second temptation was to build an oracle to which we can hand over the moral responsibility of deciding what that knowledge means and what we should do with it. And the danger is that humans will increasingly cease to bear it.”
The danger may not be that machines become human; it may be that humans become more machine-like, “preferring optimisation to judgement, probability to conviction, prediction to wisdom and freedom from error to the freedom to choose.”
The Tree of Knowledge of Good and Evil looks like an ancient warning whose time has come.
[I found it really interesting bouncing arguments made by one AI against the other and back again!]
This time two years ago:
Touched by Boris
This time three years ago
Clinging on and letting go
This time six years ago:
Out in the mid-September heat
Posts
