Every year, those of us who graduated from our medical school in a particular year set meet in different parts of the world for reunions, and occasionally we have an international reunion. This year's meeting was in Toronto, Canada, and I was asked to give a virtual presentation.
I have given presentations at previous reunions and usually try to make them light, enjoyable and memorable. I intended to do the same this year, especially since I spoke about AI at a previous meeting.
However, as I prepared this presentation, something had changed. The people developing AI — including senior executives and researchers in the technology industry — were increasingly warning about its potential risks. Some have even suggested that sufficiently advanced AI could pose an existential threat to humanity.
I therefore had to reconsider the presentation. I began to feel rather like Cassandra, the prophetess of Greek mythology, who was condemned to predict the future but never to be believed.
But perhaps we have experienced something similar before.
Y2K
Before the year 2000, computers commonly stored dates using two digits rather than four. As the millennium approached, it became clear that systems could interpret "00" as 1900 rather than 2000.
Predictions warned of serious disruption—affecting financial systems, utilities, transport, healthcare, and other critical infrastructure. Governments and companies spent enormous amounts identifying vulnerabilities, rewriting code and testing systems.
When 1 January 2000 arrived, the catastrophe largely failed to materialise.
It is tempting to conclude that Y2K was overhyped. But another interpretation is that the extensive preparation worked. The absence of catastrophe does not necessarily mean there was no risk; it may mean that the risk was identified and mitigated.
This offers an important lesson for AI.
AI Hallucinations
Think back to our days as medical students. We could sometimes give a consultant from another speciality an answer with complete confidence, despite having little or no real knowledge of the subject.
The explanation might sound intelligent. The terminology might be appropriate. We might even cite a study that sounded entirely plausible — but did not actually exist.
To anyone listening, the answer could appear authoritative. But it was simply wrong.
This is, in essence, what we mean by an AI hallucination.
AI can generate information that is fluent, coherent and authoritative-sounding, using appropriate medical terminology and apparently convincing explanations, while the underlying information is factually incorrect or entirely fabricated.
The important point is that AI does not necessarily recognise that it is wrong. It can present an incorrect answer with the same confidence and fluency as a correct one.
Therefore, particularly in medicine, AI output must be regarded as information to be assessed and verified—not as an unquestionable authority.
This is why human oversight remains essential at critical points in clinical decision-making. AI may assist with the decision, but responsibility must ultimately remain with an appropriately qualified human being.
AI Is Already Changing Society
AI is no longer simply following instructions written explicitly by programmers. Modern systems are trained on enormous quantities of data and learn complex patterns that can be difficult to interpret.
They can produce extraordinary results, but they can also produce confidently wrong answers. These are commonly described as hallucinations. They can arise from limitations in the training data, statistical generation, flawed information or the absence of a reliable mechanism for verifying whether an answer corresponds to reality.
The danger is not necessarily that AI "hates" or "wants" us. Those are human characteristics that we project onto machines. The more important issue is whether increasingly capable systems pursue objectives in ways that we did not anticipate.
This is why the warnings from researchers and technology leaders deserve attention.
But AI Is Also Transforming Medicine
At the same time, we should not lose sight of what AI is already doing for healthcare.
It can reduce administrative work — something I particularly appreciate having spent years writing letters, completing forms and documenting clinical information.
It can analyse vast quantities of patient data and identify patterns that may be difficult for an individual clinician to see. For example, it can help detect unusual patterns in hospital observations and laboratory results that could indicate an emerging infection outbreak.
In medical imaging, AI can identify subtle abnormalities on X-rays, CT and MRI scans and bring them to the clinician's attention.
It can analyse large datasets to identify previously unrecognised relationships between diseases, risk factors and outcomes.
It is also contributing to more personalised medicine, helping researchers and clinicians consider factors such as age, sex, weight, genetics, previous treatment and other patient characteristics when assessing risk and treatment.
AI is therefore not simply about replacing doctors.
In many circumstances, its greatest value may be augmenting doctors — helping us see more, analyse more and spend less time on repetitive administrative work.
The future job market
The Movies
Hollywood has been warning us about AI for decades.
The Matrix. The Terminator.
In both, intelligent machines eventually escape human control, with disastrous consequences.
Of course, Hollywood exaggerates for entertainment. But the underlying question is serious:
What happens when machines become more capable than the humans who created them?
The Human Being Remains Responsible
But there is a crucial principle.
The more powerful the technology becomes, the more important human oversight becomes.
An AI system may identify an abnormality, suggest a diagnosis or recommend a treatment. But clinical responsibility remains with appropriately qualified healthcare professionals and the governance systems around them.
We cannot simply accept an AI answer because it is confidently delivered.
We must question it, verify it and understand its limitations.
The Future
So, should we be frightened?
I think the more useful question is whether we are prepared.
Humanity has faced enormous collective challenges before — nuclear weapons, climate change, the destruction of the ozone layer and global pandemics. We have not solved all of them, but international agreements, institutions, regulation, scientific cooperation and human ingenuity have demonstrated that difficult global problems can be managed.
AI will require the same approach.
We need sensible regulation, international cooperation, technical safeguards, transparency, accountability and continued human oversight.
I remain optimistic because AI is a human creation, and therefore its future is not predetermined.
We should neither assume that AI will save humanity nor that it will destroy us.
We should recognise its extraordinary potential, understand its risks and ensure that we remain responsible for how it is developed and used.
Perhaps that is the real lesson from Y2K.
We do not have to predict exactly what will happen.
We have to prepare for what could happen.



Comments
Post a Comment