From 2020 to 2026: the same problem, two documents

Director’s Note — 2026

In May 2020, the European Commission announced within the European AI Alliance the deployment of an artificial-intelligence tool to diagnose COVID-19 from computed-tomography images across ten hospitals in nine countries. The text below is that announcement; the comment accompanying it is the one I signed at the time, as a member of the Alliance. I have not amended it: I keep it exactly as published, sealed, because its value lies precisely in its date.

I warned there of three things. That operating below a minimum sample threshold breeds singularities and dangerous identifications. That replacing clinical judgment with software leads to serious errors. And that institutional haste, no less than slowness, has a price. Six years on, those three warnings have not merely held: they have hardened into architecture. What in 2020 was an intuition written in a forum is today, in these Director’s Articles, reasoned doctrine. Medical AI must distinguish prediction from decision, assistance from authority, and uncertainty from failure; it must undergo external validation before deployment, not after harm; and it must not mistake cost for health. The Watson case —which recommended a contraindicated drug to a patient with haemorrhage, without hesitating— and the Epic Sepsis case are the bill for having ignored what was already known.

From 2020 to 2026, the problem remains the same. Only one thing has changed: we now have the cases that prove it.

Juan Antonio Lloret Egea

Author’s comment

Juan Lloret · 26 May 2020, 15:07

Thanks for sharing this information.

This subject is very topical, along with the use of data in pandemics such as COVID-19. And it is convenient to have real, highly qualified experts in the fields, or investments can be very unsuccessful for Europe. For example, using data below a minimum threshold (statistical sample universe) can lead to singularities or very dangerous identifications for citizens. Also trying to substitute qualified doctors and specialists (guided by the patient’s clinic and the experience of the healthcare provider) with software or hardware can carry serious errors. In the USA this is already done under the supervision of international standards such as the FDA and the IEEE (for example, the Computer Society and others).

If Europe walks slowly, it will be a serious problem for citizens and significant investment losses. On the contrary, if Europe walks too fast it can cause chaos in the profession of medicine: lightning technicians (X-ray technicians), radiologists, etc. Regards, Juan Antonio.

Director’s Note — 2026

This second document, also from May 2020 and from the same European AI Alliance forum, addresses the other half of the problem: not the deployment of the tool, but the data it is trained on. It concerns a COVID-19 detector based on chest X-rays, built on a set of barely some nine hundred images.

My comment at the time insisted on the essential point: in a new and poorly documented disease, knowledge of the terrain is everything, and the unknowns were —and are— many. There, already in raw form, was the thesis I have developed six years later in these Director’s Articles: a model trained on scarce or biased data does not learn to diagnose; it learns to repeat the bias of its origin, and it does so with the fluency of one who does not doubt. It is the same lesson as the Obermeyer case —the algorithm that mistook healthcare spending for health need— and the Epic Sepsis case: external validation is not a later formality; it is the prior condition that separates a clinical tool from a well-presented statistical superstition.

Nine hundred images in 2020 to decide on a pandemic. In 2026 we are still discussing the same thing, only now with the statistical casualties in plain sight.

Juan Antonio Lloret Egea

Author’s comment

Juan Lloret · 30 March 2020 · Additional comment V

COVID-19 is a novel, still poorly documented disease, and a good knowledge of the terrain is essential. The description given by Wang and Wong, of the University of Waterloo and Darwin AI, is a good starting point, though I am well aware that the scientific community is working most intensively. Nevertheless, there are still many unknowns, and this alone shows the complexity of the matter. I also attach technical bibliography for context in the references of the original thread.