Welcome to the read that explores an interplay between future vision and methods to define appropriate governance for healthcare AGI (artificial general intelligence). It projects a vision as a future state and discusses methods that help co-construct governance as part of that future state.
In large-scale tech deployment, humans have previously chosen to adopt and adapt safety and efficacy practices from more mature domains, where technology has
diffused widely and successfully. Our central thesis is that this may well happen in
healthcare AGI via the adoption of practices from regulated industries, adapted and extended to manage AGI risks.
Vision
It is the year 2100. In a hospital somewhere in the developed world, tens of thousands of patients are cared for in facilities extending over many floors and hundreds of acres. Breakthroughs in longevity in prior decades have prolonged human lives without alleviating the disease burden. Disease incidence and prevalence have been growing steadily and now include recently emerging diseases due to climate change and other causes.
Patient numbers in this and other hospitals, for shorter or longer lengths of stay and in secondary or acute care, have been steadily rising, straining human resource capacity for decades. Alongside medical and nursing staff, hundreds of robots of several commercial types now perform clinical care tasks at scale, circulating in hospital areas.
Developing the science, technology, legal, and ethical framework supporting the deployment and use of robots became a necessity due to steadily declining
healthcare staff numbers vis-à-vis expanding human populations, and a diversifying and growing disease burden. More advanced robots work in collaboration with clinicians and provide care for patients e.g. during surgery, post-surgery care, and follow-up, with personalized disease treatment planning, care delivery, and evaluation in the patient’s preferred language.
These advanced robots are called AGI healthcare workers or agents. Across primary and secondary care, they have assumed tasks based on AGI evolution and
maturation. They now offer advanced patient simulation ‘virtual human twinning’, accurately tracking current and projecting future pathophysiology and treatment options, and various AI-human interaction interfaces e.g. advanced translation and cultural immersion in healthcare settings. Due to their added value and the resulting reorganisation of care delivery, where AI has helped deliver services for more patients better while creating more jobs, a global market emerged to include manufacturers, software providers, alignment and risk professionals testers, evaluators, chips and spare parts suppliers and recyclers.
Monitoring typically takes place within a large, fully equipped monitoring room called the “control tower” that facilitates tracking behavioural patterns, alignment with human-set objectives, and location of each AGI worker active and circulating in hospital. Humans manning the control tower continuously monitor individual and group AGI agent patterns and trends. Control tower infrastructure affords them of capabilities e.g. individual agent continuous monitoring, patient outcomes review, billing (in case of externally commissioned agents), agent cohort review, agent replacement, deactivation and decommissioning, in line with the applicable legal framework.
Members of each hospital’s healthcare AGI fleet, are managed throughout their lifecycle. They are delivered by vendors, commissioned into service, monitored continuously, undergo maintenance, and are decommissioned following established standard operating procedures approved by hospital management. AI engineering and maintenance sections have been set up in hospitals in response to statutory compliance requirements set by governments across world regions and jurisdictions, adopted based on growing evidence in AGI agents’ effective contribution to clinical management alongside human clinician participation and supervision.
Governments have legislated to ensure safe operation of AGI agents in healthcare while mitigating risks. These include licensing (e.g. CE marked in the EU) of standards-based robots as products entering markets; activation, monitoring and de-activation via a specific SIM-type card called Agent Identity Module, placed in the back of the robot within a hardware part accessible to trained human personnel only; backup procedures for retaining learning acquired in case of loss, theft, or other untoward action; AGI agent monitoring of alignment, thought processes and behavior towards its user and non-user groups (patients; clinicians; nurses; family members, carers; other hospital staff).
At the national level, AGI Agencies have been set up to monitor AGI agent deployment in healthcare across its lifecycle and in all healthcare facilities. Governments are looking at legislation to extend this towards AGI agent deployment in private homes, thus enabling a new generation of telehealth and telecare services. Advanced,
robust monitoring remains key to ensure risk mitigation, particularly after reports of intended misalignment by humans (e.g. agents rewarded towards euthanasia).
Adoption from Other Industries
This vision includes the conjecture that AGI will adopt successful processes, tools, methods and other artefacts from other, mature mass technology domains.
Four examples are provided:
These imply further artefacts are likely to be adopted from other, mature regulated industries e.g. standards, standard-setting processes, expert fora, legislation approaches.
While these adoption patterns are to be expected, the more important question is: how can we as humans, citizens, polities construct an accepted vision and its governance? What are the methods available for reasoning plausibly and systematically about future options?
Methods
Published literature includes a myriad of methods tested in several policy domains. An evolution is observed towards participatory approaches.
Methods tend to coalesce around three types:
A few useful concepts emerge across these:
Technology Foresight & Forecasting
Technology foresight uses a varying range of techniques based on social science and other research methods e.g. literature reviews, interviews, simulation, extrapolation, sensitivity, pattern and risk analysis, SWOT, and technology watch.
Key issues associated with TF have been that political and other considerations may often shift the focus towards short-term policy priorities at the expense of longer-term projection. In addition, they are sometimes seen as producing unitary ( ‘monochromatic’) visions of a future state, often including the most prudent, safe approaches towards a policy outcome.
Scenario Planning
SP uses social science research methods to construct descriptions of the current situation and identify key stakeholders, trends, and uncertainties that act to transform this into future states described by each scenario. Participatory SP includes active stakeholder participation. Issues have been found in combining qualitative and quantitative data, and inadequate testing of scenario-based models.
Anticipatory Governance
Anticipatory Governance is a more recent framework combining key elements above in one single approach. Used to provide policy responses to the dynamics of science and technology innovation, it seeks to proactively address their societal impacts. It comprises three ‘pillars’: Foresight focuses on constructing and evaluating several detailed, plausible future states for reflection, planning, and capacity-building for readiness. Engagement focuses on informed exchange and reasoning among stakeholders - the public, policy and science and technology actors. Integration is about bringing stakeholders together as a basis for collective action and responses within the context of each stakeholder group. The United Nations, OECD, and
countries in the EU have looked at several AG approaches for AI governance.
Conclusion
New tech domains may be grafted through adoption and reuse of artefacts from more mature industries. However, mainstream foresight methods do not explicitly seem to account for this. Nevertheless, future states emerging from the evolution of science and technology cannot be fully detached from the current situation in a particular domain.
A key theme is the identification of biases in every instance where humans reflect and make decisions about the future. Biases acknowledgement in method selection and use, at every step, leads to a fuller critical evaluation of options for future states and governance.
Active and sustained stakeholder engagement and involvement are very key for the realization of a shared vision, based on a shared understanding and capacity building measures that respond to the subject matter requirements and the societal needs of tomorrow.
