Generative-AI based Agents to Revolutionize Medical Diagnosis and Treatment of Cancer

Scope

Background and scope

Imaging is a crucial component of cancer clinical protocols, providing detailed morphological, structural, metabolic, and functional information. However, harnessing the full potential of the data generated through medical imaging in clinical settings remains challenging. Clinicians often struggle to combine diverse and large-scale data into a comprehensive view of patient care, disease progression, and treatment efficacy. The inability to seamlessly integrate and interpret diverse data sources result in suboptimal patient outcomes and inefficiencies in the delivery of healthcare.

The integration of traditional Artificial Intelligence (AI) with medical imaging can transform healthcare, but most existing applications are still in their infancy and must overcome a number of challenges to accelerate adoption. These include AI applications being confined to single data modalities, which restricts their overall effectiveness (Monomodal Application); inadequate and insufficient data training, leading to data scarcity and a lack of generalizability, making them less reliable across diverse patient populations, including with regard to gender-sensitivity; and the lack of AI model interpretability, as many AI systems function as "black boxes," providing little insight into their decision-making processes. This lack of transparency limits trust in the systems and their usability in clinical settings.

The goal of this Pathfinder Challenge is to create interactive GenAI autonomous agents and/or a combination of them (super-agent) that provide clinicians with a holistic end to end perspective of patient care, throughout the entire clinical pathway. These agents aim to enhance pattern identification, reduce inconsistencies and errors in diagnoses as well as improve cancer treatment. While the focus is on GenAI, we also encourage the integration of other advanced AI technologies, such as topological and geometric deep learning, neural fields, graph neural networks, etc., which can complement and enhance the robustness and effectiveness of GenAI-based solutions in addressing the challenges of cancer diagnosis and therapy.

The Challenge will support early-stage groundbreaking research projects that will develop and validate novel approaches and concepts for integrating and interpreting multimodal medical imaging and health data. Additionally, it will involve generating reliable synthetic medical data, which will also be pooled to form a common database and used for the development of advanced algorithms.

Specific Objectives

Project proposals under this Challenge should focus on one (and only one) of the following diseases: breast cancer, cervical cancer, ovarian cancer, prostate cancer, lung cancer, brain cancer, stomach cancer or colorectal cancer.

Each proposal should address both the following areas (at least one sub-objective from each of the areas):

Area 1: Technological area

• GenAI-based tools for Integrating Multidimensional Multimodal health Data

Investigate groundbreaking techniques and methodologies for developing GenAI algorithms that combine multidimensional (e.g. time dimension, space dimension) and multimodal data from various sources. These include multiple imaging modalities (e.g., MRI, CT, PET, X-ray), clinical data (e.g., electronic health records, lab results, structured and unstructured clinical data, pathology results, genetics and – omics data, videos, knowledge databases, and other resources). The goal is to provide a comprehensive view of the patient’s condition. The developed algorithms should be capable of producing unified and actionable datasets that can be exploited for the development of the AI tools described in Area 2 (clinical).

• Medical Data Augmentation

Develop GenAI models based on groundbreaking techniques that are in the conceptual or initial experimental phase for medical data augmentation. These models should be capable of creating highly realistic synthetic medical data (images, genomics data, etc.) and generating complementary data from existing sources (for example producing synthetic CT images from MRI images), to support iterative cycles of model training.

• Medical Knowledge Representation and Integration

Create an initial prototype GenAI model for medical knowledge representation and integration. This model should aim to develop a comprehensive and dynamic medical knowledge base, to identify discrete medical imaging features associated with demographic information and systemic conditions, to improve the interpretability of AI-based models and extract new knowledge not previously identifiable by experts without assistance.

Area 2: Clinical Area

• Predictive Diagnosis

Develop an interactive autonomous agent capable of assessing the likelihood of a patient developing cancer by analysing their medical history, imaging data, and genetic information. The agent should provide personalised health risk predictions, enabling early detection and preventive measures.

• Enhance Personalized Treatment Selection

Develop novel AI algorithms and architectures that leverages multidimensional and multimodal data integration, along with synthetic data generation, to predict the optimal treatment pathway for specific patient conditions, as well as to forecast disease progression and treatment efficacy providing a comprehensive view of patient care.

Appropriate performance metrics should be considered for the continuous evaluation and testing of the scientific and technical robustness (including accurately quantify uncertainties) of all developed algorithms and architectures in Areas 1 and 2. Rigorous testing against diverse datasets is essential to ensure that the models perform reliably across various patient demographics and conditions, thereby reducing the risk of skewed results and ensuring precision from diagnoses to therapy.

Projects should also conduct proof of concept studies in controlled settings to demonstrate improved and more accurate diagnosis and treatment when compared to current clinical practice. The viability of the developed technologies should be evaluated, guiding further refinement and improvement. For instance, a super-agent could be validated for assisting and/or replacing clinicians through the whole clinical pathway of the patient, providing a holistic view of patient care, that is currently unachievable due to fragmented healthcare systems and associated expertise.

The focus should also be on enhancing the interpretability of AI models/agents, making their decision-making processes more transparent and understandable to clinicians. This could involve developing cutting-edge techniques such as causal inference methods, explainable AI frameworks, or novel visualization tools that provide deeper insights into AI decision-making processes.

The AI models developed under this Challenge are expected to comply with the EU concept for Trustworthy AI, relevant ethical principles, and the AI Act. In addition to focusing on performance, careful attention must be given to data quality, transparency, privacy, and security.

Proposers are encouraged to leverage the data and tools available in the Cancer Image Europe platform (deployed in the context of the European Cancer Imaging Initiative) for their proposed work. In turn they should contribute the datasets, and developed AI tools and models to the platform under agreed conditions. All datasets produced should be described where possible with metadata records in the EU dataset catalogue of the European Health Data Space (EHDS) using the Health DCAT-AP metadata standard.

Projects that address only one of the two ‘Areas’ or other cancer types will be considered "out" of scope.

Expected Outcomes and Impacts:

In support of the European AI Strategy and the Cancer Plan for Europe and the Cancer Mission this Challenge looks to support the development of the next generation models for cancer diagnosis and treatment, with Generative AI.

This Challenge aims to create a collaborative environment where diverse expertise — including for example data science, informatics, oncology, radiology, pathology, medical physics, bioinformatics, geneticists, healthcare administrators, and patient advocacy groups — converges to address the complexities of developing autonomous agents for holistic patient care, through enhanced diagnosis and personalized treatment.

The Challenge aspires to significantly improve patient care and reduce pressure on the healthcare system by leveraging advanced interactive autonomous agents for diagnosis and personalized treatment. By alleviating the burdens on clinicians and ensuring compliance with the EU concept for Trustworthy AI, the initiative will enhance the quality and reliability of medical services. Economically, it promises substantial cost reductions and cost avoidance, leading to long-term improvements in healthcare efficiency and sustainability. Ultimately, this challenge will foster innovation and establish Europe as a leader in the field, delivering profound benefits to patients, healthcare providers, and society at large.

The portfolio of selected projects will be designed to deliver a set of agents/models for improved diagnosis and personalized treatment of the above-mentioned cancers. Specifically, the projects will collaborate to:

- Create a shared database of synthetically generated images to be used across all projects for the development of their algorithms;

- Compare the use of a combination of the agents in the case of multiple cancers;

- Benchmark agents for enhanced diagnosis and personalized treatment selection;

- Define innovative clinical pathways in oncology;

- Externally validate the developed agents within a project at clinical premises of another project in the portfolio;

- Develop standardized methods and frameworks for evaluating AI- Act and Medical Device Regulation (MDR)- compliant generative AI models.

The portfolio of projects to be funded under this Challenge will be composed in such a way that they address ideally all cancers mentioned in this call, apply different technologies, and provide access to relevant clinical facilities and research infrastructures. The following categories will be used for the composition:

Category 1 – type of cancer

Category 2 – type of technology

Category 3 – access to appropriate infrastructure data and ecosystem integration.

Specific conditions

Applications for this Challenge with elements that concern the evolution of European communication networks (5G, post-5G and other technologies linked to the evolution of European communication networks) will be subject to restriction for the protection of European communication networks (see Annex II – Section B1)