Home
This wiki is intended to be used as a reference for the concepts and instructions of the project. If you are new for the project and Framework, it is important that you follow this wiki with attention. Although it is an important piece of information documentation, this wiki is not supposed to replace the official documentation of each tool, prototype or used library.
MIMBCD-UI project deals with the use of a recently proposed technique in literature: Deep Convolutional Neural Networks (CNNs). These deep networks will incorporate information from several modes: magnetic resonance imaging volumes (MRI), ultrasound images, mammographic images (both views CC and MLO) and text. The proposed algorithm, called for multimodality CNNs (MMCNNs) will have the ability to process multimodal information at a unified and sustained manner. This methodology needs to "learn" what are the masses and calcifications. So that is necessary to collect the ground-truth or notes of the masses and calcifications provided by medical experts. For the collection of these notes, the design and development of an interface is necessary to allow users (in this case, medical specialists) to display various types of image (i.e., ultrasound, MRI and mammography), and that also allows for user interaction, particularly in providing the notes of the masses and calcifications. For these reasons, it is crucial for the development of this project, cooperation with experts providing the above notes.
Medical imaging diagnosis is a routine effort performed by radiologists to help diagnose or monitor a patient's medical condition. Medical imaging diagnosis allows physicians to identify pathologies by decoding tissue characteristics while examining visual properties in medical imaging. It plays a central role in modern medicine, particularly in the prevention and diagnosis of breast cancer, one of the leading causes of mortality worldwide. Breast cancer is the most common cancer in women worldwide, with nearly 1.7 million new cases diagnosed in 2012. However, proper classification, location, detection, targeting and registration of tumors requires the use of different imaging modalities that contribute to diagnostic reliability.
With the hype of new Artificial Intelligence (AI) methods such as Machine Learning (ML) and Deep Learning (DL) algorithms, automated agents (assisted by AI) are closer to the radiology room than ever before. This proximity becomes highly relevant for the medical community, which is interested in solutions to support clinical diagnosis. Greater stakeholder engagement requires sufficient validation and continuously monitored emerging AI tools as well as available breast cancer screening data. That said, national health systems are increasingly in need of methods, such as our solutions, for ongoing clinical diagnostic support to support medical decision-making.
Our goal is to provide AI solutions for radiology. We bring radiology experts and software engineers together to optimize healthcare professionals workload through the development of high-performance AI tools.
The aim of this project is to develop a system supported by intelligent agents for diagnosing medical images in the field of breast cancer. To address the effects of lack of clarity in AI results, we provide a service where our intelligent agents are applied to the breast cancer domain at different levels of medical expertise and multiple clinical workflows. Despite the promise of assisting clinicians in the decision-making process, there are two initial challenges that our services aim to address: (i) the lack of available and cured medical data to be consumed by AI algorithms; and (ii) the fact that medical professionals often find it difficult to understand how an AI system turns its outcome into a final medical decision. The goal is to develop a disruptive platform with the introduction of various AI techniques using an intelligent agent to communicate with the various physicians in an interpretable way. Specifically, focusing on how the multimodality and assistance of an AI model can add value to the medical workflow.
In our solution, we need to consider several factors when making decisions about how to add AI to medical practice. For instance, bias is a familiar concept for clinicians, who are already trained to practice evidence-based medicine. Our solution was designed concerning the interaction of clinicians and AI in a real-world setting. Hence, we provide several explainability techniques supported by summarizing the reasons of the AI behavior, gaining the user's trust, or producing insights about the decision-making causes. These explainability techniques were verified at top scientific publications, where evidence of effectiveness and efficiency was demonstrated. A key factor to our solution is pairing control functionalities across the AI outputs with several visual explainability techniques that empower the clinicians' choice and sense of control.
With this solution, clinicians reduced about 20% of the medical error. Specifically, the developed solution provides a decrease of more than 20% for False-Positives and a decrease of 2% for False-Negatives. Moreover, this solution is reducing 34.93% of the time (i.e., about 3min/patient) for clinicians to fully diagnose a patient. Due to these results, we are providing evidence of an immense reduction of healthcare costs for governments and private institutions.
A significant number of technologies under development and in prototype or clinical trials, suggest that AI-powered diagnostic departments will feature in many future clinical institutions [2]. Diagnostic institutions are leveraging pattern recognition and DL to reduce diagnosis turnabout time [4], and improve pathology workflow accuracy and efficiency of the diagnostic.
Our solution is being used in more than nine public and private health institutions [1] in Portugal. We are currently scaling our services and solutions outside Portugal, where we already have the collaboration of more than 300 international physicians. From our networking of physicians, many belong to excellent radiology associations and institutions, such as the American College of Radiology (ACR) with collaboration with this project. Within the ACR (among other institutions), we will be able to further validate and promote our solution. Nevertheless, we already have an invention [2, 3] registered as a national patent, with the number 116801 and reference DP/01/2021/74923, as well as, registered internationally, as the number PCT/PT2021/050029.
As already mentioned, we are scaling our services and solutions outside Portugal, where we already have the collaboration of more than 300 international doctors. Of these physicians, many belong to excellent radiology associations and institutions, such as the ACR. Within the ACR (among other institutions), we will be able to further validate and promote our solution.
An approach of using AI to simulate clinical trials before human trials have also been seen, leaving plenty of scope available for what AI can create. From medical recommendations and lesion detection, to experimental clinical trials, the scope of this technology is rapidly expanding.
Our AI system has been shown to reduce errors in human observation, as well as being as good as expert radiologists in the process of detecting cancer. The solution represents a breakthrough in early detection of breast cancer. Thus, saving many lives or reducing costs for the patient by developing an AI system able to identify cancers with a degree of precision similar to radiologists. Specifically, AI can lead clinicians to reduce the number of False-Positives and False-Negatives.
As the influence and value of this technology grows, so too does the importance of developing effective patent strategies for AI inventions. In this project, we need to address the key questions faced by us in this space of intellectual property. Moreover, we need to undrstand the best strategies and considerations for protecting AI-implemented inventions for healthcare products and services.
Currently, our research work follows a TRL6, meaning that the system prototype was already demonstrated in a relevant environment. In fact, we already studied our solution with 45 clinicians recruited on a volunteer basis from nine clinical institutions in Portugal.
By using the Innovation Maturity Level (IML), defined by CIMIT, will be applied as a matrix system to measure the maturity of four domains: Technology, Regulatory, Marketing/Business, and Clinical. Currently, we have fully completed the invention cycle and are dealing with translation. For the translation cycle, we already surpassed the IML 5 (Proof of Value), as we already studied and demonstrated the potential of the solution to work and create value for all stakeholders. Now, we are resolving the IML 6 (Initial Clinical Trials), as we are surpassing the regulated production of the prototypes and collection of clinical, as well as economic data.
At the moment, we have a registered patent at national and international level. However, this only represents a tiny slice of the potential inventions in our laboratory. For now, we will be present at the next edition of Lab2Market organized by the Technology Transfer Area at Instituto Superior Técnico. Our aim is to develop and register more patents, as well as further to create an organization that promotes services and products aimed at these solutions.
Getting real-world evidence concerning AI to be paid for, we need data that shows our solution is making a difference. Hence, our project provides evidence from our findings [1] to support the argument that AI can improve the medical imaging workflow.
Every year, about 30% to 50% of diagnosed cases result in False Positives [2]. The vast majority of these numbers are translated into biopsies resulting in an enormous cost to the national health system, as well as physically and psychologically harming the patient. Other than that, about 8% to 10% of diagnosed cases are False-Negatives, which often cause the patient's death. To improve both False Positives and False Negatives, we are able to reduce costs and improve doctors' lives [1], bringing better healthcare to patients. Further, AI will become a $2 billion industry worldwide. Making it essential to have a solution, as proposed in this invention.
[1] Francisco Maria Calisto, Carlos Santiago, Nuno Nunes, Jacinto C. Nascimento, Introduction of human-centric AI assistant to aid radiologists for multimodal breast image classification, International Journal of Human-Computer Studies, Volume 150, 2021, 102607, ISSN 1071-5819, https://doi.org/10.1016/j.ijhcs.2021.102607
[2] Calisto, F. M. (2020). Breast Cancer Medical Imaging Multimodality Lesion Contours Annotating Method. Instituto Superior Técnico. https://doi.org/10.13140/RG.2.2.14792.55049
[3] Francisco Maria Calisto, Nuno Nunes, and Jacinto C. Nascimento. 2020. BreastScreening: On the Use of Multi-Modality in Medical Imaging Diagnosis. In Proceedings of the International Conference on Advanced Visual Interfaces (AVI '20). Association for Computing Machinery, New York, NY, USA, Article 49, 1–5. DOI:https://doi.org/10.1145/3399715.3399744
[4] Francisco M. Calisto, Alfredo Ferreira, Jacinto C. Nascimento, and Daniel Gonçalves. 2017. Towards Touch-Based Medical Image Diagnosis Annotation. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17). Association for Computing Machinery, New York, NY, USA, 390–395. DOI:https://doi.org/10.1145/3132272.3134111