Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation
<p dir="ltr">The ability to predict breast cancer metastases is essential for making effective clinical decisions and managing patients. Traditional models predominantly rely on structured clinical data, which often lacks essential contextual details, limiting their predictive accura...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , |
| منشور في: |
2025
|
| الموضوعات: | |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513533202399232 |
|---|---|
| author | Faseela Abdullakutty (22564814) |
| author2 | Younes Akbari (16303286) Somaya Al-Maadeed (5178131) Ahmed Bouridane (2270131) Rifat Rifat Hamoudi (22564817) |
| author2_role | author author author author |
| author_facet | Faseela Abdullakutty (22564814) Younes Akbari (16303286) Somaya Al-Maadeed (5178131) Ahmed Bouridane (2270131) Rifat Rifat Hamoudi (22564817) |
| author_role | author |
| dc.creator.none.fl_str_mv | Faseela Abdullakutty (22564814) Younes Akbari (16303286) Somaya Al-Maadeed (5178131) Ahmed Bouridane (2270131) Rifat Rifat Hamoudi (22564817) |
| dc.date.none.fl_str_mv | 2025-06-06T03:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1007/s12559-025-10474-6 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Enhancing_the_Prediction_of_Breast_Cancer_Progression_Through_Multi-modal_Data_Transformation/30540857 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Biomedical and clinical sciences Oncology and carcinogenesis Health sciences Health services and systems Information and computing sciences Artificial intelligence Machine learning Breast cancer metastasis Synthetic data generation Multi-modality Multi Co-Guided Attention (MCGA) |
| dc.title.none.fl_str_mv | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">The ability to predict breast cancer metastases is essential for making effective clinical decisions and managing patients. Traditional models predominantly rely on structured clinical data, which often lacks essential contextual details, limiting their predictive accuracy. In order to address this limitation, a multi-modal approach is introduced in which structured data is transformed into unstructured text, while contextual richness is preserved. Using this text, synthetic images are generated across three key diagnostic modalities, histopathology, mammography, and ultrasound, to enhance predictive capabilities. Based on converted text data, a pre-trained diffusion model was used to generate synthetic medical images in histopathology, mammography, and ultrasound modalities. The impact of a variety of text description variants on image quality and metastasis prediction was assessed. Comprehensive tumor descriptions or a combination of histological type and differentiation status were the most effective generation strategies. A comparison was conducted between three prediction approaches: a unimodal approach, an early fusion approach based on concatenation, and the Multi Co-Guided Attention (MCGA) approach. Through mutual attention, MCGA enhances feature alignment by addressing inter- and intra-modal heterogeneity and capturing complex relationships. Unimodal and multi-modal methods were evaluated with the application of SMOTE to mitigate the impact of data imbalance. Multi-modal fusion significantly outperforms unimodal methods, especially when class imbalances are mitigated by using SMOTE. When SMOTE was applied, Ultrasound+BERT achieved the highest level of accuracy (0.90), followed by Histopathology+BERT (0.88), and Mammogram+BERT (0.88). As compared to early fusion, MCGA demonstrated better class balance and improved minority class detection. Incorporating unstructured text with synthetic imaging modalities improves the accuracy of metastasis prediction by preserving contextual information. A MCGA fusion is particularly effective in ensuring balanced class performance, particularly for rectifying class imbalances. Through this approach, the complementary strengths of textual and visual data are leveraged to overcome limitations in multi-modality integration. These results demonstrate the potential for advancing the prediction of breast cancer metastasis, offering a more robust and context-aware framework for clinical decision-making.</p><h2>Other Information</h2><p dir="ltr">Published in: Cognitive Computation<br>License: <a href="https://creativecommons.org/licenses/by/4.0" target="_blank">https://creativecommons.org/licenses/by/4.0</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1007/s12559-025-10474-6" target="_blank">https://dx.doi.org/10.1007/s12559-025-10474-6</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_9b822f07c97d858d80b1978fdfe52a5f |
| identifier_str_mv | 10.1007/s12559-025-10474-6 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/30540857 |
| publishDate | 2025 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data TransformationFaseela Abdullakutty (22564814)Younes Akbari (16303286)Somaya Al-Maadeed (5178131)Ahmed Bouridane (2270131)Rifat Rifat Hamoudi (22564817)Biomedical and clinical sciencesOncology and carcinogenesisHealth sciencesHealth services and systemsInformation and computing sciencesArtificial intelligenceMachine learningBreast cancer metastasisSynthetic data generationMulti-modalityMulti Co-Guided Attention (MCGA)<p dir="ltr">The ability to predict breast cancer metastases is essential for making effective clinical decisions and managing patients. Traditional models predominantly rely on structured clinical data, which often lacks essential contextual details, limiting their predictive accuracy. In order to address this limitation, a multi-modal approach is introduced in which structured data is transformed into unstructured text, while contextual richness is preserved. Using this text, synthetic images are generated across three key diagnostic modalities, histopathology, mammography, and ultrasound, to enhance predictive capabilities. Based on converted text data, a pre-trained diffusion model was used to generate synthetic medical images in histopathology, mammography, and ultrasound modalities. The impact of a variety of text description variants on image quality and metastasis prediction was assessed. Comprehensive tumor descriptions or a combination of histological type and differentiation status were the most effective generation strategies. A comparison was conducted between three prediction approaches: a unimodal approach, an early fusion approach based on concatenation, and the Multi Co-Guided Attention (MCGA) approach. Through mutual attention, MCGA enhances feature alignment by addressing inter- and intra-modal heterogeneity and capturing complex relationships. Unimodal and multi-modal methods were evaluated with the application of SMOTE to mitigate the impact of data imbalance. Multi-modal fusion significantly outperforms unimodal methods, especially when class imbalances are mitigated by using SMOTE. When SMOTE was applied, Ultrasound+BERT achieved the highest level of accuracy (0.90), followed by Histopathology+BERT (0.88), and Mammogram+BERT (0.88). As compared to early fusion, MCGA demonstrated better class balance and improved minority class detection. Incorporating unstructured text with synthetic imaging modalities improves the accuracy of metastasis prediction by preserving contextual information. A MCGA fusion is particularly effective in ensuring balanced class performance, particularly for rectifying class imbalances. Through this approach, the complementary strengths of textual and visual data are leveraged to overcome limitations in multi-modality integration. These results demonstrate the potential for advancing the prediction of breast cancer metastasis, offering a more robust and context-aware framework for clinical decision-making.</p><h2>Other Information</h2><p dir="ltr">Published in: Cognitive Computation<br>License: <a href="https://creativecommons.org/licenses/by/4.0" target="_blank">https://creativecommons.org/licenses/by/4.0</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1007/s12559-025-10474-6" target="_blank">https://dx.doi.org/10.1007/s12559-025-10474-6</a></p>2025-06-06T03:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1007/s12559-025-10474-6https://figshare.com/articles/journal_contribution/Enhancing_the_Prediction_of_Breast_Cancer_Progression_Through_Multi-modal_Data_Transformation/30540857CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/305408572025-06-06T03:00:00Z |
| spellingShingle | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation Faseela Abdullakutty (22564814) Biomedical and clinical sciences Oncology and carcinogenesis Health sciences Health services and systems Information and computing sciences Artificial intelligence Machine learning Breast cancer metastasis Synthetic data generation Multi-modality Multi Co-Guided Attention (MCGA) |
| status_str | publishedVersion |
| title | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation |
| title_full | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation |
| title_fullStr | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation |
| title_full_unstemmed | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation |
| title_short | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation |
| title_sort | Enhancing the Prediction of Breast Cancer Progression Through Multi-modal Data Transformation |
| topic | Biomedical and clinical sciences Oncology and carcinogenesis Health sciences Health services and systems Information and computing sciences Artificial intelligence Machine learning Breast cancer metastasis Synthetic data generation Multi-modality Multi Co-Guided Attention (MCGA) |