AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education
<p>We introduce NeuroVerse, a framework for exploring 3D nanometric-scale reconstructions of neural and glial cellular processes in the central nervous system. Using image stacks from volume electron microscopy, NeuroVerse generates 3D mesh models through a SAM2-based segmentation pipeline and...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , , , , , , , , |
| Published: |
2025
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513547761876992 |
|---|---|
| author | Uzair Shah (15740699) |
| author2 | Marco Agus (8032898) Daniya Boges (8032901) Hamad Aldous (21392810) Vanessa Chiappini (12451155) Mahmood Alzubaidi (15740693) Markus Hadwiger (2567419) Pierre J. Magistretti (8032907) Mowafa Househ (9154124) Corrado Calí (17302723) |
| author2_role | author author author author author author author author author |
| author_facet | Uzair Shah (15740699) Marco Agus (8032898) Daniya Boges (8032901) Hamad Aldous (21392810) Vanessa Chiappini (12451155) Mahmood Alzubaidi (15740693) Markus Hadwiger (2567419) Pierre J. Magistretti (8032907) Mowafa Househ (9154124) Corrado Calí (17302723) |
| author_role | author |
| dc.creator.none.fl_str_mv | Uzair Shah (15740699) Marco Agus (8032898) Daniya Boges (8032901) Hamad Aldous (21392810) Vanessa Chiappini (12451155) Mahmood Alzubaidi (15740693) Markus Hadwiger (2567419) Pierre J. Magistretti (8032907) Mowafa Househ (9154124) Corrado Calí (17302723) |
| dc.date.none.fl_str_mv | 2025-05-09T06:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1016/j.cag.2025.104239 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/AI-guided_immersive_exploration_of_brain_ultrastructure_for_collaborative_analysis_and_education/29108669 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Biomedical and clinical sciences Neurosciences Information and computing sciences Artificial intelligence Ultrastructural analysis Immersive exploration Volume electron microscopy Brain reconstructions Metaverse for education Metaverse for neuroscience |
| dc.title.none.fl_str_mv | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p>We introduce NeuroVerse, a framework for exploring 3D nanometric-scale reconstructions of neural and glial cellular processes in the central nervous system. Using image stacks from volume electron microscopy, NeuroVerse generates 3D mesh models through a SAM2-based segmentation pipeline and integrates absorption signals for deployment in a Metaverse environment. The framework includes a SAM2 adapter optimized for biological microscopy imaging, adapted with feature enhancement blocks and dual decoders to improve the segmentation of complex cellular structures. An interactive virtual AI agent, powered by Heygen and OpenAI models with domain-specific knowledge, provides semi-real-time assistance. NeuroVerse supports education and collaborative analysis for neuroanatomy and neuroscience. It includes a pipeline for the creation of 3D models, automated segmentation, mesh reconstruction, and heatmap computation, optimized for the Spatial.io ecosystem. Contributions include a virtual anatomy lab for neuroanatomy education and collaborative sessions on spatial morphology correlation and neuroenergetic absorption models. Evaluations show that the SAM2 adapter preserves fine cellular details and manages irregular boundaries. Preliminary sessions indicate potential to enhance neuroscience education, improve remote collaboration among scientists, and provide access to advanced neuroscientific data and tools. Evaluation of the virtual AI agent confirms its ability to provide context-aware support, interpret complex cellular structures, and facilitate understanding through semi-real-time assistance for students analyzing neural and glial reconstructions. NeuroVerse combines imaging, segmentation, and AI technologies within an immersive Metaverse platform for neuroscience education and research.</p><h2>Other Information</h2> <p> Published in: Computers & Graphics<br> License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.cag.2025.104239" target="_blank">https://dx.doi.org/10.1016/j.cag.2025.104239</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_0417cd225478ddbfabb484d61e78f597 |
| identifier_str_mv | 10.1016/j.cag.2025.104239 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/29108669 |
| publishDate | 2025 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and educationUzair Shah (15740699)Marco Agus (8032898)Daniya Boges (8032901)Hamad Aldous (21392810)Vanessa Chiappini (12451155)Mahmood Alzubaidi (15740693)Markus Hadwiger (2567419)Pierre J. Magistretti (8032907)Mowafa Househ (9154124)Corrado Calí (17302723)Biomedical and clinical sciencesNeurosciencesInformation and computing sciencesArtificial intelligenceUltrastructural analysisImmersive explorationVolume electron microscopyBrain reconstructionsMetaverse for educationMetaverse for neuroscience<p>We introduce NeuroVerse, a framework for exploring 3D nanometric-scale reconstructions of neural and glial cellular processes in the central nervous system. Using image stacks from volume electron microscopy, NeuroVerse generates 3D mesh models through a SAM2-based segmentation pipeline and integrates absorption signals for deployment in a Metaverse environment. The framework includes a SAM2 adapter optimized for biological microscopy imaging, adapted with feature enhancement blocks and dual decoders to improve the segmentation of complex cellular structures. An interactive virtual AI agent, powered by Heygen and OpenAI models with domain-specific knowledge, provides semi-real-time assistance. NeuroVerse supports education and collaborative analysis for neuroanatomy and neuroscience. It includes a pipeline for the creation of 3D models, automated segmentation, mesh reconstruction, and heatmap computation, optimized for the Spatial.io ecosystem. Contributions include a virtual anatomy lab for neuroanatomy education and collaborative sessions on spatial morphology correlation and neuroenergetic absorption models. Evaluations show that the SAM2 adapter preserves fine cellular details and manages irregular boundaries. Preliminary sessions indicate potential to enhance neuroscience education, improve remote collaboration among scientists, and provide access to advanced neuroscientific data and tools. Evaluation of the virtual AI agent confirms its ability to provide context-aware support, interpret complex cellular structures, and facilitate understanding through semi-real-time assistance for students analyzing neural and glial reconstructions. NeuroVerse combines imaging, segmentation, and AI technologies within an immersive Metaverse platform for neuroscience education and research.</p><h2>Other Information</h2> <p> Published in: Computers & Graphics<br> License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.cag.2025.104239" target="_blank">https://dx.doi.org/10.1016/j.cag.2025.104239</a></p>2025-05-09T06:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1016/j.cag.2025.104239https://figshare.com/articles/journal_contribution/AI-guided_immersive_exploration_of_brain_ultrastructure_for_collaborative_analysis_and_education/29108669CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/291086692025-05-09T06:00:00Z |
| spellingShingle | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education Uzair Shah (15740699) Biomedical and clinical sciences Neurosciences Information and computing sciences Artificial intelligence Ultrastructural analysis Immersive exploration Volume electron microscopy Brain reconstructions Metaverse for education Metaverse for neuroscience |
| status_str | publishedVersion |
| title | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education |
| title_full | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education |
| title_fullStr | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education |
| title_full_unstemmed | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education |
| title_short | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education |
| title_sort | AI-guided immersive exploration of brain ultrastructure for collaborative analysis and education |
| topic | Biomedical and clinical sciences Neurosciences Information and computing sciences Artificial intelligence Ultrastructural analysis Immersive exploration Volume electron microscopy Brain reconstructions Metaverse for education Metaverse for neuroscience |