MatBind: Probing the multimodality of materials science with contrastive learning

Materials discovery depends critically on integrating information from multiple experimental and computational techniques, yet most tools today analyze these different data types in isolation. Here, we present MatBind, a model based on the ImageBind architecture that creates a unified embedding space across four key materials science modalities: density of states (DOS), crystal structures, text descriptions, and powder X-ray diffraction (pXRD) patterns. Using a hub-and-spoke architecture with crystal structure as the central modality, MatBind achieves cross-modal recall@1 performance of up to 98% between directly aligned modalities and up to 73% for pairs of modalities not explicitly trained together. ... mehrOur model demonstrates the ability to make semantically meaningful connections across modalities, enabling researchers to query one type of materials data using another. Our analysis shows that combining multiple modalities can improve the model's ability to recognize important structural features like perovskite crystal systems. This approach lays the foundation for more integrated materials research platforms that can accelerate discovery by leveraging the collective knowledge encoded in materials databases

Materials discovery depends critically on integrating information from multiple experimental and computational techniques, yet most tools today analyze these different data types in isolation. Here, we present MatBind, a model based on the ImageBind architecture that creates a unified embedding space across four key materials science modalities: density of states (DOS), crystal structures, text descriptions, and powder X-ray diffraction (pXRD) patterns. Using a hub-and-spoke architecture with crystal structure as the central modality, MatBind achieves cross-modal recall@1 performance of up to 98% between directly aligned modalities and up to 73% for pairs of modalities not explicitly trained together. ... mehrOur model demonstrates the ability to make semantically meaningful connections across modalities, enabling researchers to query one type of materials data using another. Our analysis shows that combining multiple modalities can improve the model's ability to recognize important structural features like perovskite crystal systems. This approach lays the foundation for more integrated materials research platforms that can accelerate discovery by leveraging the collective knowledge encoded in materials databases.