AI4EOSC: A federated cloud platform for Artificial Intelligence in scientific research

The rapid growth of Artificial Intelligence and Machine Learning in scientific research has highlighted a gap between industry-standard machine learning operations (MLOps) tools and platforms and the unique requirements of modern and Open Science, particularly regarding the FAIR (Findable, Accessible, Interoperable, and Reusable) principles. This paper presents AI4EOSC, a federated, open-source platform designed to operationalize the full AI/ML life-cycle within the European Open Science Cloud (EOSC) ecosystem. Our methodology tackles the fragmentation of distributed research infrastructures by integrating a modular and distributed architecture comprising an AI development platform, a serverless AI-as-a-Service layer, and a federated orchestration model that is able to integrate heterogeneous computing and storage resources from distributed e-infrastructures. AI4EOSC also introduces a ‘‘FAIR-by-design’’ approach that enforces metadata standardization (via MLDCAT-AP) and W3C PROV-compliant provenance tracking through a platform-integrated CI/CD pipeline. The added value of AI4EOSC is demonstrated through the delivery of a diverse set of community installations, which show consistent and seamless deployment across heterogeneous cloud providers. ... mehrThese installations are validated by a set of scientific cases, showing how our work reduces the manual burden on researchers while ensuring high levels of reproducibility and interoperability and providing a unified environment for the development, training, and production of AI/ML models in the EOSC.