Accessible data lineage: a scoping review on open-source data lineage platforms

In the contemporary landscape of data-driven enterprises, establishing data lineage in data transactions can be challenging yet a necessity, due to emerging compliance laws. While there are several commercial data lineage platforms, organizations are unable to successfully employ data lineage methods in their data ecosystem due to accessibility issues, insufficient information on the underlying lineage method, and lack of information on coverage of data lineage taxonomies. In this work, we conduct a structured scoping review using the PRISMA-ScR guidelines, to analyze to what extent current open-source platforms address aspects of data lineage. We adapted well-known data lineage taxonomies, and summarized which aspects of data lineage are addressed. The scoping review highlights the need for open-source lineage platforms that intelligently deduce lineage where meta-data is not available and further research to support inter-organizational data transactions. We draw insights for future areas of research in data lineage, both for practitioners and researchers.