Build complete data lineage
Level: product
Articles Addressing This Problem (6):
What it means to get your data ready for AI
The article discusses the evolving role of data engineers in the context of Agentic AI, highlighting how the interaction with data is shifting from a...
Building End-to-End Data Lineage for One-Time and Complex Queries Using Amazon Athena, Amazon Redshift, Amazon Neptune, and dbt
This article discusses the challenges and solutions for building end-to-end data lineage in enterprise data analytics, particularly for one-time and...
Capture data lineage from dbt, Apache Airflow, and Apache Spark with Amazon SageMaker
i don't know "OpenLineage standard" before, I guess Datahub should enable to support it as well.
How Meta discovers data flows via lineage at scale
Explained in the three systems (API, data warehouse, AI inference), how to efficiently collect and validate the Lineage metadata.
JD.com's Exploration and Practice of Big Data Governance (CN)
JD.com has developed a comprehensive big data governance framework to manage its extensive data infrastructure, which includes thousands of servers,...
Building end-to-end data lineage for one-time and complex queries using Amazon Athena, Amazon Redshift, Amazon Neptune and dbt
Build a process to built the complete data lineage information by merging the partial lineage generated by dbt automatically.