Filter articles by tags or search for specific topics:
Filter articles by tags or search for specific topics:
Original URL: https://www.canva.dev/blog/engineering/our-journey-to-snowflake-monitoring-mastery/
Added Date: December 22, 2024
Memo:
Original URL: https://blog.haoxp.xyz/posts/parquet-to-arrow/
Added Date: December 21, 2024
Memo:
Added Date: December 20, 2024
Memo:
Added Date: December 19, 2024
Memo: Improving the data processing efficiency by implementing Apache Iceberg's base-2 file layout for S3.
Added Date: December 17, 2024
Memo: An agent breaks down the process of answering questions into multiple steps, and uses different tools to answer different types of questions or interact with multiple data sources, is a good practise.
Added Date: December 16, 2024
Memo: Without Iceberg, there are lot of overhead works to implement WAP pattern.
Original URL: https://engineering.grab.com/metasense-v2
Added Date: December 15, 2024
Memo: Grab's Data Engineering and Data Governance teams enhanced their Large Language Model (LLM) integration to automate metadata generation and data classification. Post-rollout improvements focused on refining model accuracy, reducing manual verification, and increasing scalability across the data lake.
Original URL: https://engineering.grab.com/llm-powered-data-classification
Added Date: December 15, 2024
Memo: Grab's Data Engineering and Data Governance teams collaborated to automate metadata generation and sensitive data identification using Large Language Models (LLMs). This initiative aimed to enhance data discovery and streamline access management across the organization.
Added Date: December 13, 2024
Memo: Build a process to built the complete data lineage information by merging the partial lineage generated by dbt automatically.
Original URL: https://www.uber.com/en-AU/blog/lucene-version-upgrade/
Added Date: December 12, 2024
Memo: