AWS S3

Amazon Simple Storage Service

Web site

Tech tags:

• Data storage
• Self managed Iceberg service
• Conditional writes

Related shared contents:

• Schema Change Management at Halodoc
  spike
  2025-03-07
• How Formula 1® uses generative AI to accelerate race-day issue resolution
  project
  2025-02-18

  Very classic Glue job pipeline to feed the AWS Bedrock Knowledge Bases for a RAG use case.
• A First Look at S3 (Iceberg) Tables
  tech1
  2024-12-04

  S3 Table bucket handle the Iceberg compaction and catalog maintenance tasks for you.
• Views pwn Tables as data interfaces
  project
  2024-12-05

  Twitch has leveraged Views in their Data Lake to enhance data agility, minimize downtime, and streamline development workflows. By utilizing Views as interfaces to underlying data tables, they've enabled seamless schema modifications, such as column renames and VARCHAR resizing, without necessitating data reprocessing. This approach has facilitated rapid responses to data quality issues and supported efficient ETL processes, contributing to a scalable and adaptable data infrastructure.
• How Amazon Ads uses Iceberg optimizations to accelerate their Spark workload on Amazon S3
  project
  2024-11-22

  Improving the data processing efficiency by implementing Apache Iceberg's base-2 file layout for S3.
• Building end-to-end data lineage for one-time and complex queries using Amazon Athena, Amazon Redshift, Amazon Neptune and dbt
  product
  2024-12-12

  Build a process to built the complete data lineage information by merging the partial lineage generated by dbt automatically.
• BMW Cloud Data Hub: A reference implementation of the modern data architecture on AWS
  project
  2022-06-22
• Building a GDPR compliance solution with Amazon DynamoDB
  project
  2024-11-11

  To search user profiles to remove, we use an AWS Lambda function that queries Aurora, DynamoDB, and Athena and places those locations in a DynamoDB table specifically for GDPR requests.

In productions with:

Canva Pinterest Funding Circle