AWS Lake Formation extends table permissions to access underlying data in Amazon S3

AWS Lake Formation now enables you to read and write the underlying data files in Amazon S3 for tables registered in the AWS Glue Data Catalog. This provides you with a single set of permissions for both SQL queries and direct file access using your existing Lake Formation table grants.

With this launch, Lake Formation provides temporary, scoped credentials for registered S3 locations based on your table permissions. SELECT permissions grant read access, and SUPER permissions grant read and write access to the data at that location. This capability comes built-in with Amazon EMR 7.13 or later. As a result, you can access data files directly from your Spark jobs for tasks that require file level access such as model training, feature engineering, or debugging data quality issues.

You can also integrate your Apache Spark or Trino applications using APIs or through an open source plugin provided by AWS. Additionally, all access is logged in AWS CloudTrail to provide a unified audit trail across SQL and file-based operations on your tables.

This feature is available at no additional charge in all AWS Regions where AWS Lake Formation is available. To learn more, see Lake Formation documentation, EMR documentation, API reference, and open source plug-in.