AWS Glue Data Quality now supports anomaly detection and writing results to the AWS Glue Data Catalog
Share
Services
AWS Glue Data Quality now supports anomaly detection for Catalog-based data quality evaluations and the ability to write evaluation results to AWS Glue Data Catalog (GDC) tables. These capabilities work across both ETL jobs and Catalog evaluations, giving you a consistent data quality experience regardless of workflow type.
With anomaly detection support for GDC, you can identify unexpected changes in data statistics such as sudden drops in distinct values or row count spikes in your GDC tables using ML-powered time-series forecasting, without writing rules with explicit thresholds. This is especially valuable for data engineers monitoring hundreds of tables in the GDC who need to surface issues automatically.
With results storagein GDC, Data Quality rule outcomes, profiling metrics, and anomaly predictions (with confidence bounds) are written back to GDC tables, creating a queryable record of all quality evaluations. Whether the evaluation runs in an ETL job or directly on a Catalog table, results can be queried at any time using standard SQL.
AWS Glue Data Quality anomaly detection and Catalog results storage are available in all AWS commercial regions and AWS GovCloud (US) regions.
To get started, visit the [AWS Glue Data Quality documentation](https://docs.aws.amazon.com/glue/latest/dg/data-quality.html).
What else is happening at Amazon Web Services?
Read update
Services
Share
Amazon CloudFront now supports Origin Access Control (OAC) for Amazon S3 Multi-Region Access Points
about 19 hours ago
Services
Share
Amazon EKS now supports certificate authority (CA) rotation with automated lifecycle management
about 19 hours ago
Services
Share
Read update
Services
Share
Generative AI Inference Recommendation for Amazon SageMaker now available in the SageMaker AI Studio
about 20 hours ago
Services
Share
Amazon Redshift introduces long-term system table retention with Amazon S3 Tables integration
about 23 hours ago
Services
Share