Amazon EMR on EKS now supports Apache Spark troubleshooting agent
Share
Services
Amazon EMR on EKS now supports the Apache Spark troubleshooting agent. Data engineers can now diagnose EMR on EKS job failures through natural language, receiving automated root cause analysis and PySpark code recommendations without manually navigating distributed logs and Spark History Server data.
The agent analyzes Spark History Server data, distributed executor logs, and cluster configurations to identify issues such as memory errors, data skew, resource contention, and connectivity failures. With this launch, the Spark troubleshooting agent now covers all EMR deployment options: EMR on EC2, EMR Serverless, and EMR on EKS. The agent is accessible directly from the EMR on EKS console through a "Troubleshoot with AI" option on failed jobs. Additionally, the agent is available through MCP (Model Context Protocol) using any compatible AI coding agent, including Kiro, Claude Code, and Cursor. All operations are read-only, authenticated with IAM roles, and logged in AWS CloudTrail.
The Spark troubleshooting agent for Amazon EMR on EKS is available in all AWS Regions where the SageMaker Unified Studio is [available](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/spark-troubleshooting-features.html#available-regions). To get started, go to EMR on EKS console, or set up the MCP server in your preferred AI coding agent. For detailed guidance, see the [EMR troubleshooting agent documentation](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/spark-troubleshoot.html).
What else is happening at Amazon Web Services?
Amazon EC2 network/EBS instances now available in additional regions
about 6 hours ago
Services
Share
Read update
Services
Share
AWS DMS Schema Conversion now supports offline SQL Server conversion
about 11 hours ago
Services
Share
Read update
Services
Share
Amazon DocumentDB (with MongoDB compatibility) now supports R8g.24xlarge and R8g.48xlarge instances
about 12 hours ago
Services
Share