Maintained with ☕️ by
IcePanel logo

General availability: Spark on HDInsight

Share

Services

Today we're announcing general availability of Spark on Azure HDInsight. This release brings improvements in availability, scalability, and productivity to open-source Apache Spark. It's a common sentiment in the industry that big data becomes easier with Spark. Spark has fewer moving pieces and provides a more integrated platform for big-data workloads, from ETL to SQL to machine learning. In addition to that, Spark on Azure HDInsight brings: * An industry-leading availability SLA of 99.9 percent, which makes it suitable for demanding enterprise workloads. * A scalable storage layer via Azure Data Lake Store. * Productivity tools for every phase of data exploration and development. Jupyter notebooks with a customized Spark kernel enable interactive data exploration. Integration with BI dashboards like Power BI, Tableau, and Qlik is good for quick data sharing and continuous reporting. The IntelliJ plug-in is a reliable option for long-term development and debugging of code artifacts. For more information, check out the following links: * Documentation: [Overview of Apache Spark on HDInsight](https://azure.microsoft.com/en-us/documentation/articles/hdinsight-apache-spark-overview/) * Announcement: [Apache Spark for Azure HDInsight now generally available](https://azure.microsoft.com/en-us/blog/apache-spark-for-azure-hdinsight-now-generally-available/) * HDInsight * [ HDInsight](https://azure.microsoft.com/en-gb/products/hdinsight/)