Microsoft Brings Innovations From SQL Server To Hadoop With Stinger
Over the last two years, Microsoft has been sharing our progress in Big Data by working to bring Hadoop to the masses through the simplicity and manageability of Windows and Azure. In 2012, Microsoft communicated our expanded partnership with Hortonworks by making Hortonworks Data Platform the core of our Hadoop solution on-premise and in the cloud.
As part of this partnership, Microsoft has collaborated with Hortonworks and the open source software (OSS) community to contribute directly to Apache Hadoop. While our first wave of contributions with Hortonworks has been to port Hadoop to Windows, we’ve recently contributed to other projects like the Stinger initiative to dramatically speed up the performance of Hive and make Hadoop enterprise-ready.
About The Stinger Initiative
In collaboration with Hortonworks and others from the OSS community, Microsoft has brought some of the technological breakthroughs of SQL Server to Hadoop. In SQL Server 2012, we introduced the in-memory columnstore which included a vectorized query execution engine and a columnar format that demonstrated 10-100x performance gains on data warehouse queries. While these improvements varied by customer scenarios, some achieved upwards of 600x. With the Stinger initiative, Microsoft is collaborating with Hortonworks to bring similar query execution and columnstore technologies to Hadoop so that we can collectively improve the performance of Hive up to 100x.
The first fruits of this has already been realized with Hortonworks Data Platform 2.0 for Windows and with HDInsight previewing Hadoop 2.2 clusters. Both of these Hadoop solutions leverage phase 2 of the Stinger project which has up to 40x improvements to query response times and up to 80% in data compression.
Microsoft is pleased to be a part of the open source software (OSS) Big Data community for the past year and a half. We’ve gained a lot from the community and are delighted to continue our partnership with Hortonworks and bringing more innovations to Hadoop.
We invite you to learn more about Microsoft’s Hadoop offers below:
- Do you want to deploy Hadoop in the cloud? Try HDInsight. Read how to do this for the preview here and in this three part blog series: part 1, part 2, part 3
- Read CIO magazine’s Windows Azure HDInsight customer story spotlight
- Read more about Microsoft’s involvement with Hadoop through Stinger and partnering with Hortonworks
- Watch a Hortonworks and Microsoft webinar on the Hybrid Modern Data Architecture
- Do you want to deploy Hadoop on-premise? Read the HDP 2.0 for Windows Announcement
