When Gartner compares traditional BI to Big Data, they often use the Suits vs Hoodies metaphor. Traditional BI/DW being the Suits, and Big Data being the Hoodies.
The methodologies of the Suits are tried and true. Their requirements based top-down approach has proven its value to the enterprise time and time again. Most organizations have a significant investment in BI, and are happy with the known unknowns that traditional BI solutions answer. In contrast to the requirements based top-down approach, Big Data is driven more by opportunity oriented experimentation. Hoodies are typically using the Hadoop ecosystem to drive innovation and insights for a functional area or business initiative. Everything from the buzz words they use to their tools of choice are very different than those that a traditional BI team uses. Often times the insights the Hoodies derive never make it from the Data Lake or Big Data System to the Data Warehouse and vice versa. Microsoft’s Analytic Platform System (APS) brings the Suits and Hoodies together.
What is Analytics Platform System (APS)?
In April 2014 Microsoft announced a rebranding of their data warehouse appliance from Parallel Data Warehouse (PDW) to APS. The name change was a good idea as new features really make the appliance a new offering. In other words, the PDW aspects of the appliance aren’t going away. They have been supplemented with an awesome new feature code-named Polybase which integrates PDW seamlessly with Hadoop.
The PDW region within APS utilizes Massively Parallel Processing to process large volumes of structured data with relative ease. This is the place where the Suits reside. By migrating your Enterprise Data Warehouse and/or data marts to PDW you will see a dramatic increase in query performance for the right price. PDW boasts a lower cost per TB than any other data warehouse appliance on the market.
Inside APS, one can also take advantage of the Hadoop region. This of course is where the Hoodies reside. They can place all their log data, sensor data, etc in a Data Lake in the Hadoop region. The Hadoop Region in APS can sit inside or outside the appliance. You really want it to sit inside the appliance if possible to take advantage of the performance gains you will get from the Infiniband network. The Hadoop region supports Hortonworks on Windows and Linux, Cloudera for Linux, and Microsoft’s instance of Hadoop in the cloud, HDInsight.
Polybase allows one to write a single SQL query against structured data in PDW and non-structured data in Hadoop. Think of APS as the brain. When you ask APS a question, the brain can reach into the PDW Region and/or the Hadoop Region to compute the answer.
At Cardinal Solutions we see APS as a product that can help our customers answer some complex business questions more efficiently and effectively, and we are excited to help them on their APS journey.