Apache Spark, real-time, cloud BI & analytics, IoT, and self-service where the trends to watch in 2015, and they’ll continue to make waves into 2016.
It’s that time of year again. Here’s a look back on the big stories of 2015 and a look forward on the trends that will carry on into 2016.
1. Spark Lights A Fire. In 2015 companies and vendors started to realize that the opportunity with big data isn’t just to scale up BI and the data warehouse. Thus the Apache Spark open source framework and other analytical options that go beyond SQL were hot in 2015. Spark was embraced by scores of vendors and hundreds of big companies in 2015. IBM was the most visible vendor advocate, but plenty of other data-integration and big data platform companies joined the bandwagon.
It takes more than SQL to make sense of big data. Spark couples in-memory performance
with SQL, streaming, machine learning, graph and R-based data-analysis options.
A key driver of the interest in Spark’s ensemble of analytics – which encompasses SQL, R, graph analysis and machine learning — is the fact that in an increasingly digital world, companies are generating and need to analyze a variety of data types. As companies do more marketing and business online, for example, clickstreams, social data and mobile data become much more important. SQL is good for analyzing the transactional data behind those interactions, but graph, machine learning and other techniques shine with these new types of data. And companies want to look forward, not just back, so they can make the right moves to maximize sales and profits. That’s what is driving interest in predictive capabilities, such as those available in R.
2. Real-time gets real. Streaming data analysis (a.k.a., real-time data analysis) was another hot category in 2015. Recent announcements by Amazon, Cloudera, Confluent (the Kafka support company), Microsoft, MapR, SAP and plenty of others point to the demand for low-latency data capture and analysis capabilities. Online advertising, marketing and retail scenarios have been a big driver, as companies seek to trigger ads, launch campaigns, and serve up cross-sell and up-sell offers while customers are still online. Real-time fraud detection, risk analysis, IoT (see below) and security threat detection are other scenarios where time is of the essence. Look for the wave of real-time announcements to continue into 2016.
3. Cloud-based analytics and business intelligence options take off. Some vendors (like BIRST and BusinessObjects) were very early to cloud-based (Software-as-a-Service-style) business intelligence. But the first-generation of options that emerged seven to ten years ago didn’t exactly set the world on fire. Early pioneers including LudicEra, Oco and PivotLink didn’t survive.
This gauge of cloud-based data analysis versus on-premises-based data analysis, as measured
by Tableau Online, explains why cloud-based BI and analytics services are finally taking off. Tableau
says Amazon Redshift, Google Analytics, Google Big Query and Salesforce are the top-sources.
Times have changed. Now that huge volumes of data are originating and accumulating online (think Amazon RedShift, Google Analytics, Google BigQuery and Salesforce) cloud-based BI and analytics options are starting to take off. Vendors including IBM, GoodData, Microsoft, Oracle and SAP all upped their stakes in cloud-based BI in 2015 while upstarts Tableau and Qlik prepared to deliver deeper cloud services 2016. Stay tuned for yet more announcements in 2016 and check out these six tips for success in cloud-based data analysis.
4. IoT Services: Will they bear fruit? The question isn’t which vendor did but which didn’t introduce an IoT suite or IoT-related services portfolio in 2015? The list of players announcing new or expanded IoT capabilities in 2015 included IBM, Microsoft, Salesforce and SAP. (Others deeply invested prior to 2015 included General Electric, Cisco and Intel, among others.)
In my view, IoT is often a new marketing spin on analytics against sensor-based data – something in use in manufacturing and industrial settings for quite some time. Add in ingredients including geospatial data, automotive telematics, smart mobile devices and, of course, Internet-based apps and services, and you can give it a more modern “of things” twist.
The question for 2016 is how quickly will businesses adopt and prove out real-world, IoT-based used cases. Our deepest thinker on this topic, Andy Mulholland, points out that there’s a last-mile problem whereby the IT infrastructure and services are useless if you can’t make the final connections and make sense of the data streaming from the sensors and devices on the front lines. In Andy’s book, line-of-business people are crucial to IoT deployment success, and IT-centric suites and vendor portfolios won’t succeed without business leadership of IoT initiatives.
5. Self-service options proliferate. The trend toward self-service reporting and data analysis emerged five to seven years ago. Now that trend is moving into new areas, and it reached a fever pitch in self-service data prep in 2015, with announcements from data-integration vendors like Informatica , SnapLogic and Talend, and from BI vendors including Qlik (Smart Data Load) and Logi Analytics.
Self-service started in the BI realm with the likes of Qlik and Tableau. Vendors including Alteryx have
extended the trend to self-service data-prep and self-service advanced analytics for data-analyst types.
Interest in self-service advanced analytics is also on the rise. One of my most important reports of 2015 was “The Era of Self-Service Analytics Emerges,” which looked at leading examples including IBM Watson Analytics, SAP Lumira and SAS Visual Analytics/Visual Statistics. Other vendors and products pursuing this self-service advanced analytical trend include Alteryx, Microsoft Power BI, Qlik and Tableau.
Hope you enjoy the read on these trends as we head into 2016 and have a happy and healthy New Year!
