10 Tips From The Chief Analytics Officer Forum
10 Tips From The Chief Analytics Officer Forum
Doug Henschen is former Vice President and Principal Analyst where he focused on data-driven decision making. Henschen’s Data-to-Decisions research examines how organizations employ data analysis to reimagine their business models and gain a deeper understanding of their customers. Henschen's research acknowledges the fact that innovative applications of data analysis requires a multi-disciplinary approach starting with information and orchestration technologies, continuing through business intelligence, data-visualization, and analytics, and moving into NoSQL and big-data analysis, third-party data enrichment, and decision-management technologies. Insight-driven business models are of interest to the entire C-suite, but most particularly chief executive officers, chief digital officers,…...
Read moreChief analytics officers should be grounded in the business and deeply familiar with customer experience. From there, use more data, test, test, test and make the case for data-driven decisions.
Chief analytics officers are (or should be) just one step away from chief strategy officers.
This was just one of many pearls of wisdom heard at the January 26-27 Chief Analytics Officer Forum in New York. The event was attended by a who’s who gathering of CAOs, chief intelligence officers, analytics directors and aspirants to these roles from companies including American Express, AXA, Charles Schwab, Delta Airlines, Disney, McGraw Hill, the New York Times and State Farm Insurance.
Tips from the Chief Analytics Officer Forum
I attended only the second day of the CAOForum, but I came away with plenty of notes and my own thoughts on these 10 best practices for analytics leaders.
- Deeply understand the customer experience to reduce friction. Keynote speaker Joe DeCosmo, CAO at global lender Enova International, said his team reviews customer website and call-center journeys on a regular basis to spot opportunities for improvement. The fix may not involve analytics, but in one instance Enova analysts found that a 50-percent reduction in page-load times led to 2.25-percent conversion-rate increase and a $1.5 million boost in incremental revenue for just one product. The lesson: great models can’t overcome bad processes.
- Develop a test-and-learn culture. Amazon, CapitolOne and other leading companies have proven the value of testing. Andy Pulkstenis, director of analytics at State Farm Insurance, said willingness to test has a direct correlation to innovation. He encouraged CAOs to view failed tests as steps toward success. He cited the example of the highly successful product WD-40, which was invented only after the failed tests of water displacement formulas 1 through 39.
Most companies never progress beyond A/B or one-factor-at-a-time (OFAT) testing, Pulkstenis said. These techniques are powerful on their own, but State Farm also uses more sophisticated techniques include multivariate testing and multivariate testing with covariate analysis. In one simple A/B test of two direct-mail pieces promoting life insurance to new parents, State Farm found that a plain-text letter generated 45% higher lift than the same message and offer surrounded by baby imagery. - Abstract analytics from applications. In a rush to get to market, Enova built analytic models right into its transactional applications, but that’s not ideal, said DeCosmo. Removing analytics from the apps simplified ongoing analytical tuning and model swapping without disrupting the operational applications. In the bargain, Enova gained cloud-deliverable, real-time analytic services (for scoring, loan approval and more) that the company can now offer to non-competing lenders.
- Document your work. How many analytics teams have models named after analysts that left the company years earlier? Or how often do you discover that you recreated work that was part of a previous project you didn’t know about? There were plenty of knowing smiles in the audience when DeCosmo posed these questions. To avoid do-over scenarios, DeCosmo advised teams to thoroughly document their work – analytical and otherwise. It’s a step that takes time and effort, but it pays dividends many times over as you can reuse IP, refine past work and avoid repeating mistakes.
- Focus on “decision-time” analytics. Real-time and streaming analytics are getting lots of attention, but keep the real decision time in mind, advised Bill Franks, CAO at Teradata. The IRS, for example, doesn’t worry about real-time because it has weeks to detect fraud before it cuts refund checks. Conversely, one bank found it had to rethink overnight-batch updates when analysis revealed that three customer contacts in short succession about a single issue, like a bank fee, signal a high likelihood to churn. So if a customer’s Web query is followed by a phone call and, later that day, a branch visit, the branch manager can’t be warned to save the account if customer-service records don’t get inter-day updates.
- Move from artisanal to automated. The science of analytics is moving into its industrial age, said Franks of Teradata, with automated options for choosing algorithms. The software-based, automated approaches may not yield a perfect model, but they’re probably good enough. “The point is to maximize the aggregate impact, not optimize every decision,” said Franks.
- It’s not man versus machine; think man plus machine. Be open to new tools and technologies, but keep in mind that machines need human reality checks, advised Gina Papush, chief analytics and data officer at insurer QBE North America. “You sometimes have to override the model based on your own business knowledge or feedback from the business,” she said. Fellow panelists chimed in with many examples of machines turning out bad results that business-savvy analysts spotted right away. Undoubtedly necessary business rules and data inputs were lacking, but machines don’t know the difference.
- Use more data to drive a deeper customer relationship. It’s always better to have more data and more data sources. DeCosmo of Enova said his firm added opt-in use of customer bank records to refine offers and give customers better terms. Samih Fadli, chief intelligence officer at digital agency Razorfish, encouraged attendees to enrich their first-party data to resolve to unique IDs, using third-party data and tracking of device IDs and Web cookies.
- Be one step away from chief strategy officer. Firms start using analytics tactically, and many organization are maturing to use analytics enterprisewide. To take it level deeper, DeCosmo encouraged fellow CAOs to “put analytics at the heard of everything you do.” The aspirational goal of the analytics team at QBE, said Papush, is not just to be part of the practice, but central to the practice of identifying strategic opportunities.
- Start with the top-five imperatives. Boiling down this advice to the basics, I’d say the imperatives for analytics leaders are to know the business, know the customer experience, use more data and test, test, test to get to superior results. Finally, and most importantly, the real challenge for analytics leaders is to sell the power of data-driven insights within the organization. “You can use the data, but you can’t hide behind it,” observed Anthony Canitano, general manager of advanced analytics at Delta Airlines. “You have to understand context of how the data can be applied in each area of the business.”
