Overview

BAI Communications designs, builds, and operates state-of-the-art communications networks (cellular, Wi-Fi, broadcast, radio and IP networks) in major cities, including New York, Toronto, Hong Kong, Sydney and London. The company’s solutions keep commuters around the globe connected while they’re in transit, driving the evolution of smart cities.

Supernova Award Category

Data to Decisions

The Problem

There are more and more devices pushing data at the edge — leading to more data that needs to be captured and analyzed, which poses new challenges and increases complexity.

BAI was challenged to capture, store and analyze the immense volumes of log data generated over a lengthy period of time, across its smart city networks. The sheer volume of data BAI generated was too costly to store on-site.

For example, BAI’s Wi-Fi infrastructure in the Toronto subway alone sees upwards of 150,000 daily sessions. Capturing and analyzing this data over a longer period of time ensures imminent network failures or anomalies are easily identified before they become a problem. Maintaining the feed also enables BAI to provide overcrowding detection analytics to their transport authority partner, the Toronto Transit Commission (TTC).

Additionally, because of the various services BAI provides—such as public Wi-Fi and cellular networks—the data captured needs to adhere to government regulations and compliance, which translates to maintaining a tremendous amount of data without compromising it.

The Solution

It was vital that the chosen vendor had the ability to analyze the diverse types of data BAI generates, while also identifying trends and correlations across disparate systems. BAI also needed a solution that could support its regulatory and compliance requirements, while handling and analyzing growing volumes of complex data.

The results

ChaosSearch empowered BAI to find, access and analyze its data faster, easier, across a longer period of time, in a more cost-controlled manner. The platform allows BAI to stream its data from a centralized data lake onto its own cloud object storage, as-is, for log, SQL and machine learning analytics.

Working with ChaosSearch and using its compression index helped BAI reduce total cost and store data for a year. While they initially focused efforts on optimizing data storage, BAI has since expanded its ChaosSearch use case by building solutions on top of the stored data, like anomaly detection, penetration testing and more.

Metrics

Expanding data retention 52x - from one week to one year – was critical for meeting Payment Card Industry (PCI) and ISO 27001 security compliance, in a cost-effective manner.

Increasing data storage with ChaosSearch also allows BAI to enhance their analytics capabilities by uncovering and analyzing long-term trends.

The Technology

The ChaosSearch Data Lake Platform connects to and indexes data within a customer’s cloud storage environment, rendering it fully searchable and enabling analytics with existing analytic tools in use today, leveraging open APIs.

Disruptive Factor

Finding a suitable, cost effective solution was challenging due to BAI’s unique use case and volumes of data generated.

ChaosSearch allowed BAI to implement their own solution, adapted perfectly to their needs vs. other turnkey solutions that were not suitable and cost prohibitive.

Shining Moment

ChaosSearch’s analytics capability, coupled with BAI’s aggressive data storage strategy, helps the company uncover new value from its data. For example, insights generated from BAI’s Wi-Fi network helps transit operators monitor and maintain flow of traffic. By analyzing the data across the Wi-Fi network, transit operators are able to optimize routes based on transit capacity.