IoT; Data Flow Management – the science of getting real value from IoT Data
IoT; Data Flow Management – the science of getting real value from IoT Data
Andy Mulholland is Vice President and Principal Analyst focusing on cloud business models. Formerly the Global Chief Technology Officer for the Capgemini Group from 2001 to 2011, Mulholland successfully led the organization through a period of mass disruption. Mulholland brings this experience to Constellation’s clients seeking to understand how Digital Business models will be built and deployed in conjunction with existing IT systems. Coverage Areas Consumerization of IT & The New C-Suite: BYOD, Internet of Things, IoT, technology and business use Previous experience: Mulholland co authored four major books that chronicled the change and its impact on Enterprises starting in 2006 with the well-recognised book ‘Mashup Corporations’ with Chris Thomas of Intel. This was followed in…...
Read moreThe Business case for the Internet of Things -- indeed for everything becoming network connected -- lies in the positive outcomes enabled by leveraging the data collected from the instrumentation of the business as a whole. Its not just about collecting more data from existing applications, it’s about being able to obtain data in new ways. The combination of this new data and traditional analytics is expected to add Business value via ‘Insights’ that come from marring these two, previously isolated data sets, in real-time. Robust Data Flow management is required to handle the veracity, velocity, variety, and volume of IoT Data Streams when combining them with Insight Analytics.
Research report now available: The Foundational Elements for the Internet of Things (IoT)
When a canteen table gets knocked over triggering the response for a member of the canteen staff to go and pick it up that’s a real-time Event, requiring one type of “intelligent response†in the moment, and an additional action to add that data point to a data store, such that it’s available to existing analytics applications. In comparison an Insight is when the canteen manager uses an analytic application to understand that the same table gets knocked over when ever the queue exceeds 26 people. This combination of data at rest, as well as data in motion leads to positive business outcomes, both in the moment and over long term operations.
To achieve the above Event notification merely requires a simple IoT sensor attached to the table to send a report to a pre defined device that is able to affect the “in the moment fix†for an overturned table (perhaps notifying nearby canteen staff’s mobile devices). The Insight from relating together multiple data reports concerning the table being knocked over requires Data Flow Management with Insight Analytics to produce this type of new, enhanced value.
IoT Data Flow management might be considered the Internet equivalent of the IT Messaging Bus, which provides similar capabilities in the sense of distributing data, but in the diametrically opposed environment of Internet architecture versus the traditional “stove-piped†per-application architecture. This “Internet architecture†consists of non-deterministic, loosely-coupled, stateless data flows that require a degree of managed “flow†towards dynamic, rule-based, and analytic-driven delivery.
IoT Data Flow Management is by definition a complex activity and can involve many physical “things†as well as other more ephemeral information systems.. The following example will attempt to shed light on just how varied the set of constituents may be in a typical IoT solution Data Flow solution.
In this case the defined Business Value outcome is increased production levels achieved by focusing on factory production machines, (see previous blog defining three outcomes Machine, Process and People), specifically via an IoT connected Plastic Injection Molder. Although the Plastic Injection Molder is the single most important and critical factory production asset, it must be considered as part of the production line operation which includes the Plastic Kibbler machine which grinds plastic waste to feed the Molder with raw material, and the Carton Maker which prepares the cartons into which the output is packed.
The factory production manager will wish to increase the precision and accuracy of monitoring the Molder to discover settings for optimum speed and efficiency. Any increases in output require increased production in both cartons and kibbled plastic raw material, leading in turn to a similar efficiency exercise with the other two machines. The factory production manager will need to use all three sets of IoT/IIoT data to establish the optimum production speed that balances the capabilities of all three machines at an optimized setting.
Though Machine operational efficiency is the business value that will be measured, people are also heavily engaged in real time management of the optimization. A further Data Flow must connect to these processes in order to enable them to engage with operational enterprise impacts.
Changes in production levels need to be matched by supporting activities carried out through the ERP IT based operations too; Purchasing will need to increase call offs inline with increased consumption of raw materials; Production Planning will need to update its data on production rates to reflect in new planning schedules; Sales may be able to offer reduced lead times, better pricing and responsiveness to customers.
IoT Data Flows create a new level of connected and integrated Enterprises linking what is popularly called Front, and Back Office with Industrial Operations into a true Digital, or Smart Enterprise.
Previous blogs in this series have defined how various other elements in the connected environment, as well as in the connected enterprise work. ‘Fog Computing’ with Network Integration Groups enables complex Groupings of sensors, gateways and platforms. Sensor Alignment/Asset Mapping at the sensor deployment stage is required to discover/align the legacy data and provide graphic positioning information on sensors. Data Flow and Insight Analytics is the final building block over these technologies and supports the sharing of IoT/IIoT data and its flow between each of the three domains of Machines, Process and People systems.
Data Flow Management operates to monitor and switch data flows at high speed accordingly to both rules and outcomes for solutions were more than simple direct sensor to reporting connections are required. Understanding what and how to deliver Outcomes leads into the complexity of recognizing Insights together with how this requires a flow to be switched.
The diagram below shows all the elements of an IoT/IIoT architecture to connect sensors to the domains, Machine, Process and People; each of which has different technology as well as a leading technology vendor with a solution Platform. In the diagram this is shown as Salesforce, SAP and GE providing Platforms to manage and integrate IoT into their area of expertise. Each vendor also provides sophisticated analytics that require being fed the ‘right’ Data Flows from which they can provide powerful Business valuable Insights. It is this ‘tangential data’ that can provide the inputs that are of the greatest value in producing ‘Insights’.
Note that this diagram deliberately omits the usual use of arrows to show connectivity as all elements and groupings function as part of a common shared Internet networking environment. The Network itself is responsible for more than straight forward connectivity, the network must add a significant additional set of capabilities to be able to determine what may be connected to what and under what conditions, (see again Fog Computing previous blog).
Adding an example of an ‘Insight’ to the Machine production example above helps understanding what Data Flow Management and Insight Analytics has the power to introduce. The factory production manager gains the Insight that on each Thursday afternoon there is a regular drop in production speed on the Plastic Molder due to a shortage of raw material from the Plastic Kibbler. At the same time the Kibbler Machine slows down due to the built up of output raw material at the same time. The situation has occurred up to three times on Thursday afternoons for periods of less than fifteen minutes each time. Other data is appended from the same time period, including the indication that the forklift transporting raw material between the two machines has been out of its working area.
IoT/IIoT provides the data in support of the truism ‘you can only manage what you can measure’, but the real breakthroughs come from non-obvious tangential data alignments through new types of analysis coupled to IoT/IIoT Data Flows.
Switching the focus to the forklift truck driver rather than the machine operator reveals that on Tuesday afternoon the truck is called away to unload a particular contractor’s truck that requires a forklift with an extending carriage. The forklift providing the transport of materials between the Kibbler and the Molder is the only forklift on site with an extending carriage. Identification of the real issue to be addressed, which is no way directly associated with the symptoms shown of slow down in production at the Molder, would be a good example of key Insight benefit.
The central group of functions in the diagram described as Smart IOT Service Management contains three specific tasks that relate to the so-called ‘final mile’ challenge of IoT/IIoT. This is the mass scale challenge of connecting, identifying, collating and forwarding useful data in a useable manner to authorized Platforms such as those of Salesforce, SAP, and GE. As the role of Sensor Alignment and Asset Mapping has already been discussed the focus will be on the Data Flow Management and Insight Analytics.
Flow Data Management coupled to Insights Analytics is is a new specialized area of technology emerging in response to the high growth IoT/IIoT market place. As such it needs not only products and methodologies but also terminology to describe its new and unique manner of defining Data Flows together with an Open Source development environment and tools.
flowthings.io is a leading startup in Data Flow management, as defined above, defining their mission as; ‘the rapid creation of scalable solutions for real time data collection, complex event processing, and data delivery’.
Their website offers a set of Application Program Interfaces (APIs) to allow Developers to create scalable solutions that require complex combinations of; Real Time Data Ingestion, Complex Event Processing, Integration of Analytics, Big Data, and Knowledge systems, as well as Real Time Data Sharing across organizational boundaries. All of which are available via REST, MQTT, Webhook, and WebSockets protocols. For those serious interested in IoT/IIoT solutions building around ecosystems and enterprise wide sharing of IoT /IIoT data including intelligent distribution of data, a more detailed study of their web site is a recommended action.
The Business value of any IoT/IIoT deployment rests solely with the ability to use the resulting data to create a new level and type of detailed understanding of real time events and activities. To reach that value requires ‘final mile’ integration of Data Flows, and Insightful Outcomes from all available sensor sources with intelligent distribution to Technology vendors Platforms that will provide determination of actions. It is unlikely that any technology vender focused on delivering in their Business focus area, (machine, process, people), with the technology required will look to address ‘final mile’ Data Flow integration for a considerable period, and even then it will need to be provided in an Open Source format.
Data to Decisions Future of Work Innovation & Product-led Growth New C-Suite Tech OptimizationResearch report now available: The Foundational Elements for the Internet of Things (IoT)
.jpg)
