The Final Mile – Part 2; The Complexity of IoT Protocols and APIs The importance of APIs has grown recently with IoT increasing the diversity of Protocols.
Vice President and Principal Analyst, Constellation Research
Constellation Research
Andy Mulholland is Vice President and Principal Analyst focusing on cloud business models. Formerly the Global Chief Technology Officer for the Capgemini Group from 2001 to 2011, Mulholland successfully led the organization through a period of mass disruption. Mulholland brings this experience to Constellation’s clients seeking to understand how Digital Business models will be built and deployed in conjunction with existing IT systems.
Coverage Areas
Consumerization of IT & The New C-Suite: BYOD,
Internet of Things, IoT, technology and business use
Previous experience:
Mulholland co authored four major books that chronicled the change and its impact on Enterprises starting in 2006 with the well-recognised book ‘Mashup Corporations’ with Chris Thomas of Intel. This was followed in…...
Read more
IoT Devices broadly require a network transport to match the physical network characteristics and a data protocol able to communicate specialized data descriptions such as vibration, or pressure. Sometimes the two may be combined for simple types of sensor deployments, or specific data extensions can be made to an existing recognized protocol, either from the IT Industry, or the Industrial Sector. An accelerating number of protocols has seen a huge growth in the requirements for API integration or translation capabilities often offering reusable API libraries.
Protocols are the last of the three key element of the so-called IoT ‘Final Mile’, along with Asset Digitation and Data Stream switches (add URL). IoT sensor and solution developers are focused on what, and how, IoT protocols should be able to have specific data definitions for different sensing types, functions, even different industry sector requirements. The use of multiple and specialized Protocols also introduces the need for well-defined APIs with toolsets for integration leading to the topic of APIs becoming a topic of increasing importance.
The use of various forms of sensors is well established for ‘shop floor’ Operational Technology, or OT, in many manufacturing plants in small scale closed deployments. As such it’s a good working example of the development of a specialized sensor protocol into a common standard, one that today may be part of any manufacturing enterprise shift to large scale interactive IoT. In 1995 the popularity of the physical networking media of Ethernet led to the establishment of the ODVA as a formal industry body with hundreds of members to define suitable standardized common protocols. Amongst other work the ODVA developed the widely used, popular industrial network protocol CIP, standing for Common Industrial Protocol for Industrial use.
CIP is a mature example containing many factors of the type that will need to be considered to communicate the special data descriptions for the vast range of different monitoring situations arising from full scale IoT. CIP provides Sensors and Sensing systems with definitions for various types of industrial data describing Motion, Motors, Transducers, etc., even extending these into functionality services to cover common types of Operational Technology, OT, requirements such as Energy Optimization, Synchronization or Motion Control.
Whilst CIP is immensely popular in OT, it’s scarcely known, or applicable, to IT due to its specialized industrial data descriptions. This is the beginning of a new challenge in IoT’s ‘Final Mile’ as wider deployment of sensors to combines with wider numbers of types of sensor leading to new Protocol types, or unrecognizable extensions to existing Protocols. This same diversity will lead to the need for extensive API libraries with translation and integration platforms that may also include IT Process integration.
Most references to IoT protocols will refer to four currently widely recognized and used protocols albeit with countless extensions that require different API support. There are also seven similarly recognizable new protocols under development by different consortiums around specialized features. Beyond these there is a long, and seemingly growing longer, list of other Protocols with various claims as to their benefits and selective use. Before listing these most recognizable protocols it might be useful to consider the particular interactions in an IoT systems that Protocols may have to be able to support;
D2D The requirement for communication and interaction of IoT Devices 2 IoT Devices
D2S The dedicated relationship between an IoT Device 2 an IoT Server usually locally
S2S Server 2 Server sharing of Data often at the level of virtual machines on Clouds
Four Commonly Used and Extended Protocols
Listing these four protocols in outline suggests that there is some considerable overlap, even standardization, and this may well be true in core functionality but in practice it there are many different implementations, or extensions, built on each. However it is their simplicity that makes them simple to use and has led to their popularity for relatively small-scale closed IoT deployments.
MQTT - A popular easy to use TCP based choice that offers both request/response and publish/subscribe features for Device to Servers, now an OASIS standard.
XMPP - Similar capabilities to MQTT, also using TCP transport, but with Security making it popular for Smart Phones where a person can interface to Security element
DDS - A more specialized protocol based on UDP rather than TCP that provides a fast Bus usually used to interconnect machines in Industrial IoT
SNMP - Also UDP transport based providing a simple request/response service suitable for Device to Server
Seven Specialized Protocols being developed for IoT by Industry Consortiums
In contrast the following group of seven consortiums are developing protocols with a particular focus on large-scale sophisticated IoT applications. Each is backed by a significant collection of powerful Enterprises including sector market leaders, however the diversity of these Enterprises own businesses means that their are often backing more than one consortium;
Open Interconnect - Aims to provide an Open Source Protocol with a Wireless specification that will offer the necessary scale and capabilities to handle the billions of Devices interacting by 2020. Initially will offer Smart Home and Office capabilities. Promoted by General Electric and Intel.
Industrial Internet – A very powerful group of Global Players predominantly USA based focused on the Industrial Internet and Operational Technology rather than the consumer goods market present in other consortiums. I.I. is looking to select and combine other useful standards under the leadership of the Object Management Group, OMG. Promoted by General Electric, IBM and Intel.
Internet 4.0 – A term used to describe the connected IoT environment as the third wave of the Internet. Initially promoted by the German Government in support of key German Manufacturing companies’ initiatives to develop competitive IoT connected smart manufacturing. Its format is more of an alliance for R&D under the IOT@work title effort to develop a long term comprehensive IoT stack. Promoted by EU, German Government, Siemens, and SAP.
Thread - A low power Mesh network protocol already in use for NEST Smart Homes on the 2.4Ghz unlicensed wireless spectrum with the advantage of being based on common existing transport protocols. Promoted by Google and ARM.
Allseen Alliance – A substantial consortium with the support of many large Consumer and White Goods enterprises based on Qualcomm AllJoyn free to use protocol. Promoted by Linux Foundation, Qualcomm and Microsoft.
HomeKit – A further wireless based protocol, but incompatible with other wireless protocols, such as Thread, driven by Apple to allow a single Device (Smartphone/Tablet/PC) to control many Devices including IoT group based services, (such as lock up home), with a single command. Promoted by Apple, and Qualcomm.
Hypercat – A UK Smart Cities centric consortium aiming to provide a sophisticated protocol that allows searching for Devices around functionality and uses self-describing metadata to define resulting data flows. Promoted by UK Government and British Telecom
Three existing widely used API Integration Vendors who have added IoT capabilities
It is obvious that with so many different protocols already in use, and the promise of others coming into use, that there is an interoperability/integration challenge hence the considerable amount of interest recently in APIs. Unsurprisingly on seeing the opportunity several startups have moved to launch products using a variety of descriptions for their API engines or Platforms aimed specifically at the IoT market. The following list identifies three IoT API market contenders, and is the followed by three established integration vendors who have mature capabilities extended to include the IoT and Social Marketing environments.
Xively – Launched by LogMein, an established Technology Vendor, aimed specifically at integrating different IoT Devices using different protocols via a centralized Web Service. Xively provides easy configuration to a standardized API over HTTP, Sockets and MQTT to allow interaction and connectivity to existing IT.
Zetta – Provides a hub and spoke architecture to link to their Cloud where the intelligence for the processing of events is provided as a managed service together with API integration and protocol conversion. Apigee is the company behind Zetta.
Temboo – Described as an API Arbitration Platform this cloud based service sits over a wide range of third party APIs to offer developers a common Temboo format that hides individual API details using code elements within their cloud to bridge the differences. Function calls and Complex Event processing can take place in the same process no matter what Device with what Protocol is connected.
Three Vendors with IoT Additions to their Integration and Processing Platforms
SnapLogic – The SnapLogic Elastic Integration Platform with sub toolsets was developed specifically to integrate Cloud based Services, be they public, private or hybrid of any type using a stream architecture for quasi real-time, low latency responses to events. A library of more than 300 ‘snaps’ cover analytics and big data sources, identity management, social media, online storage, ERP, databases and technologies such as XML, JSON, Oauth, SOAP and REST plus IoT protocols. New Snaps are easily built and added to the library, with operational scalability claimed as a major advantage together as a central Cloud service architecture provides unlimited scale up power.
Automic – are a long established provider of process automation across disparate environments who entered both the Cloud and Social Media markets some years ago, and are now adding IoT integration. Automic have a very rich set of features and capabilities covering all aspects of not just protocol integration, but security, process automation and data analysis all with continuous nonstop service updates. Loose coupled ‘final mile’ IoT diversity can be integrate with close coupled Enterprise IT Applications, quasi real-time Interactive Device groups can operate and interoperate with near real-time IT transactions. Automic is sophisticated solution for large-scale mission critical integrations that allows IoT to extend the value of existing processes.
Mulesoft – started as an Open Source project in 2003 for quick assembly of Enterprise components and is now supported by over 200000 developers who have recently been added IoT integrations. The MuleSoft AnyPoint Platform provides API based integration and recent developer led projects have added, and are continuing to add to the library of specialized APIs for various IoT purposes.
Currently the IoT market is awash with different protocols, and there is a substantive risk that, as with PC operating systems, the choice will be tactical/accidental driven by the choice of a particular sensor vendor for a first small scale deployment. API integration may well enable the flexibility to allow sensor driven protocol choices, and this may well be necessary given the wide spread of IoT device types in an Enterprise.
However once a Protocol is adopted it is difficult to remove, accordingly even for first IoT pilots adding a selection criteria as to the protocol requirement is a recommended step. Similarly it is important to understand that selection of an API integration tool/platform is a long term strategic question, particular when integration with existing IT systems and processes is also required.
Resource
Footnotes
An excellent summary of the main features and uses for a list of protocols;
Kim Row; Internet of Things Protocols.
A case study on the Enterprise business benefits of API based protocol integration of Industrial Manufacturing Operational Technology, OT, with Information Technology IT.
Stanley, Black and Decker cases study
New C-Suite
