Event Report - SAP Ariba Live - The quest to make Procurement awesome

Holger Mueller

Vice President and Principal Analyst
Constellation Research

Holger Mueller is VP and Principal Analyst for Constellation Research for the fundamental enablers of the cloud, IaaS, PaaS and next generation Applications, with forays up the tech stack into BigData and Analytics, HR Tech, and sometimes SaaS. Holger provides strategy and counsel to key clients, including Chief Information Officers, Chief Technology Officers, Chief Product Officers, Chief HR Officers, investment analysts, venture capitalists, sell-side firms, and technology buyers.  Coverage Areas: Future of Work Tech Optimization & Innovation  Background: Before joining Constellation Research, Mueller was VP of Products for NorthgateArinso, a KKR company. There, he led the transformation of products to the cloud and laid the foundation for new Business Process as a…...

Read more

We had the opportunity to attend SAP Ariba’s Live user conference in Las Vegas, held from March 21^st till 23^rd 2017, at the Cosmopolitan. The event was well attended with over 3200 attendees, good partner representation and influencer selection.

 

 

 

So take a look at my musings on the event here: (if the video doesn’t show up, check here)
 

 

No time to watch – here is the 1-2 slide condensation (if the slide doesn’t show up, check here):

 

Event Report - SAP Ariba Live - The quest to make procurement awesome from Holger Mueller

 

Want to read on? 

Here you go: Always tough to pick the takeaways – but here are my Top 3:

Ariba is on a roll – A few years ago it seemed that Ariba (add an SAP in front of every time you read Ariba going forward) maybe in the state of a being the sleeping beauty of enterprise software. Always there – but not going anywhere. That has changed in the recent year and Ariba now has the momentum to show it: Even for SAP adding 10 of the Top 100 Global Businesses in 12 months is quite a feat – and both a testament to the market position that Ariba has achieved as well as the attractiveness of what Ariba has recently provided and plans offer soon. And suppliers will pay attention when hearing that Ariba has added B300US$ in 2016 in network volume. Similar to e-commerce we can see the networks synergies playing in favor of Ariba.
 

Atzberger - Make Procurement Awesome

Functionality Push – Last year Ariba unveiled the Guided Buying approach, both a simplification on the product side and usability improvement for the UX. The combination has worked well for Ariba and proven popular with customers. Spot Purchases were announced, too and are available now, the upcoming implementation at Latin American trading powerhouse Mercado Libre is a proof point that Ariba has built good functionality. When asking Mercado Libre for the reasons for selecting Ariba, they first mentioned the synergy effects of using SAP already – something that bides well for further sales of Spot Purchases into the SAP install base.

 

 

 

Ariba 2016 Momentum

That the combination of UX improvement and simplification is working out for Ariba, is also seen by the plans to bring the same concept to bear on the Supplier side with the Light Account: During the keynote, we saw the demo of onboarding a new supplier in 2 minutes, something as unthinkable as well as un-achievable in today’s business practice.

 

McDermott & Atzberger Q&A

Blockchain meets Purchasing – In a sign of times, this was also a conference with a Blockchain announcement, and SAP picked Hyperledger for its first dabs into distributed ledger technology. Certainly, a good choice, though SAP likely will also have to support other blockchain technologies, but Hyperledger is a good start. And few places lend themselves more to the blockchain scenario than purchasing, so it’s good to see SAP innovating.

 

 

MyPOV

It is remarkable how fast SAP Ariba is moving, especially when one considers (which was not much part of the public talks at the conference) that Ariba is in the midst of a major re-platform endeavor – moving off Oracle and onto SAP HANA. Usually vendors take a noticeable pause while undergoing exercises like these – not so much SAP Ariba. It is good to see the vendor doubling down on things that work, e.g. the UX improvements and the overall process simplifications, while at the same time innovating with blockchain, team productivity software (Microsoft Teams was shown) and speech recognition. Still a tall order – to make a traditionally boring administrative software like Procurement awesome… to get there Ariba has shown the willingness to partner and be an open platform and finally has embraced lofty goals such as diversity, or even more ambitious – the quest of ending modern age slavery, a topic near and dear to the heart of Ariba boss Alex Atzberger.

On the concern side, Ariba has to deliver a lot while a lot is happening. Never an easy scenario for any vendor, so we will be keeping an eye, especially on the platform side, how Ariba will progress in the next quarters.

Finally, it was good to see that SAP seems to have found the right length of ‘leash’ for the Ariba subsidiary – not too close to stop innovation, and allowing freedom (e.g. manifested in Ariba using angular.js and not Fiori) but also have leverage (e.g. with HANA). That SAP CEO McDermott said in a Q&A (see a Storify Tweet Story here) that he would be open to e.g. integrate with perennial co-opetitor Oracle, speaks signs of the flexibility and pragmatism that is lived now at the top of both companies, always a good sign for customers.
 

•  

Want to learn more? Checkout the Storify collection below (if it doesn’t show up – check here). Don't miss the Day #2 keynote Storify collection here.

 

 

Future of Work Tech Optimization Matrix Commerce Marketing Transformation Revenue & Growth Effectiveness Next-Generation Customer Experience Data to Decisions Digital Safety, Privacy & Cybersecurity Innovation & Product-led Growth SAP Supply Chain Automation Cloud Digital Transformation Disruptive Technology Enterprise IT Enterprise Acceleration Enterprise Software IoT Blockchain ERP Leadership Collaboration M&A SaaS PaaS IaaS Next Gen Apps CRM CCaaS UCaaS Enterprise Service Chief Information Officer Chief Procurement Officer Chief Supply Chain Officer Chief Experience Officer Chief Technology Officer Chief Information Security Officer Chief Data Officer Chief Executive Officer

Domo Climbs Enterprise Ladder in Cloud Business Intelligence

Doug Henschen

Former Vice President and Principal Analyst
Constellation Research

Doug Henschen is former Vice President and Principal Analyst where he focused on data-driven decision making. Henschen’s Data-to-Decisions research examines how organizations employ data analysis to reimagine their business models and gain a deeper understanding of their customers. Henschen's research acknowledges the fact that innovative applications of data analysis requires a multi-disciplinary approach starting with information and orchestration technologies, continuing through business intelligence, data-visualization, and analytics, and moving into NoSQL and big-data analysis, third-party data enrichment, and decision-management technologies. Insight-driven business models are of interest to the entire C-suite, but most particularly chief executive officers, chief digital officers,…...

Read more

Domo has graduated from analytics startup to enterprise contender, breaking new ground in cloud-scale deployments. Here’s a look inside the fast-growing company.

Domo is enterprise ready. That’s the key takeaway Domo wanted to project at Domopalooza, held March 22-23 in Salt Lake City. The event drew more than 3,000 attendees and saw keynote appearances from enterprise-scale customers including Target, GE Digital, UnitedHealth Group and Univision.

Domo has surpassed the 1,000-customer mark, and more than half of its revenue now comes from $1 billion-revenue-plus enterprise customers, according to company executives. At Domopalooza, CEO and founder Josh James announced that the company has reached a $120-million-annual-revenue run rate. That’s a fraction of the $827 million in revenue rival Tableau reported in 2016 and a rounding error compared to Microsoft’s revenue (although that company doesn’t break out PowerBI revenue, which is Domo’s competition). Nonetheless, given Domo’s claimed 100% growth rate and the list of enthused customers at Domopalooza, it’s time for a closer look.

Domo CEO Josh James focused mostly on interviewing big customers during
his keynote time at Domopalooza 2017.

Previously co-founder and CEO of Omniture, James founded Domo in 2010 shortly after selling his old company to Adobe for $1.8 billion. The Domo executive team is loaded with Omniture veterans, and they tell the story that James came up with the idea for Domo because he was so frustrated with the incumbent tools available for business insight when he was the CEO at Omniture. The intent was to build an agile, cloud-based analytics platform in the mold of Omniture, but designed to handle diverse business data sources (beyond the Web, mobile and social data analyzed in Omniture).

Domo runs in Amazon’s cloud, and it includes components to capture, prepare and visualize data and then engage in collaboration and optimize business decisions. The platform’s back-end data warehouse, the Domo Business Cloud, scales up at cloud speed and handles diverse data sources, including semi-structured and sparse data. At Domopalooza the company announced that its data store has surpassed 26 petabytes, making it the largest cloud-based analytical data store of its kind, according to James.

Domo’s largest customer, Target, spoke to the platform’s scalability during a keynote interview. Target loads data on every item and every transaction from about 2,000 stores into the Domo Business Cloud at 15-minute increments, explained Ben Schein, Director of BI & Analytics. Where store-level reporting was previously updated once a week, Schein said Domo brings near-real-time insight into store operations, purchasing and stocking trends to 1,500 to 1,700 users per week. James acknowledged that Target helped Domo learn how to scale, harden and mature its platform.

To capture data, Domo has created more than 400 pre-built connectors to popular data sources. A data-transformation tool called Magic lets users join and blend data through a drag-and-drop interface. The company partners with data-integration vendors like Informatica and Talend to support more sophisticated ETL work.

Domo’s front-end data-analysis environment combines pages, cards and applications. Pages are analogous to dashboards and cards are individual visual analyses. Pages and cards are mobile first, meaning you build them once and they dynamically render for phone, tablet or desktop viewing. The company has more than 1,000 applications, which are pre-built, but customizable visual analyses, such as a Social Index app for benchmarking brand popularity and net promoter scores or the Sales Forecast app, which measures predictions against actuals and quotas, with drill-down analysis of rep and manager performance. There’s also a SQL-like “Beast Mode” that enables power users to develop custom transformations and analyses.

Ease of deployment and administration are big selling points. The back end is entirely managed by Domo. When you add more data or more challenging analyses, Domo adds storage and compute nodes automatically. Pricing is based entirely on the number of users, not storage or compute capacity. The pricing model is designed to encourage customers to load more data and build more cards and pages.

Agile analysis is another selling point. At the event, a GE Digital executive showed off a company-wide performance dashboard she built “in one day,” complete with slick, graphical formatting created through an Adobe Illustrator plug-in to Domo. A merchandising executive from Target described how her team reviewed a prototype dashboard in the morning and got back a revise with all requested changes by the end of that day. And an executive from Sephora Southeast Asia said her company got started with Domo late last fall and had a dashboard available within two weeks — just in time for Black Friday performance analysis.

Domopalooza saw four key product announcements:

Analyzer upgrades. The Analyzer is where users do their slicing, dicing and page and card building. Top announcements here included a Data Lineage Inspector that shows where the data used in an analysis comes from and how it was transformed or altered. Data “slicer” buttons can now be added to cards to support guided analysis to the most sought-after views of data. And a new period-over-period analysis feature supports time-based comparisons that previously required Beast Mode customization.

Business-in-a-Box. This collection of pre-built, role-based dashboards is designed to support rapid delivery of the most-asked-for insights across sales, marketing, finance, operations, IT and other business functions. It’s set for release this spring.

Domo Everywhere. Also due this spring, Domo Everywhere is the company’s entry into embedded analytics, white-label licensing and publishing. The offering provides ways for customers to make Domo analytics available within their own software, through Web services or on websites under their own brand.

Mr. Roboto. Attendees got a sneak peek at a few of the advanced analytics, machine learning and natural language understanding capabilities of this offering. I was told it will be a layer of capabilities  within the platform, not a bolt-on module. Release dates weren’t offered, so it’s not something I expect to see fleshed out until late 2017 or perhaps Domopalooza 2018.

MyPOV on Domo’s Course  

I came away from Domopalooza impressed by the scale of Domo’s largest deployments and the enthusiasm of its customers. A highlight of the event was the closing general session, during which Domo previewed coming new features and then turned the mike over to customers to share feature requests. Each request was briefly discussed in a back-and-forth with Domo executives. The request was then listed on a slide (see photo above) for all to see and the audience was then asked to show their interest by a show of hands (or clapping or hoots and hollers). I’ve seen these sorts of sessions at other events, but you don’t see companies with poor customer satisfaction doing it for fear of initiating a bitch fest.

The turning point that Domo is now navigating is the same one that the likes of Tableau and Qlik ran into a few years ago, namely enterprise-grade maturity. Domo execs acknowledged that they’re now facing demands from IT for governance features and administrative capabilities for managing many users.

The Data Lineage Inspector, for example, is just a start on the data governance capabilities customers want. During the closing general session a customer asked for a card-certification feature whereby analyses could have a visual check mark or seal of approval indicating certified status. The indicator would automatically change if data sources or analyses were altered. Domo execs said they are working on such a governance scheme, and by a show of hands there was keen interest (approximated at 91% by Domo's chief product officer and session leader, Catherine Wong).

Domo offers three hybrid deployment options for large or regulated customers that don’t want to put everything in the public cloud. A Federated Query feature that’s very new will let customers query data in place, but performance is dependent upon the bandwidth of the connections and the compute power of each source. A second option let’s companies put the Domo Business Cloud data layer behind the corporate firewall while leaving the analysis layer in the cloud. A third option is running the entire Domo platform as a dedicated instance on AWS or Azure or as a private cloud instance behind a corporate firewall.

Domo is facing these enterprise challenges earlier in its lifecycle than did some of its competitors. That’s partly due to the maturation of the market and partly due to the experience of the Domo team rooted in Omniture. The upshot is that Domo is maturing quickly and punching above its actual weight.

Data to Decisions Marketing Transformation ML Machine Learning LLMs Agentic AI Generative AI AI Analytics Automation business Marketing SaaS PaaS IaaS Digital Transformation Disruptive Technology Enterprise IT Enterprise Acceleration Enterprise Software Next Gen Apps IoT Blockchain CRM ERP finance Healthcare Customer Service Content Management Collaboration Chief Customer Officer Chief Information Officer Chief Marketing Officer Chief Digital Officer Chief Technology Officer Chief Information Security Officer Chief Data Officer

Down Report – Power failure takes Azure services down - 3 Cloud Load Toads

Holger Mueller

Vice President and Principal Analyst
Constellation Research

Holger Mueller is VP and Principal Analyst for Constellation Research for the fundamental enablers of the cloud, IaaS, PaaS and next generation Applications, with forays up the tech stack into BigData and Analytics, HR Tech, and sometimes SaaS. Holger provides strategy and counsel to key clients, including Chief Information Officers, Chief Technology Officers, Chief Product Officers, Chief HR Officers, investment analysts, venture capitalists, sell-side firms, and technology buyers.  Coverage Areas: Future of Work Tech Optimization & Innovation  Background: Before joining Constellation Research, Mueller was VP of Products for NorthgateArinso, a KKR company. There, he led the transformation of products to the cloud and laid the foundation for new Business Process as a…...

Read more

 

We continue our series of IaaS downtimes – and other availability issues, see our Down Report on the recent AWS downtime here.

 

 

 

Kudos to Microsoft to share issue, impact, impact on customers, workaround, root cause mitigation and next steps on the Azure Status History (see here)

 

 

So let’s dissect the information available in our customary style:

RCA - Storage Availability in East US
Summary of impact: Beginning at 22:19 UTC Mar 15 2017, due to a power event, a subset of customers using Storage in the East US region may have experienced errors and timeouts while accessing storage accounts or resources dependent upon the impacted Storage scale unit. As a part of standard monitoring, Azure engineering received alerts for availability drops for a single East US Storage scale unit. Additionally, data center facility teams received power supply failure alerts which were impacting a limited portion of the East US region. Facility teams engaged electrical engineers who were able to isolate the area of the incident and restored power to critical infrastructure and systems. Power was restored using safe power recovery procedures, one rack at time, to maintain data integrity. Infrastructure services started recovery around 0:42 UTC Mar 16 2017. 25% of impacted racks had been recovered at 02:53 UTC Mar 16 2017. Software Load Balancing (SLB) services were able to establish a quorum at 05:03 UTC Mar 16 2017. At that moment, approximately 90% of impacted racks were powered on successfully and recovered. Storage and all storage dependent services recovered successfully by 08:32 UTC Mar 16 2017. Azure team notified customers who had experienced residual impacts with Virtual Machines after mitigation to assist with recovery.

MyPOV – Good summary of what happened, a power failure / power event. Good to see that customers were notified. Power events can always be tricky to recover, and it looks like Azure management erred on the side of caution bringing up services rack by rack and then adding services like SLB later. But the downtime for affected customers was long, best case – when in the first 25% of racks was almost four hours, and worst case 10 hours+. Remarkable it took Azure technicians 2 hours 20 or so minutes to get the power back. Microsoft needs to (and say it will) review power restore capabilities and find ways to bring storage back quicker. Luckily for customers and Microsoft this happened over night, with possibly lesser effect on customers… but that said we don’t know what kind of load was run on the infrastructure.

Rating: 3 Cloud Load Toads

 

Customer impact: A subset of customers using Storage in the East US region may have experienced errors and timeouts while accessing their storage account in a single Storage scale unit. Virtual Machines with VHDs hosted in this scale unit shutdown as expected during this incident and had to restart at recovery. Customers may have also experienced the following:
- Azure SQL Database: approx. 1.5% customers in East US region may have seen failures while accessing SQL Database.
- Azure Redis Cache: approx. 5% of the caches in this region experienced availability loss.
- Event Hub: approx. 1.1% of customers in East US region have experienced intermittent unavailability.
- Service Bus: this incident affected the Premium SKU of Service Bus messaging service. 0.8% of Service Bus premium messaging resources (queues, topics) in the East US region were intermittently unavailable.
- Azure Search: approx. 9 % of customers in East US region have experienced unavailability. We are working on making Azure Search services to be resilient to help continue serving without interruptions at this sort of incident in future.
- Azure Site Recovery: approx. 1% of customers in East US region have experienced that their Site Recovery jobs were stuck in restarting state and eventually failed. Azure Site Recovery engineering started these jobs manually after the incident mitigation.
- Azure Backup: Backup operation would have failed during the incident, after the mitigation the next cycle of backup for their Virtual Machine(s) will start automatically at the scheduled time.

MyPOV – Kudos for Microsoft to give insight into the percentage of customers affected. It looks like Azure Storage units are using mixed load – across Azure services. That has pros and cons, e.g. co-location of customer data, mixed averaged load profiles – but also means that a lot of services are affected when a storage unit goes down.

Rating 2 Cloud Load Toads

 

 

Workaround: Virtual Machines using Managed Disks in an Availability Set would have maintained availability during this incident. For further information around Managed Disks, please visit the following sites. For Managed Disks Overview, please visit https://docs.microsoft.com/en-us/azure/storage/storage-managed-disks-overview. For information around how to migrate to Managed Disks, please visit: https://docs.microsoft.com/en-us/azure/virtual-machines/virtual-machines-windows-migrate-to-managed-disks.
- Azure Redis Cache: although caches are region sensitive for latency and throughput, pointing applications to Redis Cache in another region could have provided business continuity.
- Azure SQL database: customers who had SQL Database configured with active geo-replication could have reduced downtime by performing failover to geo-secondary. This would have caused a loss of less than 5 seconds of transactions. Another workaround is to perform a geo-restore, with loss of less than 5 minutes of transactions. Please visit https://azure.microsoft.com/en-us/documentation/articles/sql-database-business-continuity/ for more information on these capabilities.

MyPOV – Good to see Microsoft explaining how customers could have avoided the downtime. But the Managed Disk option only applies VMs affected by the storage. Good to see the Redis Cache option – the question is though, how efficient (and costly) that would have been. Cache synching is chatty and therefore expensive. More importantly good to see the Azure SQL option, that is key for any transactional database system that needs higher availability. Again enterprise will have to balance cost benefits.

More of concern is that the other 4 services affected by the outage seem to have no Azure provided workaround, in case customers needed and would decide implement (and pay for one). No work around for Event Hub and Service Bus would not be a good situation, especially since event and bus infrastructures are used to make systems more resilient. Azure Search seems to lack a workaround, too, affecting customers using those services. It’s not clear what the statistic means though: Was Search itself not available or could the information of the affected storage units not be searched. Important distinction. The Azure Site Recovery affected isn’t good either, but kudos for Microsoft to start those manually. But manual starts can only be a workaround, as they don’t scale, e.g. in a greater outage. The failure of Azure Backup is probably the least severe, but in case of power failures, which may not be contained and might cascade, of equal substantial severity, as customers loose backup capability to protect them from potential further outages.

Rating: 2 cloud load toads (with workaround would be 1 / with no workaround 3 – the maximum, as we don’t have full clarity here, we use 3 as the average).
 

 

Root cause and mitigation: Initial investigation revealed that one of the redundant upstream remote power panels for this storage scale unit experienced a main breaker trip. This was followed by a cascading power interruption as load transferred to remaining sources resulting in power loss to the scale unit including all server racks and the network rack. Data center electricians restored power to the affected infrastructure. A thorough health check was completed after the power was restored, and any suspect or failed components were replaced and isolated. Suspect and failed components are being sent for analysis.

MyPOV – Always ironic how a cheap breaker can affect a lot of business. I am not a power specialist / electrician, but reading this – if one power panel fails and load has to be transferred, the system should still be operating. Maybe something was not considered in the redundant design vs remaining throughput capacity, not a good place to be.

Rating - 5 toads
 

 

Next steps: We are continuously taking steps to improve the Microsoft Azure Platform and our processes to help ensure such incidents do not occur in the future, and in this case it includes (but is not limited to):
- The failed rack power distribution units are being sent off for analysis. Root cause analysis continues with site operations, facility engineers, and equipment manufacturers.
- To further mitigate risk of reoccurrence, site operations teams are evacuating the servers to perform deep Root Cause Analysis to understand the issue
- Review Azure services that were impacted by this incident to help tolerate this sort of incidents to serve services with minimum disruptions by maintaining services resources across multiple scale units or implementing geo-strategy.

MyPOV – Kudos for the hands-on next steps. The key measure (which I am sure Microsoft is doing) is though: How many other storage system power units, or overall Azure power units may have the same issue, and when will they be fixed and have the right capacity / redundancy, so this event cannot repeat. And then we have the question of standardization, is this a local knowledge event, are other data centers setup differently – or the same and can the same incident with a higher certainty be avoided.

Out of curiosity – there was another event in Storage provisioning, a software defect, only 37 minutes before (you can find it on the Azure status page, right below the above incident) … and these two events could / may have been connected. The connection between the two is at hand: When having a storage failure in one location, customers (and IaaS technicians) may scramble to open storage accounts – at the same or other locations, if they cannot, ad hoc needed remediation and workaround cannot happen. There maybe a connection / there may not be a connection. But when hardware goes down and the software to manage accounts for the hardware – that’s an unfortunate – and hopefully highly unlikely – connection of events.

 

(Luckily) a mostly minor event

Unless someone was an affected party - this was a minor cloud down event. But it was luckily only a minor event, as power failures can quickly propagate and create cascading effects. Unfortunately for some of the services, there is no easy or no workaround at all available that when they go down, they are down. Apart from Microsoft's lessons learned - this is the larger concern going forward. I count a total of 12 toads, averaging 3 Cloud Loud Toads for this event.

 

 

Lessons for Cloud Customers

Here are the key aspects for customers to learn from the Azure outage:

Have you built for resilience? Sure, it costs, but all major IaaS providers offer strategies on how to avoid single location / data center failures. Way too many prominent internet properties did not chose to do so – so if ‘born on the web’ properties miss this – its key to check regular enterprises do not miss this. Uptime has a price, make it a rational decision, now is a good time to get budget / investment approved, when warranted and needed.

Ask your IaaS vendor a few questions: Enterprises should not be shy to ask IaaS providers if they have done a few things:

• How do you test your power system equipment?
• How much redundancy is in the power system?
• What are the single points of failure in the data center being used?
• When have you tested / taken off components of the power system?
• How do you make sure your power infrastructure remains adequate as you are putting more load through it (assuming the data center gets more utilized).
• What is the expected up in case of power failure? 
• How can we code for resilience – and what does it cost?
• What kind of renumeration / payment / cost relief can be expected with a downtime?
• What other single point of failure should we be aware of?
• How do you communicate in a downtime situation with customers? 
• How often and when do you refresh your older datacenters, power infrastructure / servers?
• How often have your reviewed and improved your operational procedures in the last 12 months? Give us a few examples how you have increased resilience.

And some key internal questions, customers of IaaS vendors have to ask themselves:

• How do you and how often do you test your power infrastructure?
• How do you ensure your power infrastructure keeps up with demand / utilization?
• How do you communicate with customers in case of power failure?
• How do you determine which systems to bring up and when?
• How do you isolate power failures and at what level to minimize downtime
• Make sure to learn from AWS (recent) and Microsoft’s mistakes – what is your exposure to the same event? 

Overall MyPOV

Power failures are always tricky. IT is full of anecdotes of independent power supplies not starting – even in the case of formal test. But IaaS vendors need to do better and learn from what went wrong with Azure. There maybe a commonality with the recent AWS downtime, that IaaS vendors can become the victims of their own success. AWS saw more usage of S3 systems, Microsoft may have seen more utilization of the servers attached to the failing power system setup. And CAPEX demands flow into opening new data centers versus refreshing and upgrading older data centers. 

 

There is learning all around for all participants – customers using IaaS services, and IaaS providers. Redundancy always comes at a cost, and the tradeoff in regards of how much redundancy an enterprise and a IaaS providers want and need will differ from use case to use case. The key aspect is that redundancy options exist and that tradeoffs are made in an ideally fully aware state of the repercussions. And get revisited on a regular basis. 

 

Ironically for the next few years – more minor IaaS failures like this can get the level of cloud resiliency up to the levels where they should be for both IaaS vendors and IaaS consuming enterprises. As long as all keep learning and then acting appropriately.

 

 

 

 

 

 

Tech Optimization Innovation & Product-led Growth Future of Work Microsoft SaaS PaaS IaaS Cloud Digital Transformation Disruptive Technology Enterprise IT Enterprise Acceleration Enterprise Software Next Gen Apps IoT Blockchain CRM ERP CCaaS UCaaS Collaboration Enterprise Service Chief Information Officer Chief Technology Officer Chief Digital Officer Chief Data Officer Chief Analytics Officer Chief Information Security Officer Chief Executive Officer Chief Operating Officer

IBM Blockchain as a Service and Hyperledger Fabric forge a new path

Steve Wilson

Former Vice President and Principal Analyst
Constellation Research

Steve Wilson is former VP and Principal Analyst at Constellation Research, leading the business theme Digital Safety and Privacy. His coverage includes digital identity, data protection, data privacy, cryptography, and trust. His advisory services to CIOs, CISOs, CPOs and IT architects include identity product strategy, security practice benchmarking, Privacy by Design (PbD), privacy engineering and Privacy [or Data Protection] Impact Assessments (PIA, DPIA). Coverage Areas: -  Identity management, frameworks & governance-  Digital identity technologies-  Privacy by Design -  Big Data; “Big Privacy”-  Identity & privacy innovation Previous experience: Wilson has worked in ICT innovation, research, development and analysis for over 25 years. With double…...

Read more

It’s been a big month for blockchain.

• The Hyperledger consortium released the Fabric platform, a state-of-the-art configurable distributed ledger environment including a policy editor known as Composer.
• The Enterprise Ethereum Alliance was announced, being a network of businesses and Ethereum experts, aiming to define enterprise-grade software (and evidently adopt business speak).
• And IBM launched its new Blockchain as a Service at the Interconnect 2017 conference in Las Vegas, where blockchain was almost the defining theme of the event.  A raft of advanced use cases were presented, many of which are now in live pilots around the world.  Examples include shipping, insurance, clinical trials, and the food supply chain.

I attended InterConnect and presented my research on Protecting Private Distributed Ledgers, alongside Paul DiMarzio of IBM and Leanne Kemp from Everledger. 

Disclosure: IBM paid for my travel and accommodation to attend Interconnect 2017.

Ever since the first generation blockchain was launched, applications far bigger and grander than cryptocurrencies have been proposed, but with scarce attention to whether or not these were good uses of the original infrastructure.  I have long been concerned with the gap between what the public blockchain was designed for, and the demands from enterprise applications for third generation blockchains or "Distributed Ledger Technologies" (DLTs).  My research into protecting DLTs  has concentrated on the qualities businesses really need as this new technology evolves.  Do enterprise applications really need “immutability” and massive decentralisation? Are businesses short on something called “trust” that blockchain can deliver?  Or are the requirements actually different from what we’ve been led to believe, and if so, what are the implications for security and service delivery? I have found the following:

In more complex private (or permissioned) DLT applications, the interactions between security layers and the underlying consensus algorithm are subtle, and great care is needed to manage side effects. Indeed, security needs to be rethought from the ground up, with key management for encryption and access control matched to often new consensus methods appropriate to the business application. 

At InterConnect, IBM announced their Blockchain as a Service, running on the “Bluemix High security business network”.  IBM have re-thought security from the ground up.  In fact, working in the Hyperledger consortium, they have re-engineered the whole ledger proposition. 

And now I see a distinct shift in the expectations of blockchain and the words we will use to describe it.

For starters, third generation DLTs are not necessarily highly distributed. Let's face it, decentralization was always more about politics than security; the blockchain's originators were expressly anti-authoritarian, and many of its proponents still are. But a private ledger does not have to run on thousands of computers to achieve the security objectives.  Further, new DLTs certainly won't be public (R3 has been very clear about this too – confidentiality is normal in business but was never a consideration in the Bitcoin world).  This leads to a cascade of implications, which IBM and others have followed. 

When business requires confidentiality and permissions, there must be centralised administration of user keys and user registration, and that leaves the pure blockchain philosophy in the shade. So now the defining characteristics shift from distributed to concentrated.  To maintain a promise of immutability when you don't have thousands of peer-to-peer nodes requires a different security model, with hardware-protected keys, high-grade hosting, high availability, and special attention to insider threats. So IBM's private blockchains private blockchains run on the Hyperledger Fabric, hosted on z System mainframes.  They employ cryptographic modules certified to Common Criteria EAL 5-plus and others that are designed to FIPS-140 level 4 (with certification underway). These are the highest levels of security certification available outside the military. Note carefully that this isn't specmanship.  With the public blockchain, the security of nodes shouldn't matter because the swarm, in theory, takes care of rogue miners and compromised machines. But the game changes when a ledger is more concentrated than distributed.  

Now, high-grade cryptography will become table stakes. In my mind, the really big thing that’s happening here is that Hyperledger and IBM are evolving what blockchain is really for. 

The famous properties of the original blockchain – immutability, decentralisation, transparency, freedom and trustlessness – came tightly bundled, expressly for the purpose of running peer-to-peer cryptocurrency.  It really was a one dimensional proposition; consensus in particular was all about the one thing that matters in e-cash: the uniqueness of each currency movement, to prevent Double Spend.

But most other business is much more complex than that.  If a group of companies comes together around a trade manifest for example, or a clinical trial, where there are multiple time-sensitive inputs coming from different types of participant, then what are they trying to reach consensus about?

The answer acknowledged by Hyperledger is "it depends". So they have broken down the idealistic public blockchain and seen the need for "pluggable policy".  Different private blockchains are going to have different rules and will concern themselves with different properties of the shared data.  And they will have different sub-sets of users participating in transactions, rather than everyone in the community voting on every single ledger entry (as is the case with Ethereum and Bitcoin).

These are exciting and timely developments.  While the first blockchain was inspirational, it’s being superseded now by far more flexible infrastructure to meet more sophisticated objectives.  I see us moving away from “ledgers” towards multi-dimensional constructs for planning and tracing complex deals between dynamic consortia, where everyone can be sure they have exactly the same picture of what’s going on. 

In another blog to come, I’ll look at the new language and concepts being used in Hyperledger Fabric, for finer grained control over the state of shared critical data, and the new wave of applications. 

 

Digital Safety, Privacy & Cybersecurity Future of Work Matrix Commerce Tech Optimization Innovation & Product-led Growth IBM AI Blockchain Chief Executive Officer Chief Financial Officer Chief Information Officer Chief Supply Chain Officer Chief Digital Officer Chief Technology Officer Chief AI Officer Chief Data Officer Chief Analytics Officer Chief Information Security Officer Chief Product Officer

Cloudera Focuses Message, Takes Fifth On Pending Moves

Doug Henschen

Former Vice President and Principal Analyst
Constellation Research

Doug Henschen is former Vice President and Principal Analyst where he focused on data-driven decision making. Henschen’s Data-to-Decisions research examines how organizations employ data analysis to reimagine their business models and gain a deeper understanding of their customers. Henschen's research acknowledges the fact that innovative applications of data analysis requires a multi-disciplinary approach starting with information and orchestration technologies, continuing through business intelligence, data-visualization, and analytics, and moving into NoSQL and big-data analysis, third-party data enrichment, and decision-management technologies. Insight-driven business models are of interest to the entire C-suite, but most particularly chief executive officers, chief digital officers,…...

Read more

Cloudera executives can’t talk about IPO or cloud-services rumors. Here what’s on the record from the Cloudera Analyst Conference.

There were a few elephants in the room at the March 21-22 Cloudera Analyst Conference in San Francisco. But between a blanket “no comment” about IPO rumors and non-disclosure demands around cloud plans -- even whether such plans exist, or not -- Cloudera execs managed to dance around two of those elephants.

The third elephant was, of course, Hadoop, which seems to be going through the proverbial trough of disillusionment. Some are stoking fear, uncertainty and doubt about the future of Hadoop. Signs of the herd shifting the focus off Hadoop include Cloudera and O’Reilly changing the name of Strata + Hadoop World to Strata Data. Even open-source zealot Hortonworks has rebranded its Hadoop Summit as  DataWorks Summit, reflecting that company’s diversification into streaming data with its Apache NiFI-based Hortonworks DataFlow platform.

Mike Olson, Cloudera's chief strategy officer, positions the company as a major vendor
of enterprise data platforms based on open-source innovation.

At the Cloudera Analyst Conference, Chief Strategy Officer Mike Olson said that he couldn’t wait for the day when people would stop describing his company as “a Hadoop software distributor” mentioned in the same breath with Hortonworks and MapR. Instead, Olson positioned the company as a major vendor of enterprise data platforms based on open-source innovation.

MapReduce (which is fading away), HDFS and other Hadoop components are outnumbered by other next-generation, open-source data management technologies, Olson said, and he noted that there are some customers who are just using Cloudera’s distributed and supported Apache Spark on top of Amazon S3, without using any components of Hadoop.

Cloudera has recast its messaging accordingly. Where years ago the company’s platform diagrams detailed the many open source components inside (currently about 26), Cloudera now presents a simplified diagram of three use-case-focused deployment options (shown below), all of which are built on the same “unified” platform.

Cloudera-developed Apache Impala is a centerpiece of the Analytic DB offering, and it competes with everything from Netezza and Greenplum to cloud-only high-scale analytic databases like Amazon Redshift and Snowflake. HBase is the centerpiece of the Operational DB offering, a high-scale alternative to DB2 and Oracle Database on the one hand and Cassandra, MapR and MemSQL on the other. The Data Science & Engineering option handles data transformation at scale as well as advanced, predictive analysis and machine learning.

Many companies start out with these lower-cost, focused deployment options, which were introduced last year. But 70% to 75% percent of customers opt for Cloudera’s all-inclusive Enterprise Data Hub license, according to CEO Tom Reilly. You can expect that if and when Cloudera introduces its own cloud services, it will offer focused deployment options that can be launched, quickly scaled and just as quickly turned off, taking advantage of cloud economies and elasticity.

Navigating around the non-disclosure requests, here are a few illuminating factoids and updates from the analyst conference:

Cloudera Data Science Workbench: Announced March 14, this offering for data scientists brings Cloudera into the analytic tools market, expanding its addressable market but also setting up competition with the likes of IBM, Databricks, Domino Data, Alpine Data Labs, Dataiku and a bit of coopetition with partners like SAS. Based on last year’s Sense acquisition, Data Science Workbench will enable data scientists to use R, Python and Scala with open source frameworks and libraries while directly and securely accessing data on Hadoop clusters with Spark and Impala. IT provides access to the data within the confines of Hadoop security, including Kerberos.

Apache Kudu: Made generally available in January, this Cloudera-developed columnar, relational data store provides real-time update capabilities not supported by the Hadoop Distributed File System. Kudu went through extensive beta use with customers, and Cloudera says it’s seeing a split of deployment in conjunction with Spark, for streaming data applications, and with Impala, for SQL-centric analysis and real-time dashboard monitoring scenarios.

MyTake On Cloudera Positioning and Moves

Yes, there’s much more to Cloudera’s platform than Hadoop, but given that the vast majority of customers store their data in what can only be described as Hadoop clusters, I expect the association to stick. Nonetheless, I don’t see any reason to demure about selling Hadoop. Cloudera isn’t saying a word about business results these days -- likely because of the rumored IPO. But consider the erstwhile competitors. In February Hortonworks, which has been public for two years, reported a 39% increase in fourth-quarter revenue and a 51% increase on full-year revenue (setting aside the topic of profitability). MapR, which is private, last year claimed (at a December analyst event) an even higher growth rate than Hortonworks.

Assuming Cloudera is seeing similar results, it’s experiencing far healthier growth than any of the traditional data-management vendors. Whether you call it Hadoop and Spark or use a markety euphemism like next-generation data platform, the upside customers want is open source innovation, distributed scalability and lower cost than traditional commercial software.

As for the complexity of deploying and running such a platform on premises, there’s no getting around the fact that it’s challenging – despite all the things that Cloudera does to knit together all those open-source components. I see the latest additions to the distribution, Kudu and the Data Science Workbench, as very positive developments that add yet more utility and value to the platform. But they also contribute to total system complexity and sprawl. We don’t seem to be seeing any components being deprecated to simplify the total platform.

Deploying Cloudera’s software in the cloud at least gives you agility and infrastructure flexibility. That’s the big reason why cloud deployment is the fastest-growing part of Cloudera’s business. If and when Cloudera starts offering its own cloud services, it would be able to offer hybrid deployment options that cloud-only providers, like Amazon (EMR) and Google (DataProc) can’t offer. And almost every software vendor embracing the cloud path also talks up cross-cloud support and avoidance of lock-in as differentiators compared to cloud-only options.

I have no doubt that Cloudera can live up to its name and succeed in the cloud. But as we’ve also seen many times, the shift to the cloud can be disruptive to a company’s on-premises offerings. I suspect that’s why we’re currently seeing introductions like the Data Science Workbench. It’s a safe bet. If and when Cloudera truly goes cloud, and if and when it becomes a public company, things will change and change quickly.

Related Reading:
Google Cloud Invests In Data Services, Scales Business
Spark Gets Faster for Streaming Analytics
MapR Ambition: Next-Generation Application Platform

 

Data to Decisions Tech Optimization Digital Safety, Privacy & Cybersecurity Innovation & Product-led Growth Future of Work SaaS PaaS IaaS Cloud Digital Transformation Disruptive Technology Enterprise IT Enterprise Acceleration Enterprise Software Next Gen Apps IoT Blockchain CRM ERP CCaaS UCaaS Collaboration Enterprise Service AI ML Machine Learning LLMs Agentic AI Generative AI Robotics Analytics Automation Quantum Computing developer Metaverse VR Healthcare Supply Chain Leadership business Marketing finance Customer Service Content Management Chief Information Officer Chief Digital Officer Chief Technology Officer Chief Information Security Officer Chief Data Officer