Larry Dignan

Editor in Chief of Constellation Insights Constellation Research

About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology.  Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...

Read more

Accenture said it will acquire Udacity, an e-learning platform, and launch LearnVantage, a digital education and reskilling platform.

Terms of the deal weren't disclosed, but Accenture plans to make a big splash in enterprise reskilling.

Udacity’s 230 employees will join LearnVantage, which Accenture said is aimed at training clients. Udacity, founded in 2011, has 21 registered users in 195 countries and a large content library. Udacity competes with Coursera, which has ramped up enterprise training. Accenture said LearnVantage will also look to partner with the ed-tech ecosystem, including Coursera.

In a statement, Accenture said it will work with AWS, Google Cloud, Microsoft on generative AI certifications as well as e-learning players such as Coursera, Pluralsight, Workera and Skillsoft.

Accenture said it will invest $1 billion over three years in the LearnVantage effort. Accenture aims to offer personalized training in generative AI, data science, cloud and security as well as CXO lessons.

• Education gets schooled in generative AI
• Constellation ShortList™ Learning Marketplaces
• Coursera: Generative AI will lead to reskilling, upskilling boom
• Arizona State, OpenAI to collaborate on ChatGPT education use cases
• Why you need to connect your hiring to data, outcomes pronto

Julie Sweet, Accenture CEO, said reskilling and training will be critical. Sweet said the goal is to help clients "become ‘talent creators’—with people at the center of their reinvention using technology, data and AI—and a critical part of that is investing in industry-specific training and technology skills development."

LearnVantage is part of a larger effort to reskill Accenture's 700,000 employees. According to Accenture, it will roll out data and AI training to 250,000 technology professionals. Fundamentals of AI has rolled out to more than 600,000 employees. 

Larry Dignan

Editor in Chief of Constellation Insights Constellation Research

About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology.  Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...

Read more

Twilio is staying in the customer data platform business. The company said that it has completed its review of Segment, will improve operations and the combination of Segment benefits its core communications customers.

The company during its most recent earnings report said Segment, which is about 7% of revenue, wasn't meeting potential. As a result, Twilio was reviewing what to do with the business.

In a statement, Twilio said the board concluded that Segment would undergo "tangible near-term operational changes to accelerate Segment’s path to break-even non-GAAP income from operations by Q2 2025 and non-GAAP operating profitability thereafter."

Khozema Shipchandler, CEO of Twilio, said Segment strengthens its communication platforms. Shipchandler also said Twilio has authorized an additional $2 billion in share repurchases.

According to Twilio, about 8,000 customers of its communications platform leverage Segment for insights and to create platforms. Segment is seen as a cog in Twilio's AI roadmap.

What's next? Twilio said Thomas Wyatt will become president of Segment March 11. Segment will also restructure, focus on AI and automation and increase its innovation velocity. These efforts will be led by Wyatt, who was previously led product and strategy at People.ai.

As for the outlook, which was on hold pending the Segment review, Twilio said it is expecting 2024 organic revenue growth of 5% to 10% with non-GAAP income from operations of $550 million to $600 million.

In fiscal 2025, Twilio is projecting GAAP operating profitability by the fourth quarter with Segment at break-even in the second quarter.

Larry Dignan

Editor in Chief of Constellation Insights Constellation Research

About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology.  Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...

Read more

Box added a new integration with Microsoft Azure OpenAI Service in a move that will bring more large language models to Box AI, which is available to customers with Enterprise Plus plans.

Box AI is built to be platform-neutral so it can connect to multiple LLMs. Microsoft's Azure OpenAI Service will bring models to Box's Content Cloud and meet various compliance regulations for financial services, life sciences, public sector and insurance companies.

The general availability of Box AI is part of a broader strategy to use the Content Cloud to surface insights from unstructured data, which represents 80% of enterprise content. Box's plan is to leverage AI for more high-level use cases across a wide range of industries.

According to the company, generative AI will enable its Box Content Cloud to enhance security, storage and collaboration of business content and enable enterprises to generate insights and content from repositories.

The bet is that high value us cases will drive growth going forward.

• Box CEO Levie on generative AI, productivity and platform neutrality
• Analysis: Box's Acquisition of Crooze Stands To Accelerate Customer Innovation
• Constellation ShortListâ„¢ Enterprise File Sharing and Cloud Content Management

In beta, Box AI was used for wealth advisory, clinical research, product marketing and human resources.

Box Enterprise Plus customers will get Azure OpenAI Service included with plans. Individual users under those plans have access to 20 queries a month. At a company level, 2,000 queries are available.

Separately, Box reported fourth quarter earnings. Box's fourth quarter revenue was $262.9 million, up 2% from a year ago. Net income was 57 cents a share. Non-GAAP earnings in the fourth quarter were 42 cents a share. Analysts had expected Box to report fourth quarter earnings of 38 cents a share on revenue of $262.8 million. 

For fiscal 2024, Box reported earnings of 67 cents a share on revenue of $1.04 billion, up 5% from a year ago. As for the outlook, Box projected first quarter revenue of $261 million to $263 million, up about 4% from a year ago, with non-GAAP earnings of 35 cents a share to 36 cents a share.

For fiscal 2025, Box projected revenue between $1.08 billion and $1.085 billion, up 5% from fiscal 2024. Non-GAAP earnings will be between $1.53 a share to $1.57 a share.

On a conference call with analysts, CEO Aaron Levie said:

"Combined with Box's upcoming workflow automation improvements, including the expected launch of Forms and Doc Gen this year, Box will be able to power end-to-end critical business processes natively without customers having to do any custom development. And with the acquisition of Crooze, Box will extend into new use cases within our current customers as well as enabling us to rip and replace legacy ECM solutions."

Regarding Box AI, Levie said:

"We were looking at a lot of the usage patterns, seeing their use cases. The first set of use cases were sort of dominated by the core functionality we have today, which is summarizing documents quickly, extracting information from documents as a user is sort of reading that. So, some powerful kinds of end-user productivity use cases. But quickly we got feedback that probably the biggest areas of excitement, and these are things that we've been working on, is one the ability to ask multiple documents a question. So that's the multi-document kind of analysis functionality that we're working on, that will be embedded in our hubs product."

Constellation Research analyst Holger Mueller said Box hit a few key milestones in the quarter.

"Box has managed to break the $1 billion in revenue mark, which is always a historical milestone. The problem is the vendor made the goal at pedestrian speeds with 3% growth. The question is what is holding up Box, as the team around Aaron Levie is doing all the right things in terms of innovation. Let's see if the AI partnership with Microsoft and the acquisition of ECM low code / no code specialist Crooze will change it in the new fiscal year. Otherwise, Box will be a showcase of a software category that is innovating well but can't generate revenue. As Box is gaining customers, it must be all about innovation keeps up price levels but not growing revenue. That's good for customers, but not so good for Box."

Larry Dignan

Editor in Chief of Constellation Insights Constellation Research

About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology.  Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...

Read more

ServiceNow CFO Gina Mastantuono said the company's new generative AI SKUs, Pro Plus, is seeing rapid adoption, but it will take time to move the revenue needle. She also provided insights on how ServiceNow decides how to price its products.

Speaking at a Morgan Stanley investor conference, Mastantuono said ServiceNow's AI SKUs were launched on Sept. 30 and it is the fastest growing product the company has launched, but "it's still new and it's still small in a $10 billion base."

According to Mastantuono, the Pro Plus lineup should make a meaningful contribution to revenue in two to three years. She added that ServiceNow is using generative AI technologies inside the company across all business functions for about 20 use cases and seeing productivity gains.

• ServiceNow delivers strong Q4, ups 2024 outlook

• ServiceNow launches Now Assist across Now Platform, pricing bundles

• ServiceNow CEO McDermott talks business transformation, generative AI, processes

• ServiceNow's latest Now Assist generative AI features highlight its strategy

• ServiceNow acquires UltimateSuite, adds task mining
   

"We're seeing millions of dollars of savings just this year by virtue of what we're doing internally," she said.

Mastantuono said:

"Most of what we're doing with our Gen AI SKUs, we're mimicking what we did with our initial AI SKUs and our Pro adoption back in 2018. And so, if you think about that Pro adoption versus the Pro Plus adoption, we're not building in accelerated adoption curve just yet. But if you think about the excitement, the understanding of the productivity gains that Gen AI can drive in 2024 versus what we knew and understood and what customers knew and understood back in 2018, there's an argument for sure that, that adoption curve will be faster."

She added that the upgrade from standard ServiceNow subscriptions will be a heavy lift to Pro Plus. Enterprises on the Pro plans already have a relatively easy upgrade path.

As for the pricing plans for Pro Plus, it's a 60% uplift relative to Pro SKUs. But there's a method to ServiceNow's pricing.

Mastantuono said:

"We did it very similar to how we price our Pro SKUs. So, when we launched Pro back in 2018, we had a 50% price uplift on the list price. And we did the same exact math, right? We looked at the productivity expectations, the efficiencies by task by employee salaries. And what we do is we give 90% of that value to customers and we keep 10%, that value equation seems to work extremely well with the customers. And based on our initial kind of one quarter out, feels like we're in the right place."

In other words, if ServiceNow can automate a task for someone in IT operations management who makes $100,000 a year and boost productivity 30% there's $30,000 in value. ServiceNow wants 10% of that sum in price increases.

• Get ready for a parade of domain specific LLMs
• Generative AI features starting to launch, next comes potential sticker shock

Mastantuono said that's rough math, but there are other moving parts to ponder.

"I don't think that 30% of one person's job is going away immediately. It's specific tasks. How many of the tasks that they do are going to be automated? I think companies are going to really have to lean into reskilling some employees, because it does mean that they have more time. I don't think that full jobs go away, but we make them much more productive. And reskilling in areas there's more value add."

Over time, Mastantuono said pricing will have seats, which are actually increasing with Pro SKUs, and there's consumption via tokens for large language models. ServiceNow is building capabilities for usage-based models too. Consumption models can work better as enterprises consolidate platforms. She said:

"I think there are a few different ways of thinking about potential pricing in the future. What I think is important is that efficiency and productivity is not just about time. It's about better resolution. It's about getting those IT issues solved faster, which means the employee who's either with the customer or building code and driving innovation can be more productive to drive better top line, better innovation across the board."

On the back end, ServiceNow is focused on domain specific language models, which use less computing power.

Larry Dignan

Editor in Chief of Constellation Insights Constellation Research

About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology.  Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...

Read more

Snowflake CFO Michael Scarpelli said it's foolish to pigeonhole new CEO Sridhar Ramaswamy as a technology when the business he ran at Google, the ad unit, had more employees than Snowflake has today.

Speaking at a JMP Securities conference, Scarpelli talked about last week's big news that Snowflake CEO Frank Slootman was stepping down and Ramaswamy was taking over. The transition news rattled Snowflake's market cap as shares tanked.

A week later, Scarpelli put some perspective on the CEO change. First, Scarpelli noted he has been through many CEO transitions before, and succession is part of being a chief executive. He noted that the board of Snowflake had been talking about succession for two years.

"When we acquired Neeva last May, I remember telling Frank, you've got to spend some time with Sridhar, and Frank quickly did. And we brought Sridhar onto our executive staff, and he quickly was running all of our AI and ML initiatives and the more time Frank, and the Board spent with him, we realized that he could be a successor to Frank," said Scarpelli, who just signed on for another three years as CFO. "If Sridhar were not the CEO here, I would just tell you, he'd be working at a very large company, maybe running all their AI Initiatives."

Scarpelli added that Ramaswamy scaled Google's ad business to $120 billion in revenue and had roughly 12,000 employees. Snowflake is just north of 7,000 employees. "A lot of people mistake that he's a technologist and doesn't have operational chops. He ran a very large team at Google," said Scarpelli, who noted later that Ramaswamy has a PhD in databases.

According to Scarpelli, Ramaswamy is going to be a good recruiter. "With where the company is and where it's going, you need to have someone with strong technical background to be able to run the company to help be able to attract good people as well too. He's very, very good at attracting the top technical talent," said Scarpelli.

Ramaswamy recently hired 5 core engineers from Microsoft, said Scarpelli, who quipped that he cringes as a CFO over the pay packages. "Some of these people you're talking $3 million, $4 million a year. A lot of that's equity," said Scarpelli.

With the extra talent, Snowflake is planning to speed up its path to product GA. "We need to shorten that time frame," he said. "I personally think the selection of Sridhar as the CEO is the right choice for what the company really needs."

Larry Dignan

Editor in Chief of Constellation Insights Constellation Research

About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology.  Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...

Read more

The impacts from a Feb. 21 Change Healthcare ransomware attack continue to be felt as physicians still cannot file claims in many cases and practices are beginning to feel the squeeze.

Change Healthcare, which is owned by UnitedHealth Group, was hit with a ransomware attack that disconnected systems. Change Healthcare processes claims and other financial processes and was attacked by ALPHV/Blackcat.

As Constellation Research analyst Chirag Mehta noted, this ransomware attack had a real impact at the human level. Prescriptions couldn't get filled and physician practices are running out of cash. The American Medical Association issued a letter to the US Department of Health and Human Services to ensure practices can continue to function.

The AMA said in a letter to HHS Secretary Xavier Becerra:

"The cybersecurity incident that has impacted Change Healthcare and resulted in disruptions in claims processing, eligibility checks, and other impacts on day-to-day practice operations that has severely hampered physicians’ ability to care for patients..."

"The AMA is concerned about the undue financial hardships facing physician practices if this incident is not resolved quickly. It is especially challenging financially at the beginning of the year since many practices do not carry over reserves. We are particularly concerned about small, safety net, rural, and other less-resourced practices that often serve underserved patient communities. We urge HHS to utilize any available emergency funds and authorities to provide critical financial resources to physicians, ensuring they can continue to deliver essential health care services during these challenging times."

Christine Meyer, MD, a physician with her own practice based in Exton, PA, outlined the pain from the Change Healthcare attack on LinkedIn. She noted that Optum/UnitedHealth Group is providing a $4,000 bridge loan but without the ability to submit bills her practice, which has 84 employees, is running out of cash.

Meyer is also an example of the admin burden that the AMA is also complaining about.

UnitedHealth Group has said it is working with Mandiant and Palo Alto Systems to remediate the ransomware damage and "patient care is our top priority, and we have multiple workarounds to ensure people have access to the medications and the care they need." The company also added links for funding options.

Optum, UnitedHealthcare and UnitedHealth Group systems are unaffected, according to the company. However, it was noted in feedback to Meyer's post that those systems aren't helping much. And UnitedHealth Group's workarounds are a burden. The burden briefly can be found on this FAQ:

"Change Healthcare recommends that providers use the applicable payer’s portal to check claim status, as well as complete eligibility verifications and prior authorizations. If a portal is not available, the recommended approach is calling the appropriate payer’s provider service line."

In its letter, AMA blasted those workarounds.

Practices are being instructed to use direct data entry and portals to submit claims, which are quite labor-intensive compared with using their regular practice management systems that prepopulate data. These “workarounds” are adding extensive administrative burdens as well as substantial costs to physician practices for this extra manual work. In addition, practices are filing claims on paper when available, but many insurance companies no longer accept paper claims."

AMA also said HHS needs to prod Change Healthcare to be more transparent with communication. For now, UnitedHealth Group doesn't have a timeline on when Change Healthcare's systems will return.

"We will not take shortcuts or any additional risk when it comes to safeguarding our information, customer and consumer information, or the connectivity back to our systems. Our systems remain offline because of our diligence, not because of compromise. They will remain offline until we are certain we can turn them back on safely. We took action so you did not have to disconnect."

UnitedHealth completed the acquisition of Change Healthcare in 2022 after winning in court against the US Justice Department, which was against the deal due to antitrust concerns. UnitedHealth's Optum unit said the Change Healthcare can achieve "a simpler, more intelligent and adaptive health system for patients, payers and care providers."

More healthcare:

• Themes from the healthcare data, AI, disruption front lines
• 8 takeaways from Constellation Research's Healthcare Transformation Summit
• Medtronic aims to leverage medical device data for AI-enabled care
• Oracle's plan for Cerner: Cloud shift, generative code rewrite
• CVS Health’s Transformation Rides on Data, AI and Customer Experiences

Larry Dignan

Editor in Chief of Constellation Insights Constellation Research

About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology.  Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...

Read more

PC giants are all betting that AI-enabled PCs will get sales growing again and create new use cases, but it's unclear whether technology buyers will play along.

Apple's launch of the new 13- and 15-inch MacBook Air with the M3 processor dedicated a good bit of space to AI applications. Apple noted:

"Combined with the unified memory architecture of Apple silicon, MacBook Air can also run optimized AI models, including large language models (LLMs) and diffusion models for image generation locally with great performance. In addition to on-device performance, MacBook Air supports cloud-based solutions, enabling users to run powerful productivity and creative apps that tap into the power of AI, such as Microsoft Copilot for Microsoft 365, Canva, and Adobe Firefly."

Get used to similar statements from other PC vendors. In fact, the AI PC talk is already starting as companies hope for a second half sales rebound. Here's a sampling of recent comments from earnings calls from PC giants HP, Lenovo and Dell.

HP CEO Enrique Lores:

"We remain extremely excited about the opportunity that AI PCs will bring in terms of both the customer value that they will deliver in terms of security, in terms of latency, in terms of cost and also the impact it will have over time in the company."

Lenovo CEO Yuanqing Yang:

"AI PC could represent an inflection point for the PC industry. It could drive a new product cycle with premium pricing, which is key to delivering premium to market growth, strong ASP and sustainable profitability."

Dell Technologies Chief Operating Officer Jeff Clarke:

"We have hardware enabled AI PCs with an application base coming that should make it an exciting opportunity to own an AI PC."

All of these executives said their take on an improving second half 2024 for PC sales was based on third party validation, notably an IDC report. IDC expects AI PCs will represent 60% of all PC shipments worldwide and grow from 50 million units in 2024 to 167 million in 2027.

The argument for the ramp of AI PC shipments is that the installed base is already long in the tooth and AI can change the future of work. These AI PCs will be able to improve productivity, offer faster performance and lower inferencing costs as they leverage compressed language models.

All that's missing is the killer application, which Microsoft is working to provide in the latest versions of Windows. Lenovo said it is starting to ship AI PCs and the portfolio will expand through 2025. Yang said AI PCs will be a win for users as well as their employers due to hybrid AI that will leverage local compute.

"AI PC comes with a personal intelligent agent using natural language user interface, a compressed local large language model, heterogeneous computing with CPU, GPU, NPU, privacy and security protection and our AI application ecosystem. We believe this trend will stimulate another industry refresh cycle, as the users require devices designed for more creativity and productivity."

Lores said HP is working with partners to make sure applications can leverage the new PC capabilities. Lores noted that HP has seen more stability in small business sales and education and AI PCs will drive gradual sales improvements over the next two years.

For HP and Lenovo, AI PCs will have to appeal to consumers and enterprises to move the needle. HP's PC sales in the first quarter skewed commercial. 

Dell is mostly focused on sales to corporate customers. The bulk of Dell's PC sales go to enterprises and the focus pays off as total revenue per unit is nearly twice as large as the competition.

Clarke said AI PCs will drive sales, but Dell's guidance doesn’t include them. Simply put, Dell isn't seeing the upside yet. Clarke's bet is that the Windows upgrade cycle is likely to drive corporate sales.

"Do I expect the PC market to be bigger in calendar 24 than 23? Yes. Do I think the PC market is likely bigger in the second half of 24 than it is in the first half? Absolutely. We believe the opportunity in PCs is second half driven. That is primarily a result of an aging install base. It has never been bigger or older than it is today. We have a version of Windows that is retiring.”

What can derail the AI PC dream? Inflation and constrained budgets for both consumers and enterprises. Consumers are stretching out used cars well beyond historical life spans and PCs aren't going to be much different. Enterprises have steadier refresh cycles, but will want to see productivity gains from AI PCs, which will cost more than regular PCs. Enterprises will also be looking to leverage those pandemic-era PCs as long as possible because IT budget priorities rest elsewhere.

Bottom line: There's hope that PC sales will improve in the second half of 2024, but it's telling that major players aren't including any uplift in their financial guidance. In the end, that approach makes sense. Hope isn't strategy.

Larry Dignan

Editor in Chief of Constellation Insights Constellation Research

About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology.  Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...

Read more

This post first appeared in the Constellation Insight newsletter, which features bespoke content weekly and is brought to you by Hitachi Vantara.

The lowly enterprise data center may be making a comeback as companies look to leverage generative AI but keep corporate data secure.

This burgeoning trend may not come as a complete surprise. Constellation Research analyst Dion Hinchcliffe noted in a report last year that CIOs were scrutinizing public cloud workloads because they could get better total cost of ownership on-premises. What's emerged is a hybrid cloud approach where generative AI may tip the scales toward enterprise data centers.

On C3 AI's third quarter earnings call, CEO Tom Siebel talked generative AI workloads. He said: "The interesting trend we're seeing--and this is really counterintuitive--is we're going to see the return of in-house data centers where people have the GPUs inside. I'm hearing this in the marketplace and even my engineers have talked about it. Some are saying, 'it's going to be cost effective for us to build.'"

Siebel isn't the only one seeing generative AI making the data center cool again.

Nutanix CEO Rajiv Ramaswami said the company is "seeing more examples of people repatriating" workloads from the public cloud to data centers and private cloud arrangements. Enterprise concerns about generative AI are likely to equate to more data center buildouts than you'd think.

"I'm talking about enterprise workloads, because what has gone to the public cloud largely has been net new applications. So, for these workloads are still sitting in the enterprise environment, I think CIOs are being a lot more careful about how much of that do they take to the public cloud," said Ramaswami. "There's some repatriation happening from the public cloud back on-prem. But there's also a lot more scrutiny and forethought being applied to what should go into the public cloud."

Nutanix, which just reported earnings, is also seeing traction with its GPT-in-a-box offering, which accelerates generative AI across an enterprise while keeping data secure. Many of the industries interested in GPT-in-a-box cited by Ramaswami were noted by C3. Federal agencies, defense, pharmaceutical, and finance.

It's not surprising that defense, federal agencies and banking and finance are driving on-premises generative AI workloads. These companies are well-capitalized, have critical security needs and operate as technology companies in many ways.

Enterprises in these industries, along with nation-states, are the ones most likely to build their own data centers for AI workloads. Nvidia CEO Jensen Huang equates these next-gen data centers as AI factories.

Huang explained on Nvidia's most recent earnings call:

"You see us talking about industrial generative AI. Now our industries represent multi-billion-dollar businesses, auto, health, financial services. In total, our vertical industries are multi-billion-dollar businesses now. And of course, sovereign AI. The reason for sovereign AI has to do with the fact that the language, the knowledge, the history, the culture of each region are different, and they own their own data.

They would like to use their data, train it to create their own digital intelligence and provision it to harness that raw material themselves. It belongs to them, each one of the regions around the world. The data belongs to them. The data is most useful to their society. And so, they want to protect the data. They want to transform it themselves, value-added transformation, into AI and provision those services themselves."

The rest of us just want better TCO on AI workloads, but you get the idea. Dell Technologies appears to be capitalizing on AI-optimized servers along with SuperMicro. 

How AI workloads will reshape data center demand

Jeff Clarke, Vice Chairman and Chief Operating Officer at Dell Technologies, said orders for AI-optimized servers grew 40% sequentially and backlog doubled to $2.9 billion.

"83% of all data is on-prem. More data will be created outside of the data center going forward that inside the data center today, that's going to happen at the edge of the network, a smart factory and oil derrick. We believe AI will ultimately get deployed next to where the data is created driven by latency.

We sold to education customers, manufacturing customers, and governments. We've sold the financial services, business, engineering and consumer services. Companies are seeing vast deployments, proving out the technology. Customers quickly find that they want to run AI on-prem because they want to control their data. They want to secure their data. It's their IP and they want to run domain specific and process specific models to get the outcomes they're looking for."

Power problems

What could derail the march to in-house data centers? Today, it's procuring GPUs, but that'll work itself out. Power will be the big issue going forward. Siebel explained:

"The constraint today is you have to call up Jensen and beg for GPUs. Let's say we can figure out how to do that. Maybe we know somebody who knows Jensen. Okay, the hard part is we can't get power. You cannot get the power to build the data center in Silicon Valley. Power companies will not give you power for a data center. So, I think this GPU constraint is ephemeral and soon it's going to be power."

Pure Storage Charles Giancarlo cited power as a limiting factor for AI adoption on the company's fourth quarter earnings call. "The energy demands of AI are outstripping the availability of power in many data center environments," he said.

The power issue was also noted at Constellation Research's Ambient Experience conference this week. Rob Tarkoff, Executive Vice President & GM, CX Applications at Oracle, said sustainable and cost-effective power is of generative AI's big challenges.

"Where are we going to get the power cost effectively and environmentally sustainable power to be able to run all the models we want to run? How are we going to keep the cost at a level that companies can realistically invest in?" asked Tarkoff, who added that scaling data and compute isn't cheap. "All of this sounds really cool in production, but it's an expensive and energy consuming value proposition."

Constellation Research's take

This thread about enterprises potentially building out data centers for generative AI spurred a bit of debate internally. Here's what a few analysts had to say.

• Doug Henschen: "Data warehouse/lakehouse workloads are selectively remaining on or moving back on premises. Warehouse and lakehouse workloads are often data- and compute-intensive, and in many cases, organizations that have moved these workloads to the cloud have had sticker-shock experiences. Among those firms that have maintained on-premises systems and data center capacity, we've seen some instances where they've either delayed migration of data warehouse workloads to the cloud or even repatriated workloads to on-premises platforms. When workloads are stable and predictable and there's no need for cloud elasticity, it can be more cost effective to run these workloads on premises."
• Dion Hinchcliffe: "Here's the problem with public cloud and today's increasingly always-on, high performance computing workloads (basically, a lot of ML + AI, both training and ops): 1) The hyperscalers' profit margins, 2) R&D costs for I/PaaS, and 3) the massive build-out Capex to support high growth. All three numbers are high right now. Public cloud customers cover all three expenses as major cost components in their monthly bills. Many CIOs have now realized they can bring in a good percent of workloads back in, sweat their private DC assets to the last penny, and largely avoid paying these markups (that at the end of the day aren't really their problem.) So, the big question: Is the pendulum swinging back to private data centers? Not hardly, but as my CIO interviews and research on the topic last year showed, there is definitely an active rebalancing of workloads to where they run best. That means a good bit of AI, especially training, will go back in-house or to specialty partners that are highly optimized for this, that will also keep the private training data away from the hyperscalers."
• Chirag Mehta: "The reality doesn't quite add up. Even with AI, running GPU on-prem is Capex intensive and requires highly skilled people. Other than a few companies this is not a practical thing to do."
• Andy Thurai: "The repatriation/migration is happening on a selected workload basis. For AI workloads, especially the training, it is a bit complicated and requires a ton of MLOps and scaling tools that the private data centers don't have. The GPU shortage and scale needed for model training are also constraints. While it is easy to move ML workloads, AI workloads are still a challenge from what I hear."

Relevant Shortlists:

• Next-Generation Computing Platforms
• Artificial Intelligence and Machine Learning Best Of Breed Platforms
• Artificial Intelligence & Machine Learning Cloud Platforms
• Cloud-Based Data Science and Machine Learning Platforms
• Hybrid-Cloud and MultiCloud NoSQL Databases
• Hybrid-Cloud and Multicloud Analytical Data Platforms
• Infinite Connectivity Vendors