Why generative AI workloads will be distributed locally

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

This post first appeared in the Constellation Insight newsletter, which features bespoke content weekly.

Generative AI workloads have been dominated by Nvidia, a massive cloud buildout and compute that comes at a premium. I'm willing to bet that in a year, we'll be talking about distributed compute for model training and more workloads on edge devices ranging from servers to PCs to even smartphones.

On earnings conference calls, generative AI is still a common theme, but there's a subtle shift focusing on commoditization of the compute behind large language model (LLM) training and a hybrid approach that leverages devices that are being built with processors generative AI capable.

"There's a market shift towards local inferencing. It's a nod to both the necessity of data privacy and an answer to cloud-based inference cost," said Intel CEO Pat Gelsinger on the company's third quarter earnings conference call.

Here's a quick tour of what's bubbling up for local compute powered generative AI.

• Samsung VP of Mobile Experience Daniel Araujo, speaking on Samsung's earnings call, said "smartphones are poised to be some of the most important access points for AI." He added that "we want to establish a new standard for experiences that mobile devices can provide through hybrid AI, which encompasses both on device and server-based AI solutions."
• Google's Pixel 8 Pro was touted as being capable of running the company's LLM on device.
• Akamai is building an infrastructure-as-a-service network that revolves around edge computing nodes.
• Apple's latest MacBook Pro launch, which includes new M3 processors, touted faster model training as a use case. PC makers are already prepping PCs with higher specs (and prices) to manage local generative AI workloads.
• Amazon's Echo device launch was more about LLMs than devices, but the hardware can handle processing locally in many cases.
• Simply put, you won't be hurting for local compute resources for generative AI use cases.

Amazon CEO Andy Jassy said:

"In these early days of generative AI, companies are still learning which models they want to use, which models they use for what purposes and which model sizes they should use to get the latency and cost characteristics they desire. In our opinion, the only certainty is that there will continue to be a high rate of change."

Indeed, the change coming for generative AI is going to revolve around local compute that's distributed.

Here's why I think we may get to distributed model training sooner than the industry currently thinks:

• Enterprises are building out generative AI infrastructure that often revolves around Nvidia, who needs competition but right now has an open field and the margins to prove it.
• The generative AI price tag is tolerated today because the low-hanging productivity gains are still being harvested. If you can improve software development productivity by 50% who is going to sweat the compute costs? Pick your use case and the returns are there at the moment.
• But those easy returns are likely to disappear in the next 12 months. There will be more returns on investment, but compute costs will begin to matter.
• Companies will also gravitate to smaller models designed for specific use cases. These models, by the way, will need less compute.
• Good enough processors and accelerators will be used to train large language models (LLMs)--especially for cases where a fast turnaround isn't required. Expect AWS' Inferentia and Trainium to garner workloads as well as AMD GPUs. Intel, which is looking to cover the spectrum of AI use cases, can even benefit.
• The good enough model training approach is likely to extend to leveraging edge devices for compute. For privacy and lower costs, smartphones, PCs and other edge devices are going to be equipped and ready to leverage local compute.

Ultimately, I wouldn't be surprised if we get to a peer-to-peer or Hadoop/MapReduce-ish approach to generative AI compute.

Disrupting yourself can hurt even if it makes sense, just ask Paycom

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

A common mantra among technology companies is that it's better to disrupt yourself than let a competitor do it. On the whiteboard that mantra makes long-term sense. In reality, a business model transition can crush your stock.

Just ask Paycom, a human capital management software provider that created Beti, a service that dramatically reduces payroll errors and drives value for customers. The concept is shockingly simple: Give employees access to payroll to fix errors before the checks are cut.

Paycom shares were hammered this week because Beti is doing away with unscheduled payroll runs and error fixes. What's wrong with generating returns for customers? A more perfect payroll means fewer billable items for Paycom, which charged for corrections and unscheduled payrolls.

Here’s a look at the carnage (hint: lower right on chart is where “PAYC” wound up.)

To its credit, Paycom is playing the long game with customers. Sure, the business model transition hurts a bit, but other companies have gone through it. Think about how software companies transitioned from licenses to subscriptions after Adobe led the way.

Beti launched in 2021 and now two-thirds of Paycom's customer base is using it. Paycom is rolling out Beti to Mexico to complement US and Canada.

Paycom CEO Chad Richison explained the Beti effect.

"For most employees, the value of the perfect payroll is oftentimes immeasurable. If their check is perfect, they don't need to borrow money from a friend or family member to get through the weekend or make a bill payment. How do you measure the value of that?

We're getting better and better at helping employers measure the full value available to them when payrolls are perfect. A portion of that value is easy to calculate because it's the value they receive by the elimination of after-the-fact payroll errors that require correction payroll runs, manual checks, voided checks, direct deposit reversals, additional wires, tax adjustments, W2Cs, et cetera, et cetera.

Perfect payrolls eliminate these common after-the-fact payroll corrections that would otherwise be billable. So the more employees do their own payroll, the greater the savings delivered to the client from Paycom future billings, which results in lower related revenue recognized by Paycom."

Paycom isn't hurting. The company delivered third-quarter revenue of $406 million, up 22% from a year ago, with net income of $75 million, or $1.30 a share. Non-GAAP earnings were $1.77 a share. Third-quarter sales missed guidance because Beti was working too well. In addition, the outlook was light as Paycom projected revenue between $420 million to $425 million in the fourth quarter.

In other words, the more customers use Beti, the more value they receive at the expense of Paycom revenue. Paycom is expecting revenue growth in 2024 between 10% and 12%. A customer that had to ran 19 payrolls a quarter due to errors can now run 13.

The big question: Should Paycom have disrupted its own services business? It's a conundrum faced by many companies and the decision isn't easy. Here are a few thoughts about answering that question.

Paycom's long game. Richison said that he's focused on "the client value and the differential between what they're paying and what they're actually achieving." Overall, I don't think you can go wrong providing value to customers.

If Paycom didn't launch Beti someone else would have. I'd argue that payroll errors aren't a feature but a bug. Some startup would have disrupted the error-ridden payroll process with something similar to Beti anyway. Paycom would have lost the services revenue and customers too over time. Shortly after Beti launched, Richison said on Paycom's fourth quarter 2021 conference call:

"For years, I have been predicting the end of the old model whereby HR and payroll personnel’s routine of inputting data for employees is replaced by a self-service model that provides employees direct access to the database.

The old model is dying and that is good for both the business and the employee."

Here's a fun fact: Paycom's fourth quarter revenue in 2021 was $285 million, well below the projected $420 million or so two years later.

Don't forget word of mouth. Paycom received a good amount of attention as shares fell this week. Enterprise buyers who dig a bit will quickly find a "man bites dog" headline as Paycom sacrificed revenue for customer value. Something tells me this Paycom stock tale may wind up being good marketing.

Paycom is adding new customers. Should Beti ramp even further Paycom will have more enterprise customers on its platform. These customers tend to buy other services later if the vendor delivers.

"New business sales as well as cross-selling within our base has always been a mitigating factor to any type of transition shift, we make like this," said Richison. "New business sales remained strong. In fact, most of the calls we get in are about Beti. We've got our first enterprise rep and they're only targeting deals that have greater than 25,000 employees. And they've got plenty of leads."

Could Paycom have managed this transition better? Perhaps. But Beti appears to be a hit. In other words, take the win with a bit of pain. Disrupt yourself or be disrupted.

Google Cloud CEO Thomas Kurian on DisrupTV: Generative AI will revamp businesses, industries

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Google Cloud CEO Thomas Kurian said companies are starting to reimagine their businesses for artificial intelligence one process at a time. The big question is what industries hit scale first. 

Speaking on DisrupTV, Kurian said "in virtually every field, we're seeing people take the persona that they had, and then creating a digital version of that persona using AI and this is happening incredibly quickly."

"The common theme is taking the skills of the model that represents what human beings can do, but now creating a digital persona and assisting a function," he said.

Kurian cited use cases across industries including healthcare, insurance, cosmetics and media to name a few. Many of these industries started out with AI use cases that revolved around classification and categorization. The next phase for AI was prediction using models based on different parameters. Generation is the new phase.

"Generation is the next skill as you train a model with a set of inputs and it can generate output," said Kurian. "You can now put together and automate a complete workflow for the whole company. And this is happening in many places. We see it happening at scale at many around the world."

Related: Cloud customers still optimizing spend and should forever | Google Cloud delivered third quarter revenue of $8.4 billion, up 22% from a year ago, with an operating profit of $266 million

• Google Cloud Next everything announced: Infusing generative AI everywhere
• At Google I/O 2023, Google Cloud launches Duet AI, Vertex AI enhancements | Generative AI features starting to launch, next comes potential sticker shock

• Enterprises seeing savings, productivity gains from generative AI

Constellation Research CEO Ray Wang asked Kurian whether incumbent companies or startups had advantages in generative AI. Kurian said:

"We're very early in the market. To do AI well, we need high quality data sets to fine tune the state-of-the-art models. For high quality datasets, you need state of the art models, and you obviously need the infrastructure to serve the models. But then just as importantly, you need to integrate these into the application surface.

The companies that succeed will have capabilities for state-of-the-art models that are driven by high quality datasets that they own, and the ability to activate these models. within account within the context of an application surface.

There will obviously be disruptors and they will take a function of process that was done in one way and fundamentally change it. You will see disruptions in different industries, where the fundamental business model itself may change.

"The winners are always those that solve a fundamental problem materially."

Going forward, generative AI will be democratized and simplified. AI will be an enabler for new technologies and access. In other words, we'll all become programmers to some degree. Kurian said:

"The code generation model will generate the skeleton code and create the environment. You can say 'hey, I want to support a million and a half users with less than 10 seconds latency of 10 milliseconds latency and I need to guarantee four nines of availability' and the system will do it for you. There's no reason that that cannot be done with where models are. And by doing that, we again change how widely accessible these things are. We are also very encouraged by the fact that we can make these models work not just for people in affluent countries, but also people in emerging markets."

Kurian also touched on other topics.

Security. Kurian said AI is already detecting threats, but also prioritizing them. "We built a model that can look at the threats that are emerging in your infrastructure, which one needs priority and how does that threat affect you," he said. "Models don't have that emotional bias. So, they can look at many more patterns to detect what's going on what is the attack the attack surface. It can remediate it and automate the creation of the rulebook to resolve the problem. We're applying AI to the whole spectrum."

AI-driven cybersecurity attacks. "We also see AI being used by bad actors to create new types of threats and we're also building our platform to thwart new types of threats," he said.

Running Google Cloud. Kurian said Google had great products but needed to build out enterprise capabilities as an independent unit.

"We're the fifth largest software company in the world, which is a long, you know, huge credit to the team. But when we looked at it, we felt we needed to do four things really well. You need to take great technology but convert it into solutions that people can use. Just having technology that's not accessible is a challenge.

Second, we need to build a great go to market function. What kind of structure do you have? How do you focus? We started with a set number of industries and countries.

It's an ecosystem game. It's not your company against another company. It's your ecosystem. So, we made decisions very early to partners. We started with 100 partners today there are 100,000 partners. And part of that is we wanted to bring that ecosystem so that people realize it's a bigger pie that they are creating, not slicing off the same pie. That's the third one.

And then you have to do something really well. It's just like sports. In order to play really well, you have to do the grunt work of training. We have to do a lot of the things below the surface of the water such as the systems, the legal contracting, and the frameworks to be more efficient as an organization. Those were all put in place so that you can go faster. Unless you have a strong core, you can't really play well. We've been super fortunate that we've been blessed with such a great team of people that have done so much of the work to get us where we are today."

Apple Q4 better than expected, but it has a Mac sales sink

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Apple's fourth quarter results were better-than-expected but revenue was down for the fourth consecutive quarter. Mac sales in the quarter were weaker than expected, but may get a lift with new MacBook Pro models on tap. 

The company reported fourth quarter earnings of $1.46 a share on revenue of $89.5 billion, down 1% from a year ago.

Wall Street was expecting Apple to report fourth quarter earnings of $1.39 a share on revenue of $89.28 billion.

Here's a breakdown of Apple fourth quarter results by product line and their targets via LSEG.

• iPhone revenue: $43.8 billion; Estimate: $43.81 billion
• Mac revenue: $7.61 billion; Estimate: $8.63 billion
• iPad revenue: $6.44 billion; Estimate: $6.07 billion
• Wearables, Home and Accessories revenue: $9.33 billion; Estimate: $9.43 billion
• Services revenue: $22.31 billion; Estimate: $21.35 billion

In a statement, CEO Tim Cook said the company has a strong lineup for the holiday season including new Macs. Overall, Apple's game plan is to monetize its user base with services and subscriptions.

For fiscal 2023, Apple reported net income of $96.99 billion, or $6.13 a share, on revenue of $298.08 billion.

Other items to know:

• China revenue in the fourth quarter was $15.08 billion, down from $15.47 billion a year ago.
• Americas revenue was $40.11 billion, up from $39.8 billion a year ago.
• Europe and Japan revenue was down slightly from a year ago.

Constellation Resarch CEO Ray Wang said:

"Despite continued revenue decline, Apple is a digital giant and flight to safety stock in good times and bad. The macro conditions are elongating the iPhone replacement cycle. China is the challenge as iPhone 15 sales slow and Huawei has revamped its offerings. The elongation of iPhone replacement cycles is the headwind. 

The new Mac Lineup provides cost savings and higher margins and could revitalize sales. The vertically integrated strategy is working For Mac, iPhone, Watch, and ultimately Vision Pro. Services is the bright spot. All eyes on the holiday forecast for the December quarter." 
 

More:

• Apple's Double Tap gesture on Apple Watch sets up Vision Pro UI handoff

Palantir's commercial business scales with help of AI boot camps

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Palantir's commercial annual revenue run rate is closing in on the company's government business as Palantir Artificial Intelligence Platform (AIP) gains enterprise traction.

The upcoming parity point between commercial revenue and Palantir's core government business is worth watching. In the third quarter, Palantir commercial revenue grew 23% to $251 million with US revenue growing 33% to $116 million. Government revenue grew 12% to $308 million in the third quarter.

Overall, Palantir reported third quarter earnings of $72 million, or 3 cents a share, on revenue of $558 billion, up 17% from a year ago. Adjusted earnings were 7 cents a share. The results handily topped expectations.

Palantir has been trying to grow its enterprise revenue base for years and now has 181 commercial customers, up 37% from a year ago. As for the outlook, Palantir is projecting 2023 revenue between $2.216 billion and $2.22 billion. J.D. Power, Palantir team up on generative AI apps for auto value chain

On a conference call with analysts, Ryan Taylor, Palantir's Chief Revenue Officer, said the company closed 80 deals including 12 worth $10 million or more across 11 industries.

Taylor said:

"We're also seeing the acceleration of larger deals and shorter times to conversion and expansion, including a multiyear deal in excess of $40 million with one of the largest home construction companies in the U.S. to start up pilot and converted all within Q3. This growth is in part due to AIP's continued transformation of the way we partner with and deliver value for our customers, and we expect AIP's impact to continue to intensify."

Palantir has seen traction with "AIP boot camps," which deliver real workflows on customer data in 5 days or less. That approach is driving contract expansions. "We're on track to conduct boot camps for more than 140 organizations by the end of November, nearly half of those are taking place this month alone, which is more than the number of U.S. commercial pilots we conducted all of last year," said Taylor. "We almost tripled the number of AIP users last quarter and nearly 300 distinct organizations have used AIP since our launch just five months ago. We will continue investing meaningfully in boot camps as our go-to-market strategy for AIP."

Of course, Taylor noted that Palantir's government business remains strong and should accelerate going forward.

Shyam Sankar, Palantir's CTO, said AIP boot camps are driving the point home that you can't use LLMs without tools to provide algorithmic reasoning. Sankar recently gave a talk on the topic.

According to Palantir CEO Alex Karp the commercial success isn't surprising if you zoom out and consider the company's military experience. Karp said:

"AIP and U.S. commercial is not only is disrupting the market, it's setting a standard that I don't believe any other software company will be able to reach partly because they misunderstood the value of LLMs and their relative importance, and lack of importance, partly because they don't have decades of experience on the frontline as we do in the military with managing the core ways in which you make these things precise, the way in which you provide governance."

Karp added that AIP enables enterprises to manage LLMs and "basically pen test your enterprise." "My view of what we should do is build products that are so good that the competition stops competing, whether that's in commercial or on the battlefield and that's what we're doing. And that's what we're seeing in AIP," said Karp.

SAP Build Code adds generative AI tools, bridges to process data, Java, JavaScript

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

SAP launched SAP Build Code development tools that leverage generative AI and court Java and JavaScript developers as well as connect to data stores across the enterprise.

At SAP's TechEd 2023 conference, SAP outlined a series of tools that aim to enable developers to build applications and connect them to the SAP Business Technology Platform. These pro code tools, which build on last year's low code/no code rollout, feature the following:

• Generative AI productivity tools to connect heterogenous systems.
• Vector embedding capability in SAP HANA Cloud to handle unstructured data and provide context to models.
• SAP AI Foundation, a developer repository to build AI-driven business apps.
• Development tools that are optimized for Java and JavaScript and interoperate with ABAP Cloud development.
• Code generation via SAP's generative AI assistant Joule.
• Unification of design and run-time services to build interfaces using Java and Node.js.
• The ability to work in Visual Studio cloud hosted environments.
• Simplified API management so developers can connect to third party systems, link processes and keep business context.

SAP Build Code will be generally available in the first quarter.

The ERP giant's other big play was to connect its SAP Build Code developer base to its process optimization tools including SAP Build Process Automation, SAP Signavio, which will get a large language model specific to business processes or large process model, and SAP Integration Suite. By connecting development and process optimization, SAP is looking to keep data on its platform while extending into other third-party systems. To that end, SAP also said that models built in DataRobot can be hosted inside SAP AI Core and used in SAP Build applications and extensions.

More:

• SAP's 2023 outlook steady, S/4HANA revenue nears €4 billion annual run rate
• SAP launches Joule generative AI copilot, goes to 2-year S/4HANA Cloud major release cycle
• SAP buys LeanIX, aims to couple it with Signavio, system transformation
• SAP's AI ambitions depend on migrating customers to cloud, S/4HANA
• SAP user group DSAG rips S/4HANA innovation plans, maintenance increases
• SAP aims to infuse generative AI throughout its applications: Here's everything from SAP Sapphire 2023

In addition, SAP Datasphere will get updates to import business context from SAP S/4HANA Cloud, SAP S/4HANA, SAP Business Warehouse and SAP BW/4HANA for data integration. Datasphere will also have simplified data sharing as well as prebuilt business content with a release planned for the fourth quarter.

Constellation Research's take

Holger Mueller, Constellation Research analyst, handicapped SAP's announcements. He said:

"SAP is doing right by its developer community with the launch of Build Code – which took a temporary back seat with all the attention the low code/no code offering had in the last 12 months. It is of strategic importance for both SAP customers and SAP alike that existing customers can move their ECC based customizations and extensions to the cloud future, where they need to upgrade to S/4HANA by 2027.

Given that most custom SAP code is in ABAP though, the SAP developer community wonders why the focus was Java/JavaScript. Certainly, a more en vogue platform today, but it is easier to move alike code / programming language – than re-writing. It would have been a big win if SAP could have delivered all pro-code tooling in 2023.

On the HANA side it is important that HANA gets the now 'standard' AI vector feature – what it will mean for in memory usage will have to be seen – but a key step to keep HANA and HANA data (where all of S/4 HANA data is) relevant in the AI age. What needs to be clarified is the basic working of SAP’s AI platform – that would have been a prime-time topic for TechEd. We may now have to wait till Sapphire."

ServiceNow CEO McDermott talks business transformation, generative AI, processes

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

ServiceNow CEO Bill McDermott said workflow and process optimization are blending as enterprises shift away from siloed data and infrastructure to focus on transforming operations with new technologies such as generative AI.

McDermott, speaking during ServiceNow’s industry analyst day, closed out sessions with ServiceNow executives who outlined generative AI plans, strategy, target industries and returns on investment.

He said enterprises have created "a hornet's nest of complexity." "The number one reason that digital transformation hasn't delivered positive ROI to businesses is integration. And that's why 85% of the businesses that have gone on transformation didn't get a positive ROI, because nothing integrates," said McDermott.

ServiceNow reports strong Q3, ups outlook for 2023 

• ServiceNow launches Now Assist across Now Platform, pricing bundles
• ServiceNow outlines generative AI roadmap, bring your own LLMs
• ServiceNow expands finance, supply chain process, workflow automation

What's changed is an inflection point where enterprises have to cut costs, leverage new technologies and be agile enough to navigate a bevy of unknowns.  McDermott said IT budgets are likely to rise and "generative AI is the main cause of the increase." "I've tested this theory with CEOs that they're going to lean even more heavily on SG&A functions. They're not going to do what they've always done, which is upgrading older technology and 1,000 points of dim light point solutions. They're going to start stopping things in favor of investing in platforms that matter. There's about a half a dozen initiatives that matter and Gen AI is number one on the list," he said.

McDermott added that boosting efficiency requires a focus on processes and a move away from siloed operations.

"We're moving away from the siloed operation. Operations today have to be done in teams. It has to be done across silos. It has to be done in business processes, whether that sorts of pay cash and so forth. So, what you're seeing now is a complete renaissance in where the work needs to be done," he said. "We have to put a bit of an action layer above this mess. Straighten out the mess, go into the process, go into the end integration and then provide a UX that hides the complexity."

According to McDermott, C-level executives are pivoting to business transformation management "where you rethink the whole value chain."

Other key points from McDermott include:

Speed and time to ROI is critical. McDermott said CEOs have to rev innovation cycles even as they are navigating multiple concerns--geopolitical issues as well as economic concerns--and have to become more productive. "Leaders can't be focused on time consuming projects with high risks," said McDermott. "They have to get the job done. Customers need positive ROI. They need costs out and they need productivity up and keep an eye on growth."

He added that CEOs "really need to increase the clock speed of their companies."

Work is still complicated. "What I try to explain to CEOs in the context of what we do is companies still waste a lot of time," said McDermott. "The average worker today swivels between 13 disparate applications in the enterprise. And that's forfeiting more than 20% of their productivity. Some cases are a lot higher than that. That seems pretty ridiculous."

The intersection of data and AI. CEOs are starting to realize that to leverage generative AI they need to focus on their data. "The enlightened CEOs are getting the picture that they really need to focus on data," said McDermott, adding that AI is the next step. "If I can improve the productivity of my workers or my engineers by 30% to 40% combined with digital transformation, we're now moving into a whole different category--business transformation."

Generative AI's impact on employment. "I absolutely believe that this might be the great unlock in human productivity and enterprise productivity. And it actually might be a force multiplier for increasing the world's GDP. Because if I'm right on digital transformation, I would also be right that we're going to need more people in the workforce. They're going to do different things. They're going to be retrained. A lot of people forget we're in the tightest labor market that we've been in for two decades," said McDermott. "It's the biggest thing that we've come across in the enterprise, probably since the internet."

Snowflake lays out AI managed service vision with Cortex

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Snowflake consolidated its AI and machine learning efforts with Snowflake Cortex, a managed service that provides access to large language models (LLMs), AI models and vector search in one place and includes Snowflake Copilot, Document AI and Universal Search.

Cortex, announced at Snowflake's Snowday developer conference, is in private preview. Snowflake Cortex will include LLMs including Meta's Llama 2 model as well as task-specific models. For Snowflake, the move to Cortex highlights a vision that revolves around providing AI building blocks without expertise or GPU infrastructure management.

To go with Cortex, Snowflake expanded its Snowflake Horizon built-in governance platform. The company added data quality monitoring, data lineage UI, differential privacy policies, enhanced classification of data and an upcoming Trust Center that will streamline cross-cloud security and compliance modeling.

Snowflake also plans to provide a cost management interface so admins can manage costs in Snowflake.

Related: Snowflake Q2 better than expected | Snowflake launches Snowpark Container Services, linchpin to generative AI strategy | Snowflake, Nvidia team up to enable custom enterprise generative AI apps

Although the announcements are notable for Snowflake customers, it's worth noting that many of these services are in private preview.

Doug Henschen, Constellation Research analyst, provided his take on Snowflake's barrage of announcements and private previews. He said:

"Snowflake is making lots of private preview announcements. Snowflake's general pattern is to make more "heads up it’s coming" announcements during Snowday and more "it's now public preview or GA" at Snowflake Summit in June, but I don't tend to get too get excited about private preview announcements that are six to 12 months from being available to customers.

Horizon and Cortex are, indeed, two of the higher-profile announcements. Horizon is really consolidating everything Snowflake already has and has planned in the way of metadata management, cataloging and governance. It’s partly a response to competitors, like Databricks and Google, that have put cataloging at the forefront with their Unity Catalog and Dataplex offerings, respectively. Snowflake already had a catalog and multiple governance capabilities, but they’re now pulling everything together under the Horizon umbrella and providing a clearer and more comprehensive vision of what’s ahead. Cortex is entirely in private preview but, here too, we're seeing the vision for how GenAI and more conventional AI/ML will be made available on the Snowflake platform."

Research: 

• Google Sets BigQuery Apart With GenAI, Open Choices, and Cross-Cloud Querying
• Constellation ShortList™ Self-Service Data Science & Machine Learning
• How Data Catalogs Will Benefit From and Accelerate Generative AI
• Analytics and Business Intelligence Trends in Cloud, Embedding, and Generative AI
• Constellation ShortList™ Automated Cloud Data Warehouse Services

Instead, Henschen said Snowflake customers should focus on things that are closer to launch. Snowflake announced the following additions that are closer to prime time.

• Snowpark ML Modeling API, which will be generally available soon, will give developers and data scientists tools to simplify model training faster.
• Snowpark Model Registry, public preview soon, will have expanded support for models from Hugging Face.
• Snowflake Native App Framework, generally available on AWS soon and in public preview soon on Microsoft Azure.
• Snowflake Container Services, public preview in select AWS regions.
• Support for Iceberg Tables, which will be in public preview soon.

Henschen said Snowflake customers need to focus on what's coming within weeks more than months. He added:

"The standouts for me are the things that are entering public preview or that will soon be generally available. Support for Iceberg Tables, for example, is now in public preview, and it promises interoperability of data managed in Snowflake for open access and usability of data for other workloads and applications. It’s a win for customers. Another example is the Snowpark ML Modeling API, which will enable the many customers using Snowpark to more easily tap into Snowflake data for model preprocessing and training.  Similarly, the Snowpark Model Registry, which is going into public preview, will improve support for ML OPS within Snowflake."

The biggest takeaway from Snowflake's Snowpark conference is that the company is consolidating its efforts to provide a unified platform. Snowflake's core themes at Snowpark were to simplify the data foundation, accelerate AI adoption and scale applications.

"It's a good set of announcements, but with so many elements being in private preview, my sense is that both Databricks and Google are ahead of them on having ML/AI and GenAI capabilities internally within the platform and in supporting customers who want to develop their own ML/AI and GenAI capabilities. I’d like to see either a shorter lag time between private preview and public preview stages or a shift by Snowflake towards only announcing capabilities that are nearly ready for public preview. There are capabilities, like Unistore, for example, that were announced at Snowflake Summit 2022 that still aren’t available. In my opinion it’s not a good idea to announce things unless they’ll definitely be available within six to eight months."

AMD sees strong data center demand for Q4

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

AMD's third quarter financial results were better than expected, but the outlook for the fourth quarter was lighter than expected. AMD CEO Dr. Lisa Su said "our data center business is on a significant growth trajectory" due to its EPYC server processors and Instinct MI300 accelerator shipments.

The company reported third quarter earnings of $299 million, or 18 cents a share, on revenue of $5.8 billion, up 4% from a year ago. Non-GAAP earnings for the quarter were 70 cents a share.

Wall Street was expecting AMD to report third quarter earnings of 68 cents a share on revenue of $5.69 billion.

As for the outlook, AMD projected fourth quarter revenue of $6.1 billion, give or take $300 million. That outlook would equate to 9% revenue growth from the fourth quarter a year ago. Non-GAAP gross margin guidance of 51.5% was in line with expectations. That outlook is based on strong growth in AMD's data center unit and momentum in the client business with lower sales in gaming and embedded markets.

Like Intel, AMD saw stabilization in the PC business. Su said AMD saw record sales of its Ryzen 7000 series PC processors as well as server processors.

• AMD sees AI interest, AMD Instinct MI300A and MI300X GPUs on track
• AMD makes its case for generative AI workloads vs. Nvidia
• Oracle Cloud Infrastructure launches AMD 4th gen EPYC instances
• Why enterprises will want Nvidia competition soon
• Oracle’s Data Platform Gets an EPYC™ Boost
• Intel's Q3 better than expected as PC business stabilizes

Key items from AMD's analyst conference call:

• "Based on the rapid progress we are making with our AI road map execution and purchase commitments from cloud customers, we now expect Data Center GPU revenue to be approximately $400 million in the fourth quarter and exceed $2 billion in 2024 as revenue ramps throughout the year. This growth would make MI300 the fastest product to ramp to $1 billion in sales in AMD history. I look forward to sharing more details on our progress at our December AI event," said Su. 
• "We've been planning the supply chain for the last year and we're always planning for success. So certainly, for the current forecast of greater than $2 billion, we have adequate supply. But we have also planned for a supply chain forecast that could be significantly higher than that, and we would continue to work with customers to build that out," said Su. 
• "Looking at the next couple of quarters, we expect strong growth in our Data Center business, driven by both EPYC and Instinct processors. This growth will be partially offset by softening demand in our Embedded business and lower semi-custom revenue, given where we are in the console cycle," said Su. "As the PC market returns to seasonal patterns, we believe we are well positioned to gain profitable share in the premium and commercial portions of the market based on the strength of our product offerings."
• Su said the fourth quarter revenue for its Instinct processors will initially be HPC, but then become mostly AI through 2024. "Within the AI space, we've had very good customer engagement across the board from hyperscalers to OEMs, enterprise customers and some of the new AI start-ups that are out there," she said. "From a workload standpoint, we would expect MI300 to be on both training and inference workloads. We're very pleased with the inference performance on MI300, so especially for large language model inference, given some of our memory bandwidth and memory capacity. We think that's going to be a significant workload for us. But I think we would see a broad set of workloads as well as broad customer adoption."

By the numbers for the third quarter:

• Data center revenue was $1.6 billion, flat on a year ago. 4th generation AMD EPYC CPU sales were up 21% sequentially but offset by system-on-a-chip data center products. AMD said it added almost 100 new instances for its AMD EPYC processors across cloud hyperscalers.
• AMD Instinct MI300A and MI300X GPUs are on track for volume production in the fourth quarter.
• Client revenue was $1.5 billion, up 42% from a year ago. AMD Ryzen 7000 Series sales were up 46% from a year ago.
• Gaming revenue was down 8% from a year ago to $1.5 billion in the quarter.
• Embedded sales were $1.2 billion, down 5% from a year ago.
• AMD acquired Nod.ai and Mipsology to build out its AI software offerings.