ServiceNow's latest Now Assist generative AI features highlight its strategy

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

ServiceNow launched a series of Now Assist generative AI services across its platform.

The features, available today, are part of a systematic effort to infuse generative AI use cases across workflows courtesy of Nvidia GPUs, ServiceNow large language models (LLMs) and the Now Platform.

Specifically, ServiceNow launched Now Assist in Virtual Agent, flow generation and Now Assist for Field Service Management (FSM). ServiceNow's generative AI strategy revolves around what it calls "practical generative AI applications" that focus on use cases and smaller models that are more efficient.

ServiceNow CEO McDermott talks business transformation, generative AI, processes

During a recent analyst briefing, CJ Desai, ServiceNow's Chief Product Officer, said:

"If we have smaller LLMs, they are cost efficient. They serve a particular use case. Small is better and a specific use case will run faster and provide a great experience. Our strategy is simple: Have ServiceNow specific small language models."

Among the ServiceNow generative AI features now available:

• Now Assist in Virtual Agent makes it easier to create and deploy generative AI chat experiences. Updates include Q&A in knowledge management and multi-turn conversations for service requests and catalog orders.
• Flow generation, a feature that enables a broader user base of admins and developers to automate more workflows using generative AI, which will take text prompts and create workflows. Workflows can then be tweaked and refined using App Engine's no-code interface.
• Now Assist for FSM uses generative AI to access activity, parts and incidental data to summarize field work and provide more proactive service.

The three additions are available in the ServiceNow Store.

Also see:

• ServiceNow reports strong Q3, ups outlook for 2023
• ServiceNow launches Now Assist across Now Platform, pricing bundles
• ServiceNow outlines generative AI roadmap, bring your own LLMs
• ServiceNow expands finance, supply chain process, workflow automation

Cisco sees weak Q2 as customers digest, implement shipped orders

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Cisco Systems reported better-than-expected first quarter results, but its fiscal 2024 outlook fell short of expectations. Cisco said it saw "a slowdown of new product orders in the first quarter of fiscal 2024."

The company reported first quarter earnings of 89 cents a share on revenue of $14.7 billion, up 8% from a year ago. Non-GAAP earnings were $1.11 a share. CEO Chuck Robbins said the first quarter was a "solid start" to the fiscal year. 

Wall Street was expected to report first quarter earnings of $1.03 a share on revenue of $14.63 billion. Cisco recently announced plans to acquire Splunk to expand its observability, AI and cybersecurity footprint.

The issue for Cisco is its second quarter outlook. CFO Scott Herren said customers are implementing "large amounts of recently shipped product." "We expect to see product order growth rates accelerate in the second half of the year," he said.

Cisco added that it believes the primary reason for the slowdown hitting the second quarter is that customers are digesting "exceptionally strong product delivery over the past three quarters." Cisco estimated that there are one to two quarters of shipped product orders waiting to be implemented. 

As for the outlook, Cisco projected second quarter revenue of $12.6 billion to $12.8 billion with non-GAAP earnings of 82 cents a share to 84 cents a share. For fiscal 2024, Cisco is projecting revenue of $55.8 billion to $55 billion with non-GAAP earnings of $3.87 a share to $3.93 a share. Analysts were expecting second quarter revenue of $14.2 billion and fiscal 2024 revenue of $57.8 billion and non-GAAP second quarter earnings of 99 cents a share and $4.05 for the fiscal year. 

By the numbers:

• Americas revenue was up 14% with EMEA flat from a year ago and APJC down 3%.
• Networking revenue in the first quarter was up 10%.
• Security revenue in the first quarter was up 4%.
• Observability revenue was up 21%.
• Collaboration was up 3%.

Cisco closed five acquisitions in the first quarter: Accedian, Working Group Two, Oort Inc., SamKnows, and Code BGP.

Microsoft launches Azure Models as a Service

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Microsoft CEO Satya Nadella said the company will launch Azure Models as a Service, which will aim to build out its OpenAI model offerings with a broader selection.

Nadella, speaking at Microsoft's Ignite 2023 conference, said the company wants to "bring the best selection of open-source models to Azure and do so responsibly."

He added that Azure's models-as-a-service offering will include Stable Diffusion, Llama2 from Meta, Mistral and Jais, the world's largest Arabic language model. Command from Cohere will also be available.

Microsoft launches AI chips, Copilot Studio at Ignite 2023

"With models-as-a-service, developers won't have to provision GPUs so they can focus on development and not back-end operations," said Nadella, who noted that customers can fine tune foundational models with their own data. "We want to support models in every language in every country," said Nadella.

Specifics include:

• Models as a Service will offer inference APIs and hosted fine tuning for Llama 2 in Azure AI model catalog.
• PayGo inference APIs are billed by the numbers of tokens used.
• Llama2 can be fine-tuned with your own data.
• Content moderation is built into the service.

Microsoft launches AI chips, Copilot Studio at Ignite 2023

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Microsoft fleshed out its generative AI infrastructure plans with the launch of its custom processors for model training and inference along with Copilot Studio for new use cases.

At its Ignite 2023 conference, Microsoft Azure joined its rivals in offering custom processors for model training. Nvidia has been dominating the field, but now has competition from AMD along with custom processors such as Amazon Web Services Trainium and Inferentia and Google Cloud TPUs.

Microsoft launched Azure Maia 100 AI Accelerator, the company's first in-house custom AI system on a chip. Azure Maia is designed to optimize training and inferencing efficiency for large language models (LLMs). The launch of Azure Maia highlights how Microsoft is becoming a systems company and more than software. Microsoft in a blog post reiterated that it will work closely with Nvidia and will offer its H200 GPUs as well as AMD's MI300x

"We need to be even the world's best systems company across heterogeneous infrastructure," said Microsoft CEO Satya Nadella during his Ignite 2023 keynote. "We work closely with our partners across the industry to incorporate the best innovation. In this new age, AI will be defining everything across the fleet in the data center. As a hyperscaler, we see the workloads, we learn from them, and then get this opportunity to optimize the entirety of the stack."

According to Microsoft, Azure Maia features:

• Support for open-standard MX sub-8bit data types.
• Industry standard ethernet connectivity.
• 4x Maia per server with liquid-cooled design.
• 5nm process technology from TSMC.
• The ability to run OpenAI models like GPT3.5 with testing for GitHub Copilot and Bing.

Rani Borkar, commercial vice president, Azure Hardware Systems and Architecture, said Microsoft is taking a systems approach to its infrastructure and optimizing for AI workloads. Azure has 60 data center regions today.

Microsoft uses Oracle Cloud Infrastructure for Bing conversational workloads

Borkar said Microsoft Azure is supporting a wide range of models from OpenAI and others. She also emphasized that Azure would continue to roll out instances based on the latest GPUs from Nvidia and AMD. Borkar said:

"To support these models, we are in that reimagining the entire cloud infrastructure, with a systems approach top to bottom end to end systems approach with a deep understanding of model architecture and workload requirements. We are optimizing our systems with our custom silicon so that our customers can benefit from performance, power efficiency, and the cloud promise of trust and reliability."

Borkar said that Azure plans to optimize every layer of the AI stack to reduce costs and improve energy efficiency.

To that end, Microsoft also outlined Azure Cobalt 100 CPU, the first Arm-based compute system on a chip from the company. Cobalt has up to a 40% performance/core improvement over previous Azure Arm servers. Features include:

• Microsoft designed security and power management.
• 128 cores; 12 channels of DDR.
• Built on 5nm process technology.
• Powers Microsoft Teams and Azure SQL Server.

When asked about better price/performance with Azure Maia, Borkar said each workload is different. The game plan is to give customers options to choose from a wide range of chips and ecosystems.

Constellation Research analyst Andy Thurai said:

"If Microsoft can prove that training and inferencing LLM custom models on their chipset can be more cost/power effieicient than the competitors there may be an opportunity to get customers to engage. At this point, I don’t see much traction for the chips when they are released next year."

Copilot Studio

Along with Microsoft's custom processors, the company also launched Copilot Studio. Copilot Studio will be of interest to enterprises looking to customize copilots for new use cases.

GitHub plans to infuse Copilot throughout its platform | OpenAI launches GPTs as it courts developers, models for use cases | Software development becomes generative AI's flagship use case

The game plan for Microsoft and Copilot Studio is straightforward: Build copilots for every Microsoft experience. Inspire features copilot hooks for everything from Teams to PowerApps to security and various business applications.

Omar Aftab, Vice President of Microsoft Conversational AI, said Microsoft is building out a copilot ecosystem and copilot Studio is the next iteration of the strategy. "Copilot Studio allows users to extend various first party Copilots or build their own custom enterprise Copilots," said Aftab. "You essentially have a number of conversational tools at your fingertips."

Microsoft's Copilot Studio launch will also enable customers to build custom GPTs from OpenAI. Copilot Studio is also integrated with Microsoft Azure's various services.

Key points about Copilot Studio include:

• End-to-end lifecycle management.
• Managed components, APIs, Azure services and low-code tools and plug-ins in one place.
• Ability to supplement LLMs with business-critical topics.
• Built-in analytics and telemetry on Copilot performance.
• Access to corporate documents as well as public searches.
• Publishing on websites, SharePoint and other options.
• Ability to segment access by roles.

Ultimately, Microsoft plans to build a Copilot marketplace, but that will happen over time, said Aftab.

Celonis launches Process Intelligence Graph, makes case process enables automation, AI applications

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Celonis is betting that process mining data will be the enabler for automation and generative AI across enterprises with the launch of its Process Intelligence (PI) graph.

The argument is worth considering as the intersection of process automation, intelligence, machine learning and artificial intelligence is getting crowded. Multiple vendors are gunning to be that platform that connects the systems and processes behind business transformation.

At Celonis' Celosphere conference, the company outlined how it wants PI Graph to be the "Wikipedia of Process Intelligence." The idea here is that the system agnostic PI Graph, which was introduced a day after Celonis acquired Symbio, will leverage process knowledge from customer deployments and provide an intelligence layer that will form a digital twin of the business and ultimately value chains. Disclosure: I used to work for Celonis. 

This process layer will orchestrate systems, processes and optimizations to continually improve operations. According to Celonis, PI Graph will be the common process language to unify enterprises. Alex Rinke, co-CEO of Celonis, noted that the PI Graph is "the connective tissue that’s been missing in modern enterprises." The data from the PI Graph could give AI and automation models process knowhow. 

Multiple vendors are aiming to be this connective tissue between enterprise processes. The game is to meld process models, KPIs, insights and workflows into one experience. For instance, Microsoft bought Minit in 2022 and then wrapped that process mining technology into a Power Automate wrapper. SAP acquired Signavio and has its platform as well as Datasphere as a wrapper with additions such as LeanIX. UIPath has built out its automation platform that puts process mining together with other technologies including its AutoPilot generative AI.

Constellation Research CEO Ray Wang said Celonis' PI Graph gives companies that ability to embed process intelligence into "their operating models and technology stacks, enabling a new wave of powerful applications and use cases."

At Celosphere, Celonis also launched the following:

• Process Copilot, a system to identify value via the PI Graph.
• A new Celonis Studio to enhance app testing, create dashboards and launch actions.
• Transformation Hub, which provides a hub to measure the value from Celonis implementations.
• Process Adherence Manager, formerly Process Sphere, is generally available.   

New Relic brings observability to AI stack

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

New Relic launched New Relic AI monitoring, which aims to bring observability to AI operations and applications. New Relic also expanded its partnership with AWS to integrate Amazon Bedrock with its AI monitoring platform.

The company’s launch is timely given that boards of directors are pressuring CXOs to deliver generative AI applications and productivity gains, but enterprises are trying to avoid large language models (LLMs) and applications that aren't tracked. New Relic recently went private in a deal valued at $6.5 billion. Meanwhile, Vendors are scrambling to provide a generative AI magic bullet with fun names, domain specific LLMs and add-ons that add up, but the reality has been more challenging.

According to New Relic, AI Monitoring (AIM) will bring visibility across AI applications so enterprises can optimize performance, quality and costs. New Relic AIM will have more than 50 integrations and include LLM model comparisons and response tracing. New Relic said AIM is designed to monitor LLMs and vector databases to surface inaccuracies, biases, security issues and telemetry data to give engineers insights to the AI stack.

Constellation ShortList™ Observability | Constellation ShortList™ AIOps | A CIO's Guide to Observability

Since New Relic already has application performance monitoring (APM) tools the extension into AI gives enterprises a suite to observe the entire enterprise stack. Cisco’s acquisition of Splunk is driven by the expanding observability market.

New Relic AIM will monitor the following AI platforms:

• Orchestration framework: LangChain
• LLM: OpenAI, PaLM2, HuggingFace
• Machine learning libraries: Pytorch, TensorFlow
• Model serving: Amazon SageMaker, AzureML
• Vector databases: Pinecone, Weaviate, Milvus, FAISS
• AI infrastructure: Azure, AWS, Google Cloud

Features in New Relic AIM include auto instrumentation, a holistic view across AI applications integrated with application performance monitoring, deep trace insights for LLM responses, performance and cost comparisons and tools to enable responsible use of AI.

Related:

• Why generative AI workloads will be distributed locally
• Software development becomes generative AI's flagship use case
• Enterprises seeing savings, productivity gains from generative AI

• LogicMonitor Adds AIOps Capabilities to Its Hybrid Observability Platform
• The Urgent Case for a Chief AI Officer
• An Executive Guide to Faster Incident Resolution

Dell Technologies, Hugging Face aim to make on-premises generative AI deployments easier

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Dell Technologies and Hugging Face are teaming up to target on-premises generative AI deployments to make the jump from enterprise proof-of-concepts to production easier.

While cloud hyperscalers have dominated the generative AI discussion, enterprises are wary of costs as well as protecting data and see hybrid approaches more viable. In addition, CXOs are being asked by boards of directors to produce generative AI projects, but the move to production use cases is challenging. Vendors are scrambling to provide a generative AI magic bullet with fun names, domain specific LLMs and add-ons that add up, but the reality has been more challenging.

"Customers are reporting--and we observe this every day--that they're frustrated by the complexity associated with building AI that includes applications in closed ecosystems," said Dell Technologies' Matt Baker, Senior Vice President of AI Strategy. "If you recall with the era of big data, there was a real challenge progressing from proof of concept to production."

• Why generative AI workloads will be distributed locally
• Software development becomes generative AI's flagship use case
• Enterprises seeing savings, productivity gains from generative AI

The theory behind the Dell and Hugging Face partnership is that AI is likely to go to where the data lives--mostly on premises--instead of the other way around. Dell has been publishing validated designs that include storage, servers and accelerators.

Jeff Boudier, Head of Product and Growth at Hugging Face, added that "going from a model to a solution you can deploy is a completely different ballgame."

Under the partnership, Dell customers will be able to select open-source models for use cases on Hugging Face, pick optimized infrastructure for that workload and get help tuning models within a container. The model from there is on-premises and owned by the customer.

Boudier added that enterprises are looking to take control of their machine learning and AI destiny. "The only way to do that is to become a builder," he said.

The two companies will also aggregate libraries, data sets and tutorials on training. Baker said that there will be templates and models designed for specific outcomes. Some models will be trained for use cases and an enterprise would then couple it with proprietary data in a container for tuning.

Initially, Dell Technologies and Hugging Face will focus on PowerEdge and data center gear designed for models. Baker, however, noted that workstations will also be included. The Dell and Hugging Face connection will also be available through Apex.

Nvidia launches H200 GPU, shipments Q2 2024

Larry Dignan

Editor in Chief of Constellation Insights
Constellation Research

Larry Dignan is Editor in Chief of Constellation Insights at Constellation Research, where he leads editorial coverage focused on enterprise technology, digital transformation, and emerging trends shaping the future of business. He oversees research-driven news, analysis, interviews, and event coverage designed to help technology buyers and vendors navigate complex markets with clarity and context. ...

Read more

Nvidia launched its Nvidia H200 GPU, which will offer faster memory and more bandwidth for generative AI workloads. The H200 will ship in the second quarter of 2024.

The launch comes as Wall Street waits for Nvidia's earnings and a read on whether the company could meet demand. In addition, Nvidia is about to see competition from AMD and hyperscale cloud players have their own proprietary chips for model training.

Nvidia's H200 is the first to offer HBM3e, which gives the H200 the ability to deliver 141GB of memory at 4.8 terabytes per second. That tally is a big jump in capacity and bandwidth relative to the Nvidia A100. The H200 serves as the base for the Nvidia HGX H200 AI computing platform based on the company's Hopper architecture.

Related:

• Microsoft uses Oracle Cloud Infrastructure for Bing conversational workloads
• Why generative AI workloads will be distributed locally
• Enterprises seeing savings, productivity gains from generative AI
• AMD sees AI interest, AMD Instinct MI300A and MI300X GPUs on track
• AMD makes its case for generative AI workloads vs. Nvidia
• Why enterprises will want Nvidia competition soon

According to Nvidia, H200 will nearly double the inference speed on the Llama 2 large language model. Nvidia said software updates will boost performance more.

The latest Nvidia GPU will be available across server makers such as ASRock Rack, ASUS, Dell Technologies, Eviden, GIGABYTE, Hewlett Packard Enterprise, Ingrasys, Lenovo, QCT, Supermicro, Wistron and Wiwynn.

Amazon Web Services, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure will also deploy H200-based instances as will CoreWeave, Lambda and Vultr.

One thing to watch going forward will be the supply of Nvidia GPUs and whether it can meet demand. Also watch what vendors get allocations of GPUs relative to others.

For instance, Super Micro CEO Charles Liang said on the company's fiscal first quarter earnings call.

"We navigated tight AI GPU and key components supply conditions to deliver total solutions and large compute clusters, especially for generative AI workloads where our backorders continue to expand faster than our forecast.

During the first quarter, demand for our leading AI platforms in plug-and-play rack-scale, especially for the LLM-optimized NVIDIA HGX-H100 solutions, was the primary growth driver.

Many customers have started to request direct-attached cold-plate liquid-cooling solutions to address the energy costs, power grid constraints and thermal challenges of these new GPU infrastructures."