BT150 CxO zeitgeist: AI agent growing pains abound
CxOs in Constellation Research's BT150 had a spirited debate about the merits of agentic AI, its ability to scale today and where it can deliver returns.
Yes, we know we're in a sector that's all agentic AI all the time, but enterprise-scale agents aren't anywhere near turnkey. And that's why you need all of those forward deployed engineers to get AI agents working. The AI agent game is still in the early innings.
The BT150 meetup was held Jan. 30. This Constellation Research CxO call operates under Chatham House rules so the takeaways aren't attributed.
Although CxOs have previously noted complaints about agentic AI consumption models, SaaS vendors' platform pitch and AI agent washing, this conversation was more about practical deployment takeaways.
Here are the key points:
- Large enterprises aren't depending on autonomous agents for core production workloads yet. Enterprise AI agents have left the lab, but remain on a tight leash.
- Most successful projects today are deterministic or orchestrated flows with small, well-bounded probabilistic components.
- AI agents are hamstrung by context window instability, memory drift and over-prompting and overuse of RAG. Shoving more stuff into an agentic AI framework can often degrade quality in the field.
- Context engineering is critical and CxOs need to move beyond prompt engineering to precisely define which data, files and tools are in scope for a given task.
- Governance will need to include runtime context control not just security and entitlements.
- AI agents today are best treated as minions that are good at simple repetitive tasks not fully trusted teammates.
- The highest value for AI agents in the near term is cost takeout, process acceleration and error reduction.
- One big return use case for AI agents is observability layers and intercepting problematic docs before they land in a repository.
- Enterprises are focused on process-level returns and discrete processes with clear metrics such as stock-outs, cash flow, replenishment and service turnaround. These tasks may be better suited for small language models.
- The real work in agentic AI deployments is process redesign and not just data cleanup and model selection. Technology and process debt requires redesign not just an AI wrapper.
- 2026 is seen as a year of AI agent experimentation with a few production use cases. There was a healthy debate on AI agents and their impact on SaaS companies. One CxO noted that CRM will disappear as a category. Others noted that take was unrealistic given the historical investment in CRM. The more likely outcome is SaaS as a category is reshaped for agentic AI.
- It is more important than ever to start projects with "what are we trying to do?" Too often, AI projects are starting with which model to buy.
More:
About Larry Dignan: Dignan was most recently Celonis Media’s Editor-in-Chief, where he sat at the intersection of media and marketing. He is the former Editor-in-Chief of ZDNet and has covered the technology industry and transformation trends for more than two decades, publishing articles in CNET, Knowledge@Wharton, Wall Street Week, Interactive Week, The New York Times, and Financial Planning. He is also an Adjunct Professor at Temple University and a member of the Advisory Board for The Fox Business School's Institute of Business and Information Technology. Constellation Insights does the following: Cover the buy side and sell side of enterprise tech with news, analysis, profiles, interviews, and event coverage of vendors, as well as Constellation Research's community and…...
Read morePublished
Author