New Relic launched New Relic AI monitoring, which aims to bring observability to AI operations and applications. New Relic also expanded its partnership with AWS to integrate Amazon Bedrock with its AI monitoring platform.
The company’s launch is timely given that boards of directors are pressuring CXOs to deliver generative AI applications and productivity gains, but enterprises are trying to avoid large language models (LLMs) and applications that aren't tracked. New Relic recently went private in a deal valued at $6.5 billion. Meanwhile, Vendors are scrambling to provide a generative AI magic bullet with fun names, domain specific LLMs and add-ons that add up, but the reality has been more challenging.
According to New Relic, AI Monitoring (AIM) will bring visibility across AI applications so enterprises can optimize performance, quality and costs. New Relic AIM will have more than 50 integrations and include LLM model comparisons and response tracing. New Relic said AIM is designed to monitor LLMs and vector databases to surface inaccuracies, biases, security issues and telemetry data to give engineers insights to the AI stack.
Constellation ShortList™ Observability | Constellation ShortList™ AIOps | A CIO's Guide to Observability
Since New Relic already has application performance monitoring (APM) tools the extension into AI gives enterprises a suite to observe the entire enterprise stack. Cisco’s acquisition of Splunk is driven by the expanding observability market.
New Relic AIM will monitor the following AI platforms:
- Orchestration framework: LangChain
- LLM: OpenAI, PaLM2, HuggingFace
- Machine learning libraries: Pytorch, TensorFlow
- Model serving: Amazon SageMaker, AzureML
- Vector databases: Pinecone, Weaviate, Milvus, FAISS
- AI infrastructure: Azure, AWS, Google Cloud
Features in New Relic AIM include auto instrumentation, a holistic view across AI applications integrated with application performance monitoring, deep trace insights for LLM responses, performance and cost comparisons and tools to enable responsible use of AI.
Related:
- Why generative AI workloads will be distributed locally
- Software development becomes generative AI's flagship use case
- Enterprises seeing savings, productivity gains from generative AI
- LogicMonitor Adds AIOps Capabilities to Its Hybrid Observability Platform
- The Urgent Case for a Chief AI Officer
- An Executive Guide to Faster Incident Resolution