Want Reliable AI? It Starts with Knowing What’s Going On

Abstract

As AI becomes mission-critical, traditional monitoring falls short on AI-specific metrics like token usage, model drift, and response quality. The challenge intensifies when comparing models (OpenAI GPT 4, 4o-mini vs Open Source Models like Mistral) and architectures (simple chat vs agentic AI with tools). How do you choose? How do you optimize for performance and cost?

New Relic's AI Monitoring provides full-stack visibility across any AI platform or architecture, from simple LLM calls to complex multi-agent workflows, enabling you to detect failures, bottlenecks, and runaway costs before they impact users or budgets.

This hands-on workshop progresses from foundational AI applications to cutting-edge agentic systems. You'll build with Azure OpenAI, compare models, explore GitHub Models, and implement modern agentic AI using Microsoft's Autogen and Model Context Protocol (MCP). New Relic AI Monitoring provides the observability to understand performance, debug issues, and make data-driven architectural decisions at every stage.

You'll learn to:

- Build and monitor AI applications across Azure OpenAI and GitHub Models

- Compare models and platforms with performance, cost, and capability trade-offs

- Implement agentic AI with autonomous reasoning and tool usage

- Trace complex workflows from user request through agent reasoning to tool execution

- Optimize AI applications using data-driven insights

Prerequisites:

• Required: A free New Relic account. Sign up here.
• Recommended: Azure OpenAI (https://ai.azure.com/) API access to deploy OpenAI and DALL-E 3 base models.
• Recommended: GitHub Personal Access Token with models.read scope

By invitation only