# Evaluating Agentic AI: DeepEval, RAGAS & TruLens Frameworks Compared

In this episode of Memriq Inference Digest - Leadership Edition, we unpack the critical frameworks for evaluating large language models embedded in agentic AI systems. Leaders navigating AI strategy will learn how DeepEval, RAGAS, and TruLens provide complementary approaches to ensure AI agents perform reliably from development through production.

In this episode:

- Discover how DeepEval’s 50+ metrics enable comprehensive multi-step agent testing and CI/CD integration

- Explore RAGAS’s revolutionary synthetic test generation using knowledge graphs to accelerate retrieval evaluation by 90%

- Understand TruLens’s production monitoring capabilities powered by Snowflake integration and the RAG Triad framework

- Compare strategic strengths, limitations, and ideal use cases for each evaluation framework

- Hear real-world examples across industries showing how these tools improve AI reliability and speed

- Learn practical steps for leaders to adopt and combine these frameworks to maximize ROI and minimize risk

Key Tools & Technologies Mentioned:

- DeepEval

- RAGAS

- TruLens

- Retrieval Augmented Generation (RAG)

- Snowflake

- OpenTelemetry

Timestamps:

0:00 Intro & Why LLM Evaluation Matters

3:30 DeepEval’s Metrics & CI/CD Integration

6:50 RAGAS & Synthetic Test Generation

10:30 TruLens & Production Monitoring

13:40 Comparing Frameworks Head-to-Head

16:00 Real-World Use Cases & Industry Examples

18:30 Strategic Recommendations for Leaders

20:00 Closing & Resources

Resources:

- Book: "Unlocking Data with Generative AI and RAG" by Keith Bourne - Search for 'Keith Bourne' on Amazon and grab the 2nd edition

- This podcast is brought to you by Memriq.ai - AI consultancy and content studio building tools and resources for AI practitioners.

Discover how the Model Context Protocol (MCP) is revolutionizing AI system integration by simplifying complex connections between AI models and external tools. This episode breaks down the technical and strategic impact of MCP, its rapid adoption by industry giants, and what it means for your AI strategy.

In this episode:

- Understand the M×N integration problem and how MCP reduces it to M+N, enabling seamless interoperability

- Explore the core components and architecture of MCP, including security features and protocol design

- Compare MCP with other AI integration methods like OpenAI Function Calling and LangChain

- Hear real-world results from companies like Block, Atlassian, and Twilio leveraging MCP to boost efficiency

- Discuss the current challenges and risks, including security vulnerabilities and operational overhead

- Get practical adoption advice and leadership insights to future-proof your AI investments

Key tools & technologies mentioned:

- Model Context Protocol (MCP)

- OpenAI Function Calling

- LangChain

- OAuth 2.1 with PKCE

- JSON-RPC 2.0

- MCP SDKs (TypeScript, Python, C#, Go, Java, Kotlin)

Timestamps:

0:00 - Introduction to MCP and why it matters

3:30 - The M×N integration problem solved by MCP

6:00 - Why MCP adoption is accelerating now

8:15 - MCP architecture and core building blocks

11:00 - Comparing MCP with alternative integration approaches

13:30 - How MCP works under the hood

16:00 - Business impact and real-world case studies

18:30 - Security challenges and operational risks

21:00 - Practical advice for MCP adoption

23:30 - Final thoughts and strategic takeaways

Resources:

• "Unlocking Data with Generative AI and RAG" by Keith Bourne - Search for 'Keith Bourne' on Amazon and grab the 2nd edition
• This podcast is brought to you by Memriq.ai - AI consultancy and content studio building tools and resources for AI practitioners.

In this episode of Memriq Inference Digest - Leadership Edition, we explore how Retrieval-Augmented Generation (RAG) systems maintain quality and trust at scale through advanced evaluation methods. Join Morgan, Casey, and special guest Keith Bourne as they unpack the game-changing RAGAS framework and the emerging practice of reference-free evaluation that enables AI to self-verify without costly human labeling.

In this episode:

- Understand the limitations of traditional evaluation metrics and why RAG demands new approaches

- Discover how RAGAS breaks down AI answers into atomic fact checks using large language models

- Hear insights from Keith Bourne’s interview with Shahul Es, co-founder of RAGAS

- Compare popular evaluation tools: RAGAS, DeepEval, and TruLens, and learn when to use each

- Explore real-world enterprise adoption and integration strategies

- Discuss challenges like LLM bias, domain expertise gaps, and multi-hop reasoning evaluation

Key tools and technologies mentioned:

- RAGAS (Retrieval Augmented Generation Assessment System)

- DeepEval

- TruLens

- LangSmith

- LlamaIndex

- LangFuse

- Arize Phoenix

Timestamps:

0:00 - Introduction and episode overview

2:30 - What is Retrieval-Augmented Generation (RAG)?

5:15 - Why traditional metrics fall short for RAG evaluation

7:45 - RAGAS framework and reference-free evaluation explained

11:00 - Interview highlights with Shahul Es, CTO of RAGAS

13:30 - Comparing RAGAS, DeepEval, and TruLens tools

16:00 - Enterprise use cases and integration patterns

18:30 - Challenges and limitations of LLM self-evaluation

20:00 - Closing thoughts and next steps

Resources:

- "Unlocking Data with Generative AI and RAG" by Keith Bourne - Search for 'Keith Bourne' on Amazon and grab the 2nd edition

- Visit Memriq AI at https://Memriq.ai for more AI engineering deep-dives, guides, and research breakdowns

Thanks for tuning in to Memriq AI Inference Digest - Leadership Edition. Stay ahead in AI leadership by integrating continuous evaluation into your AI product strategy.