Test AI & LLM App with DeepEval, RAGAs & more using Ollama
Become an AI QA Engineer! Learn to test LLMs & AI apps with DeepEval, RAGAs, HF Evaluate & local LLMs. Roadmap to AI testing expertise.
Course Content
Test AI & LLM App with DeepEval, RAGAs & more using Ollama is an all-levels Udemy course taught by Karthik KK. It runs 10h 29m, is taught in English, and includes a certificate of completion. Learners rate it 4.5 out of 5 from 200 reviews, with 1,617 students enrolled. It is a paid course, last updated August 2025.
Master the art of testing AI applications and Large Language Models (LLMs) with this comprehensive guide. Learn practical techniques to evaluate AI performance, identify potential biases, and build robust AI applications. This course is designed for QA professionals, AI QA engineers, developers, data scientists, and AI practitioners looking to stay ahead in the rapidly evolving field of artificial intelligence.
- Foundations of AI Application Testing: Get a solid understanding of LLM testing, different AI application types, key evaluation metrics, and powerful LLM evaluation libraries.
- Local LLM Deployment with Ollama: Discover how to deploy LLMs locally using Ollama, a simple and effective tool. Explore various AI models, learn to run LLMs locally, and set up Ollama as an API using both GUI and CLI interfaces.
- Environment Setup: Prepare your testing environment with Jupyter Notebooks for efficient testing and learn to set up Confident AI for streamlined evaluation.
- DeepEval Basics: Dive into DeepEval, a powerful framework for LLM testing. Learn to write your first DeepEval code for Answer Relevance and Context Precision. Evaluate your models within Confident AI and test with local LLMs. Understand LLMTestCases and Goldens for effective evaluation.
- Advanced LLM Evaluation: Explore advanced LLM evaluation techniques using LangChain. Focus on evaluating Answer Relevance, Context Precision, and detecting bias in your models. Create custom criteria with GEval for tailored evaluations.
- RAG Testing with DeepEval: Understand Retrieval-Augmented Generation (RAG) and how to test RAG applications effectively with DeepEval. Create GEval for RAG to assess conciseness and completeness.
- Advanced RAG Testing with DeepEval: Develop sophisticated RAG testing strategies by creating multiple test datasets and utilizing Goldens in Confident AI. Analyze actual output and retrieval context to ensure optimal performance. Generate LLMTestCases directly from your datasets.
- Testing AI Agents and Tool Callings: Learn to test AI Agents and their ability to utilize external tools. Work with agents, test them with and without actual systems, and evaluate performance across multiple datasets.
- Evaluating LLMs using RAGAS: Introduce yourself to RAGAS, a dedicated framework for evaluating RAG systems. Understand key metrics like Context Recall, Noise Sensitivity, MultiTurnSample, and other general-purpose metrics for summaries and harmfulness.
- Testing RAG Applications with RAGAS: Get hands-on experience with RAGAS, including setup, creating retrievers and vector stores, utilizing the MultiTurnSample dataset, and evaluating RAG applications for optimal performance.
This course provides the skills you need to ensure the quality and reliability of your AI applications and LLMs.
Want AI in your product? Hire the team behind TutorialSearch.io
LLM features, RAG, agents and automation wired into a real product — built by the devs who shipped this site.