LLM Performance Testing
Welcome to the practical part of the performance testing lesson. This lab is divided into two separate tasks. Each one is designed to give you practical experience with performance testing. Below, you’ll find a brief overview of each task, along with links to detailed instructions. If you want to dive deeper into how to evaluate the performance of LLM models, check out the Performance Evaluation section.
Task 1: Benchmarking Azure OpenAI Models
In this task, you’ll get hands-on experience with the Azure OpenAI Benchmarking Tool. This tool is an invaluable asset for gauging the performance of Azure OpenAI deployments. It proves particularly useful in the initial phases of a project, helping developers determine if the model deployment is appropriately scaled. Moreover, it enables comparisons between various Azure OpenAI deployments.
For a step-by-step guide on how to proceed with this task, please follow the Task 1 Instructions.
Task 2: Load Testing LLM Apps
In this task, you’ll be working with a reference Language Model (LLM) application. This application, based on the RAG pattern, is available in a separate repository. Your objective is to deploy this application and then carry out load testing using Azure Load Testing. This hands-on task will provide you with practical experience in managing load testing for LLM applications, mirroring real-world scenarios.
For a step-by-step guide on how to proceed with this task, please follow the Task 2 Instructions.