From the course: ChatGPT: Crafting Exceptional GPTs for Enhanced Productivity and Innovation

Unlock this course with a free trial

Join today to access over 24,900 courses taught by industry experts.

GPTs and ChatGPT: Building benchmarks to measure AI performance

GPTs and ChatGPT: Building benchmarks to measure AI performance - ChatGPT Tutorial

From the course: ChatGPT: Crafting Exceptional GPTs for Enhanced Productivity and Innovation

Start my 1-month free trial Buy for my team

GPTs and ChatGPT: Building benchmarks to measure AI performance

“

- Anytime we change the instructions to our GPT or we change the knowledge base, it can have unexpected effects. And so one of the important things that we want to do when we start building a custom GPT and really thinking about how do we build the best custom GPT, a very important thing to do is to build yourself a simple benchmark. Now, why do you do this? Well, you do this so that as you make changes, you can make sure that it's still reasoning effectively, that it's not regressing in some area and starting to give bad answers to something it used to do well on. But the other reason you want to do it is because you want to make sure that it really is as good as you think it is. And often when we go and sort of ad hoc do this, we may miss areas where we will say, oh, well it should be able to do this, and we don't actually test it. And then it turns out it doesn't do a very good job of it. So building yourself a benchmark is a really simple thing that you can do to one, make sure it…

Contents