Exploring the playground

- [Speaker] Let's take a closer look at the playground for GitHub models. Starting on the marketplace, you can choose a model that you want to experiment with. Let's use ChatGPT this time. From the detail page, you can learn more information about the model itself and with the buttons on the top, you can get started with a sample question. Let's go with a question on gothic architecture. (breathes deeply) You can see that the question is sent to me as the user, and that the response of the model starts flowing back to me. This flow of text as a response is called streaming. We can see that we got back information to answer my question. This shows how much information is available in the model about generic topics like in this case, architecture. It retrieved this information from the model by predicting the likeliness of the next word in a sentence. With some clever engineering in the model. GitHub can use this flow to show it as a conversation, which is in the form of question and answers here. This is also referred to as turns. A turn is made up out of the question or prompt that I asked, and then the response of the model. Keep in mind that the response is the result of a probabilistic calculation. The response is made up out of the most likely words after my question, so the response is not based on any real facts. The model has seen these terms like pointed arches and red defaults the most in its training data. The words were in the vicinity of the gothic topic, and that is why it makes it very likely to be used in the response. So don't forget that the response cannot be accepted as a fact. You will have to determine what you want to do with it. In this case, the response makes perfect sense to me, but then again, I am not a history expert. We can interact with the response using the buttons at the bottom, we can give the response a thumbs up or down, copy the entire response, or even regenerate the response. Regenerate means that we send the exact same prompt to the model and see if we get a different result. Since interacting with a large language model is non-deterministic, we will get a different result back each time, although it might seem mostly the same most of the times. Since this is still part of our conversation, we can now follow-up with more questions and thus, execute more turns. We can use the playground to discover more about the gothic period. You can keep exploring the conversational capabilities of the model and test what happens when you change the settings on the right to see their effects. For example, we can add a system prompt and say, only use French in your response. Now, ask a follow-up question and your system prompt will be in the result as well. This also highlights the translation capabilities of the models as well as the fact that you can communicate with it in your native language. On the top-right, you can still get the model detail card from the details button and then read again the capabilities of this model. The playground is really meant for testing out the conversational capabilities. None of the other options are available at the time of this recording. On the top-left, we can also easily change the model we are testing with. This lets you try out your prompts with different models and compare the results. You can see that the previous chat conversation also gets cleared out. You can also clear it out when needed with the restart button on the top-right. After testing, you probably are ready to start with actual code and build things into your own application. There are two main ways to continue from here. First of all, we can use the buttons on the top to see how the playground is set up to give us the chat window and what the moving parts of it are. On the right hand side, there's also an entry point to get started as well. We will have a look at that in the next video. After looking at the options in the playground itself first. From the code tab, we get an example of the code that is needed to send prompts and responses in our own application in the same way that the playground is doing. The options here that you switch between different language examples where Python, JavaScript, and C# are languages and REST will show you the raw rest calls using a standard web request. The SDK lets you select between the model Vendors SDK, if there is any, and the standard Azure AI inference SDK. I would recommend the ladder as that enables you to switch models by changing just a model string. Let's take a look at the coding sample shown here. At the top, we see the dependencies needed to work with this code. The install command is even mentioned in line four. You can see that we can automate with the GitHub token on line 11. All these three options are linked to your GitHub account, which is used in the playground as well. In the main function. An instance of a client that can talk to the models is needed. Notice the generic URL used for GitHub models here. That URL will be different if you are using your own Azure OpenAI instance. Next up, you can see that a request is set to the slash chat slash completions endpoint of the OpenAI instance. In the body, the user prompt is added as well as the model string and the other parameters from the settings on the right. You can even change the settings, like respond in Dutch in the system prompt, and you can see the code example update directly, and the last part of the code is showing your user the output of the response. Next up, I want to show you that the playground also shows you directly what is happening with the streaming mode of the models. When I switch back to the chat and select a question, you can see on the raw tab that the streaming response is immediately shown there as well. You can see that the playground already gives you plenty of options to experiment with the different models to find out what options could work in your own application. Join me in the next video to get some actual code running on your local machine against GitHub models.