From the course: Hands-On Introduction to Transformers for Computer Vision
Join today to access over 24,900 courses taught by industry experts.
What is an attention heatmap? - PyTorch Tutorial
From the course: Hands-On Introduction to Transformers for Computer Vision
What is an attention heatmap?
- [Instructor] Welcome to chapter seven, video one: What Is an Attention Heatmap? We're going to get into the nitty gritty here when it comes to understanding exactly how your vision transformer is looking at your images and help you understand and explain why that model is making the right prediction. Let's remember back. Flip back to chapter two or chapter four. We patch images before we pass 'em into our vision transformer. The model then looks at the individual patches and helps do the self-attention mechanisms to understand how each patch relates to other patches. We think back to our slide, image is worth 16-by-16 words. We pass in our patches, they go through the linear projection layer, and afterwards, we'll be able to get our model output. Attention maps help us understand which patches the model has decided are important based on the decision it made at the end of the model's prediction. In this case, I'm going to pass in this picture of a dog. I'm going to ask, "What type…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.