From the course: Generative AI: Introduction to Diffusion Models for Text Generation
Join today to access over 24,900 courses taught by industry experts.
Diffusion vs. autoregressive models for text generation - Gemini Tutorial
From the course: Generative AI: Introduction to Diffusion Models for Text Generation
Diffusion vs. autoregressive models for text generation
- [Instructor] We've explored various generative models for text generation, including an introduction to diffusion models and the recognition of transformer-based architectures as the current state of the art. Transformers, which follow an autoregressive paradigm, largely dominates today's NLP landscape. Autoregressive means that the model generates text one token at a time, with each new token conditioned on the previously generated ones. To better contextualize the rise of diffusion models for text generation, it's important to compare them. This will highlight key differences in methodology, strengths, limitations, and the specific roles each model type is suited to play in the future of AI-driven text generation. Understanding this distinction helps to clarify why diffusion models are attracting increasing attention. Now, let's compare autoregressive versus diffusion models across different criteria. Generation process. Autoregressive models like transformers are sequential. They…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.