Downloaded 340 times
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 8
How Generative AI works (admittedly oversimplified)
The system generates text or images using its previously built model of the
statistical distributions of tokens (= “chunks” of words, punctuation marks,
pixels, etc.) created from its very large training dataset.
Image from Pattern Recognition. GeeksforGeeks. Retrieved May 16, 2023 from https://www.geeksforgeeks.org/pattern-recognition-introduction/
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
Doc
Chat
Image](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-8-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 9
How Generative AI works (admittedly oversimplified)
It might make mistakes or “hallucinate” based on the limitations of its
process, but the output still might look like what you wanted.
Ted Chiang’s analogy = “unreliable photocopier” or a “blurry JPEG”
Ted Chiang. 2023. ChatGPT Is a Blurry JPEG of the Web. The New Yorker. Retrieved May 10, 2023 from https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
Doc
Chat
Image](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-9-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 11
How Generative AI works (admittedly oversimplified)
We can ask it
questions - but a
very specific type
of question known
as prompts,
following this
structure:
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
“Here’s a fragment of text.
Tell me how this fragment might <continue on in
this language, or suggest a particular image>.
According to your model of the statistics of
<human language, or human-handled images>,
what <words, or pixels> are likely to come next?”](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-11-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 12
How Generative AI works (admittedly oversimplified)
The prompts are converted into tokens (= “chunks” of words, punctuation marks,
pixels, etc.), then the system analyzes what is likely to come next, based on the
tokens in its own dataset (as many as 32,000 in GPT-4!).
It then generates a tokenized output.
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
Vector of
probabilities from
own tokens](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-12-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 13
How Generative AI works (admittedly oversimplified)
With each output, it keeps re-analyzing the probabilities to decide next tokens.
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and shopped
Vector of
probabilities from
own tokens](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-13-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 14
HERE’S THE REALLY COOL PART!!!
Transformers (the “T in “GPT”) know how to direct attention to specific parts
of the input to guide their selection of the output - such as verb tenses, objects.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/1706.03762
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and shopped
Vector of
probabilities from
own tokens](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-14-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 15
How Generative AI works (admittedly oversimplified)
The system can give you different answers to the same inputs:
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and bought
Vector of
probabilities from
own tokens](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-15-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 16
How Generative AI works (admittedly oversimplified)
The system can give you different answers to the same inputs:
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and clocked in
Vector of
probabilities from
own tokens](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-16-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 17
How Generative AI works (admittedly oversimplified)
The system can give you different answers to the same inputs:
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and danced
Vector of
probabilities from
own tokens
huh?](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-17-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 18
How Generative AI works (admittedly oversimplified)
“Hallucinations” - when the output doesn’t seem to make sense - are why it is
important not to accept everything it outputs at face value.
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and danced
Vector of
probabilities from
own tokens
huh?](https://image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-18-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 8
How Generative AI works (admittedly oversimplified)
The system generates text or images using its previously built model of the
statistical distributions of tokens (= “chunks” of words, punctuation marks,
pixels, etc.) created from its very large training dataset.
Image from Pattern Recognition. GeeksforGeeks. Retrieved May 16, 2023 from https://www.geeksforgeeks.org/pattern-recognition-introduction/
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
Doc
Chat
Image](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-8-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 9
How Generative AI works (admittedly oversimplified)
It might make mistakes or “hallucinate” based on the limitations of its
process, but the output still might look like what you wanted.
Ted Chiang’s analogy = “unreliable photocopier” or a “blurry JPEG”
Ted Chiang. 2023. ChatGPT Is a Blurry JPEG of the Web. The New Yorker. Retrieved May 10, 2023 from https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
Doc
Chat
Image](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-9-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 11
How Generative AI works (admittedly oversimplified)
We can ask it
questions - but a
very specific type
of question known
as prompts,
following this
structure:
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
“Here’s a fragment of text.
Tell me how this fragment might <continue on in
this language, or suggest a particular image>.
According to your model of the statistics of
<human language, or human-handled images>,
what <words, or pixels> are likely to come next?”](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-11-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 12
How Generative AI works (admittedly oversimplified)
The prompts are converted into tokens (= “chunks” of words, punctuation marks,
pixels, etc.), then the system analyzes what is likely to come next, based on the
tokens in its own dataset (as many as 32,000 in GPT-4!).
It then generates a tokenized output.
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
Vector of
probabilities from
own tokens](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-12-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 13
How Generative AI works (admittedly oversimplified)
With each output, it keeps re-analyzing the probabilities to decide next tokens.
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and shopped
Vector of
probabilities from
own tokens](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-13-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 14
HERE’S THE REALLY COOL PART!!!
Transformers (the “T in “GPT”) know how to direct attention to specific parts
of the input to guide their selection of the output - such as verb tenses, objects.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/1706.03762
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and shopped
Vector of
probabilities from
own tokens](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-14-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 15
How Generative AI works (admittedly oversimplified)
The system can give you different answers to the same inputs:
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and bought
Vector of
probabilities from
own tokens](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-15-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 16
How Generative AI works (admittedly oversimplified)
The system can give you different answers to the same inputs:
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and clocked in
Vector of
probabilities from
own tokens](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-16-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 17
How Generative AI works (admittedly oversimplified)
The system can give you different answers to the same inputs:
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and danced
Vector of
probabilities from
own tokens
huh?](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-17-2048.jpg)
![An Introduction to Generative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 18
How Generative AI works (admittedly oversimplified)
“Hallucinations” - when the output doesn’t seem to make sense - are why it is
important not to accept everything it outputs at face value.
Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551
Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/
n tokens in 1 token out
She went to the store and danced
Vector of
probabilities from
own tokens
huh?](https://crownmelresort.com/image.slidesharecdn.com/anintroductiontogenerativeai-230517181740-5bb25b49/75/An-Introduction-to-Generative-AI-May-18-2023-18-2048.jpg)
More Related Content
Similar to An Introduction to Generative AI - May 18, 2023
An Introduction to Generative AI - May 18, 2023
- 1. An Introduction to GenerativeAI Cori Faklaris Assistant Professor, Dept. of Software and Information Systems Charlotte AI Institute for Smarter Learning, UNC Charlotte Dubois Center, May 18, 2023
- 2. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 2 Key takeaways from this talk ● Generative AI tools are great for PRODUCTIVITY - they can be nifty shortcuts to dispose of low-value tasks and / or to jumpstart creativity ● Generative AI tools should always be used - and taught to be used - with a critical mind, because they are prone to mistakes and “hallucinations”
- 3.
- 4. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 4 Lots of hype - and doom /gloom - around AI right now … It’s difficult to know where to look or how to start to understand AI We tend to be afraid of things that we don’t understand #evilbrag
- 5. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 5 When you hear “AI,” think “statistical pattern-matching” ● Oracle describes AI this way: AI has become a catchall term for applications that perform complex tasks that once required human input, such as communicating with customers online or playing chess. The term is often used interchangeably with … machine learning (ML) and deep learning. Text from What is Artificial Intelligence (AI)? Oracle, n.d. Retrieved May 16, 2023 from https://www.oracle.com/artificial-intelligence/what-is-ai/ Image from Pattern Recognition. GeeksforGeeks. Retrieved May 16, 2023 from https://www.geeksforgeeks.org/pattern-recognition-introduction/ The data is “tokenized” (= made into “chunks” of words, punctuation marks, pixels, etc.) during this process - remember this for later
- 6. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 6 AI has been with us for years, whether “generative” or not Google and other search engines Social media recommendations Conversational user interfaces such as Siri and Alexa Sensor-informed driver assistants in cars and trucks “Auto-complete” and “smart replies” for email and text messaging Tiktok screenshots from J. D. Biersdorfer. 2022. The Latecomer’s Guide to TikTok. The New York Times. Retrieved May 16, 2023 from https://www.nytimes.com/2022/10/26/technology/personaltech/tiktok-guide-latecomers.html ADAS images from Wikipedia contributors. 2023. Advanced driver-assistance system. Wikipedia, The Free Encyclopedia. Retrieved from https://en.wikipedia.org/w/index.php?title=Advanced_driver-assistance_system&oldid=1150142876
- 7. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 7 Now, AI can synthesize part or all of a creative work ● McKinsey defines generative AI as: … Algorithms (such as ChatGPT) that can be used to create new content, including audio, code, images, text, simulations, and videos. Recent breakthroughs in the field have the potential to drastically change the way we approach content creation. Text and image from What is generative AI? McKinsey. Retrieved May 16, 2023 from https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-generative-ai
- 8. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 8 How Generative AI works (admittedly oversimplified) The system generates text or images using its previously built model of the statistical distributions of tokens (= “chunks” of words, punctuation marks, pixels, etc.) created from its very large training dataset. Image from Pattern Recognition. GeeksforGeeks. Retrieved May 16, 2023 from https://www.geeksforgeeks.org/pattern-recognition-introduction/ Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ Doc Chat Image
- 9. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 9 How Generative AI works (admittedly oversimplified) It might make mistakes or “hallucinate” based on the limitations of its process, but the output still might look like what you wanted. Ted Chiang’s analogy = “unreliable photocopier” or a “blurry JPEG” Ted Chiang. 2023. ChatGPT Is a Blurry JPEG of the Web. The New Yorker. Retrieved May 10, 2023 from https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ Doc Chat Image
- 10. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 10 How Generative AI works (admittedly oversimplified) Ted Chiang’s analogy = “unreliable photocopier” / blurry JPEG image https://commons.wikimedia.org/wik i/File:Blurry_eiffel.jpg - shared under CC-SA 4.0 license
- 11. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 11 How Generative AI works (admittedly oversimplified) We can ask it questions - but a very specific type of question known as prompts, following this structure: Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 “Here’s a fragment of text. Tell me how this fragment might <continue on in this language, or suggest a particular image>. According to your model of the statistics of <human language, or human-handled images>, what <words, or pixels> are likely to come next?”
- 12. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 12 How Generative AI works (admittedly oversimplified) The prompts are converted into tokens (= “chunks” of words, punctuation marks, pixels, etc.), then the system analyzes what is likely to come next, based on the tokens in its own dataset (as many as 32,000 in GPT-4!). It then generates a tokenized output. Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ n tokens in 1 token out Vector of probabilities from own tokens
- 13. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 13 How Generative AI works (admittedly oversimplified) With each output, it keeps re-analyzing the probabilities to decide next tokens. Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ n tokens in 1 token out She went to the store and shopped Vector of probabilities from own tokens
- 14. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 14 HERE’S THE REALLY COOL PART!!! Transformers (the “T in “GPT”) know how to direct attention to specific parts of the input to guide their selection of the output - such as verb tenses, objects. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/1706.03762 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ n tokens in 1 token out She went to the store and shopped Vector of probabilities from own tokens
- 15. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 15 How Generative AI works (admittedly oversimplified) The system can give you different answers to the same inputs: Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ n tokens in 1 token out She went to the store and bought Vector of probabilities from own tokens
- 16. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 16 How Generative AI works (admittedly oversimplified) The system can give you different answers to the same inputs: Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ n tokens in 1 token out She went to the store and clocked in Vector of probabilities from own tokens
- 17. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 17 How Generative AI works (admittedly oversimplified) The system can give you different answers to the same inputs: Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ n tokens in 1 token out She went to the store and danced Vector of probabilities from own tokens huh?
- 18. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 18 How Generative AI works (admittedly oversimplified) “Hallucinations” - when the output doesn’t seem to make sense - are why it is important not to accept everything it outputs at face value. Murray Shanahan. 2022. Talking About Large Language Models. arXiv [cs.CL]. Retrieved from http://arxiv.org/abs/2212.03551 Bea Stollnitz. How generative language models work. Retrieved May 10, 2023 from https://bea.stollnitz.com/blog/how-gpt-works/ n tokens in 1 token out She went to the store and danced Vector of probabilities from own tokens huh?
- 19. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 19 Examples of publicly available Generative AI tools Crowdsourced list of available AI tools: https://bit.ly/UsefulLLMs
- 20. Using Generative AIas an academic
- 21. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 21 Use DALL-E 2 to create images for course slides - Goal 1: Quickly source visuals that add interest and reinforce content - Goal 2: Demonstrate limits of AI output with limited inputs or prompts
- 22. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 22 Use ChatGPT to create first draft of biography text - Goal 1: Cut the time spent on low-value but necessary job tasks - Goal 2: Goof around with fellow academics on social media
- 23. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 23 Use BingChat to draft a grant proposal - Goal 1: Overcome “analysis paralysis”, make yourself laugh in the process - Goal 2: Experiment with a sequence of prompts for sophisticated outputs
- 24. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 24 Assign students to pick/use a tool, then critique the output - Goal 1: Give permission and encouragement to play around with new tech - Goal 2: Mentor class members in how to think critically use of AI tools
- 25. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 25 My syllabus policy on “Use of AI and Other Creative Tools” In this course, students are permitted to use tools such as Stable Diffusion, DALL-E, ChatGPT, and BingChat. In general, permitted use of such tools is consistent with permitted use of non-AI assistants such as Grammarly, templating tools such as Canva, or images or text sourced from the internet or others’ files. No student may submit an assignment or work on an exam as their own that is entirely generated by means of an AI tool. If students use an AI tool or other creative tool to generate, draft, create, or compose any portion of any assignment, they must (a) credit the tool, (b) identify what part of the work is from the AI tool and what is from themselves, and (c) briefly summarize why they decided to use the tool and include its output. Cori Faklaris. 2023. Policy on Use of AI Tools for my course syllabus, version 1.0. Cori Faklaris’ blog – HeyCori. Retrieved May 16, 2023 from https://blog.corifaklaris.com/2023/03/17/policy-on-use-of-ai-tools-for-my-course-syllabus-version-1-0/#.ZGPtWezMJqs
- 26. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 26 Some actual and/or realistic risks of using generative AI ● Violations of data privacy ○ Some students told me they do not feel comfortable giving up any data to such services, such as may be required for creating an account. For these students, I created an alternate assignment for Slide 24, using a search engine. ● Violations of intellectual property ○ Check the Terms of Service - will your inputs or prompts be used as training data? ● Violations of academic integrity ○ Do a spot check of outputs, using a search engine, to see if any are wholly from another work ○ Analyze submitted work using Open AI’s AI Text Classifier or the multi-service GPTZero
- 27. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 27 Humans’ #1 skill set will continue to be communication Screenshot from https://twitter.com/TheRealOllieLaw/status/ 1656605938374307840?s=20
- 28. An Introduction toGenerative AI | Cori Faklaris, Assistant Professor, Dept. of Software and Info. Systems | Page 28 Key takeaways ● Generative AI tools can be nifty shortcuts to dispose of low-value tasks and / or to jumpstart creativity. ● Generative AI tools should always be used with your “thinking cap” on because they are prone to mistakes and “hallucinations.” Thank you for listening! Crowdsourced list of available AI tools: https://bit.ly/UsefulLLMs