Download as PDF, PPTX
Uploaded byMárton Kodok
Build applications with generative AI on Google Cloud
The document outlines a presentation on building applications with generative AI on Google Cloud, featuring the Gemini model and Vertex AI platform. It discusses multimodal capabilities, pricing, and various use cases, including function calling and integration options. The presentation emphasizes the ease of use for developers and the comprehensive model garden available for machine learning applications.
More Related Content
![[SHARED ONLINE] - GDG Cloud Iasi - Make it happen with Google AI - Sponsored ...](https://cdn.slidesharecdn.com/ss_thumbnails/sharedonline-gdgcloudiasi-makeithappenwithgoogleai-sponsoredbyqodea-250505062541-8c86ec1d-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Build applications with generative AI on Google Cloud
![[BuildWithAI] Introduction to Gemini.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/buildwithaiintroductiontogemini-240507191145-3c85d2d0-thumbnail.jpg?width=640&height=640&fit=bounds)
![[SHARED ONLINE] - GDG Cloud Iasi - Build with AI - Sponsored by Qodea.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/sharedonline-gdgcloudiasi-buildwithai-sponsoredbyqodea-250505062307-4cd143cd-thumbnail.jpg?width=640&height=640&fit=bounds)
Build applications with generative AI on Google Cloud
- 1. BuildapplicationswithgenerativeAI onGoogleCloud Márton Kodok Software Architectat REEA.net June 13-14, 2024 Barcelona
- 2. 1. Intro 2. Buildwith Gemini 3. What is Vertex AI? 4. Foundation Models + Model Garden 5. Multimodal use cases + Demo 6. Function calling with Gemini + Demo Agenda Build applications with generative AI on Google Cloud @martonkodok
- 3. ● Google DeveloperExpert on Cloud technologies (2016→) ● Champion of Google Cloud Innovators program (2021→) ● Among the Top 3 romanians on Stackoverflow 206k reputation ● Crafting Cloud Architecture+ML backends at REEA.net Articles: martonkodok.medium.com Twitter: @martonkodok Slideshare:martonkodok StackOverflow: pentium10 GitHub: pentium10 Build applications with generative AI on Google Cloud @martonkodok About me
- 4.
- 5.
- 6.
- 7.
- 8. @martonkodok list the top10 incidents from email for service name "trademark", group results in table
- 9.
- 10. @martonkodok find all emailsfrom this year from Hidroelectrica that contain "factura" and list all invoice values for this year, with dates, in a table, order by descending
- 11.
- 12. @martonkodok Gemini comes indifferent sizes Gemini 1.0 Pro Multimodal reasoning across a wide range of tasks GA Gemini 1.5 Flash fastest, most cost-efficient model yet for high volume tasks GA Gemini 1.5 Pro Multimodal reasoning for longer prompts, 1 million context window GA 1M high volume 1000 (RPM), lower latency and cost 1M Large prompt, 1000+ pages of PDF 32K Text, 16 images, 1 video
- 13. * May 2024@martonkodok A bit about pricing For 1M tokens Gemini 1.5 Flash Llama 3 70B C3 Haiku GPT-4o Input Token $0.35 - $0.70 $0.59 $0.25 $5.00 Output Token $1.05 - $2.10 $0.79 $1.25 $15.00 1 million tokens (for prompts up to 128K tokens | for prompts longer than 128K)
- 14. @martonkodok Where to use Buildwith Gemini Google AI Studio aistudio.google.com - API key - 60 QPM - free outside of EU - Libraries in Python/Go/Node/Dart/Android console.cloud.google.com - service account key - Vertex AI SDK - LangChain integration + libraries - access to 120 models via Model Garden - available for individual/edu/business Vertex AI Studio
- 15. “VertexAI is theGoogle Cloud product group for ML End-to-end ML Platform @martonkodok
- 16. Gen AI andModel Garden on Vertex AI @martonkodok Google Cloud AI Portfolio Foundation Models Vertex AI End-to-End ML Platform Text Chat Code Image Video Google Cloud Infrastructure - GPUs/TPUs Contact Center AI Healthcare AI Discovery AI Document AI Business Users AI Practitioners Developers Audio and Music Vertex AI Studio Generative AI APIs Model Garden Gemini Business Google Workspace Duet Al for Google Cloud AI Platform: Fine tuning | Distill | Serve
- 17.
- 18. Build applications withgenerative AI on Google Cloud @martonkodok Model Garden Task Specific AutoML and APIs Open Source Models Foundation Models Model Garden
- 19. Build applications withgenerative AI on Google Cloud @martonkodok Model Garden Foundation Models Multi-task Large-scale Minimal training
- 20. Build applications withgenerative AI on Google Cloud @martonkodok Language Foundation Models Gemini 1.0 Pro Multimodal reasoning across a wide range of tasks Gemini 1.5 Flash fastest, most cost-efficient model yet for high volume tasks Gemini 1.5 Pro Multimodal reasoning for longer prompts, 1 million context window Imagen 2 Generate images from Text prompts Multimodal Embeddings Extract semantic information Chirp for Speech to Text Build voice enabled applications
- 21. Build applications withgenerative AI on Google Cloud @martonkodok Introduction Generative AI Studio Console-tool Test generative models Rapid prototyping
- 22. Build applications withgenerative AI on Google Cloud @martonkodok Generative AI Studio
- 23. Build applications withgenerative AI on Google Cloud @martonkodok Custom language Prompt samples 1. Summarization 2. Classification 3. Extraction 4. Writing 5. Ideation cloud.google.com/vertex-ai/docs/generative-ai/learn/prompt-samples
- 24.
- 25. Build applications withgenerative AI on Google Cloud @martonkodok Prompt structure 1. Text: 2. Question: 3. Answer: 4. Categories: 5. Options:
- 26. Build applications withgenerative AI on Google Cloud @martonkodok Best practices for the contextfield of your prompt
- 27.
- 28. Build applications withgenerative AI on Google Cloud @martonkodok Introduction Multi model Image Video Text Function Calling
- 29. Build applications withgenerative AI on Google Cloud @martonkodok Multi model use cases 1. Info seeking 2. Object recognition 3. Digital content understanding 4. Structured content generation 5. Captioning / description 6. Extrapolation
- 30. Build applications withgenerative AI on Google Cloud @martonkodok Multi model use cases
- 31.
- 32. Build applications withgenerative AI on Google Cloud @martonkodok Multimodal - Embeddings Multi model - Embeddings Image Video Text Audio
- 33. Build applications withgenerative AI on Google Cloud @martonkodok Language Foundation Models Gemini 1.0 Pro Multimodal reasoning across a wide range of tasks Gemini 1.5 Flash fastest, most cost-efficient model yet for high volume tasks Gemini 1.5 Pro Multimodal reasoning for longer prompts, 1 million context window Imagen 2 Generate images from Text prompts Multimodal Embeddings Extract semantic information Chirp for Speech to Text Build voice enabled applications
- 34. Google Cloud Next‘24 Proprietary Joint Embedding Vector Space Image: “gray tabby cat laying in front of a Christmas tree” Text: size color living Multimodal embeddings ● Embed text, image, and video in the same semantic space with the same dimensionality ● Applications: image/video content search, classification, recommendation
- 35. 035 Google Cloud Next‘24 Proprietary Resources BQ Embedding Generation Documentation | Blog | Video BQ Vector Search Documentation | Blog| LangChain Demo Assets BQ LangChain Notebook | Gen AI Repo Vector search and multimodal embeddings in BigQuery goo.gle/next24-ana302
- 36. “ Cloud Next’24demo app in action @martonkodok
- 37.
- 38. Build applications withgenerative AI on Google Cloud @martonkodok Function calling in Gemini A framework to connect LLMs to real-time data delegate certain data processing tasks to functions
- 39.
- 40. Gen AI …repository Build applications with generative AI on Google Cloud @martonkodok
- 41. @martonkodok Sample code forGenAI on Google Cloud goo.gle/gen-ai-github
- 42. Build applications withgenerative AI on Google Cloud @martonkodok https://goo.gle/gen-ai-github -> Gemini folder
- 43.
- 44.
- 45. Build applications withgenerative AI on Google Cloud @martonkodok Open Source models Llama 2 on Vertex AI Meta's Llama 2 models on Vertex AI Code Llama Designed for general code synthesis and understanding, designed for Python Gemma 2B and 7B Family of lightweight, state-of-the-art open models Claude on Vertex AI Claude 2, Instant 1.2, and more
- 46. “ VertexAI enablesmodel inference via API Build applications with generative AI on Google Cloud @martonkodok
- 47. Build applications withgenerative AI on Google Cloud @martonkodok VertexAI:Managed unified ML platform Fine-tuning 1 click deploy
- 48. @martonkodok Explore AI modelswith Colab samples cloud.google.com/vertex-ai/docs/start/explore-models#test-promptsub
- 49. Conclusion Build applications withgenerative AI on Google Cloud @martonkodok
- 50. 1. Build withthe groundbreaking ML tools that power Google 2. Model Garden provides a curated collection of 120+ models 3. Approachable from the non-ML developer perspective (managed models, fine-tuning training) 4. Accelerate ML with tooling for pre-trained, open source and custom models 5. Deploy to applications with just one-click 6. End-to-end integration for data and AI with build pipelines that outperform and solve complex ML tasks Vertex AI: Enhanced ML developer experience Build applications with generative AI on Google Cloud @martonkodok
- 51. Twitter: @martonkodok Thank you.Q&A. Reea.net - Integrated web solutions driven by creativity to deliver projects. Follow for articles: martonkodok.medium.com Slides available on: slideshare.net/martonkodok