Download as PDF, PPTX
Uploaded byTimothy Spann
2024 Nov 05 - Linux Foundation TAC TALK With Milvus
The document introduces Milvus, an open-source vector database designed to manage and utilize embedding vectors generated by deep learning models. It highlights key features, deployment options, and various use cases including semantic search and anomaly detection, while emphasizing its scalable architecture optimized for high-dimensional vector data. Additionally, it provides insights into integration frameworks and tools for managing and monitoring Milvus deployments.
More Related Content
Similar to 2024 Nov 05 - Linux Foundation TAC TALK With Milvus
2024 Nov 05 - Linux Foundation TAC TALK With Milvus
- 1. Talk with Milvus TimSpann @ Zilliz
- 2. 2 | ©Copyright 10/22/23 Zilliz 2 | © Copyright 2024 Zilliz Tim Spann Principal Developer Advocate Zilliz tim.spann@zilliz.com https://www.linkedin.com/in/timothyspann/ https://x.com/PaaSDev
- 3. 3 | ©Copyright 8/16/23 Zilliz 3 | © Copyright 8/16/23 Zilliz Introduction
- 4. 4 | ©Copyright 9/27/23 Zilliz 4 | © Copyright 9/27/23 Zilliz Vector search is the new paradigm
- 6. or Apple or Rising dough or Change cartire Rising Dough Proofing Bread ✔ ❌ Why is Semantic Search Difficult?
- 7. Why is SemanticSearch Important? 10% Other newly generated data in 2025 will be unstructured data 90% Data Source: The Digitization of the World by IDC
- 8.
- 9. New Challenge: Searchin Vector Spaces How to Index and Search? ● High-dimensional ● > 1000 dims How to Scale? ● 10-100 million vectors? ● Billions? ● Trillions? ● Billions of users? Multiple Data Types? ● Text ● Images ● Audio ● Graphs ● …
- 10. | © Copyright9/25/23 Zilliz 10 Milvus is an Open-Source Vector Database to store, index, manage, and use the massive number of embedding vectors generated by deep neural networks and LLMs. contributors 400 stars 30K docker pulls 66M forks 2.7K + Milvus: High-performance, scalable vector database
- 11. Integrations Framework Hardware Infrastructure Embedding Models LLMs SoftwareInfrastructure Vector Database
- 12.
- 13. How Does SimilaritySearch Work?
- 14. Benchmarks Shows 3-20x fastercomparing with open source Milvus At least 6x faster than other vector databases https://github.com/zilliztech/VectorDBBench
- 15. Rich functionality Bulk ImportGPU, Intel & ARM CPU support Disk Based Index Tiered Storage Million+ level tenant support Hybrid Search Dense & Sparse RBAC, TLS, Encryption Float, Binary, & Sparse Vector Tag+Vector Optimized Filtering Dynamic Schema
- 16. 16 | ©Copyright 8/16/23 Zilliz 16 | © Copyright 8/16/23 Zilliz Getting Started
- 17. Deployment Options Milvus Lite ●Locally hosted ● Suitable for prototyping and demos Milvus Standalone ● Single remote/local server ● “Medium” scale ● Simplified setup, maintenance, etc. compared to cluster Milvus Cluster ● Distributed system ● Many different types of nodes ● Scales to 100s of billions of vectors
- 19. 19 | ©Copyright 8/16/23 Zilliz 19 | © Copyright 8/16/23 Zilliz Explore Tools
- 20. Attu is anall-in-one milvus administration tool.
- 21. Attu – GUIInterface
- 22.
- 23. Attu – hybridsearch
- 24.
- 25. https://github.com/zilliztech/milvus_cli Milvus_CLI🚀 ● CLI ● Admin ●View ● Query ● Search https://milvus.io/docs/cli_commands.md
- 26.
- 27. https://milvus.io/tools/sizing Sizing Tool Good forsizing your first production cluster For early workloads this is more than you need.
- 28. https://github.com/zilliztech/milvus-backup Backup ● Swagger RESTAPI ● CLI ● API ● Backup ● Restore
- 29. 29 | ©Copyright 8/16/23 Zilliz 29 | © Copyright 8/16/23 Zilliz Users and Use Cases
- 30. | © Copyright9/25/23 Zilliz 30 Retrieval Augmented Generation RAG Expand LLMs' knowledge by incorporating external data sources into LLMs and your AI applications Match user behavior or content features with other similar ones to make effective recommendations Recommender System RecSys Search for semantically similar texts across vast amounts of natural language documents Text/Semantic Similarity Search Molecular Similarity Search Search for similar substructures, superstructures, and other structures for a specific molecule Fraud & Anomaly Detection Detect data points, events, and observations that deviate significantly from the usual pattern Multimodal Similarity Search Search over multiple types of data simultaneously, e.g. text, audio, images, video Milvus powers search across various types of apps
- 31.
- 32.
- 33. 33 | ©Copyright 8/16/23 Zilliz 33 | © Copyright 8/16/23 Zilliz Architecture
- 34. Milvus 🤝 Open-Source MINIO StoreVectors and Indexes Enables Milvus’ stateless architecture Kafka/ Pulsar Handles Data Insertion stream Internal Component Communications Real-time updates to Milvus Prometheus / Grafana Collects metrics from Milvus Provides real-time monitoring dashboards Kubernetes Milvus Operator CRDs
- 35.
- 36. Stateless Architecture Stateless ComponentsAll Milvus components are deployed Stateless. Object Storage Milvus relies on Object Storage (MinIO, S3, etc) for data persistence. Vectors are stored in Object Storage, Metadata is in etcd. Scaling and Failover Scaling and failover don't involve traditional data rebalancing. When new pods are added or existing ones fail, they can immediately start handling requests by accessing data from the shared object storage.
- 37.
- 38. ● Subscribe tothe log broker for real-time querying ● Convert new data into Growing Segments - temporary in-memory structures for the latest information. ● Access Sealed Segments from object storage for comprehensive searches. ● Perform hybrid searches combining vector and scalar data for accurate retrieval. Query Node: Serving Search Requests
- 39.
- 40. Array Fields • Addarray field • Search array field in filter Support functions like ARRAY_CONTAINS、ARRAY_CONTAINS_ALL、ARRAY_CONTAINS_ANY、 ARRAY_LENGTH • Support inverted index or bitmap index to accelerate search performance
- 41. JSON Fields • Addarray field • Search array field in filter Support functions like JSON_CONTAINS、JSON_CONTAINS_ALL、 JSON_CONTAINS_ANY • Support inverted index for specific key WIP
- 42. Partition • Partition cando help for data pruning during search, which can boost read performance • User need to manage it manually, create/drop partition and insert/search with explicitly define partition option, which introduce complexity
- 43. Partition Key • Hashbased on the key values, manage partition inside automatically make it easier for user to use • Default 64 partitions by hash
- 44. Replica • Improve readQPS • Provide higher availability
- 45. 45 | ©Copyright 8/16/23 Zilliz 45 | © Copyright 8/16/23 Zilliz Integration
- 46. Framework Hardware Infrastructure Embedding Models LLMs SoftwareInfrastructure Vector Database Integration Integration
- 47.
- 48. Contributing Your FirstIssue You can also contribute to Milvus directly: • Issues · milvus-io/milvus · GitHub by following the contribution instructions, and see • Contributing to Open Source Milvus: A Beginner’s Guide
- 49. 49 | ©Copyright 10/22/23 Zilliz 49 | © Copyright 2024 Zilliz Milvus Open Source Self-Managed Zilliz Cloud SaaS Fully-Managed github.com/milvus-io/milvus Getting Started with Vector Databases zilliz.com/cloud
- 50.