MongoDB, profile picture
Uploaded byMongoDB
PPTX, PDF245 views

MongoDB.local DC 2018: Tips and Tricks for Avoiding Common Query Pitfalls

The document discusses strategies for optimizing query performance in MongoDB, emphasizing the importance of indexes for both retrieval and update operations. It covers various index types, performance implications of different query structures like $and vs. $or, and practical examples of creating and using indexes to improve execution efficiency. Additionally, it highlights the role of the performance advisor in identifying slow queries and suggests best practices for managing indexes effectively.

Embed presentation

Downloaded 14 times
1 / 142
2 / 142
3 / 142
4 / 142
5 / 142
6 / 142
7 / 142
8 / 142
9 / 142
10 / 142
11 / 142
12 / 142
13 / 142
14 / 142
15 / 142
16 / 142
17 / 142
18 / 142
19 / 142
20 / 142
21 / 142
22 / 142
23 / 142
24 / 142
25 / 142
26 / 142
27 / 142
28 / 142
29 / 142
30 / 142
31 / 142
32 / 142
33 / 142
34 / 142
35 / 142
36 / 142
37 / 142
38 / 142
39 / 142
40 / 142
41 / 142
42 / 142
43 / 142
44 / 142
45 / 142
46 / 142
47 / 142
48 / 142
49 / 142
50 / 142
51 / 142
52 / 142
53 / 142
54 / 142
55 / 142
56 / 142
57 / 142
58 / 142
59 / 142
60 / 142
61 / 142
62 / 142
63 / 142
64 / 142
65 / 142
66 / 142
67 / 142
68 / 142
69 / 142
70 / 142
71 / 142
72 / 142
73 / 142
74 / 142
75 / 142
76 / 142
77 / 142
78 / 142
79 / 142
80 / 142
81 / 142
82 / 142
83 / 142
84 / 142
85 / 142
86 / 142
87 / 142
88 / 142
89 / 142
90 / 142
91 / 142
92 / 142
93 / 142
94 / 142
95 / 142
96 / 142
97 / 142
98 / 142
99 / 142
100 / 142
101 / 142
102 / 142
103 / 142
104 / 142
105 / 142
106 / 142
107 / 142
108 / 142
109 / 142
110 / 142
111 / 142
112 / 142
113 / 142
114 / 142
115 / 142
116 / 142
117 / 142
118 / 142
119 / 142
120 / 142
121 / 142
122 / 142
123 / 142
124 / 142
125 / 142
126 / 142
127 / 142
128 / 142
129 / 142
130 / 142
131 / 142
132 / 142
133 / 142
134 / 142
135 / 142
136 / 142
137 / 142
138 / 142
139 / 142
140 / 142
141 / 142
142 / 142
Tips and Tricks for Avoiding Common Query Pitfalls With emphasis on improving query performance
Gary Taylor Senior Solutions Architect MongoDB
• Motivation • Who am I • What we're going to talk about Roadmap
Motivation The Power of Query Optimization Query tuning results in: • Improved performance • Reduced resource utilization This may lead to: • Improved stability and predictability • A smaller hardware footprint • End-user happiness
• Senior Solutions Architect, MongoDB Federal • 3 year tenure with MongoDB • Where I've worked • Large scale data processing emphasis Who am I?
1. Indexes • What they are and why use them • Examples of using them o Single, Compound, Case-insensitive • Using "explain" 2. Blocking Stages • What they are • Methods to avoid them 3. $and vs. $or operator • Why so different in performance 4. Tools that can help Performance areas we're going to talk about
Here are my Documents HOW DO I FIND THE SPECIFIC DOCUMENT(S) I WANT?
USE AN INDEX IT IS A WAY TO LOCATE YOUR DATA QUICKLY
What are indexes? Indexes are special data structures that store a small portion of the collection’s data set in an easy to traverse form.
Why indexes? It’s about performance Indexes are the single biggest tunable performance factor for an application TIP: • Use for frequently accessed queries • Use when low latency response time needed
Why indexes? (cont.) • ⅔ of all performance problems are due to a missing or incorrect secondary index • No secondary index ⇒ full collection scan
MongoDB Indexes Some of the indexes MongoDB supports Single-field indexes Compound-field indexes Multi-Key indexes Geospatial indexes Text indexes TTL indexes Hashed indexes
Show Me BASIC MONGODB INDEXING EXAMPLES
Simple example We have a twitter database that has a tweets collection in MongoDB that contains a set of tweets and we run a query. > use twitter > db.tweets.find( { "user.followers_count" : 1000 } )
How does our query behave? We can see what the query does by using ".explain(…)" > db.tweets.find( { "user.followers_count" : 1000 } ).explain("executionStats") • TIP: Using "executionStats", MongoDB runs the query optimizer to choose the winning plan, executes the winning plan to completion, and returns statistics describing the execution of the that plan.
What’s this explain() do? The most important values we we’re interested for indexing in are • nReturned : number of documents returned by the query • totalDocsExamined : number of documents touched during the query • totalKeysExamined : number of index keys scanned TIP: A totalKeysExamined or totalDocsExamined value much higher than nReturned indicates we could use or need a better index.
How does our query behave? … "executionStats" : { "executionSuccess" : true, "nReturned" : 8, "executionTimeMillis" : 107, "totalKeysExamined" : 0, "totalDocsExamined" : 51428, "executionStages" : { "stage" : "COLLSCAN", "filter" : { "user.followers_count" : { "$eq" : 1000 } }, "nReturned" : 8, "executionTimeMillisEstimate" : 100, … "direction" : "forward", "docsExamined" : 51428 } ... } Ugh! We just did a “full collection scan” of 51428 documents to find 8 documents.
Can we do better for reading? We put a single-field index on the tweets collection and see if that improves our performance. > db.tweets.createIndex( { "user.followers_count" : 1 } ) > db.tweets.find( { "user.followers_count" : 1000 } ).explain("executionStats")
How does our read query behave with this index? … "executionStats" : { "executionSuccess" : true, "nReturned" : 8, "executionTimeMillis" : 0, "totalKeysExamined" : 8, "totalDocsExamined" : 8, "executionStages" : { "stage" : "FETCH", "nReturned" : 8, "executionTimeMillisEstimate" : 0, "inputStage" : { "stage" : "IXSCAN", "nReturned" : 8, "invalidates" : 0, "direction" : "forward", "docsExamined" : 51428 … "indexName" : "user.followers_count_1" Wow! Our query used the index to find our 8 documents and didn't have to scan the whole collection.
Another example Now try a compound-field index. > db.tweets.createIndex( {"user.time_zone" : 1, "user.followers_count" : 1 } ) > db.tweets.find({"user.time_zone" : "Tokyo", "user.followers_count": 10}).explain("executionStats")
How does our read query behave? … "executionStats" : { "executionSuccess" : true, "nReturned" : 17, "executionTimeMillis" : 5, "totalKeysExamined" : 17, "totalDocsExamined" : 17, "executionStages" : { "stage" : "FETCH", "nReturned" : 17, "executionTimeMillisEstimat e" : 0, "works" : 19, "advanced" : 17, "needTime" : 0, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "docsExamined" : 17, "alreadyHasObj" : 0, "inputStage" : { "stage" :"IXSCAN", "nReturned" : 17, … "indexName" : "user.time_zone_1_user.followers_cou nt_1", … Still looking good!
Do I have to do exact match queries? NO! You can do range queries. > db.tweets.find( { "user.followers_count" : {$gte : 10, $lte : 100} } ).explain("executionStats")
How does our range query behave? … "executionStats" : { "executionSuccess" : true, "nReturned" : 17643, "executionTimeMillis" : 21, "totalKeysExamined" : 17643, "totalDocsExamined" : 17643, "executionStages" : { "stage" : "FETCH", "nReturned" : 17643, "executionTimeMillisEsti mate" : 22, "docsExamined" : 17643, "alreadyHasObj" : 0, "inputStage" : { "stage" : "IXSCAN", "nReturned" : 17643, … "indexName" : "user.followers_count_1" … Still looking good!
Indexes for writing? Can indexes improve write performance? > db.tweets.explain("executionStats") .update({"user.followers_count" : 1000}, {$set : {"large_following" : true}})
How does our update query behave? … "executionStats" : { "executionSuccess" : true, "nReturned" : 0, "executionTimeMillis" : 0, "totalKeysExamined" : 8, "totalDocsExamined" : 8, "executionStages" : { "stage" : "UPDATE", "nReturned" : 0, "executionTimeMillisEsti mate" : 0, … "inputStage" : { "stage" : "IXSCAN", "nReturned" : 8, … "indexName" : "user.followers_count_1", Still looking good!
What about case-insensitivity? What happens if you search for "TOKYO" vs. "Tokyo"? > db.tweets.find({"user.time_zone" : "Tokyo", "user.followers_count": {$gte : 10, $lte : 100}}).count() 841 > db.tweets.find({"user.time_zone" : "TOKYO", "user.followers_count": {$gte : 10, $lte : 100}}).count() 0 You get different results! This query did not find any matching documents!
Creating Case-insensitive indexes? TIP: To use a case insensitive index on a collection with no default collation, create an index with a collation and set the strength parameter to 1 or 2. > db.tweets.createIndex( {"user.time_zone" : 1, "user.followers_count" : 1}, { name : "user_timezone_user_followers_1_insensitive", collation: { locale : 'en', strength : 2 } } ) > db.tweets.find({"user.time_zone" : "Tokyo", "user.followers_count": {$gte : 10, $lte : 100}}).count() 841 > db.tweets.find({"user.time_zone" : "TOKYO", "user.followers_count": {$gte : 10, $lte : 100}}).collation({locale : "en", strength : 2}).count() 841 Now, the results are the same when using the case-insensitive index we created.
Deleting indexes Listing the indexes to get index name > db.tweets.getIndexes() [ {"v" : 2,"key" : {"_id" : 1},"name" : "_id_","ns" : "twitter.tweets"}, {"v" : 2,"key" : {"user.followers_count" : 1},"name" : "user.followers_count_1","ns" : "twitter.tweets"}, {"v" : 2, "key" : { "user.time_zone" : 1, "user.followers_count" : 1 }, "name" : "user.time_zone_1_user.followers_count_1", "ns" : "twitter.tweets" } ]
Deleting indexes Here is how to remove an existing index. > db.tweets.dropIndex("user.followers_count_1") { "nIndexesWas" : 3, "ok" : 1 } > db.tweets.getIndexes() [ {"v" : 2,"key" : {"_id" : 1},"name" : "_id_","ns" : "twitter.tweets"}, {"v" : 2, "key" : { "user.time_zone" : 1, "user.followers_count" : 1 }, "name" : "user.time_zone_1_user.followers_count_1", "ns" : "twitter.tweets" } ]
TIP: Delete indexes when they are not used by your applications for given queries. Indexes aren't a free lunch as they • Consume space • Affect write operations Why delete indexes?
REMEMBER EASY AS …
Index Early TIP: Indexes deserve first-tier consideration in the design process. Efficiency at the data access level has historically been offloaded to a DBA-like role which prompts the kind of post-design optimization layers that the document-oriented database stack still has the opportunity to avoid.
TIP: Indexed queries perform better by several orders of magnitude, even on small data. While an un-indexed query may take 10 seconds, the same query can take as little as ~0 milliseconds given a proper index. Index Often
Queries generally make use of indexes from left to right. TRICK: An compound index can be utilized to the extent that a query searches a "prefix" subset of those fields.  db.tweets.createIndex( {"user.followers_count" : 1, "user.lang" : 1, "user.time_zone" : 1} )  db.tweets.find({"user.time_zone" : "Tokyo", "user.followers_count": {$gte : 10, $lte : 100}, "user.lang" : "en"}) ✖ db.tweets.find({"user.time_zone" : "Tokyo", "user.lang" : "en"})  db.tweets.find({ "user.followers_count": {$gte : 10, $lte : 100}})  db.tweets.find({ "user.followers_count": {$gte : 10, $lte : 100}, "user.time_zone" : "en"}) ✖ db.tweets.find({ "user.lang" : "en", "user.time_zone" : "Tokyo"})  Uses the index ✖ Does not use the index Index Fully
Leverage Index Sorts • TIP: If your queries will contain a sort, add the sorted field to the end of your index. Useful Commands • .explain("executionStats") ➡ its output can be analyzed to see if an index might be appropriate ➡ its output will show what index (if any) has been used for a given query • .createIndex(…), .dropIndex(…) ➡ are used for creating and dropping indexes • .getIndexes() or .getIndexKeys() ➡ will show you what indexes you have • .hint(…) ➡ allows you to indicate what index to use Other
Compound Indexing – Rules of Thumb Order fields in a compound index from most selective (most unique) to least selective (least unique) • Usually, this means equality fields (fields queried for a specific value) before range fields (e.g., date fields queried for a particular time period) TIP: When dealing with multiple equality values, start with the most selective Other .
Work Smarter Not Harder • Understand the business logic • Index appropriately • Is it the right index to support the query? • Return only the data you require • TIP: Leverage the Performance Advisor • The Performance Advisor (in Ops Mangager and Atlas), monitors queries that MongoDB considers slow and suggests new indexes to improve query performance.
Blocking Operations
Acme Games Introduces... ShortFite! Brand new Battle Royale game Launching October 31st
• Game nearly complete • Developers have learned a lot from their MongoDB champion Stakeholder Concerns • App being stress tested • Concerns over current performance 
Stakeholder Concern #1 Developers created index db.games.createIndex({ gamerTag: 1 }) This query takes several seconds to execute: db.games.find( { gamerTag: "Ace" } ).sort({score:-1}) Adding an index on score does not help! db.games.createIndex({ score: -1 })
Blocking
Blocking Operation ● Formally: ■ “An operation which must process all input before it can begin to produce any output.” ● Opposite of the often desirable “fully pipelined” plan which can stream results back as soon as they are found. ● TIP: Commonly observed when a sort is added to a query
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting with blocking
Sorting without blocking
Sorting without blocking
Sorting without blocking
Sorting without blocking
Blocking Stages • $sort • In aggregation and find • $group • $bucket • $count • $facet
Working with blocking stages For sorting: TRICK: Add a supporting index Worth the overhead in almost all circumstances For other stages: Determine if you really need the blocking stage? TRICK: Offload to secondary member (where possible)
Original performance of db.games.find( { gamerTag: "Ace" } ).sort({score:-1}) without an index is poor. “Clearly MongoDB is not webscale!” Example #1 (recap)
Adding an index makes it fast db.games.find( { gamerTag: "Ace" } ).sort({score:-1}) db.games.createIndex({ gamerTag: 1, score:-1 }) db.games.find( { gamerTag: "Ace" } ).sort({score:-1}) "That’ll work great!” Example #1 (recap)
The $and version of a query returns quickly: db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) But the $or version is slow: db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Stakeholder Concern #2
The $and version of a query returns quickly: db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) But the $or version is slow: db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) But we just created an index with both those fields… Isn't it being used with $or? Stakeholder Concern #2
Why is $and grand but $or poor?
$and example Query on games: db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Matching games: { gamerTag: "Ace", score: 9500 } Non-matching games: { gamerTag: "Ace", score: 500 }, { gamerTag: "Bob", score: 9500 }, { gamerTag: "Bob", score: 500 }
Groups of documents score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 }
$and Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) { gamerTag: "Bob", score: 500 }
$and Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) { gamerTag: "Bob", score: 500 }
$and Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$and Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$and Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$and Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$and Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$and Venn Diagram (logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1 , score:-1} 5009500 9500 500
$and Venn Diagram (logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 9500 500 "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000.0)" ] } {gamerTag:1 , score:-1}
$and Venn Diagram (logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000.0)" ] }
$and Venn Diagram (logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000.0)" ] }
$and Venn Diagram (logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 5009500 9500 500 {gamerTag:1 , score:-1} "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000.0)" ] }
$or example Query on games: db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Matching games: { gamerTag: "Ace", score: 9500 }, { gamerTag: "Ace", score: 500 }, { gamerTag: "Bob", score: 9500 } Non-matching games: { gamerTag: "Bob", score: 500 }
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or Venn Diagram (logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or (single) Index visualization Ace Bob {gamerTag:1 , score:-1} 9500 500 9500 500 db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or (single) Index visualization Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1 , score:-1} 9500 500 9500 500
$or (single) Index visualization Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or (single) Index visualization Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or (single) Index visualization Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or (single) Index visualization Ace Bob 9500 500 5009500 {gamerTag:1 , score:-1} Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or (single) Index visualization Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } Actual (Hinted) Index Bounds: "indexBounds" : { "gamerTag" : [ "[MinKey, MaxKey]" ], "score" : [ "[MaxKey, MinKey]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 5009500 {gamerTag:1 , score:-1}
$or (single) Index visualization Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } Actual (Hinted) Index Bounds: "indexBounds" : { "gamerTag" : [ "[MinKey, MaxKey]" ], "score" : [ "[MaxKey, MinKey]" ] } So is there anything we can do to improve the performance of this query? db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 5009500 {gamerTag:1 , score:-1}
TIP Use multiple indexes! db.data.createIndex({gamerTag: 1}) db.data.createIndex({score: -1})
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1 , score:-1} 9500 500 9500 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1}
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1} 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1} 9500 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) {gamerTag:1} {score:-1} Ace Bob 9500 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
$or (multiple) Index visualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
TIP Use multiple indexes! db.data.createIndex({gamerTag: 1}) db.data.createIndex({score: -1}) But we already have the {gamerTag:1, score:-1} index. Do we need both of these new ones?
TIP Use multiple indexes! db.data.createIndex({gamerTag: 1}) db.data.createIndex({score: -1}) But we already have the {gamerTag:1, score:-1} index. Do we need both of these new ones? No we don't.
TIP Use multiple indexes! db.data.createIndex({gamerTag: 1}) db.data.createIndex({score: -1}) Works with sorting Generate a SORT_MERGE plan
ENTERPRISE TOOLS WHAT'S AVAILABLE TO HELP
Compass THE GUI FOR MONGODB  VISUALLY EXPLORE YOUR DATA  RUN AD HOC QUERIES IN SECONDS  INTERACT WITH YOUR DATA WITH FULL CRUD FUNCTIONALITY  VIEW AND OPTIMIZE YOUR QUERY PERFORMANCE
MongoDB Compass Debug and optimize: Visual explain plans Understand how queries are running using a GUI that allows you to easily identify and resolve performance issues. • View key information about the execution plan of a query • Visualize different explain stages in an easy-to-understand tree format Visual explain plans are currently in beta.
Does this query look familiar? Notice these 3 values? What do they indicate?
MongoDB Compass Debug and optimize: Index details MongoDB Compass allows you to view index details for a given collection: • Type of index: regular, text, geospatial, or hashed • Size of index: how much space the index uses • Index utilization: how many times the index has been used • Special properties: unique index, compound index, etc.
Ops/Cloud Manager THE BEST WAY TO RUN MONGODB Has lots of functionality … including Query Optimization to identify slow-running queries, get index suggestions, automate index builds.
Visual Query Profiler Identify your slow-running queries with the click of a button
Index Suggestions Index recommendations to improve your deployment
• INDEXES ARE A GREAT WAY TO IMPROVE MONGODB QUERY PERFORMANCE  Remember that roughly 2/3 of performance problems are due to missing or incorrect indexes  Use "explain" to see what (if any) indexes are being used • INDEXES ARE NOT A PANACEA FOR ALL PERFORMANCE OR DESIGN ISSUES  For example, they are not a substitute for poor schema design (a band-aid at most) • INDEXES HAVE PRICE TO PAY IN OPERATIONS AND STORAGE; USE THEM WISELY  As your data changes the indexes have to be updated – but automatically  They take up space • BLOCKING STAGES CAN IMPACT PERFORMANCE  Use indexes where possible, AVOID when Possible  Secondary nodes may be a solution • DIFFERENT OPERATORS OR STAGES IN MONGODB CAN USE (OR NOT USE) INDEXES DIFFERENTLY  Different or additional indexes may help • THERE ARE MONGODB TOOLS THAT CAN HELP
WHAT WE WANT OUR DATABASE TO BE!If your query performance is poor, there is likely something that you can do to improve it; Sometimes by 99%
WRAPUP WHAT ELSE WOULD YOU LIKE TO KNOW?
Thank You!

More Related Content

PDF
Indexing and Query Optimizer (Mongo Austin)
byMongoDB
 
PPT
Fast querying indexing for performance (4)
byMongoDB
 
PPTX
Indexing & Query Optimization
byMongoDB
 
PPTX
Indexing and Query Optimization
byMongoDB
 
PPTX
Reducing Development Time with MongoDB vs. SQL
byMongoDB
 
PPTX
Indexing with MongoDB
byMongoDB
 
PPTX
Webinar: Index Tuning and Evaluation
byMongoDB
 
PDF
MongoDB World 2016: Deciphering .explain() Output
byMongoDB
 
Indexing and Query Optimizer (Mongo Austin)
byMongoDB
 
Fast querying indexing for performance (4)
byMongoDB
 
Indexing & Query Optimization
byMongoDB
 
Indexing and Query Optimization
byMongoDB
 
Reducing Development Time with MongoDB vs. SQL
byMongoDB
 
Indexing with MongoDB
byMongoDB
 
Webinar: Index Tuning and Evaluation
byMongoDB
 
MongoDB World 2016: Deciphering .explain() Output
byMongoDB
 

What's hot

PPTX
MongoDB and Indexes - MUG Denver - 20160329
byDouglas Duncan
 
PPTX
Rx 101 Codemotion Milan 2015 - Tamir Dresher
byTamir Dresher
 
PPTX
Elastic search
byVinaykumar Hebballi
 
PPTX
Mythbusting: Understanding How We Measure the Performance of MongoDB
byMongoDB
 
PDF
Sanjar Akhmedov - Joining Infinity – Windowless Stream Processing with Flink
byFlink Forward
 
PDF
집단지성 프로그래밍 08-가격모델링
byKwang Woo NAM
 
PDF
MongoDB
byHemant Kumar Tiwary
 
PDF
Full Text Search In PostgreSQL
byKarwin Software Solutions LLC
 
PDF
Indexing and Performance Tuning
byMongoDB
 
PDF
Query planner
byMiguel Angel Nieto
 
PPTX
Indexing and Query Optimizer (Aaron Staple)
byMongoSF
 
PDF
Windows 8 Training Fundamental - 1
byKevin Octavian
 
PDF
webScrapingFunctions
byHellen Gakuruh
 
PDF
PostgreSQL: Advanced indexing
byHans-Jürgen Schönig
 
PPTX
Indexing Strategies to Help You Scale
byMongoDB
 
ODP
Mongo indexes
byparadokslabs
 
PDF
Gruter_TECHDAY_2014_01_SearchEngine (in Korean)
byGruter
 
PDF
Backendless apps
byMatteo Bonifazi
 
PDF
Webinar: Building Your First App with MongoDB and Java
byMongoDB
 
MongoDB and Indexes - MUG Denver - 20160329
byDouglas Duncan
 
Rx 101 Codemotion Milan 2015 - Tamir Dresher
byTamir Dresher
 
Elastic search
byVinaykumar Hebballi
 
Mythbusting: Understanding How We Measure the Performance of MongoDB
byMongoDB
 
Sanjar Akhmedov - Joining Infinity – Windowless Stream Processing with Flink
byFlink Forward
 
집단지성 프로그래밍 08-가격모델링
byKwang Woo NAM
 
MongoDB
byHemant Kumar Tiwary
 
Full Text Search In PostgreSQL
byKarwin Software Solutions LLC
 
Indexing and Performance Tuning
byMongoDB
 
Query planner
byMiguel Angel Nieto
 
Indexing and Query Optimizer (Aaron Staple)
byMongoSF
 
Windows 8 Training Fundamental - 1
byKevin Octavian
 
webScrapingFunctions
byHellen Gakuruh
 
PostgreSQL: Advanced indexing
byHans-Jürgen Schönig
 
Indexing Strategies to Help You Scale
byMongoDB
 
Mongo indexes
byparadokslabs
 
Gruter_TECHDAY_2014_01_SearchEngine (in Korean)
byGruter
 
Backendless apps
byMatteo Bonifazi
 
Webinar: Building Your First App with MongoDB and Java
byMongoDB
 

Similar to MongoDB.local DC 2018: Tips and Tricks for Avoiding Common Query Pitfalls

PDF
Indexing and Query Performance in MongoDB.pdf
byMalak Abu Hammad
 
PPTX
Query Optimization in MongoDB
byHamoon Mohammadian Pour
 
PDF
Mongo db a deep dive of mongodb indexes
byRajesh Kumar
 
PDF
Nosql part 2
byRuru Chowdhury
 
PPTX
Indexing In MongoDB
byKishor Parkhe
 
PPTX
Mongodb Performance
byJack
 
PPTX
MongoDB's index and query optimize
bymysqlops
 
PPTX
Indexing documents
byMongoDB
 
PPT
Indexing & query optimization
byJared Rosoff
 
PPTX
Guidelines for indexing and tools
byNagaVarthini
 
PPTX
MongoDB - Indexing- Types iNDEXING.pptx
byrevathyvr4
 
PPTX
Automated Slow Query Analysis: Dex the Index Robot
byMongoDB
 
DOCX
unit 4,Indexes in database.docx
byRaviRajput416403
 
PPTX
#MongoDB indexes
byDaniele Graziani
 
PDF
Mongophilly indexing-2011-04-26
bykreuter
 
PPT
Mongo Performance Optimization Using Indexing
byChinmay Naik
 
PPTX
New Indexing and Aggregation Pipeline Capabilities in MongoDB 4.2
byAntonios Giannopoulos
 
PDF
Mongo db improve the performance of your application codemotion2016
byJuan Antonio Roy Couto
 
PPTX
MongoDB Aggregations Indexing and Profiling
byManish Kapoor
 
PPTX
Mongo db index
byzarigatongy
 
Indexing and Query Performance in MongoDB.pdf
byMalak Abu Hammad
 
Query Optimization in MongoDB
byHamoon Mohammadian Pour
 
Mongo db a deep dive of mongodb indexes
byRajesh Kumar
 
Nosql part 2
byRuru Chowdhury
 
Indexing In MongoDB
byKishor Parkhe
 
Mongodb Performance
byJack
 
MongoDB's index and query optimize
bymysqlops
 
Indexing documents
byMongoDB
 
Indexing & query optimization
byJared Rosoff
 
Guidelines for indexing and tools
byNagaVarthini
 
MongoDB - Indexing- Types iNDEXING.pptx
byrevathyvr4
 
Automated Slow Query Analysis: Dex the Index Robot
byMongoDB
 
unit 4,Indexes in database.docx
byRaviRajput416403
 
#MongoDB indexes
byDaniele Graziani
 
Mongophilly indexing-2011-04-26
bykreuter
 
Mongo Performance Optimization Using Indexing
byChinmay Naik
 
New Indexing and Aggregation Pipeline Capabilities in MongoDB 4.2
byAntonios Giannopoulos
 
Mongo db improve the performance of your application codemotion2016
byJuan Antonio Roy Couto
 
MongoDB Aggregations Indexing and Profiling
byManish Kapoor
 
Mongo db index
byzarigatongy
 

More from MongoDB

PDF
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
byMongoDB
 
PDF
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
byMongoDB
 
PDF
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
byMongoDB
 
PDF
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
byMongoDB
 
PDF
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
byMongoDB
 
PDF
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
byMongoDB
 
PDF
MongoDB SoCal 2020: MongoDB Atlas Jump Start
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
byMongoDB
 
PDF
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
byMongoDB
 
PDF
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
byMongoDB
 
PDF
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
byMongoDB
 
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
byMongoDB
 
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
byMongoDB
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
byMongoDB
 
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
byMongoDB
 
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
byMongoDB
 
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
byMongoDB
 
MongoDB SoCal 2020: MongoDB Atlas Jump Start
byMongoDB
 
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
byMongoDB
 
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
byMongoDB
 
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
byMongoDB
 
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
byMongoDB
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
byMongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
byMongoDB
 
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
byMongoDB
 
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
byMongoDB
 
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
byMongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
byMongoDB
 
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
byMongoDB
 
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
byMongoDB
 
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
byMongoDB
 

Recently uploaded

PDF
Beyond Basics: How to Build Scalable, Intelligent Imagery Pipelines
bySafe Software
 
PPTX
Opening Plenary - Esri UK Welsh Conference 2025
byEsri UK
 
PDF
Dev Dives: Build smarter agents with UiPath Agent Builder
byUiPathCommunity
 
PPTX
AI: Beyond Generative AI and LLM | Harrie de Groot (harrie.dev)
byHarrie de Groot
 
PDF
KMWorld - KM & AI Bring Collectivity, Nostalgia, & Selectivity
byJonathan Ralton
 
PDF
Top Crypto Supers 15th Report November 2025
byStephen Perrenod
 
PDF
AWS Cloud Technical Essentials - AWS Certificate
byVICTOR MAESTRE RAMIREZ
 
PDF
IDSECCONF2025 - Aan Wahyu - Maze–Malware Analysis Platform.pdf
byidsecconf
 
PPTX
Neo4j Fraud GraphTalk Singapore Nov 2025
bygourisachdeva2
 
PDF
SELinux Basics: Managing SELinux Modes and Context - RHCSA+.pdf
byLinuxCert Guru
 
PDF
Oracle MySQL HeatWave - Complete - Version 3
byLuca Bonesini
 
PPTX
Explaining ourselves – people, computers and AI
byAlan Dix
 
PDF
Running Non-Cloud-Native Databases in Cloud-Native Environments_ Challenges a...
byAlkin Tezuysal
 
PPTX
How Design Systems and AI Agents Accelerate Product Delivery Sixt
byBoyan Dimitrov
 
PDF
Introduction to Shell Scripting - RHCSA+.pdf
byLinuxCert Guru
 
PDF
MINDCTI Revenue Release Quarter 3 2025 - Financial Presentation
byMIND CTI
 
PPTX
Understanding the digital experience of FE teaching staff in 2024/25
byJisc
 
PDF
November 2025 OpenMetadata Community Spotlight - Astronomer & Airflow 3
byOpenMetadata
 
PDF
Linux Foundation Certified System Administrator (LFCS) Exam.pdf
byLinuxCert Guru
 
PDF
PMI PMBOK8 Released on November 13, 2025
byScott M. Graffius
 
Beyond Basics: How to Build Scalable, Intelligent Imagery Pipelines
bySafe Software
 
Opening Plenary - Esri UK Welsh Conference 2025
byEsri UK
 
Dev Dives: Build smarter agents with UiPath Agent Builder
byUiPathCommunity
 
AI: Beyond Generative AI and LLM | Harrie de Groot (harrie.dev)
byHarrie de Groot
 
KMWorld - KM & AI Bring Collectivity, Nostalgia, & Selectivity
byJonathan Ralton
 
Top Crypto Supers 15th Report November 2025
byStephen Perrenod
 
AWS Cloud Technical Essentials - AWS Certificate
byVICTOR MAESTRE RAMIREZ
 
IDSECCONF2025 - Aan Wahyu - Maze–Malware Analysis Platform.pdf
byidsecconf
 
Neo4j Fraud GraphTalk Singapore Nov 2025
bygourisachdeva2
 
SELinux Basics: Managing SELinux Modes and Context - RHCSA+.pdf
byLinuxCert Guru
 
Oracle MySQL HeatWave - Complete - Version 3
byLuca Bonesini
 
Explaining ourselves – people, computers and AI
byAlan Dix
 
Running Non-Cloud-Native Databases in Cloud-Native Environments_ Challenges a...
byAlkin Tezuysal
 
How Design Systems and AI Agents Accelerate Product Delivery Sixt
byBoyan Dimitrov
 
Introduction to Shell Scripting - RHCSA+.pdf
byLinuxCert Guru
 
MINDCTI Revenue Release Quarter 3 2025 - Financial Presentation
byMIND CTI
 
Understanding the digital experience of FE teaching staff in 2024/25
byJisc
 
November 2025 OpenMetadata Community Spotlight - Astronomer & Airflow 3
byOpenMetadata
 
Linux Foundation Certified System Administrator (LFCS) Exam.pdf
byLinuxCert Guru
 
PMI PMBOK8 Released on November 13, 2025
byScott M. Graffius
 

MongoDB.local DC 2018: Tips and Tricks for Avoiding Common Query Pitfalls

  • 1.
    Tips and Tricksfor Avoiding Common Query Pitfalls With emphasis on improving query performance
  • 2.
  • 3.
    • Motivation • Whoam I • What we're going to talk about Roadmap
  • 4.
    Motivation The Power ofQuery Optimization Query tuning results in: • Improved performance • Reduced resource utilization This may lead to: • Improved stability and predictability • A smaller hardware footprint • End-user happiness
  • 5.
    • Senior SolutionsArchitect, MongoDB Federal • 3 year tenure with MongoDB • Where I've worked • Large scale data processing emphasis Who am I?
  • 6.
    1. Indexes • Whatthey are and why use them • Examples of using them o Single, Compound, Case-insensitive • Using "explain" 2. Blocking Stages • What they are • Methods to avoid them 3. $and vs. $or operator • Why so different in performance 4. Tools that can help Performance areas we're going to talk about
  • 7.
    Here are my Documents HOW DOI FIND THE SPECIFIC DOCUMENT(S) I WANT?
  • 8.
    USE AN INDEX IT ISA WAY TO LOCATE YOUR DATA QUICKLY
  • 9.
    What are indexes? Indexesare special data structures that store a small portion of the collection’s data set in an easy to traverse form.
  • 10.
    Why indexes? It’s aboutperformance Indexes are the single biggest tunable performance factor for an application TIP: • Use for frequently accessed queries • Use when low latency response time needed
  • 11.
    Why indexes? (cont.) •⅔ of all performance problems are due to a missing or incorrect secondary index • No secondary index ⇒ full collection scan
  • 12.
    MongoDB Indexes Some ofthe indexes MongoDB supports Single-field indexes Compound-field indexes Multi-Key indexes Geospatial indexes Text indexes TTL indexes Hashed indexes
  • 13.
    Show Me BASICMONGODB INDEXING EXAMPLES
  • 14.
    Simple example We havea twitter database that has a tweets collection in MongoDB that contains a set of tweets and we run a query. > use twitter > db.tweets.find( { "user.followers_count" : 1000 } )
  • 15.
    How does ourquery behave? We can see what the query does by using ".explain(…)" > db.tweets.find( { "user.followers_count" : 1000 } ).explain("executionStats") • TIP: Using "executionStats", MongoDB runs the query optimizer to choose the winning plan, executes the winning plan to completion, and returns statistics describing the execution of the that plan.
  • 16.
    What’s this explain()do? The most important values we we’re interested for indexing in are • nReturned : number of documents returned by the query • totalDocsExamined : number of documents touched during the query • totalKeysExamined : number of index keys scanned TIP: A totalKeysExamined or totalDocsExamined value much higher than nReturned indicates we could use or need a better index.
  • 17.
    How does ourquery behave? … "executionStats" : { "executionSuccess" : true, "nReturned" : 8, "executionTimeMillis" : 107, "totalKeysExamined" : 0, "totalDocsExamined" : 51428, "executionStages" : { "stage" : "COLLSCAN", "filter" : { "user.followers_count" : { "$eq" : 1000 } }, "nReturned" : 8, "executionTimeMillisEstimate" : 100, … "direction" : "forward", "docsExamined" : 51428 } ... } Ugh! We just did a “full collection scan” of 51428 documents to find 8 documents.
  • 18.
    Can we dobetter for reading? We put a single-field index on the tweets collection and see if that improves our performance. > db.tweets.createIndex( { "user.followers_count" : 1 } ) > db.tweets.find( { "user.followers_count" : 1000 } ).explain("executionStats")
  • 19.
    How does ourread query behave with this index? … "executionStats" : { "executionSuccess" : true, "nReturned" : 8, "executionTimeMillis" : 0, "totalKeysExamined" : 8, "totalDocsExamined" : 8, "executionStages" : { "stage" : "FETCH", "nReturned" : 8, "executionTimeMillisEstimate" : 0, "inputStage" : { "stage" : "IXSCAN", "nReturned" : 8, "invalidates" : 0, "direction" : "forward", "docsExamined" : 51428 … "indexName" : "user.followers_count_1" Wow! Our query used the index to find our 8 documents and didn't have to scan the whole collection.
  • 20.
    Another example Now trya compound-field index. > db.tweets.createIndex( {"user.time_zone" : 1, "user.followers_count" : 1 } ) > db.tweets.find({"user.time_zone" : "Tokyo", "user.followers_count": 10}).explain("executionStats")
  • 21.
    How does ourread query behave? … "executionStats" : { "executionSuccess" : true, "nReturned" : 17, "executionTimeMillis" : 5, "totalKeysExamined" : 17, "totalDocsExamined" : 17, "executionStages" : { "stage" : "FETCH", "nReturned" : 17, "executionTimeMillisEstimat e" : 0, "works" : 19, "advanced" : 17, "needTime" : 0, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "docsExamined" : 17, "alreadyHasObj" : 0, "inputStage" : { "stage" :"IXSCAN", "nReturned" : 17, … "indexName" : "user.time_zone_1_user.followers_cou nt_1", … Still looking good!
  • 22.
    Do I haveto do exact match queries? NO! You can do range queries. > db.tweets.find( { "user.followers_count" : {$gte : 10, $lte : 100} } ).explain("executionStats")
  • 23.
    How does ourrange query behave? … "executionStats" : { "executionSuccess" : true, "nReturned" : 17643, "executionTimeMillis" : 21, "totalKeysExamined" : 17643, "totalDocsExamined" : 17643, "executionStages" : { "stage" : "FETCH", "nReturned" : 17643, "executionTimeMillisEsti mate" : 22, "docsExamined" : 17643, "alreadyHasObj" : 0, "inputStage" : { "stage" : "IXSCAN", "nReturned" : 17643, … "indexName" : "user.followers_count_1" … Still looking good!
  • 24.
    Indexes for writing? Canindexes improve write performance? > db.tweets.explain("executionStats") .update({"user.followers_count" : 1000}, {$set : {"large_following" : true}})
  • 25.
    How does ourupdate query behave? … "executionStats" : { "executionSuccess" : true, "nReturned" : 0, "executionTimeMillis" : 0, "totalKeysExamined" : 8, "totalDocsExamined" : 8, "executionStages" : { "stage" : "UPDATE", "nReturned" : 0, "executionTimeMillisEsti mate" : 0, … "inputStage" : { "stage" : "IXSCAN", "nReturned" : 8, … "indexName" : "user.followers_count_1", Still looking good!
  • 26.
    What about case-insensitivity? Whathappens if you search for "TOKYO" vs. "Tokyo"? > db.tweets.find({"user.time_zone" : "Tokyo", "user.followers_count": {$gte : 10, $lte : 100}}).count() 841 > db.tweets.find({"user.time_zone" : "TOKYO", "user.followers_count": {$gte : 10, $lte : 100}}).count() 0 You get different results! This query did not find any matching documents!
  • 27.
    Creating Case-insensitive indexes? TIP:To use a case insensitive index on a collection with no default collation, create an index with a collation and set the strength parameter to 1 or 2. > db.tweets.createIndex( {"user.time_zone" : 1, "user.followers_count" : 1}, { name : "user_timezone_user_followers_1_insensitive", collation: { locale : 'en', strength : 2 } } ) > db.tweets.find({"user.time_zone" : "Tokyo", "user.followers_count": {$gte : 10, $lte : 100}}).count() 841 > db.tweets.find({"user.time_zone" : "TOKYO", "user.followers_count": {$gte : 10, $lte : 100}}).collation({locale : "en", strength : 2}).count() 841 Now, the results are the same when using the case-insensitive index we created.
  • 28.
    Deleting indexes Listing theindexes to get index name > db.tweets.getIndexes() [ {"v" : 2,"key" : {"_id" : 1},"name" : "_id_","ns" : "twitter.tweets"}, {"v" : 2,"key" : {"user.followers_count" : 1},"name" : "user.followers_count_1","ns" : "twitter.tweets"}, {"v" : 2, "key" : { "user.time_zone" : 1, "user.followers_count" : 1 }, "name" : "user.time_zone_1_user.followers_count_1", "ns" : "twitter.tweets" } ]
  • 29.
    Deleting indexes Here ishow to remove an existing index. > db.tweets.dropIndex("user.followers_count_1") { "nIndexesWas" : 3, "ok" : 1 } > db.tweets.getIndexes() [ {"v" : 2,"key" : {"_id" : 1},"name" : "_id_","ns" : "twitter.tweets"}, {"v" : 2, "key" : { "user.time_zone" : 1, "user.followers_count" : 1 }, "name" : "user.time_zone_1_user.followers_count_1", "ns" : "twitter.tweets" } ]
  • 30.
    TIP: Delete indexeswhen they are not used by your applications for given queries. Indexes aren't a free lunch as they • Consume space • Affect write operations Why delete indexes?
  • 31.
  • 32.
    Index Early TIP: Indexesdeserve first-tier consideration in the design process. Efficiency at the data access level has historically been offloaded to a DBA-like role which prompts the kind of post-design optimization layers that the document-oriented database stack still has the opportunity to avoid.
  • 33.
    TIP: Indexed queriesperform better by several orders of magnitude, even on small data. While an un-indexed query may take 10 seconds, the same query can take as little as ~0 milliseconds given a proper index. Index Often
  • 34.
    Queries generally makeuse of indexes from left to right. TRICK: An compound index can be utilized to the extent that a query searches a "prefix" subset of those fields.  db.tweets.createIndex( {"user.followers_count" : 1, "user.lang" : 1, "user.time_zone" : 1} )  db.tweets.find({"user.time_zone" : "Tokyo", "user.followers_count": {$gte : 10, $lte : 100}, "user.lang" : "en"}) ✖ db.tweets.find({"user.time_zone" : "Tokyo", "user.lang" : "en"})  db.tweets.find({ "user.followers_count": {$gte : 10, $lte : 100}})  db.tweets.find({ "user.followers_count": {$gte : 10, $lte : 100}, "user.time_zone" : "en"}) ✖ db.tweets.find({ "user.lang" : "en", "user.time_zone" : "Tokyo"})  Uses the index ✖ Does not use the index Index Fully
  • 35.
    Leverage Index Sorts •TIP: If your queries will contain a sort, add the sorted field to the end of your index. Useful Commands • .explain("executionStats") ➡ its output can be analyzed to see if an index might be appropriate ➡ its output will show what index (if any) has been used for a given query • .createIndex(…), .dropIndex(…) ➡ are used for creating and dropping indexes • .getIndexes() or .getIndexKeys() ➡ will show you what indexes you have • .hint(…) ➡ allows you to indicate what index to use Other
  • 36.
    Compound Indexing –Rules of Thumb Order fields in a compound index from most selective (most unique) to least selective (least unique) • Usually, this means equality fields (fields queried for a specific value) before range fields (e.g., date fields queried for a particular time period) TIP: When dealing with multiple equality values, start with the most selective Other .
  • 37.
    Work Smarter NotHarder • Understand the business logic • Index appropriately • Is it the right index to support the query? • Return only the data you require • TIP: Leverage the Performance Advisor • The Performance Advisor (in Ops Mangager and Atlas), monitors queries that MongoDB considers slow and suggests new indexes to improve query performance.
  • 38.
  • 39.
    Acme Games Introduces... ShortFite! Brandnew Battle Royale game Launching October 31st
  • 40.
    • Game nearlycomplete • Developers have learned a lot from their MongoDB champion Stakeholder Concerns • App being stress tested • Concerns over current performance 
  • 41.
    Stakeholder Concern #1 Developerscreated index db.games.createIndex({ gamerTag: 1 }) This query takes several seconds to execute: db.games.find( { gamerTag: "Ace" } ).sort({score:-1}) Adding an index on score does not help! db.games.createIndex({ score: -1 })
  • 42.
  • 43.
    Blocking Operation ● Formally: ■“An operation which must process all input before it can begin to produce any output.” ● Opposite of the often desirable “fully pipelined” plan which can stream results back as soon as they are found. ● TIP: Commonly observed when a sort is added to a query
  • 44.
  • 45.
  • 46.
  • 47.
  • 48.
  • 49.
  • 50.
  • 51.
  • 52.
  • 53.
  • 54.
  • 55.
  • 56.
  • 57.
  • 58.
  • 59.
  • 60.
  • 61.
  • 62.
  • 63.
  • 64.
  • 65.
  • 66.
  • 67.
  • 68.
  • 69.
  • 70.
  • 71.
  • 72.
    Blocking Stages • $sort •In aggregation and find • $group • $bucket • $count • $facet
  • 73.
    Working with blockingstages For sorting: TRICK: Add a supporting index Worth the overhead in almost all circumstances For other stages: Determine if you really need the blocking stage? TRICK: Offload to secondary member (where possible)
  • 74.
    Original performance of db.games.find({ gamerTag: "Ace" } ).sort({score:-1}) without an index is poor. “Clearly MongoDB is not webscale!” Example #1 (recap)
  • 75.
    Adding an indexmakes it fast db.games.find( { gamerTag: "Ace" } ).sort({score:-1}) db.games.createIndex({ gamerTag: 1, score:-1 }) db.games.find( { gamerTag: "Ace" } ).sort({score:-1}) "That’ll work great!” Example #1 (recap)
  • 77.
    The $and versionof a query returns quickly: db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) But the $or version is slow: db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Stakeholder Concern #2
  • 78.
    The $and versionof a query returns quickly: db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) But the $or version is slow: db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) But we just created an index with both those fields… Isn't it being used with $or? Stakeholder Concern #2
  • 79.
    Why is $andgrand but $or poor?
  • 80.
    $and example Query ongames: db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Matching games: { gamerTag: "Ace", score: 9500 } Non-matching games: { gamerTag: "Ace", score: 500 }, { gamerTag: "Bob", score: 9500 }, { gamerTag: "Bob", score: 500 }
  • 81.
    Groups of documents score:{$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 }
  • 82.
    $and Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) { gamerTag: "Bob", score: 500 }
  • 83.
    $and Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) { gamerTag: "Bob", score: 500 }
  • 84.
    $and Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 85.
    $and Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 86.
    $and Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 87.
    $and Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 88.
    $and Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 89.
    $and Venn Diagram(logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1 , score:-1} 5009500 9500 500
  • 90.
    $and Venn Diagram(logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 9500 500 "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000.0)" ] } {gamerTag:1 , score:-1}
  • 91.
    $and Venn Diagram(logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000.0)" ] }
  • 92.
    $and Venn Diagram(logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000.0)" ] }
  • 93.
    $and Venn Diagram(logical) db.games.find({ $and : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 5009500 9500 500 {gamerTag:1 , score:-1} "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000.0)" ] }
  • 94.
    $or example Query ongames: db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Matching games: { gamerTag: "Ace", score: 9500 }, { gamerTag: "Ace", score: 500 }, { gamerTag: "Bob", score: 9500 } Non-matching games: { gamerTag: "Bob", score: 500 }
  • 95.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 96.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 97.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 98.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 99.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" { gamerTag: "Ace", score: 9500 } { gamerTag: "Ace", score: 500 } { gamerTag: "Bob", score: 9500 } { gamerTag: "Bob", score: 500 } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 100.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 101.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 102.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 103.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 104.
    $or Venn Diagram(logical) score: {$gt: 9000}gamerTag: "Ace" db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 105.
    $or (single) Indexvisualization Ace Bob {gamerTag:1 , score:-1} 9500 500 9500 500 db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 106.
    $or (single) Indexvisualization Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1 , score:-1} 9500 500 9500 500
  • 107.
    $or (single) Indexvisualization Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 108.
    $or (single) Indexvisualization Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 109.
    $or (single) Indexvisualization Ace Bob 9500 500 9500 500 {gamerTag:1 , score:-1} Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 110.
    $or (single) Indexvisualization Ace Bob 9500 500 5009500 {gamerTag:1 , score:-1} Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 111.
    $or (single) Indexvisualization Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } Actual (Hinted) Index Bounds: "indexBounds" : { "gamerTag" : [ "[MinKey, MaxKey]" ], "score" : [ "[MaxKey, MinKey]" ] } db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 5009500 {gamerTag:1 , score:-1}
  • 112.
    $or (single) Indexvisualization Expected Index Bounds: "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ], "score" : [ "[inf.0, 9000]" ] } Actual (Hinted) Index Bounds: "indexBounds" : { "gamerTag" : [ "[MinKey, MaxKey]" ], "score" : [ "[MaxKey, MinKey]" ] } So is there anything we can do to improve the performance of this query? db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob 9500 500 5009500 {gamerTag:1 , score:-1}
  • 113.
  • 114.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1 , score:-1} 9500 500 9500 500
  • 115.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] })
  • 116.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace
  • 117.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob
  • 118.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1}
  • 119.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1} 500
  • 120.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) Ace Bob {gamerTag:1} 9500 500
  • 121.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) {gamerTag:1} {score:-1} Ace Bob 9500 500
  • 122.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
  • 123.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
  • 124.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
  • 125.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
  • 126.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
  • 127.
    $or (multiple) Indexvisualization db.games.find({ $or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ] }) "indexBounds" : { "gamerTag" : [ "["Ace", "Ace"]" ] } "indexBounds" : { "score" : [ "[inf.0, 9000]" ] } {gamerTag:1} {score:-1} Ace Bob 9500 500
  • 128.
    TIP Use multiple indexes! db.data.createIndex({gamerTag:1}) db.data.createIndex({score: -1}) But we already have the {gamerTag:1, score:-1} index. Do we need both of these new ones?
  • 129.
    TIP Use multiple indexes! db.data.createIndex({gamerTag:1}) db.data.createIndex({score: -1}) But we already have the {gamerTag:1, score:-1} index. Do we need both of these new ones? No we don't.
  • 130.
    TIP Use multiple indexes! db.data.createIndex({gamerTag:1}) db.data.createIndex({score: -1}) Works with sorting Generate a SORT_MERGE plan
  • 131.
  • 132.
    Compass THE GUIFOR MONGODB  VISUALLY EXPLORE YOUR DATA  RUN AD HOC QUERIES IN SECONDS  INTERACT WITH YOUR DATA WITH FULL CRUD FUNCTIONALITY  VIEW AND OPTIMIZE YOUR QUERY PERFORMANCE
  • 133.
    MongoDB Compass Debug andoptimize: Visual explain plans Understand how queries are running using a GUI that allows you to easily identify and resolve performance issues. • View key information about the execution plan of a query • Visualize different explain stages in an easy-to-understand tree format Visual explain plans are currently in beta.
  • 134.
    Does this query lookfamiliar? Notice these 3 values? What do they indicate?
  • 135.
    MongoDB Compass Debug andoptimize: Index details MongoDB Compass allows you to view index details for a given collection: • Type of index: regular, text, geospatial, or hashed • Size of index: how much space the index uses • Index utilization: how many times the index has been used • Special properties: unique index, compound index, etc.
  • 136.
    Ops/Cloud Manager THE BEST WAYTO RUN MONGODB Has lots of functionality … including Query Optimization to identify slow-running queries, get index suggestions, automate index builds.
  • 137.
    Visual Query Profiler Identifyyour slow-running queries with the click of a button
  • 138.
  • 139.
    • INDEXES AREA GREAT WAY TO IMPROVE MONGODB QUERY PERFORMANCE  Remember that roughly 2/3 of performance problems are due to missing or incorrect indexes  Use "explain" to see what (if any) indexes are being used • INDEXES ARE NOT A PANACEA FOR ALL PERFORMANCE OR DESIGN ISSUES  For example, they are not a substitute for poor schema design (a band-aid at most) • INDEXES HAVE PRICE TO PAY IN OPERATIONS AND STORAGE; USE THEM WISELY  As your data changes the indexes have to be updated – but automatically  They take up space • BLOCKING STAGES CAN IMPACT PERFORMANCE  Use indexes where possible, AVOID when Possible  Secondary nodes may be a solution • DIFFERENT OPERATORS OR STAGES IN MONGODB CAN USE (OR NOT USE) INDEXES DIFFERENTLY  Different or additional indexes may help • THERE ARE MONGODB TOOLS THAT CAN HELP
  • 140.
    WHAT WE WANTOUR DATABASE TO BE!If your query performance is poor, there is likely something that you can do to improve it; Sometimes by 99%
  • 141.
    WRAPUP WHAT ELSEWOULD YOU LIKE TO KNOW?
  • 142.