QUERY HANDLING
OPTIMIZER’S ROLE

• The optimizer should generate an efficient
  plan to access the data that we need to
  work with.
WHY DO WE NEED IT?

• SQL Statement defines what we are looking
  for. It doesn’t define how to get the data.
• The optimizer has to generate a query plan
  which tells the server how to get the data.
OPTIMIZER CHALENGES
NUMBER OF JOIN ORDERS
OPTIMIZER

• The optimizer doesn't necessarily look for
  the best query plan.
• The optimizer searches for a good enough
  plan.
WORK STAGES (HIGH LEVEL)

•   Create a logical plan
•   Apply transformation rules
•   Apply heuristics
•   Select the cheapest one
STEPS OF QUERY PROCESSING
PARSING

• Checks that the syntax is correct
• Creates a logical tree that represents the
  query.
DEMO 1

• See logical tree that was produced by parse
  step.
BINDIND

• Gets the logical tree that was produced by
  the parsing step
• Makes sure that all objects referenced by
  the logical tree exist and that the user can
  see them.
OPTIMIZATION

• The most important step. Generates the
  query plan that will be executed
TRIVIAL PLAN

• Very simple queries with very simple logical
  tree won’t get full optimization
• The optimizer will generate a plan that is
  called a trivial plan
• The trivial plan is very cheap to generate
  and it won’t be inserted into the plan cache
TRIVIAL PLAN

• If the trivial plan’s cost is more then the
  value that is configured as “cost threshold
  for parallelism”, the query will get full
  optimization
• You can disable the trivial plan by using
  trace flag 8757
SIMPLIFICATION

• The OPTIMIZER rewrites the tree to make it
  more simple. There are few simplification
  methods:
  – Constant folding
  – Domain simplification
  – Contradiction detection
DEMO 2

• Simplification and trivial plan
OPTIMIZATION LEVELS

• If no trivial plan was found, SQL Server
  starts optimizing the query.
• SQL Server has 3 stages that are called:
  – Search 0 (Transaction processing)
  – Search 1 (Quick plan)
  – Search 2 (Full optimization)
OPTIMIZATION LEVELS

• Each level has an entry and termination
  condition.
• Termination condition can be good enough
  plan was found, or to much time passed
• Optimization can begin at a low search step
  and get to a higher search step.
PROPERTIES

• Each node in the logical tree has properties
  attached to it.
• There are 2 types of properties
  – Logical properties (Node cost, output columns,
    Type information and nullability, etc)
  – Physical properties (Sort order, partition
    information etc’)
RULES

• The optimizer has set of more then 350
  rules that is using to optimize the query.
• The rules help the optimizer to modify the
  logical tree in a way that doesn’t effect the
  query’s results
• The rules also dictate the physical
  implementation of the logical tree
RULES

• There are four types of rules that can be
  used:
  – Simplification rules
  – Exploration rules
  – Implementation rules
  – Physical properties enforcement rules
MEMO

• All the trees are stored in memory in a
  structure that is called Memo
• Each optimization has its own memo
• A single memo can get to the size of 1.6 GB
OPTIMIZATION PROBLEMS

• On rare occasions there can be a timeout
  for the optimization process
• Most times when it happens, it will be on
  search 2 stage, and the server will use the
  query plan that was produced on search 1
  stage.
• Sometime the server stops optimization
  because of memory pressure
DMVs

• There are 2 DMVs that can give us
  information about the optimizer:
  – sys.dm_exec_query_optimizer_info, shows
    information about optimization process
  – sys.dm_exec_query_transformation_stats, sho
    ws information about rules usage
Demo 3
QUSTIONS?

• Additional reading:
  – http://sqlblog.com/blogs/paul_white/archive/
    2012/04/28/query-optimizer-deep-dive-part-
    1.aspx
  – http://www.red-
    gate.com/community/books/inside-sql-server-
    query-optimizer

Query handlingbytheserver

  • 1.
  • 2.
    OPTIMIZER’S ROLE • Theoptimizer should generate an efficient plan to access the data that we need to work with.
  • 3.
    WHY DO WENEED IT? • SQL Statement defines what we are looking for. It doesn’t define how to get the data. • The optimizer has to generate a query plan which tells the server how to get the data.
  • 4.
  • 5.
  • 6.
    OPTIMIZER • The optimizerdoesn't necessarily look for the best query plan. • The optimizer searches for a good enough plan.
  • 7.
    WORK STAGES (HIGHLEVEL) • Create a logical plan • Apply transformation rules • Apply heuristics • Select the cheapest one
  • 8.
    STEPS OF QUERYPROCESSING
  • 9.
    PARSING • Checks thatthe syntax is correct • Creates a logical tree that represents the query.
  • 10.
    DEMO 1 • Seelogical tree that was produced by parse step.
  • 11.
    BINDIND • Gets thelogical tree that was produced by the parsing step • Makes sure that all objects referenced by the logical tree exist and that the user can see them.
  • 12.
    OPTIMIZATION • The mostimportant step. Generates the query plan that will be executed
  • 13.
    TRIVIAL PLAN • Verysimple queries with very simple logical tree won’t get full optimization • The optimizer will generate a plan that is called a trivial plan • The trivial plan is very cheap to generate and it won’t be inserted into the plan cache
  • 14.
    TRIVIAL PLAN • Ifthe trivial plan’s cost is more then the value that is configured as “cost threshold for parallelism”, the query will get full optimization • You can disable the trivial plan by using trace flag 8757
  • 15.
    SIMPLIFICATION • The OPTIMIZERrewrites the tree to make it more simple. There are few simplification methods: – Constant folding – Domain simplification – Contradiction detection
  • 16.
    DEMO 2 • Simplificationand trivial plan
  • 17.
    OPTIMIZATION LEVELS • Ifno trivial plan was found, SQL Server starts optimizing the query. • SQL Server has 3 stages that are called: – Search 0 (Transaction processing) – Search 1 (Quick plan) – Search 2 (Full optimization)
  • 18.
    OPTIMIZATION LEVELS • Eachlevel has an entry and termination condition. • Termination condition can be good enough plan was found, or to much time passed • Optimization can begin at a low search step and get to a higher search step.
  • 19.
    PROPERTIES • Each nodein the logical tree has properties attached to it. • There are 2 types of properties – Logical properties (Node cost, output columns, Type information and nullability, etc) – Physical properties (Sort order, partition information etc’)
  • 20.
    RULES • The optimizerhas set of more then 350 rules that is using to optimize the query. • The rules help the optimizer to modify the logical tree in a way that doesn’t effect the query’s results • The rules also dictate the physical implementation of the logical tree
  • 21.
    RULES • There arefour types of rules that can be used: – Simplification rules – Exploration rules – Implementation rules – Physical properties enforcement rules
  • 22.
    MEMO • All thetrees are stored in memory in a structure that is called Memo • Each optimization has its own memo • A single memo can get to the size of 1.6 GB
  • 23.
    OPTIMIZATION PROBLEMS • Onrare occasions there can be a timeout for the optimization process • Most times when it happens, it will be on search 2 stage, and the server will use the query plan that was produced on search 1 stage. • Sometime the server stops optimization because of memory pressure
  • 24.
    DMVs • There are2 DMVs that can give us information about the optimizer: – sys.dm_exec_query_optimizer_info, shows information about optimization process – sys.dm_exec_query_transformation_stats, sho ws information about rules usage
  • 25.
  • 26.
    QUSTIONS? • Additional reading: – http://sqlblog.com/blogs/paul_white/archive/ 2012/04/28/query-optimizer-deep-dive-part- 1.aspx – http://www.red- gate.com/community/books/inside-sql-server- query-optimizer