What is the difference between max_depth and max_leaf_nodes parameter in decision tree classifier.
if depth is 4, then the number of leaf nodes will be 2^4 = 16.
So providing max_depth = 4 or max_leaf_nodes = 16 should create the same tree right ?
2 Answers
$\begingroup$
$\endgroup$
Not necessarily. Some branch may terminate earlier (e.g., because there are not enough observations in the node to split, or some other criterion for stopping has been met). In other words, if depth is 4 the number of leaves will not always be 16. Hope it helps.
$\begingroup$
$\endgroup$
Agreeing with and adding to @D F 's answer:
While the maximum number of leaves at depth = 4 is, of course, 16, the maximum depth with 16 nodes is much higher than 4, and depends on both the size of your sample and your minimum node size. If you have N = 1000 and a minimum node size of 10, you could have (in theory) a depth of almost 100. Of course, that isn't going to happen in real life.
But I've certainly seen trees where one branch is much longer than others. 16 nodes and a depth of 7 or 8 wouldn't be all that weird.
