I'm not sure what kind of explanation you're hoping for here, but the gist is that this is standard "first-time setup" work.
I imagine this only happens in the editor, though I haven't verified that. Unless I'm completely misunderstanding what's being done, the property drawers don't need to be created in a built game. Perhaps try timing there to see if the issue persists.
That said, here's some more info if you're curious:
Comparing the first and second frames in the profiler, you can see that the calls under the Material constructor are quite similar. The critical difference comes in MaterialPropertyHandler.GetHandler().
First frame, that method also calls MaterialPropertyHandler.GetShaderPropertyHandler(), under which are many other calls, including a fair amount of memory allocation. (231.3KB on my machine).
A quick google search turns up what happens in GetHandler() that makes a difference:
if (MaterialPropertyHandler.s_PropertyHandlers.TryGetValue(propertyString, out materialPropertyHandler))
return materialPropertyHandler;
materialPropertyHandler = MaterialPropertyHandler.GetShaderPropertyHandler(shader, name);
I.e., that dictionary (apparently) has to be populated the first time you create a Material. After that, it will skip the call to GetShaderPropertyHandler().
You can adjust when this happens (e.g. in Start() or Awake(), etc.) if you need to, but I don't know of any way to avoid it entirely.