I can't speak to integration with SQLAlchemy or what Azure SQL requires from an Azure Active Directory access token (you may find sqlalchemy/6031 helpful). But I can say that yes, DefaultAzureCredential can do what you want in terms of token acquisition, given a few preconditions are met: your Functions app must be configured for managed identity, you must be signed in to Azure in VS Code, and the environment variables you mention must not be set in either environment (they're used to configure a service principal, i.e. not what you want).
DefaultAzureCredential iterates over a chain of credentials until one provides a token (the package readme documents the ordering of these credentials). Given the above preconditions, managed identity will succeed in your Azure Function but fail locally, causing the credential to try VS Code. Note that you will be authenticating different identities, so you'll need to ensure both the managed identity and user signed in to VS Code are authorized to access Azure SQL.