Download as PDF, PPTX
Uploaded byJimmy Lai
Annotate types in large codebase with automated refactoring
The document discusses an automated refactoring process for enhancing type annotations in a large Python codebase at Carta, which consists of 1.8 million lines and 120,000 functions. It highlights tools like libcst for modifying Python code and mentions strategies such as using static and runtime analysis to improve type coverage. Additionally, it encourages continuous improvement through automated weekly updates and the engagement of developers.
![[PyCon US 2025] Scaling the Mountain_ A Framework for Tackling Large-Scale Te...](https://cdn.slidesharecdn.com/ss_thumbnails/pyconus2025scalingthemountainaframeworkfortacklinglarge-scaletechdebt-250517122757-38c6df76-thumbnail.jpg?width=640&height=640&fit=bounds)
![[LDSP] Solr Usage](https://cdn.slidesharecdn.com/ss_thumbnails/searchengineldspsolr-140127194826-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
![[LDSP] Search Engine Back End API Solution for Fast Prototyping](https://cdn.slidesharecdn.com/ss_thumbnails/searchengineldsp-140126193834-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Annotate types in large codebase with automated refactoring
- 1. Annotate Types inLarge Codebase with Automated Refactoring Jimmy Lai, Software Engineer at Carta Feb. 9, 2022
- 4.
- 5. A Large PythonCodebase Python code 1.8 million lines 27,000 files 120,000 functions ~200 active developers Lots of TypeError, AttributeError, ValueError
- 6. Type Annotation andMypy Mypy: Argument 1 to "add" has incompatible type "str"; expected "int"
- 7. Automated Refactoring Automated codechanges for fixing large scale tech debt (Code Formatting, Type Annotation, Dead Code Cleanup) LibCST Features: ● Concrete Syntax Tree ● Transformer and Matcher API ● Metadata with static analysis Recommended tool: LibCST A library for modifying Python code easily.
- 8. Code Review withPull Requests Pull Request Pull Request Pull Request Pull Request
- 13. Add missing typesbased on static analysis
- 16. MonkeyType: add missingtypes based on runtime data 1. Collect types by running Python program. 2. Aggregate collected types and apply to the code using LibCST. Run test cases and apply types:
- 17. Make it morefun! Automated weekly updates and leaderboards!
- 18. Fully Typed FunctionCoverage 2018 2021 automated refactoring
- 19. Production Type ErrorImprovement
- 20. 20  Carta We are hiring!https://tinyurl.com/carta-jobs Carta Engineering Blog https://medium.com/building-carta Contact: jimmy.lai@carta.com