YI
Uploaded byYusuke Izawa
PDF, PPTX77 views

Stack Hybridization: A Mechanism for Bridging Two Compilation Strategies in a Meta Compiler Framework

This document summarizes Yusuke Izawa's master's thesis defense on stack hybridization, a mechanism for bridging two compilation strategies - tracing and method-based - in a meta compiler framework. The proposal extends a meta-tracing just-in-time (JIT) compiler to apply different compilation strategies to different parts of a program based on call context. A proof-of-concept implementation in OCaml showed the hybrid approach was about 1.1x faster than a method-based only approach and over 100x faster than a tracing only approach.

Embed presentation

Download as PDF, PPTX
Stack Hybridization: A Mechanism for Bridging Two Compilation Strategies in a Meta Compiler Framework Yusuke Izawa (伊澤侑祐), Tokyo Institute of Technology (東京工業大学), Master's thesis defence, 2020/1/31
Creating a modern VM is a hard task 2 Context JIT Compiler Interpreter Optimizer FFI GC ... 10K lines or more Virtual Machine Debugger How to reduce engineering efforts?
Generate a VM by merely writing an interpreter 3 Context: How to reduce engineering efforts? JIT Compiler Interpreter Optimizer FFI GC ... Interpreter Interpreter ... Focus VM Meta JIT Compiler Framework Interpreter Debugger Written by language designers
Apply JIT compilation to a pair of meta-interpreter and base-program 4 Context: How a Meta JIT Compiler Works Meta-interpreter (Meta) JIT compiler Base-program sum fib Native code sum: ... ... ... ... fib: ... ... ... ... Next: Two mainstream meta JIT compiler frameworks
RPython and Truffle/Graal 5 Context: State-of-the-arts RPython with Meta-tracing[C.F.Bolz et al., '09] Truffle/Graal with Partial Evaluation[T.Wurthinger et al., '12] S t r a t e g y Trace-based Method-based ● crossing function calls ● one-side of a conditional branch
Trade-offs between trace- and method-based JIT 6 Problem trace-based S t r a t e g y method-based fib sum nest-fun fib sum nest-fun G o o d B a d Path-Divergence Problem[R.Huang et al., '16]
● Approach: ○ Extend a meta-tracing JIT compiler framework ● Proof-of-concept: ○ BacCaml, based on the MinCaml[E.Sumii, '05] compiler ■ RPython is too complicated ○ Implemented in OCaml (6000 SLOC), C (300 SLOC), Assembly (200 SLOC) Meta-hybrid JIT Compiler Framework 7 Proposal Next: Our previous work
8 Previous Work and its Problem trace-based compilation method-based compilation by tracing Step1: Method JIT by Tracing (Bachelor’s thesis) Goal: Apply different strategies to different area Different management for return address meta-interpreter for trace-based comp. meta-interpreter for method-based comp.
9 This Work trace-based compilation method-based compilation by tracing Step2: Stack Hybridization (This work) Goal: Apply different strategies to different area S.H. meta-interpreter for both comp.
10 Problem on Previous Work Native from trace-JIT Native from method-JIT def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] ret_addr = pc + 1 user_stack.push(ret_addr) pc = addr elsif == RET: v = user_stack.pop() addr = user_stack.pop() user_stack.push(v) pc = addr Different management for return address def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] r = interp(addr) user_stack.push(r) pc += 1 elsif == RET: v = user_stack.pop() return v managed by host-stack Meta-interpreter for trace-JIT Meta-interpreter for method-JIT User-stack style Host-stack style managed by user-stack
A meta-interpreter design using two kinds of stack 11 Our Solution: Stack Hybridization (1) ● Design choice: Embed two styles into a single definition if instr == CALL: addr = bytecode[pc] if mj_context(addr): user_stack.push(HS) # call under method-JIT ctx else user_stack.push(US) # call under trace-JIT ctx elsif == RET: … ctx = user_stack.pop() if ctx == US: # return under trace-JIT ctx else: # return under method-JIT ctx Put flag Check flag For changing the JIT context Native from trace-JIT Native from method-JIT
Example compiled code: sum-fib 12 Our Solution: Stack Hybridization (2) sum (trace-JIT) fib (method-JIT) @method_jit def fib(n) = if n < 2 then 1 else fib(n-1) + fib(n-2) @trace_jit def sum(n) = if n < 1 then 1 else fib(n) + sum(n-1) sum(100) .globl tracetj1.890 tracetj1: movl ebx, %ecx subl $3, %ecx … movl %ebx, %ecx addl $1, %ecx movl $200, (%eax,%ecx,4) addl $2, %ebx jmp tracemj0.876 … .globl tracemj0.876 tracemj0.876: movl ebx, %ecx subl $2, %ecx … movl $100, (%eax,%ebx,4) movl %ebx, %ecx movl $2, %ecx … addl $16, %ebp call tracemj0.876 subl $16, %ebp … movl (%eax,%esi,4), %esi cmpl $100, %edx jne jne_else.868 movl %ecx, %eax ret je_else.868: movl $56, %edx … movl %esi, %eax jmp guard_tracemj0.846 Put flag Put flag Check flag
.globl tracemj0.876 tracemj0.876: movl ebx, %ecx subl $2, %ecx … movl $100, (%eax,%ebx,4) movl %ebx, %ecx movl $2, %ecx … addl $16, %ebp call tracemj0.876 subl $16, %ebp … movl (%eax,%esi,4), %esi cmpl $100, %edx jne jne_else.868 movl %ecx, %eax ret je_else.868: movl $56, %edx … movl %esi, %eax jmp guard_tracemj0.876 Example compiled code: sum-fib 13 Our Solution: Stack Hybridization (2) .globl tracetj1.890 tracetj1: movl ebx, %ecx subl $3, %ecx … movl %ebx, %ecx addl $1, %ecx movl $200, (%eax,%ecx,4) addl $2, %ebx jmp tracemj0.876 … .globl interp interp: movl %ebx, 0(%ebp) … sum (trace-JIT) fib (method-JIT) @method_jit def fib(n) = if n < 2 then 1 else fib(n-1) + fib(n-2) @trace_jit def sum(n) = if n < 1 then 1 else fib(n) + sum(n-1) sum(100)
Benchmarking the performance of BacCaml 14 Evaluation: Overview ● Preparation: ○ Write a small programming language almost equivalent to MinCaml in BacCaml ● Evaluation 1: ○ Compare trace-JIT, method-JIT with MinCaml ■ Run a part of shootout benchmark https://dada.perl.it/shootout/index.html ● Evaluation 2: ○ Compare hybrid-JIT, trace-JIT, method-JIT with MinCaml ■ Combine functions that are suitable for trace-JIT and method-JIT respectively based on eval. 1 Environment: Arch Linux, CPU; Intel 2.7GHz Core i5, Mem; 8GB
15 Evaluation 1 better ● Tracing JIT is suitable for branching control flow (1.28x faster than method JIT) ● Method JIT is good for any programs (around 10x ~ 100x faster than tracing JIT)
16 Evaluation 2 better ● Hybrid JIT is about 1.1x better than method JIT only, and over 100x better than tracing JIT only
● Summary ○ Proposal: a meta-hybrid JIT compiler framework and its poc implementation, BacCaml ○ Performance: hybrid JIT is about 1.1x faster than method JIT only (and about 100x faster than tracing JIT only) ● Future work ○ Add some optimizations for resulting traces ○ Investigate profiling scheme ○ Apply our approach to RPython 17 Conclusion
18 Further Future work trace-JIT + method-JIT in Meta-tracing (PoC) Support language designers w/ Language Workbench [M.Fowler, '05] ● DynSem for Truffle[V.Vergu, '19] ● PySem for RPython region-JIT[G.Ottoni, '18] in Meta-tracing (PoC) trace-JIT + method JIT in Meta-tracing (Practical)
Method JIT by Tracing: Overview ● Create method-based compilation by customizing following points: ○ tracing entry/exit points ○ conditional branch ○ loop ○ function calls 19 Appendix: Our Previous Work
Why We Need Two Different Styles 20 Appendix: Our Previous Work CALL RET def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] r = interp(addr) user_stack.push(r) pc += 1 elsif == RET: v = user_stack.pop() return v base-program meta-interpreter Not inline Emit "CALL" to log log
Method JIT by Tracing 21 Appendix: Our Previous Work CALL RET RET b a c k tr a c k base-program
Why cannot unify into one style: Trace-JIT and Host-stack style 22 Appendix: Integration Problem in Meta-interpreter (1) def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] r = interp(addr, user_stack) user_stack.push(r) pc += 1 elsif == RET: v = user_stack.pop() return v ● cannot convert a non-tail recursive call into a loop ● why?: stack frame isn't mached recursive call
Why cannot unify into one style: Method-JIT and User-stack style 23 Appendix: Integration Problem in Meta-interpreter (2) def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] ret_addr = pc + 1 user_stack.push(ret_addr) pc = addr elsif == RET: v = user_stack.pop() addr = user_stack.pop() user_stack.push(v) pc = addr ● difficult to trace the successor of a deep fun. call ● why?: too much time to reach the RET
● GraalSqueak [F. Niephaus, et al., '18] ○ Idea: designing a meta-interpreter to enable trace-based compilation on Truffle/Graal ○ Methodology: adding runtime annotations to a meta-interpreter definition written in Truffle ○ Performance: about 1.4x faster than OpenSmalltalk VM ● HHVM JIT Compiler [G.Ottoni, '18] ○ Idea: performing JIT compilation based on the combination of basic blocks ○ Performance: about 1.2x efficient than normal trace-based compilation (HHVM 1st gen.) 24 Related work
25 Our Solution: Stack Hybridization (2) Example Compiled Code: fib-sum ... Call Guard fail ... ... ... ... ... ... Ret @tracing_jit def sum(n) = if n < 2 then 1 else fib(n-1) + fib(n-2) @method_jit def fib(n) = if n < 1 then sum(1000) else fib(n-1) + fib(n-2) fib(30)
26 Context: Building a virtual machine Current Situation w/ Meta JIT Compiler Framework Prototype Build a virtual machine (VM) Improve runtime system Continue to improve... Define syntax and abstract syntax tree (AST), AST interp. Still using AST interpreter, build a lot of optimizer like GC... Define bytecode format, and bytecode interpreter Performance still bad, thus write JIT compiler Prototype Define syntax and abstract syntax tree (AST) Integrate with Meta JIT Write an interpreter with a meta JIT compiler framework Already it is fast Empowered by Meta JIT compiler
Integration Problem: Failing to find return address in naively mixed execution 27 Technical Problem in Previous Work: Details (1) ● Example 1: calling a trace-compiled function (sum) from a method-compiled function (fib_s)
Integration Problem: Failing to find return address in naively mixed execution 28 Technical Problem in Previous Work: Details (2) ● Example 2: calling a method-compiled function (fib) from a trace-compiled function (sum_f)
Trade-offs between trace- and method-based JIT 29 Problem ● straight-line control flow ● smaller compiled code ● automatic function inlining ● matured technique ● apply to any programs trace-based ● bigger compiled code ● careful management for function inlining ● performance goes worse in complex control flow[R.Huang et al., '16] S t r a t e g y method-based Our Solution: Combine trace- and method-based compilation
Trade-offs between trace- and method-based JIT 30 Problem ● branching control flow ● smaller compiled code ● good for a deep-nested fun. call ● matured technique ● apply to any programs trace-based ● bigger compiled code ● careful management for function inlining ● performance goes worse in complex control flow[R.Huang et al., '16] S t r a t e g y method-based fib sum nest-fun fib sum nest-fun G o o d B a d
How Stack Hybridization works in sum-fib 31 Our Solution: Stack Hybridization (2) ... ... ... ... JUMP sum fib ？？？ @method_jit def fib(n) = if n < 2 then 1 else fib(n-1) + fib(n-2) @tracing_jit def sum(n) = if n < 1 then 1 else fib(n) + sum(n-1) sum(100) ...
● Tracing JIT is suitable for straight-line control flow (1.28x faster than method JIT) ● Method JIT is good for any programs (around 10x ~ 100x faster than tracing JIT) 32 Evaluation 1: Comparing tracing and method JIT performance better
● Hybrid JIT is 1.36x better than method JIT, and over 100x better than tracing JIT 33 Evaluation 2: Hybrid JIT performance better

More Related Content

PDF
Two-level Just-in-Time Compilation with One Interpreter and One Engine
byYusuke Izawa
 
ODP
GCC, GNU compiler collection
byAlberto Bustamante Reyes
 
PPT
Lecture 1
byguest6c6268
 
PDF
eBPF Debugging Infrastructure - Current Techniques
byNetronome
 
PPTX
RaVioli: A Parallel Vide Processing Library with Auto Resolution Adjustability
byMatsuo and Tsumura lab.
 
PPT
GEM - GNU C Compiler Extensions Framework
byAlexey Smirnov
 
PDF
Ekon 25 Python4Delphi_MX475
byMax Kleiner
 
PDF
Challenges in GPU compilers
byAnastasiaStulova
 
Two-level Just-in-Time Compilation with One Interpreter and One Engine
byYusuke Izawa
 
GCC, GNU compiler collection
byAlberto Bustamante Reyes
 
Lecture 1
byguest6c6268
 
eBPF Debugging Infrastructure - Current Techniques
byNetronome
 
RaVioli: A Parallel Vide Processing Library with Auto Resolution Adjustability
byMatsuo and Tsumura lab.
 
GEM - GNU C Compiler Extensions Framework
byAlexey Smirnov
 
Ekon 25 Python4Delphi_MX475
byMax Kleiner
 
Challenges in GPU compilers
byAnastasiaStulova
 

What's hot

PPTX
[Sitcon2018] Analysis and Improvement of IOTA PoW Implementation
byZhen Wei
 
PDF
Introduction to the LLVM Compiler System
byzionsaint
 
PDF
Pascal script maxbox_ekon_14_2
byMax Kleiner
 
PPT
Debugging Applications with GNU Debugger
byPriyank Kapadia
 
PDF
Directive-based approach to Heterogeneous Computing
byRuymán Reyes
 
PPT
Al2ed chapter5
byAbdullelah Al-Fahad
 
PDF
International Journal of Engineering Research and Development
byIJERD Editor
 
PPTX
Mixing C++ & Python II: Pybind11
bycorehard_by
 
PDF
Everything You Need to Know About the Intel® MPI Library
byIntel® Software
 
PDF
Open cl programming using python syntax
bycsandit
 
PPT
C under Linux
byn/a - currently unemployed
 
PPT
Nug2004 yhe
byYassine Rafrafi
 
PDF
EKON 25 Python4Delphi_mX4
byMax Kleiner
 
PDF
Take advantage of C++ from Python
byYung-Yu Chen
 
PDF
Notes about moving from python to c++ py contw 2020
byYung-Yu Chen
 
PDF
Cray XT Porting, Scaling, and Optimization Best Practices
byJeff Larkin
 
PDF
Rcpp
byAjay Ohri
 
PDF
A closure ekon16
byMax Kleiner
 
PDF
1 Vampir Overview
byPTIHPA
 
[Sitcon2018] Analysis and Improvement of IOTA PoW Implementation
byZhen Wei
 
Introduction to the LLVM Compiler System
byzionsaint
 
Pascal script maxbox_ekon_14_2
byMax Kleiner
 
Debugging Applications with GNU Debugger
byPriyank Kapadia
 
Directive-based approach to Heterogeneous Computing
byRuymán Reyes
 
Al2ed chapter5
byAbdullelah Al-Fahad
 
International Journal of Engineering Research and Development
byIJERD Editor
 
Mixing C++ & Python II: Pybind11
bycorehard_by
 
Everything You Need to Know About the Intel® MPI Library
byIntel® Software
 
Open cl programming using python syntax
bycsandit
 
C under Linux
byn/a - currently unemployed
 
Nug2004 yhe
byYassine Rafrafi
 
EKON 25 Python4Delphi_mX4
byMax Kleiner
 
Take advantage of C++ from Python
byYung-Yu Chen
 
Notes about moving from python to c++ py contw 2020
byYung-Yu Chen
 
Cray XT Porting, Scaling, and Optimization Best Practices
byJeff Larkin
 
Rcpp
byAjay Ohri
 
A closure ekon16
byMax Kleiner
 
1 Vampir Overview
byPTIHPA
 

Similar to Stack Hybridization: A Mechanism for Bridging Two Compilation Strategies in a Meta Compiler Framework

PDF
JIT compilation in modern platforms – challenges and solutions
byaragozin
 
PDF
Stranger in These Parts. A Hired Gun in the JS Corral (JSConf US 2012)
byIgalia
 
PDF
Presentation slides for "A formal foundation for trace-based JIT compilation"
bymvdcamme
 
PDF
How difficult is to get a JIT right? Talk from ESGU 2024
byESUG
 
PDF
A Better Python for the JVM
byTobias Lindaaker
 
PDF
VMs, Interpreters, JIT
byMarcus Denker
 
PDF
JRuby 9000 - Optimizing Above the JVM
byCharles Nutter
 
PDF
Tail Call Elimination in Open Smalltalk
byESUG
 
PDF
PyPy's approach to construct domain-specific language runtime
byNational Cheng Kung University
 
PDF
Oh the compilers you'll build
byMark Stoodley
 
PDF
JRuby 9000 - Taipei Ruby User's Group 2015
byCharles Nutter
 
PDF
J9's MethodHandle Compilation Pipeline #JFokus2015
byDanHeidinga
 
PPT
OCAT: Object Capture based Automated Testing (ISSTA 2010)
bySung Kim
 
PDF
Efficient call path detection for android os size of huge source code
bycsandit
 
PDF
EFFICIENT CALL PATH DETECTION FOR ANDROID-OS SIZE OF HUGE SOURCE CODE
bycscpconf
 
PDF
Invoke dynamic your api to hotspot
byBoundary
 
PPTX
Building High-Performance Language Implementations With Low Effort
byStefan Marr
 
PDF
«Python на острие бритвы: PyPy project» Александр Кошкин, Positive Technologies
byit-people
 
PDF
Lafauci dv club oct 2006
byObsidian Software
 
PDF
Ola Bini Evolving The Java Platform
bydeimos
 
JIT compilation in modern platforms – challenges and solutions
byaragozin
 
Stranger in These Parts. A Hired Gun in the JS Corral (JSConf US 2012)
byIgalia
 
Presentation slides for "A formal foundation for trace-based JIT compilation"
bymvdcamme
 
How difficult is to get a JIT right? Talk from ESGU 2024
byESUG
 
A Better Python for the JVM
byTobias Lindaaker
 
VMs, Interpreters, JIT
byMarcus Denker
 
JRuby 9000 - Optimizing Above the JVM
byCharles Nutter
 
Tail Call Elimination in Open Smalltalk
byESUG
 
PyPy's approach to construct domain-specific language runtime
byNational Cheng Kung University
 
Oh the compilers you'll build
byMark Stoodley
 
JRuby 9000 - Taipei Ruby User's Group 2015
byCharles Nutter
 
J9's MethodHandle Compilation Pipeline #JFokus2015
byDanHeidinga
 
OCAT: Object Capture based Automated Testing (ISSTA 2010)
bySung Kim
 
Efficient call path detection for android os size of huge source code
bycsandit
 
EFFICIENT CALL PATH DETECTION FOR ANDROID-OS SIZE OF HUGE SOURCE CODE
bycscpconf
 
Invoke dynamic your api to hotspot
byBoundary
 
Building High-Performance Language Implementations With Low Effort
byStefan Marr
 
«Python на острие бритвы: PyPy project» Александр Кошкин, Positive Technologies
byit-people
 
Lafauci dv club oct 2006
byObsidian Software
 
Ola Bini Evolving The Java Platform
bydeimos
 

Recently uploaded

PDF
Oracle AI Database 26ai _ AI-Native Database for Enterprises.pdf
byAstuteBusiness
 
PPTX
Short Film Script Development Process HARMONIA
byzdolezalmedia
 
PDF
A new Vision of Software Sustainability and its Engineering: Reflections and ...
byPatricia Lago
 
PDF
The Evolution of Project Portfolio Management - What Modern PMOs Are Doing Di...
byOnePlan Solutions
 
PDF
AI/ML Infra Meetup | Open Source Michelangelo: Uber's Predictive to Generativ...
byAlluxio, Inc.
 
PDF
NYC ACE 12-Nov-2025-Combined Presentation.pdf
byAUGNYC
 
PDF
Threat Hunting with Garuda: From Manual Analysis to AI-Driven Hunting By Monn...
byCysinfo Cyber Security Community
 
PDF
Combinatorial Interview Problems with Backtracking Solutions - From Imperativ...
byPhilip Schwarz
 
PDF
DSD-INT 2025 The Singapore Regional Model in Delft3D FM - Zijl
byDeltares
 
PDF
AWS Certified Developer - Associate certificate.pdf
byGabriellaSantosRodri
 
PDF
2025_11_19 - OpenMetadata Community Meeting.pdf
byOpenMetadata
 
PDF
The Ultimate Serial Port Detective – Baudowl
byCysinfo Cyber Security Community
 
PDF
Smarter Testing Safer Systems Balancing AI and Oversight in Regulated Environ...
byApplitools
 
PDF
AI/ML Infra Meetup | Unlock the Future of Generative AI: TorchTitan's Latest ...
byAlluxio, Inc.
 
PPTX
Performance Testing Transformation - LTB 2024
byAlexander Podelko
 
PPTX
BI and Reporting Software Turning Clinic Data Into Smarter, More Compassionat...
byEasy Clinic
 
PDF
Hyperon Chain – Smart Contract Security Audit Report by EtherAuthority
byEtherAuthority1
 
PDF
Speculative Automated Refactoring of Imperative Deep Learning Programs to Gra...
byRaffi Khatchadourian
 
PDF
DSD-INT 2025 3D Modeling of Shallow Water Dynamics Using Delft3D FM - Martins
byDeltares
 
PDF
vMix Overview & Key Features Easily Produce, Record and Live Stream
byIGM Soraing
 
Oracle AI Database 26ai _ AI-Native Database for Enterprises.pdf
byAstuteBusiness
 
Short Film Script Development Process HARMONIA
byzdolezalmedia
 
A new Vision of Software Sustainability and its Engineering: Reflections and ...
byPatricia Lago
 
The Evolution of Project Portfolio Management - What Modern PMOs Are Doing Di...
byOnePlan Solutions
 
AI/ML Infra Meetup | Open Source Michelangelo: Uber's Predictive to Generativ...
byAlluxio, Inc.
 
NYC ACE 12-Nov-2025-Combined Presentation.pdf
byAUGNYC
 
Threat Hunting with Garuda: From Manual Analysis to AI-Driven Hunting By Monn...
byCysinfo Cyber Security Community
 
Combinatorial Interview Problems with Backtracking Solutions - From Imperativ...
byPhilip Schwarz
 
DSD-INT 2025 The Singapore Regional Model in Delft3D FM - Zijl
byDeltares
 
AWS Certified Developer - Associate certificate.pdf
byGabriellaSantosRodri
 
2025_11_19 - OpenMetadata Community Meeting.pdf
byOpenMetadata
 
The Ultimate Serial Port Detective – Baudowl
byCysinfo Cyber Security Community
 
Smarter Testing Safer Systems Balancing AI and Oversight in Regulated Environ...
byApplitools
 
AI/ML Infra Meetup | Unlock the Future of Generative AI: TorchTitan's Latest ...
byAlluxio, Inc.
 
Performance Testing Transformation - LTB 2024
byAlexander Podelko
 
BI and Reporting Software Turning Clinic Data Into Smarter, More Compassionat...
byEasy Clinic
 
Hyperon Chain – Smart Contract Security Audit Report by EtherAuthority
byEtherAuthority1
 
Speculative Automated Refactoring of Imperative Deep Learning Programs to Gra...
byRaffi Khatchadourian
 
DSD-INT 2025 3D Modeling of Shallow Water Dynamics Using Delft3D FM - Martins
byDeltares
 
vMix Overview & Key Features Easily Produce, Record and Live Stream
byIGM Soraing
 

Stack Hybridization: A Mechanism for Bridging Two Compilation Strategies in a Meta Compiler Framework

  • 1.
    Stack Hybridization: AMechanism for Bridging Two Compilation Strategies in a Meta Compiler Framework Yusuke Izawa (伊澤侑祐), Tokyo Institute of Technology (東京工業大学), Master's thesis defence, 2020/1/31
  • 2.
    Creating a modernVM is a hard task 2 Context JIT Compiler Interpreter Optimizer FFI GC ... 10K lines or more Virtual Machine Debugger How to reduce engineering efforts?
  • 3.
    Generate a VMby merely writing an interpreter 3 Context: How to reduce engineering efforts? JIT Compiler Interpreter Optimizer FFI GC ... Interpreter Interpreter ... Focus VM Meta JIT Compiler Framework Interpreter Debugger Written by language designers
  • 4.
    Apply JIT compilationto a pair of meta-interpreter and base-program 4 Context: How a Meta JIT Compiler Works Meta-interpreter (Meta) JIT compiler Base-program sum fib Native code sum: ... ... ... ... fib: ... ... ... ... Next: Two mainstream meta JIT compiler frameworks
  • 5.
    RPython and Truffle/Graal 5 Context:State-of-the-arts RPython with Meta-tracing[C.F.Bolz et al., '09] Truffle/Graal with Partial Evaluation[T.Wurthinger et al., '12] S t r a t e g y Trace-based Method-based ● crossing function calls ● one-side of a conditional branch
  • 6.
    Trade-offs between trace-and method-based JIT 6 Problem trace-based S t r a t e g y method-based fib sum nest-fun fib sum nest-fun G o o d B a d Path-Divergence Problem[R.Huang et al., '16]
  • 7.
    ● Approach: ○ Extenda meta-tracing JIT compiler framework ● Proof-of-concept: ○ BacCaml, based on the MinCaml[E.Sumii, '05] compiler ■ RPython is too complicated ○ Implemented in OCaml (6000 SLOC), C (300 SLOC), Assembly (200 SLOC) Meta-hybrid JIT Compiler Framework 7 Proposal Next: Our previous work
  • 8.
    8 Previous Work andits Problem trace-based compilation method-based compilation by tracing Step1: Method JIT by Tracing (Bachelor’s thesis) Goal: Apply different strategies to different area Different management for return address meta-interpreter for trace-based comp. meta-interpreter for method-based comp.
  • 9.
    9 This Work trace-based compilationmethod-based compilation by tracing Step2: Stack Hybridization (This work) Goal: Apply different strategies to different area S.H. meta-interpreter for both comp.
  • 10.
    10 Problem on PreviousWork Native from trace-JIT Native from method-JIT def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] ret_addr = pc + 1 user_stack.push(ret_addr) pc = addr elsif == RET: v = user_stack.pop() addr = user_stack.pop() user_stack.push(v) pc = addr Different management for return address def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] r = interp(addr) user_stack.push(r) pc += 1 elsif == RET: v = user_stack.pop() return v managed by host-stack Meta-interpreter for trace-JIT Meta-interpreter for method-JIT User-stack style Host-stack style managed by user-stack
  • 11.
    A meta-interpreter designusing two kinds of stack 11 Our Solution: Stack Hybridization (1) ● Design choice: Embed two styles into a single definition if instr == CALL: addr = bytecode[pc] if mj_context(addr): user_stack.push(HS) # call under method-JIT ctx else user_stack.push(US) # call under trace-JIT ctx elsif == RET: … ctx = user_stack.pop() if ctx == US: # return under trace-JIT ctx else: # return under method-JIT ctx Put flag Check flag For changing the JIT context Native from trace-JIT Native from method-JIT
  • 12.
    Example compiled code:sum-fib 12 Our Solution: Stack Hybridization (2) sum (trace-JIT) fib (method-JIT) @method_jit def fib(n) = if n < 2 then 1 else fib(n-1) + fib(n-2) @trace_jit def sum(n) = if n < 1 then 1 else fib(n) + sum(n-1) sum(100) .globl tracetj1.890 tracetj1: movl ebx, %ecx subl $3, %ecx … movl %ebx, %ecx addl $1, %ecx movl $200, (%eax,%ecx,4) addl $2, %ebx jmp tracemj0.876 … .globl tracemj0.876 tracemj0.876: movl ebx, %ecx subl $2, %ecx … movl $100, (%eax,%ebx,4) movl %ebx, %ecx movl $2, %ecx … addl $16, %ebp call tracemj0.876 subl $16, %ebp … movl (%eax,%esi,4), %esi cmpl $100, %edx jne jne_else.868 movl %ecx, %eax ret je_else.868: movl $56, %edx … movl %esi, %eax jmp guard_tracemj0.846 Put flag Put flag Check flag
  • 13.
    .globl tracemj0.876 tracemj0.876: movl ebx,%ecx subl $2, %ecx … movl $100, (%eax,%ebx,4) movl %ebx, %ecx movl $2, %ecx … addl $16, %ebp call tracemj0.876 subl $16, %ebp … movl (%eax,%esi,4), %esi cmpl $100, %edx jne jne_else.868 movl %ecx, %eax ret je_else.868: movl $56, %edx … movl %esi, %eax jmp guard_tracemj0.876 Example compiled code: sum-fib 13 Our Solution: Stack Hybridization (2) .globl tracetj1.890 tracetj1: movl ebx, %ecx subl $3, %ecx … movl %ebx, %ecx addl $1, %ecx movl $200, (%eax,%ecx,4) addl $2, %ebx jmp tracemj0.876 … .globl interp interp: movl %ebx, 0(%ebp) … sum (trace-JIT) fib (method-JIT) @method_jit def fib(n) = if n < 2 then 1 else fib(n-1) + fib(n-2) @trace_jit def sum(n) = if n < 1 then 1 else fib(n) + sum(n-1) sum(100)
  • 14.
    Benchmarking the performanceof BacCaml 14 Evaluation: Overview ● Preparation: ○ Write a small programming language almost equivalent to MinCaml in BacCaml ● Evaluation 1: ○ Compare trace-JIT, method-JIT with MinCaml ■ Run a part of shootout benchmark https://dada.perl.it/shootout/index.html ● Evaluation 2: ○ Compare hybrid-JIT, trace-JIT, method-JIT with MinCaml ■ Combine functions that are suitable for trace-JIT and method-JIT respectively based on eval. 1 Environment: Arch Linux, CPU; Intel 2.7GHz Core i5, Mem; 8GB
  • 15.
    15 Evaluation 1 better ● TracingJIT is suitable for branching control flow (1.28x faster than method JIT) ● Method JIT is good for any programs (around 10x ~ 100x faster than tracing JIT)
  • 16.
    16 Evaluation 2 better ● HybridJIT is about 1.1x better than method JIT only, and over 100x better than tracing JIT only
  • 17.
    ● Summary ○ Proposal:a meta-hybrid JIT compiler framework and its poc implementation, BacCaml ○ Performance: hybrid JIT is about 1.1x faster than method JIT only (and about 100x faster than tracing JIT only) ● Future work ○ Add some optimizations for resulting traces ○ Investigate profiling scheme ○ Apply our approach to RPython 17 Conclusion
  • 18.
    18 Further Future work trace-JIT+ method-JIT in Meta-tracing (PoC) Support language designers w/ Language Workbench [M.Fowler, '05] ● DynSem for Truffle[V.Vergu, '19] ● PySem for RPython region-JIT[G.Ottoni, '18] in Meta-tracing (PoC) trace-JIT + method JIT in Meta-tracing (Practical)
  • 19.
    Method JIT byTracing: Overview ● Create method-based compilation by customizing following points: ○ tracing entry/exit points ○ conditional branch ○ loop ○ function calls 19 Appendix: Our Previous Work
  • 20.
    Why We NeedTwo Different Styles 20 Appendix: Our Previous Work CALL RET def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] r = interp(addr) user_stack.push(r) pc += 1 elsif == RET: v = user_stack.pop() return v base-program meta-interpreter Not inline Emit "CALL" to log log
  • 21.
    Method JIT byTracing 21 Appendix: Our Previous Work CALL RET RET b a c k tr a c k base-program
  • 22.
    Why cannot unifyinto one style: Trace-JIT and Host-stack style 22 Appendix: Integration Problem in Meta-interpreter (1) def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] r = interp(addr, user_stack) user_stack.push(r) pc += 1 elsif == RET: v = user_stack.pop() return v ● cannot convert a non-tail recursive call into a loop ● why?: stack frame isn't mached recursive call
  • 23.
    Why cannot unifyinto one style: Method-JIT and User-stack style 23 Appendix: Integration Problem in Meta-interpreter (2) def interp(pc, user_stack): … while True: … if instr == CALL: addr = bytecode[pc] ret_addr = pc + 1 user_stack.push(ret_addr) pc = addr elsif == RET: v = user_stack.pop() addr = user_stack.pop() user_stack.push(v) pc = addr ● difficult to trace the successor of a deep fun. call ● why?: too much time to reach the RET
  • 24.
    ● GraalSqueak [F.Niephaus, et al., '18] ○ Idea: designing a meta-interpreter to enable trace-based compilation on Truffle/Graal ○ Methodology: adding runtime annotations to a meta-interpreter definition written in Truffle ○ Performance: about 1.4x faster than OpenSmalltalk VM ● HHVM JIT Compiler [G.Ottoni, '18] ○ Idea: performing JIT compilation based on the combination of basic blocks ○ Performance: about 1.2x efficient than normal trace-based compilation (HHVM 1st gen.) 24 Related work
  • 25.
    25 Our Solution: StackHybridization (2) Example Compiled Code: fib-sum ... Call Guard fail ... ... ... ... ... ... Ret @tracing_jit def sum(n) = if n < 2 then 1 else fib(n-1) + fib(n-2) @method_jit def fib(n) = if n < 1 then sum(1000) else fib(n-1) + fib(n-2) fib(30)
  • 26.
    26 Context: Building avirtual machine Current Situation w/ Meta JIT Compiler Framework Prototype Build a virtual machine (VM) Improve runtime system Continue to improve... Define syntax and abstract syntax tree (AST), AST interp. Still using AST interpreter, build a lot of optimizer like GC... Define bytecode format, and bytecode interpreter Performance still bad, thus write JIT compiler Prototype Define syntax and abstract syntax tree (AST) Integrate with Meta JIT Write an interpreter with a meta JIT compiler framework Already it is fast Empowered by Meta JIT compiler
  • 27.
    Integration Problem: Failingto find return address in naively mixed execution 27 Technical Problem in Previous Work: Details (1) ● Example 1: calling a trace-compiled function (sum) from a method-compiled function (fib_s)
  • 28.
    Integration Problem: Failingto find return address in naively mixed execution 28 Technical Problem in Previous Work: Details (2) ● Example 2: calling a method-compiled function (fib) from a trace-compiled function (sum_f)
  • 29.
    Trade-offs between trace-and method-based JIT 29 Problem ● straight-line control flow ● smaller compiled code ● automatic function inlining ● matured technique ● apply to any programs trace-based ● bigger compiled code ● careful management for function inlining ● performance goes worse in complex control flow[R.Huang et al., '16] S t r a t e g y method-based Our Solution: Combine trace- and method-based compilation
  • 30.
    Trade-offs between trace-and method-based JIT 30 Problem ● branching control flow ● smaller compiled code ● good for a deep-nested fun. call ● matured technique ● apply to any programs trace-based ● bigger compiled code ● careful management for function inlining ● performance goes worse in complex control flow[R.Huang et al., '16] S t r a t e g y method-based fib sum nest-fun fib sum nest-fun G o o d B a d
  • 31.
    How Stack Hybridizationworks in sum-fib 31 Our Solution: Stack Hybridization (2) ... ... ... ... JUMP sum fib ？？？ @method_jit def fib(n) = if n < 2 then 1 else fib(n-1) + fib(n-2) @tracing_jit def sum(n) = if n < 1 then 1 else fib(n) + sum(n-1) sum(100) ...
  • 32.
    ● Tracing JITis suitable for straight-line control flow (1.28x faster than method JIT) ● Method JIT is good for any programs (around 10x ~ 100x faster than tracing JIT) 32 Evaluation 1: Comparing tracing and method JIT performance better
  • 33.
    ● Hybrid JITis 1.36x better than method JIT, and over 100x better than tracing JIT 33 Evaluation 2: Hybrid JIT performance better