Transpiler

Summary

Milestones

1979

Intel publishes details of a converter called CONV86. It can convert from 8080/8085 assembly source code to 8086 assembly source code. Since the latter is a typed language, the converter assigns types to each symbol. Note that the term converter is used. The term transpiler possibly came later.

1980

Tim Paterson writes TRANS.COM to translate Z80 assembly code into .ASM assembly code for the Intel 8086 processor. The process is not fully automated and requires manual corrections.

1981

Gary Kildall writes a program called XLT86 to transpile .ASM assembly code written for Intel 8080 processor into .A86 assembly code for the Intel 8086. The idea is to automatically port P/M-80 and MP/M-80 programs to CP/M-86 and MP/M-86 platforms.

1983

Bjarne Stroustrup, the creator of C++, creates Cfront to transpile C++ code to C code since C++ doesn't have its own compiler at the time. Development of Cfront is discontinued in 1993.

2010

Facebook creates HipHop for PHP (HPHPc) for transpiling PHP to C++ for better runtime performance. This is deprecated in 2013 by HipHop Virtual Machine (HHVM) that uses just-in-time compilation. HPHPc did not fully support PHP, its performance flattened out and its deployment across all of Facebook's servers was not trivial.

Sep
2014

Sebastian McKenzie creates a transpiler named 6to5, which translates ES6 code to ES5 code. In February 2015, this becomes Babel.

Discussion

• How is a transpiler different from a compiler?

  

  Transpilation is not the same as compilation. Source: Ackerman 2016.

  A compiler converts high-level human-readable code to low-level instructions that can run on a computer. For example, a C compiler converts C code to machine executable code. A Java compiler converts Java code to what's called Java bytecode that's interpreted at runtime to execute on the target machine.

  A transpiler on the other hand usually works at the abstraction of high-level languages. The output code is still human readable. It cannot be executed directly unless its own compiler or interpreter is invoked. For example, a transpiler can convert Java code to C++ code. Programmer will still need to invoke a C++ compiler before executing the resultant machine code.

  However, some folks use these terms interchangeably. For example, VOC is stated to be transpiler from Python source code to Java bytecode when in actual fact these two are at different levels of abstraction. However, it's acceptable to call translations at the same level of abstraction as transpilation, such as from .ASM assembly code to .A86 assembly code.

  ISO/IEC/IEEE 24765 defines five generations of languages that can be related to abstractions described above.

• Why would I want to use a transpiler?

  

  Critical Python code transpiled to Fortran for performance. Source: Bysiek et al. 2016, fig. 1.

  Here are a few reasons for using a transpiler:

  • Migration: Migrate legacy code to a modern language.
  • Compatibility: Generate code conforming to an older version while developers benefit from features of a newer version, such as with browsers not updated to latest JavaScript standards.
  • Coding Skills: Codebase can be transpiled to a language in which the skills are readily available. Or it might be a matter of preference. For example, those who are from OOP background prefer TypeScript. Python coders prefer CoffeeScript instead. In both cases, code is transpiled to JavaScript.
  • Performance: Initial code was written for quick prototyping or project is moving to a different platform where another language is more suitable. Perhaps the target language has a better compiler that can generate more optimized code. For example, critical parts of a Python codebase can be transpiled to Fortran and then called from Python.
  • Tooling: Source language has better IDEs, testing and debugging tools, such as with .NET and Visual Studio.
• How does a transpiler work internally?

  

  Steps that a transpiler takes. Source: Samuels 2017.

  A transpiler parses the code and extracts the tokens, which are the basic building blocks of the language. Language keywords, variables, literals and operators are examples of tokens. This step is a combination of lexical analysis and syntax analysis. Transpiler knows how to do this because it understands the syntax rules of the input language. Given this understanding, the transpiler builds what is called Abstract Syntax Tree (AST).

  The next step is to transform the AST to suit the target language. This is then used to generate code in the target language.

  AST Explorer is an online tool to help you view the AST given a piece of code in many popular languages.

• Could you name some transpilers out there?

  JavaScript is a language that's been evolving at a regular pace but there are also lots of browsers out on the web using older versions of the language. Transpilers are therefore used commonly in the JS world to transpile code to ES5, a version that's supported by most browsers. There are lots of transpilers for JS but the popular ones are Babel, TypeScript and CoffeeScript.

  ClojureScript transpiles from Clojure to JS. JSweet transpiles from Java to TypeScript or JS. GWT Web Toolkit, formerly known as Google Web Toolkit, enables Java programmers to use JS frontend for browser-based applications. GWT includes a transpiler from Java to JS. Script# transpiles from C# to JS. C2Rust and Corrode are two alternatives for transpiling C to Rust.

  VOC is a transpiler from Python to Java, which means that even Android apps can be built from a Python codebase. Python itself has two incompatible versions: 2 and 3. However, tools are available to convert from one version to the other: 2to3, modernize, futurize and pasteurize. These could be termed as transpilers.

References

1. Ackerman, David. 2016. "WebAssembly: Finally Freed from JavaScript?" Simple Programmer, May 25. Accessed 2018-06-03.
2. Bysiek, Mateusz, Aleksandr Drozd, and Satoshi Matsuoka. 2016. "Migrating Legacy Fortran to Python While Retaining Fortran-Level Performance through Transpilation and Type Hints." 6th Workshop on Python for High-Performance and Scientific Computing (PyHPC), via Semantic Scholar. Accessed 2018-06-03.
3. ISO/IEC/IEEE. 2017. "Systems and software engineering — Vocabulary." ISO/IEC/IEEE 24765:2017, International Standard, Second edition, September. Accessed 2019-01-24.
4. Intel. 1979. "MCS-86, Assembly Language Converter, Operating Instructions for ISIS-II Users." Accessed 2018-06-05.
5. Keith-Magee, Russell. 2017. "How to write a Python transpiler." BeeWare, PyCon US 2017, Portland, Oregon, May 17-25.
6. Kyle, James. 2015. "Not Born to Die." Babel Blog, February 15. Accessed 2018-06-03.
7. Mancini, Alberto. 2015. "The GWT Toolkit: Build Powerful JavaScript Front Ends Using Java." Toptal. Accessed 2018-06-03.
8. Pawlak, Renaud. 2016. "Comparing the GWT transpiler with JSweet." JSweet, February 10. Accessed 2018-06-03.
9. Samuels, Charles. 2017. "A World of Javascript Transpilers." Site, May 31. Accessed 2018-06-03.
10. Schulz, Marius. 2015. "The Era of Transpilers." Blog, May 04. Accessed 2018-06-03.
11. Sengstacke, Peleke. 2016. "JavaScript Transpilers: What They Are & Why We Need Them." Scotch, April 25. Accessed 2018-06-03.
12. Sharp, James. 2018. "c2rust vs Corrode." non-O(n) musings blog, June 30. Accessed 2018-07-02.
13. Wikipedia. 2018a. "Source-to-source compiler." Wikipedia, May 31. Accessed 2018-06-03.
14. Wikipedia. 2018b. "Cfront." Wikipedia, April 18. Accessed 2018-06-03.
15. Wikipedia. 2018c. "HipHop for PHP." Wikipedia, March 25. Accessed 2018-06-03.
16. Zacina, Faris. 2015. "How Babel compares against TypeScript." Ministry of Programming Blog, August 23. Accessed 2018-06-03.
17. Zhu, Henry. 2016. "The State of Babel." Babel Blog, December 7. Accessed 2018-06-03.
18. nikhilk GitHub. 2013. "scriptsharp." Accessed 2018-06-03.
19. pybee GitHub. 2018. "VOC." Accessed 2018-06-03.