The State of Babel

December 7, 2016

Henry Zhu

Previous issues: Babel Roadmap #4130, 6.0 #2168

Please check out the Community section if nothing else.

Also published as part of Mariko Kosaka's 2016 Web Advent Calendar!

Some History

Sebastian created "6to5" in September of 2014. Interestingly enough, he made it to scratch an itch he had with understanding programming languages and how they work. You might have assumed whoever created the project already knew how compilers worked and understood JavaScript perfectly... but you would be wrong! Check out his post for a great read of his story: ~2015 in Review.

6to5 did exactly that: easily turn ES6 code into ES5 code. When 6to5 became Babel as mentioned in Not Born to Die, it became a platform: a suite of tools designed to create the next generation of JavaScript tooling. No longer did it just compile ES6 to ES5, it allowed developers to build tools on top of it.

Here are some of our milestones:

  • In 5.0.0, Babel aligned more with the TC39 process by introducing stages, added a .babelrc config option, and created a plugin system for custom transforms.
  • In 6.0.0, Babel became modular (a pretty controversial idea at the time). This was a huge change that led to opt-in functionality (no defaults) and the concept of Presets and Plugin Options.
    • As mentioned in his article, Sebastian joined Facebook in July 2015 and worked on the entire development of Babel 6 on company time.
  • 6.3.13 Sebastian extracted our monorepo build/publish tools into what is now Lerna. (What's funny is James rewrote it 3 times and I had to review everything)
    • After this was around when both Sebastian and James got burned out on Babel, and a few contributors stepped up.
    • We struggled to find direction and deal with the bugs/requests coming in but we got a lot of stuff done!
  • 6.13.0 finally added Preset Options.
  • 6.14.0 added a latest-preset that keeps up to date with the yearly JavaScript specification.
  • 6.16.0 allowed changing out the parser or code-generator.
  • In August, we released Babili, a minifier based on Babel.
  • In September, we released the first version of babel-preset-env (read on for details).
  • After a year on Phabricator, we moved back to to GitHub issues thanks solely to @danez and his amazing (and underappreciated) work.

If you're using Babel, let us know with a PR to our users page!

Now babel-core is downloaded over 5 million times per month and almost 60 million times total and used at huge companies like Facebook/Netflix/Airbnb and other OSS projects like React/Yarn.

Thanks everyone for your continued support! We want to continue acting as the foundation of the JavaScript toolchain: compilation, linting, minification, codemods, code coverage, etc.

Current Status

If you're interested in helping out please check out the issues linked below!

Maintaining Babel plugins for each proposal in TC39 starting from Stage 0

TC39 stands for Ecma International, Technical Committee 39: it's the committee that makes JavaScript.

Babel uses TC39's concept of stages to categorize its experimental plugins. Users should be able to easily use features before they are all implemented in browsers in stage 4 of the TC39 process.

Babel is fundamental in the process given its place in the ecosystem: it is significantly easier for a developer to update a .babelrc file than a browser flag and much faster to write a Babel plugin than to implement the feature natively in the browser. This is the core of Babel.

But the proposals process involves significant iteration: proposals can change in syntax or even get dropped. Because TC39 meets every 2 months, plugins should be updated as least as often as each change in stage so that users can be in sync.

Early feedback to the proposal champion and committee is extremely valuable although it is recommended to use Stage 0/1/2 features with caution. Although companies like Facebook take advantage of these kinds of features, they have created codemods to allow easy change.

There isn't enough time or resources to maintain each plugin, especially when there are spec updates.

Next we're going to be working with:

Relevant Issues:

Check out thefeedbackloop.xyz for more info on TC39!

Maintaining other ecosystem plugins: JSX/Flow

Babel is vital to the React and Flow ecosystems, and we work closely with the relevant teams at Facebook.

Relevant Issue Labels:

babel-preset-env ("autoprefixer" for Babel)

JavaScript compilation is a moving target: There are yearly updates to the spec, browser vendors are constantly updating to that spec, and users may drop support for earlier browsers. At first glance, there doesn't seem to be a fixed target for what we should compile our JavaScript down to.

The compat-table is updated constantly and is used for this preset.

This is where babel-preset-env comes in: it's a Babel preset that automatically determines the correct Babel plugins to use based on the provided environments.

Its goal is both simplicity in use and efficiency in output: you only need to worry about your target environments to be able to take advantage of native code. The preset decides for you the required plugins.

Some example configs

Targeting Chrome 55 + last 2 versions of other browsers via browserslist

{
  "presets": [
    ["env", {
      "targets": {
        "chrome": 55,
        "browsers": ["last 2 versions"]
      }
    }]
  ]
}

Targeting the current Node.js version (uses process.versions.node)

{
  "presets": [
    ["env", {
      "targets": {
        "node": "current"
      }
    }]
  ]
}

Chrome 55 useBuiltIns + webpack 2

{
  "presets": [
    ["env", {
      "targets": {
        "chrome": 55
      },
      "modules": false,
      "useBuiltIns": true
    }]
  ]
}

In

import "babel-polyfill";

Out (different based on environment)

import "core-js/modules/es7.string.pad-start";
import "core-js/modules/es7.string.pad-end";

Relevant Issues:

Linting via babel-eslint

example of eslint

ESLint doesn't support new language features until they reach Stage 4 of the proposals process. For this reason we maintain babel-eslint (a custom ESLint parser) so you can continue to lint JavaScript with experimental syntax.

This project was one of the hardest projects to work on: because it is just a compatibility layer between Babel and ESLint there is inherently a constant need for updates when either projects update and a high risk of unexpected changes due to monkey-patching. It was unfortunate to get issues like babel/babel-eslint#243 or babel/babel-eslint#267.

To that end, we'd like to lessen the maintenance burden of this project by improving our scope and traversal interop with ESLint. It might even be interesting to be able to write ESLint rules using babel APIs or vice versa?

Relevant Issues:

Minification

Babili is our new Babel-powered minifier, enabling you to produce minified code while targeting the latest browsers.

In

class Mangler {
  constructor(program) {
    this.program = program;
  }
}
new Mangler();

Out

// ES2015 code -> Babili -> Minified ES2015 Code
class a{constructor(b){this.program=b}}new a;

Check out our blog post for more info.

Since it was recently released, we're looking for some new contributors! There's a lot of small bugs and things that could be improved for those looking for a new project to help out with!

Codemods / Refactoring / eslint --fix

A codemod is a tool-assisted code modification; a fancy find-and-replace.

If you wanted to change merge({}) to Object.assign({}) (and maybe later object rest) you might do a regex replace. But you don't want to replace the other parts of the code that also might be called merge such as imports/exports, strings, comments, and local variables. To do it safely you'll want something more powerful that only changes the specific code you need.

Although Babel already handles transforming code into other code, it doesn't really care about the styling of the input code as a compiler. After using Babel with its default options to do a codemod, git diff looks really messy.

Enter Recast: a tool that preserves the formatting of unmodified code while also pretty-printing any new AST nodes.

recast

In the screenshot above, the top-left pane is the input code and the bottom-right pane is the output code after the babel plugin is run. In this case it's preserving the whitespace of the input code when possible.

By passing in Recast in the options, Babel can power the future of codemods.

.babelrc

{
  "parserOpts": {
    "parser": "recast"
  },
  "generatorOpts": {
    "generator": "recast"
  }
}

Run the relevant Babel transform(s) on the source code and overwrite it:

babel src -d src

This feature was just made possible so we're looking forward to making it easier to use and seeing the transformations it can enable. Check out the 6.16.0 blog post for more info!

Other relevant projects: JSCodeshift, js-codemod, Lebab.

Relevant Issues:

Code Coverage / Instrumentation

We want to support tools like nyc and babel-plugin-istanbul.

Plugin Ecosystem

Thanks to our vibrant community, new plugins are constantly being created: whether it be a new way to write your css in jsx or rewire your tests.

Currently there are > 1200 babel-plugins on npm.

We've had some interesting discussions on how we can grow and support the plugin ecosystem. We could try to watch all the repos but that is obviously overwhelming.

It might be interesting to create some bots to automate a few tasks: create specific Babel plugins/ESLint rules for babel-plugins, write codemods to update API changes, and integrate plugins into our smoke test.

  • Should we create a newsletter for new/useful plugins?
  • How can we teach people about plugins and how to write them?
  • How can we make ASTExplorer better?

Documentation (this website!)

Docs contributions have definitely been lacking in the last year.

However just recently we've done a lot of awesome stuff:

We've also added new collaborators:

  • @STRML: Added Discourse to all github pages via #875
  • @xtuc: Added support for reading the README from the babel repo so we don't have to sync 2 copies of docs via #990
  • @fredericmarx: Added support for a copy to clipboard button for code snippets via #998
  • @seedofjoy: Added a resize ability for the REPL via #1003

Some Ideas

  • All plugins should have examples. Can also embed RunKit or the REPL.
  • Update FAQ with common errors
  • API docs / make babel-handbook better

Relevant Issues:

The Future

NOTE: Everything below can be changed or dropped. Some might be already in the works and others are just suggestions that need a proper discussion/champion.

Priority should be determined on what the community needs: not because it would be nice to have.

Plugin API Changes

There is a lot of confusion around how plugins/presets interact regarding ordering. This results in bugs and issues with the config which require users to place plugins before/after others in a non-intuitive way.

We’re currently discussing API changes that could reduce the confusion. However, since this is a fundamental change to the core of Babel, it might take a while to figure out the best approach.

Versioning

Since Babel 6 we've used a "fixed" mode of versioning via Lerna. This is what allows us to release multiple packages at the same time all under the same version (if that package changes). This is nice because you don't have to manually set a version for each package but everything moves together. The only time this could cause an error is when one on the packages makes a breaking change: then every package also will bump major versions.

This is explained more in babel/notes but we still need to figure out the best plan of action for the project.

What happens when we need to update a Stage 0 spec to Stage 1 and it's a breaking change to the parser? Are we just going to bump the major version, wait to batch some changes up, or figure out how to do it via multiple versions of plugins?

Discussion Issue

Changing the Mindset Around Stage X Presets

Relevant Issues:

Speed

Performance is a feature! Other things can be more of a priority at times (bug fixes, spec compliancy, etc) but it's still important in a few different aspects.

  • How can we reduce the size/time of install, especially for a project with multiple packages? (this is helped by yarn)
  • How can we parse faster?
  • How can we make faster plugins (and measure them individually)?
  • How can we generate the transformed code faster?
  • How can we generate code that runs fast in the browser (https://fhinkel.github.io/six-speed/)?

If you to read the compiled output and see issues, then report it and ask for help in making a PR!

Previous Issues:

Possible TypeScript Support?

Maybe Babel could learn to understand TypeScript syntax (like we do with Flow)? We could add a plugin to strip TypeScript types for better interop.

This would mean parsing TypeScript specific syntax and stripping it out. However TypeScript does have non-type syntax, so for things like enum we will would have to discuss more.

Use Type Information?

Integrate with a type system such as Flow/TypeScript to make optimizations. This just means that Babel should be able to gain the knowledge through those tools that an identifier arr is actually an Array or not.

There are a few plugins that actually do type checking: babel-plugin-typecheck and babel-plugin-tcomb

Taking in a Dependency Graph / Operate Multi-file?

Then we could integrate with other tools like Webpack better. This would allow cross file transformations or whole codebase optimizations. The main reason for this would be for the minifier (being able to remove properties based on checking the usage across the whole application) or for example providing errors for missing/invalid import/exports.

Parser Errors

Better error messages from the parser like in Compiler Errors for Humans.

It is obvious that we all want to see helpful errors!

We can do better inference/guessing on user intention to prevent vague errors. Let us know in what cases that happens!

Relevant Issues:

babel-init

Basically a way to set up Babel more easily like create-react-app does.

  • Set up a .babelrc from scratch, prompt with questions

Possible Idea:

  • Ask about target environments (browsers, node) and pass to babel-preset-env
  • Ask about experimental features (add specific plugins)
  • Update the babel npm package to do something again: Make it the default/opt-in/opinionated babel that Babel 5 was. It can default to using env and say the latest 2 browsers (without any config).

Relevant Issues:

Run tc39/test262

test262 tests conformance to the continually maintained draft future ECMAScript standard found at tc39.github.io/ecma262, together with any Stage 3 or later TC39 proposals. It is maintained by Tom Care (@tcare) with significant contributions from many in the ECMAScript community.

Running the official spec tests against Babel can make sure we comply to the spec or at least know when we don't. We'll want to figure out how to do filtering on things we can't compile (proxy, TCO, etc) and set up an easy way to check failing tests and file issues and PRs for them.

Relevant Issues:

Smoke/Integration Tests

It would be useful to either have a reverse Greenkeeper or to run tests with master branch of Babel so that we can catch any major regressions before any release (node has the citgm project for this). In theory we would want to take the biggest open source projects that use Babel and run their tests.

motiz88/babel-smoke-tests is a good start! It would also be great if there's already something like this out there!

Program Analysis

There was a great talk by Alan Shreve called "Idealized Commit Logs: Code Simplification via Program Slicing" which inspired @kentcdodds to try it out in JavaScript via slice-js.

The general idea is that we have a lot of tools to help us write code but not a lot for helping us understand/read code. You can think of code-slicing as a form of targeted dead-code elimination.

slice-js

A program slice basically cuts away from the source code the code that isn't used for a test case that you run. If there are lots of if statements and loops that aren't run during your usecase then it won't show up in the program slice.

  • Semantic (AST aware) Grepping tool?

Similar to graspjs, I think it would be interesting to be able to do a find-replace with the input being an AST. It would allow us to create other analysis tools: the ability to find all IIFE's in our code, the number of times a method is called, or even how many Classes we have in our codebase.

babel --settings

This command would print out all info (also when erroring). It would also include performance metrics on how long each plugin takes.

Parser Unity

There have also been some discussions around parser/AST unity, in TheLarkInn/js-parser-discussions and previously with ESTree.

Unfortunately with Babel 6, we have "forked" and have a few differences in our AST than ESTree. Babel aims to support stage x features while other parsers may only want to support stage 4 features. We all might prioritize certain things more than others regarding spec compliancy, performance, stage x features, error messages, extensibility, releases, etc. However it's important for us to be open to breaking changes that may lead to better interop and a better community.

Sweet.js Interop?

Previous issue. Maybe we can just figure out how to have better interop instead?

Node.js Support

Should we drop support according to the EOL of Node.js versions? How long should we wait to do this in general?

  • Do we want to continue to support our users who haven't updated yet?

  • There are certain transforms/PRs that are blocked by this, due to tools that have also dropped older versions.

  • Many other build-time projects like ESLint have done so already.

  • Are we going to make a major version just for this or plan out other changes in addition?

  • Discussion Issue

Babel 5 to 6 transition / Upgrade Paths

Babel 6 was really difficult for the community to upgrade to. The initial release was a bit rushed. Although we did have a 6.0 release post, a setup guide soon after, a even a tool (now deprecated) to help with the transition it was still difficult for users to understand the changes.

There are still a good amount of users on Babel 5 that we don't want to leave behind. What can we do to help them upgrade? What steps do we need to take in the future to make sure the same thing doesn't happen with a Babel 7? Are there other projects/communities we should be reaching out to and helping?

Relevant Issues:

What else?

Anything else not already brought up here? Please send us a tweet @babeljs, a message on our slack (join at https://slack.babeljs.io/, comment on this post, or create an discussion issue in our repo!)

Are there projects or communities we should partner with more? How can we make the contributing experience more welcome? What can we do to make the development process more transparent?

Community

Old Issues:

You might think that as a project gets more widely used that more people show up to help out. But like most OSS projects that aren't backed by a company, there is a constant issue with maintenance and sustainability; people get burned out, move on to other cool projects, or get busy with work/family/etc.

Like James describes in Dear JavaScript, the current Babel team is pretty small.

Babel isn't a company, a special team at Facebook, or corporate-funded OSS project. It's a community-driven effort currently held up by a few people and we want that to grow.

So if you're interested in contributing to a tool you use, we're hoping this could be the one!

What issues should I look at or contribute to?

Many of our projects have both beginner-friendly and help-wanted labels. You can also check out discussion.

Team

Our core team is:

And just in the last 3 months a lot more collaborators:

Babili core team:

As mentioned above we have a lot of website collaborators:

Inactive but still a resource:

How can I contact the team?

GitHub

For bug reports/PRs you can check our repositories.

Twitter

You can ping us on Twitter with @babeljs - or mention us individually.

I myself joined Twitter to be able to talk with users and help out. Being able to post new features and changelogs is really useful and can help give us feedback!

Slack

We have a relatively active community on there!

You'll find a lot of awesome community members willing to help such as Jordan Harband, @ljharb, Jessica Franco, @Jessidhia, Jimmy Jia, @taion, Denis Pushkarev, @zloirock and more!

If you just have questions join #discussion and if you want to help or listen in check out #development.

We try not to discuss in private if there's no need to: I myself usually post the issues/PRs I'm working on for people to review and talk about.

Other

How else can we interact with the community? Should we go and start meetups, go to conferences, or manage hackathons?

How can we make Babel sustainable? Should we setup a Open Collective or seek a foundation? Should we pay for a project manager?

Let us know your thoughts! What do you want out of Babel?

See typos/issues? Please send a PR or comment on babel/babel.github.io#1014

If anything we want to let you know that many of us got our start with Babel in order to learn JavaScript rather than helping because we already knew it. Personally, I knew nothing about compilers and had just learned what ES6 was when I found the project. I got here because of a tiny bit of curiosity and some encouragement from a lot of people. I want to carry that forward and hope that we can all succeed together.

Thanks so much for reading!

Henry Zhu (@hzoo) (@left_pad)

Thanks to way too many folks for their review and thoughts: @DrewML, @mrjoelkemp, @kentcdodds, @existentialism, @jdalton, @gaearon, @nolanlawson, @jayphelps, @montogeek, @TheLarkInn, @jasonLaster, @benjamn, @addyosmani, @Daniel15, @loganfsmyth, @gr2m, @mathiasbynens, @chicoxyzzy, @bvaughn, @bcoe.

Recent Posts