Tall style implementation

[munificent]

This meta issue tracks work remaining to be done to ship the new "tall style" proposal (#1253).

Implement tall style:

• Implement all language syntax using the new internal representation and formatter. There should be no more throw UnimplementedError in the codebase.

• (Make the formatter aware of the language version of what it's formatting #1402) Add support for detecting the language version of each file being formatted.

• Address remaining TODO(tall) and TODO(perf) comments in code. For many of these, we may ultimately decide to do nothing, keep the current behavior, and simply remove the TODOs. But once we have migrated the regression tests and run the new formatter on a large corpus, we should be in a better position to make those decisions.

• Migrate the regression tests. Since we'll support both the short and tall styles for a while, we'll want to fork them and keep the old ones. There is a test/regression_tall/ directory already, so we can just fork and migrate the test/regression/ subdirectories into there and reformat them using the new tall style. We'll probably want to do it a subdirectory at a time (since it's so much code). The important part is to manually review the formatted results and see if the new style is producing the output we want.

• Run the new formatter on a large corpus of code. Look at the diffs and evaluate the results. Does it look like we expect? Does it produce weird output on unusual code? Are there style rules or heuristics we want to tweak? Does the performance degrade on some kinds of code?

• Close issues that are now fixed or invalid. There are likely a bunch of long-standing style issues that the new style either directly fixes or where the issue is now irrelevant because the new style takes an entirely different approach. We can close those out.

• Get feedback. We can do much of the evaluation ourselves, and we know that most users are generally in favor of the new style. But to minimize churn after it ships, it would help to send out a request for anyone who cares to try out the new style on their code and file issues for output they don't like.

Make migration easier:

When the new style ships, users will start mass migrating code. That can change the indentation of lines, which may in turn cause long strings and comments to go past the page width. Before this feature ships, we may want to consider:

• (Wrap strings automatically with dartfmt #909) Support splitting long strings.

• (Support for re-wrapping comments? #982) Support re-wrapping comments.

• (Request: more sensible formatting for lists that require custom formatting #1045) Add the ability to opt regions out code out of automated formatting. Some folks on the Flutter team have expressed a strong desire to still be able to hand-format some code where the automated formatting doesn't preserve alignment across rows in things like large table-like collection literals.

• Add support for project-wide style configuration to opt old code into the new style. Project-wide configuration of line length is a long-standing issue (support the ability to configure the formatting line length from within the file #1157). If we have that, we can also use it to let users opt into the tall style even for files whose language version is older than when the tall style ships. This can be useful for package maintainers who are deliberately staying on an older language version to be compatible with users on older SDKs but still want to use the newer style.

  This one doesn't necessarily need to be done before we ship the new style.

Ship support for tall style:

• Remove support for dart format --fix. The tall style doesn't support it and now that dart fix supports all of the same changes and can be run in a faster mode with no static analysis, we have no intention of re-implementing it. Instead, we declare a breaking change and remove support for it for both short and tall styles. This means bumping dart_style to 3.0.0.

• Remove the old dartformat and dartfmt CLI commands and the old CLI. A few years back when we changed the way to run the formatter from dartfmt (its own top-level shell command) to dart format, we also designed a new better set of command line options. That's what you get when you run dart format. But if you globally activate the dart_style package, that still exposes two executables that (for compatibility) continue to use the old CLI.

  Since we are making a 3.0.0 breaking change to dart_style anyway, and (as far as I know), those commands are rarely used, now is a good time to remove them.

• Ship the experiment. Instead of relying on an experiment flag to determine which style you get, use the language version of the file being formatted. If it's above 3.x (where x is whatever Dart SDK version the new formatter will ship in), you get the tall style. Otherwise, you get the short style.

• Roll the new formatter into the Dart SDK.

Remove support for short style:

For some number of Dart SDK releases, dart format will support both short and tall styles (and use language version and project configuration to pick which one a user gets where). But we don't indent to support short style on older code in perpetuity. Eventually, we will remove support for short style entirely and even older language version files will be formatted using the tall style.

• Decide what version of the Dart SDK will no longer support the short style. Coordinate with leads and PMs to decide when we want to remove support.

• Announce the intent to stop supporting the older style. Send a notification on all the various channels letting users know that the upcoming Dart SDK release will change the formatting of their older language versioned files. Encourage users to pre-emptively configure their project to use the new style on all of their files to minimize disruption.

• Remove support for short style. Delete the old AST visitor, Chunk internal representation, line splitter, and short style tests. Rejoice at the simpler, cleaner formatter codebase.

• Roll it into the Dart SDK.