`YamlEditor.update` should never throw `YamlException`, it should also
never throw `AssertionError`, but we know that it has bugs, and in those
cases it does throw `AssertionError`.
Change-Id: I941d88063effc8c46ba270da8531978a72e7a806
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/472562
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Auto-Submit: Jonas Jensen <jonasfj@google.com>
Commit-Queue: Jonas Jensen <jonasfj@google.com>
Added support for the `hosted` field in plugin definition,
so that analyzer plugins can be sourced from pub servers other
than pub.dev.
TEST=pkg/analysis_server/test/src/plugin2/generator_test.dart
TEST=pkg/analyzer/test/src/options/analysis_options_test.dart
Change-Id: Ie99693ab39c77b3d814843963613798f72e27f62
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/475580
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Auto-Submit: Albert Moravec <albert.moravec@gmail.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Konstantin Shcheglov <scheglov@google.com>
Remove the analyzer-only `scanner/reader.dart` re-export and refactor
`Scanner` to accept source text directly.
* Delete `pkg/analyzer/src/dart/scanner/reader.dart`, which only re-exported
`_fe_analyzer_shared` reader types.
* Replace the `Scanner(CharacterReader, ...)` factory with
`Scanner(String contents, ...)`, aligning the API with the underlying
`scanString` implementation.
* Update analyzer, analysis_server, analyzer_cli, analyzer_plugin, and scrape
call sites to pass the source string directly instead of constructing
`CharSequenceReader`.
This removes an unnecessary abstraction layer, reduces
imports/indirection, and makes scanner usage simpler and more uniform
across the repo.
Change-Id: I380b365676c8a62971a3700deadb695d1519d653
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/477600
Reviewed-by: Paul Berry <paulberry@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Konstantin Shcheglov <scheglov@google.com>
This release is so we can publish a version of analysis_server_plugin,
and then start depending on that published version in DAS.
In addition, we get a highly requested fix out to analyzer_testing
users.
Change-Id: I694f47aeee59367c1fc066bc8f7a865406b09917
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/476620
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Changes the signature of the following constructors:
- Scanner
- Scanner.fasta
- Parser
So that they accept a `DiagnosticReporter` object rather than a
`DiagnosticListener` object. This brings the scanner and the parser
into alignment with the majority of the rest of the analyzer (which
reports errors using `DiagnosticReporter` rather than by talking to
`DiagnosticListener` directly).
It also makes the `source` parameter of these constructors
unnecessary, because the `source` can be obtained from the
`DiagnosticReporter`.
Change-Id: I6a6a6964607ffb52a8332a6e618dc0c9a1e48c5c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/473442
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
When we landed null-safety we accidentally disabled internal self-testing in YamlEditor.
We fixed this in: https://github.com/dart-lang/tools/pull/2284
But this was reverted because this test broke. With this we should able to land:
https://github.com/dart-lang/tools/pull/2299
TL;DR: `YamlEditor.update` may throw `AssertionError` if it has an internal error.
Internal errors in `YamlEditor` should not happen, but we have bugs.
There is open PRs to fix some of those bugs, but in practice it's not unlikely
that there will always be bugs. We're modifying YAML source using `SourceSpan`s
from the YAML parser. Changes in the parser, surprising corner cases, complex modifications
or combinations of these can probably cause internal errors.
These internal errors happens when:
* The result YAML output is invalid YAML.
* The resulting YAML doesn't match the same semantic modification on the original YAML structure, when compared with deep equals.
Change-Id: I159b37e9a9f039f92c82881ccac2c4826332f816
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/471460
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Jonas Jensen <jonasfj@google.com>
Changes the initialization logic for `PluginConfiguration` so that it
always populates the `diagnosticConfigs` field with a map that ignores
case when matching keys. This insures that diagnostic codes in a
plug-in configuration are matched in a case-insensitive fashion.
Also changes the logic in `pkg/analysis_server_plugin` so that when it
looks up names in this map, it uses `DiagnosticCode.lowerCaseName`
rather than `DiagnosticCode.name`. This in itself has no effect (since
the map ignores case), but it paves the way for deprecating (and
eventually removing) the `DiagnosticCode.name` getter.
Change-Id: I6a6a6964f8ba149f883d1e9b2c9168a0929f7803
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/466184
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Changes the logic in `pkg/analysis_server_plugin` to use
`DiagnosticCode.lowerCaseName` instead of `DiagnosticCode.name`. This
ensures that diagnostic codes are matched in a case-insensitive
fashion.
This paves the way for deprecating (and eventually removing) the
`DiagnosticCode.name` getter.
Change-Id: I6a6a6964f4733c550f9c1060377e198808190acc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/466129
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Adjusts the logic in the lint `Registry` class so that lint names are
matched in case-insensitive fashion. For the most part this is
accomplished by adding calls to `.toLowerCase()` inside the `Registry`
class, preserving its API.
For the `enabled` method, preserving the API would have been a pain (I
would have had to translate the keys in the `ruleConfigs` parameter to
lower case). So instead I added an assertion to verify that the keys
were lower case, and pushed the responsibility to the callers to
create lower case keys.
This paves the way for a follow-up CL that will translate diagnostic
codes to `lower_snake_case` conventions.
It also solves a longstanding problem with the mixed case lint rules
`no_runtimeType_toString`, `prefer_for_elements_to_map_fromIterable`,
`prefer_iterable_whereType`. Previously, the user had to carefully
imitate the capitalization of the lint rules when specifying them in
the `analysis_options.yaml` file, even though `// ignore:` comments
for the lints matched in case insensitive fashion. With this CL, the
lint rule names in `analysis_options.yaml` are matched in case
insensitive fashion as well.
Change-Id: I6a6a6964d83241e49878bbf96ef9b94cbb12098b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/465964
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Prior to this CL, the analyzer's set of unignorable diagnostic code
names (`AnalysisOptionsImpl.unignorableDiagnosticCodeNames`) was
constructed in the following way:
- For each entry in the `cannot-ignore` section of the
`analysis_options.yaml` file that matches the name of a severity
code, the names of all diagnostic codes with matching severities
were added (1).
- All other entries in the `cannot-ignore` section were converted to
upper case and then added (2).
However, diagnostic codes associated with lints are named using
`lower_snake_case`, while analyzer diagnonstic codes are named using
`UPPER_SNAKE_CASE`.
Some of the logic that consumed
`AnalysisOptionsImpl.unignorableDiagnosticCodeNames` didn't account
for this, resulting in some subtle bugs:
- When the resolved correction producer base class
`_BaseIgnoreDiagnostic` attempted to figure out if a diagnostic was
unignorable, it compared elements of
`unignorableDiagnosticCodeNames` to `DiagnosticCode.name`, which
meant that it would successfully recognize non-lint codes as
unignorable, but it would only recognize that a lint code was
unignorable if it was included in `unignorableDiagnosticCodeNames`
by putting the name of a severity in the `cannot-ignore` section of
the `analysis_options.yaml` file.
- When `LibraryAnalyzer._filterIgnoredDiagnostics` attempted to block
unignorable diagnostics from being ignored, it compared elements of
`unignorableDiagnosticCodeNames` to `DiagnosticCode.name`,
`DiagnosticCode.uniqueName`, and
`DiagnosticCode.name.toUpperCase()`. This worked, however the
comparison to `DiagnosticCode.uniqueName` had no effect. Note that
values of `DiagnosticCode.uniqueName` always take the form
`ClassName.snake_case_diagnostic_code` or
`ClassName.SNAKE_CASE_DIAGNOSTIC_CODE`. It's impossible for any of
the values added to
`AnalysisOptionsImpl.unignorableDiagnosticCodeNames` to ever match
this, because (1) always adds values of `DiagnosticCode.name` (which
never contains a `.`), and (2) always adds strings that have been
converted to upper case.
- When `IgnoreValidator.reportErrors` attempted to report unignorable
and duplicate entries, it compared elements of
`unignorableDiagnosticCodeNames` to `IgnoredDiagnosticName.name`,
which is always lower case. That meant that it would only recognize
that a diagnostic code was unignorable if the diagnostic code was
associated with a lint and was included in
`unignorableDiagnosticCodeNames` by putting the name of a severity
in the `cannot-ignore` section of the `analysis_options.yaml`
file. (Note, however, that this bug was unobservable because the
reporting of the `unignorable_ignore` diagnostic is currently
disabled; I will address this in a follow-up CL.)
- Additionally, the logic to populate
`AnalysisOptionsImpl.unignorableDiagnosticCodeNames` based on a
severity code had a bug in its handling of error processors: if one
or more error processors were used to change the severity of a
diagnostic, then entries would be added to
`AnalysisOptionsImpl.unignorableDiagnosticCodeNames` corresponding
to both the original and the new severity.
These buggy behaviors have been fixed by:
- Streamlining and simplifying the logic that builds
`AnalysisOptionsImpl.unignorableDiagnosticCodeNames`, and ensuring
that all strings added to it are all lower case.
- Changing all logic that checks whether a string is contained in
`AnalysisOptionsImpl.unignorableDiagnosticCodeNames` so that it
first converts that string to lower case.
- Removing the ineffective logic in
`LibraryAnalyzer._filterIgnoredDiagnostics` that attempted to
compare elements of `unignorableDiagnosticCodeNames` to
`DiagnosticCode.uniqueName`.
Change-Id: I6a6a6964d89c139492dbb11d8ba3b2d33c0e2ee8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/462864
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Currently, diagnostic codes associated with lints are named using
`lower_snake_case`, while analyzer diagnonstic codes are named using
`UPPER_SNAKE_CASE`.
However, when the analyzer builds instances of the `ErrorProcessor`
class, it always uses `UPPER_SNAKE_CASE` names.
Some pieces of logic that matched up `ErrorProcessor`s to diagnostic
codes accounted for this difference; others didn't.
This led to a some buggy behaviors:
- If an instance of `ErrorProcessor` got constructed outside of the
analyzer (by an analyzer client using the analyzer public API), and
it supplied a `lower_snake_case` name, then
`ErrorProcessor.appliesTo` would only successfully match if the name
referred to a lint.
- The resolved correction producer base class `_BaseIgnoreDiagnostic`
(which forms the basis for the quick fixes "Ignore '...' in
`analysis_options.yaml`", "Ignore '...' for this line", and "Ignore
'...' for the whole file") would only notice that a diagnostic was
unignorable if the case matched exactly. In practice, this meant
that when operating on instances of `ErrorProcessor` created by the
analyzer, it wouldn't properly handle lints.
These buggy behaviors have been fixed by:
- Changing the `ErrorProcessor` constructor to always convert the
`code` to lower case.
- Changing all references to `ErrorProcessor.code` to assume lower
case.
Change-Id: I6a6a69645284f646e0c070fc2b55c4a90203d74a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/462863
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
No more flag, always parse into the new AST, always visit new AST nodes,
always return them as child entities, parent-child structure reflects
the new AST.
So, use `namePart` and `body` where possible. Deprecate previous
properties.
This is still de jure a breaking change, because `parent` of deprecated
properties changes. De facto this required very few changes in google3.
Once this CL lands, I will publish `analyzer 10.0.0`, migrate everything
to new properties, delete deprecated properties, and publish `analyzer
11.0.0`.
Maybe deprecate `NamedCompilationUnitMember.name` and migrate to
subclass specific `name` or `namePart` properties before publishing
`analyzer 10.0.0`. This part is not breaking per se.
* Deprecations in `ClassDeclaration`:
* Properties `leftBracket`, `members`, `rightBracket` are deprecated, use `body` instead.
* Properties `name`, `typeParameters` are deprecated, use `namePart` instead.
* Deprecations in `EnumDeclaration`:
* Properties `leftBracket`, `constants`, `members`, `rightBracket` are deprecated, use `body` instead.
* Properties `name`, `typeParameters` are deprecated, use `namePart` instead.
* Deprecations in `ExtensionDeclaration`:
* Properties `leftBracket`, `members`, `rightBracket` are deprecated, use `body` instead.
* Deprecations in `ExtensionTypeDeclaration`:
* Properties `leftBracket`, `constants`, `members`, `rightBracket` are deprecated, use `body` instead.
* Properties `constKeyword`, `name`, `representation`, `typeParameters` are deprecated,
use `primaryConstructor` instead.
* **Breaking Change:** While the deprecated members mentioned above still exist in the AST,
their parent nodes have changed. This means that code relying on specific parent-child
relationships for these nodes might break.
Bug: https://github.com/dart-lang/sdk/issues/61701
Change-Id: Ic48104da8b029c9b454bbd2336574b7823025565
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/461841
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Konstantin Shcheglov <scheglov@google.com>
Changes the analyzer and related packages so that when they refer to
diagnostic constants, they do so via the import prefix `diag`, which
refers to the appropriate `diagnostic.dart` file containing the top
level diagnostic constant declarations, rather than the static
declarations inside `DiagnosticCode`-derived classes (which will soon
be removed).
This CL was created by the following steps:
- Run the script
`pkg/analyzer_utilities/tool/messages/switch_to_toplevel_diagnostics.dart`.
- Execute `dart fix --apply --code=unused_import,unnecessary_import`
on the following directories (this removes imports that are no
longer necessary due to the change):
- `pkg/analysis_server`
- `pkg/analyzer`
- `pkg/linter`
- `pkg/analysis_server_plugin`
- `pkg/analyzer_plugin`
- `pkg/analyzer_testing`
- `pkg/front_end`
- `pkg/analyzer_cli`
- Execute `dart format` on the following files and directories:
- `pkg/analysis_server`
- `pkg/analyzer`
- `pkg/linter`
- `pkg/analysis_server_plugin`
- `pkg/analyzer_plugin`
- `pkg/analyzer_testing`
- `pkg/front_end/test/scanner_test.dart`
(Note that `pkg/front_end` and `pkg/analyzer_cli` are not
re-formatted as whole directories because they contain `.dart` files
that are test cases rather than source code, and reformatting those
files might change test expectations.)
- Manually add `diag` to
pkg/front_end/test/spell_checking_list_tests.txt.
- Manually fix the ignore comment in
`pkg/analyzer_testing/lib/src/analysis_rule/pub_package_resolution.dart`. (The
script `switch_to_toplevel_diagnostics.dart` automatically adds it
after `import 'package:analyzer/src/diagnostic/diagnostic.dart' as
diag;`, but then executing `dart format` bumps the ignore comment to
the following line, where it has no effect.)
Change-Id: I6a6a69643022aab2b5a6224fb4124eead243260d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/461521
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Replaces "is" tests that refer to classes derived from
`DiagnosticCode` with tests on `DiagnosticCode.type`:
- `x is ScannerErrorCode || x is ParserErrorCode` is replaced with
`x.type == DiagnosticType.SYNTACTIC_ERROR`. This is exactly
equivalent, because `ScannerErrorCode` and `ParserErrorCode` are the
only two `DiagnosticCode`-derived classes whose `type` method returns
`SYNTACTIC_ERROR`.
- `x is TodoCode` is replaced with `x.type ==
DiagnosticType.TODO`. This is exactly equivalent, because `TodoCode`
is the only `DiagnosticCode`-derived class whose `type` method
returns `TODO`.
- In `statement_completion.dart`, `x is! HintCode && x is!
WarningCode` is replaced with `x.type ==
DiagnosticType.SYNTACTIC_ERROR`. The new test is less accepting: for
example, it previously accepted diagnostics of type
`StaticWarningCode`. This test is used to short-cut the generation
of statement completions such as completing an incomplete `do`
statement. These completions are only necessary to generate if there
is a syntax error, so there should be no user-visible behavior
change.
- In `fix_processor.dart`, `x is LintCode || x is HintCode || x is
WarningCode` is replaced with `x.type == DiagnosticType.LINT ||
x.type == DiagnosticType.STATIC_WARNING`. The new test is more
accepting: it now accepts any diagnostic code whose type is
`STATIC_WARNING`, when previously it only accepted warnings of type
`WarningCode` (and rejected warnings of types like
`StaticWarningCode`). This test determines when quick fixes like
"ignore diagnostic on this line" are offered, so the behavior change
is appropriate.
- Also replaces a documentation reference to `TodoCode` (in
`todo_codes.dart`) with a reference to `DiagnosticType.TODO`.
These changes pave the way for a follow-up CL in which I plan to
eliminate these derived classes entirely.
Change-Id: I6a6a69647ff62dfe06b0219d8b292da53427cf0d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/461140
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Removes nearly all of the special handling of `uniqueName` in
constructors in the `DiagnosticCode` class hierarchy.
With three exceptions, the value passed for the named parameter
`uniqueName` is the same as the value that will eventually be stored
in the `DiagnosticCode.uniqueName` field. The three exceptions are the
constructors for `LintCode` and `SecurityLintCode`, and the unnamed
constructor for `LinterLintCode`. These constructors allow
`uniqueName` to be `null`, and if `null` is supplied, they replace it
with `LintCode.$name`.
These exceptions are needed for analyzer API compatibility in the case
of `LintCode`, and for compatibility with google3 code in the case of
`SecurityLintCode` and `LinterLintCode`.
In follow-up CLs, I will be updating google3 code to make the
exceptions in `SecurityLintCode` and `LinterLintCode` unnecessary.
Change-Id: I6a6a6964d828c3a8db6563d47347d5ca8f550ac3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/459001
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Work towards https://github.com/dart-lang/sdk/issues/61876
* Make a few notes about support beginning with Dart 3.10.
* Make a few notes indicating that plugins can only analyze Dart
sources.
* Use the handy `> [!NOTE]` and `> [!WARNING]` syntax.
* Add a note about how plugins are resolved, and the dependency on
analysis_server_plugin ^0.3.0.
Change-Id: I6ed5b3611a0ab4ff005917b0c5cff1b6fa9d5c95
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/459240
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Adds the constructor parameter `uniqueNameCheck` to `DiagnosticCode`
and derived classes. If a non-null value is supplied for this
parameter, an assertion checks that it's equal to
`uniqueName`. (Passing a `null` value disables the assertion; this is
necessary to preserve analyzer API compatibility).
The reason for this change is that the value supplied for `uniqueName`
when calling constructors in the `DiagnosticCode` class hierarchy is
not necessarily the same as the value that is eventually stored in the
`uniqueName` field; it goes through a sequence of manipulations while
being passed up the superclass chain that make its actual behavior
difficult to follow. The `uniqueNameCheck` field allows me to test the
theory that the actual values that wind up in `uniqueName` are simply
the analyzer codes, with the exception that `LinterLintCode.` is
replaced with `LintCode.`.
In a follow-up CL, I plan to eliminate nearly all the manipulations of
`uniqueName` that occur while being passed up the superclass chain,
which will simplify the code generation logic for computing
`uniqueName` values. Adding `uniqueNameCheck` now allows me to test
out the new, simpler logic, and verify that it will produce the same
result as today's more complex logic.
The `uniqueNameCheck` parameter is marked as `@deprecated` to prevent
clients from trying to use it; this ensures that when I remove it in a
follow-up CL, it won't be necessary to bump the analyzer major
version.
Change-Id: I6a6a696440bce62a31b988d12064244e54e37b6b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/458820
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
This changes the API of writing an analysis rule test. Instead of
overriding the `String get analysisRule` property, and registering the
rule in `setUp` (which requires importing a private type), the
developer can now just set the `AbstractAnalysisRule rule` field in
`setUp`. This both removes the requirement to manually register the
rule, and reduces the risk of typos, by removing the String API.
Fixes https://github.com/dart-lang/sdk/issues/61793
Change-Id: Ic7753d3157e06906ba5ccbccfea67aaa3179dcc8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/459340
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Changes several references to derived diagnostic types
(`CompileTimeErrorCode`, `WarningCode`, etc.) to simply
`DiagnosticCode`.
This paves the way for a follow-up CL that will change the static
types of the generated analyzer diagnostic code objects to more
general types. This is part of a longer-term effort to simplify the
`DiagnosticCode` class hierarchy.
Change-Id: I6a6a69642561aebf2607e834fa09fa608cb4fa10
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/458587
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Observation 1: The legacy protocol answered completion requests via a
`server.resolveForCompletion` call, and the LSP protocol answered
completion requests via a `server.getResolvedUnit` call where the idea
is that resolving for completion requires less and is therefore faster.
Solution: Make the LSP protocol use `server.resolveForCompletion` too.
Observation 2: Completion requests often come right after change request
making timing important and the `await driver.applyPendingFileChanges()`
call done in the analysis server "pushed" the timing making the
`server.resolveForCompletion` actually finished after it had already
resolved the whole thing.
Solution: Don't do that - the driver adds it to the queue of work, the
work is done later in `performWork` called from
`AnalysisDriverScheduler._run` where `_applyPendingFileChanges` is
always called anyway (which is the call that completes the
`applyPendingFileChanges`call).
Observation 3: If there is no change yet to be processed, a
`resolveForCompletion` call is slower than a `getResolvedUnit` because
the resolved unit is cached (assuming it's a priority file) and the
`resolveForCompletion` call always parses the file again.
Solution: Respond to the `resolveForCompletion` call with the resolved
unit data if it's available in the cache. The cache is always cleared
when changes happen anyway.
Benchmarks on this stuff is a bit weird because it's timing related - so
while I'd say this is overall just better there are also runs where we
get "bad timing" and the runtimes are therefore not better. In an
attempt to clear it up I've run the benchmarks 25 times each, and
attempted to put the data in two different buckets as needed.
*lsp_type_in_big_file of size 16,000*
```
Fully done after last type (ms):
Difference at 95.0% confidence
-1419.2 +/- 322.074
-16.8866% +/- 3.83225%
(Student's t, pooled s = 566.238)
Whole typing time (ms):
Difference at 95.0% confidence
-1418.88 +/- 322.037
-13.9661% +/- 3.16983%
(Student's t, pooled s = 566.172)
Uncancelled completion response time (ms):
Difference at 95.0% confidence
-1590.48 +/- 682.753
-28.0194% +/- 12.028%
(Student's t, pooled s = 1200.35)
```
The `Uncancelled completion response time` has a big "+/-" so attempting
to "good and bad bucketize" it I get:
good bucket:
```
Difference at 95.0% confidence
-1929.8 +/- 347.712
-35.9528% +/- 6.47797%
(Student's t, pooled s = 543.261)
```
bad bucket (though truthfully there wasn't a clear cutoff before):
```
No difference proven at 95.0% confidence
```
which sort of makes sense: If the completion runs before a (new) change
starts processing we now `resolveForCompletion` instead which is faster,
but if completion runs after the change has started processing we
essentially - both before and after - do nothing (except wait for the
calculation to finish) because we just load the data from cache.
*lsp_type_in_big_file_ask_for_completion, 16,000*
```
Completion #1 (ms):
No difference proven at 95.0% confidence
```
Ehh. There's a clear cutoff in the now, so taking the 8 (how the cutoff
happens to be) fastest from each I get
```
Difference at 95.0% confidence
-1034.12 +/- 75.3617
-37.673% +/- 2.74542%
(Student's t, pooled s = 70.2673)
```
Moving on.
```
Completion #2 (ms):
Difference at 95.0% confidence
-1076.28 +/- 178.446
-37.2601% +/- 6.17769%
(Student's t, pooled s = 313.726)
```
here 2 in the "now" has bad timing, removing them from the statistics
gives
```
Difference at 95.0% confidence
-1182.39 +/- 104.543
-40.9334% +/- 3.6192%
(Student's t, pooled s = 179.748)
```
Moving on.
```
Completion #3 (ms):
Difference at 95.0% confidence
-735.6 +/- 292.553
-25.6471% +/- 10.2%
(Student's t, pooled s = 514.336)
and removing the 8 bad ones:
Difference at 95.0% confidence
-1195.69 +/- 99.305
-41.6884% +/- 3.46232%
(Student's t, pooled s = 156.306)
```
Continuing like this:
```
Completion #4 (ms):
Difference at 95.0% confidence
-948.4 +/- 265.704
-32.9887% +/- 9.24214%
(Student's t, pooled s = 467.134)
and removing the 5 bad ones:
Difference at 95.0% confidence
-1252.87 +/- 86.5425
-43.5793% +/- 3.01026%
(Student's t, pooled s = 143.022)
Completion #5 (ms):
Difference at 95.0% confidence
-1067.88 +/- 199.266
-37.0011% +/- 6.90439%
(Student's t, pooled s = 350.329)
and removing the 3 bad ones:
Difference at 95.0% confidence
-1236.76 +/- 69.5776
-42.8527% +/- 2.4108%
(Student's t, pooled s = 118.18)
```
Moving on to the "Completion without change" I realize just now that the
benchmark for the first entry is broken - it doesn't wait until the
previous change has been processed, meaning that in the 3 cases where we
got bad timing in "Completion #5 (ms)" we see about the same result as
before, but in the 22 other cases we see bad results because it has to
wait until the previous change has been processed. For the remaining
(2-5) there is no virtually change which makes sense because both before
and now it just fetches the resolved unit from cache.
*legacy_type_in_big_file_ask_for_completion*
```
Completion #1 (ms):
Difference at 95.0% confidence
-1817.08 +/- 60.6836
-48.0638% +/- 1.60515%
(Student's t, pooled s = 106.688)
Completion #2 (ms):
Difference at 95.0% confidence
-2208.56 +/- 48.4844
-55.4647% +/- 1.21761%
(Student's t, pooled s = 85.2403)
Completion #3 (ms):
Difference at 95.0% confidence
-2159.68 +/- 69.4145
-53.0717% +/- 1.70578%
(Student's t, pooled s = 122.037)
Completion #4 (ms):
Difference at 95.0% confidence
-2264.44 +/- 73.2112
-53.5455% +/- 1.73117%
(Student's t, pooled s = 128.712)
Completion #5 (ms):
Difference at 95.0% confidence
-2147.4 +/- 68.5023
-50.9254% +/- 1.62453%
(Student's t, pooled s = 120.434)
```
The first "Completion without change" suffers from the same as before
and I will skip it here.
```
Completion without change #2 (ms):
Difference at 95.0% confidence
-416.28 +/- 28.2024
-61.3512% +/- 4.15646%
(Student's t, pooled s = 49.5826)
Completion without change #3 (ms):
Difference at 95.0% confidence
-687.24 +/- 21.9848
-92.6499% +/- 2.96387%
(Student's t, pooled s = 38.6514)
Completion without change #4 (ms):
Difference at 95.0% confidence
-637.32 +/- 26.482
-95.2703% +/- 3.95868%
(Student's t, pooled s = 46.5579)
Completion without change #5 (ms):
Difference at 95.0% confidence
-702.32 +/- 18.7277
-95.8301% +/- 2.55535%
(Student's t, pooled s = 32.925)
```
I don't know why there doesn't appear to be any timing related issues
here (maybe sending and receiving the entire big file (in legacy vs in
lsp where a small 'diff' is send) taking more time pushes the timing,
but I'm guessing) - nor do I know why now "Completion without change #2"
is slower (~250 ms) than the subsequent ones (~30 ms).
Change-Id: I4c21d658efccbcf197eedb69f466b2942b78c4b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/457364
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Konstantin Shcheglov