Remove the analyzer-only `scanner/reader.dart` re-export and refactor
`Scanner` to accept source text directly.
* Delete `pkg/analyzer/src/dart/scanner/reader.dart`, which only re-exported
`_fe_analyzer_shared` reader types.
* Replace the `Scanner(CharacterReader, ...)` factory with
`Scanner(String contents, ...)`, aligning the API with the underlying
`scanString` implementation.
* Update analyzer, analysis_server, analyzer_cli, analyzer_plugin, and scrape
call sites to pass the source string directly instead of constructing
`CharSequenceReader`.
This removes an unnecessary abstraction layer, reduces
imports/indirection, and makes scanner usage simpler and more uniform
across the repo.
Change-Id: I380b365676c8a62971a3700deadb695d1519d653
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/477600
Reviewed-by: Paul Berry <paulberry@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Konstantin Shcheglov <scheglov@google.com>
Changes the signature of the following constructors:
- Scanner
- Scanner.fasta
- Parser
So that they accept a `DiagnosticReporter` object rather than a
`DiagnosticListener` object. This brings the scanner and the parser
into alignment with the majority of the rest of the analyzer (which
reports errors using `DiagnosticReporter` rather than by talking to
`DiagnosticListener` directly).
It also makes the `source` parameter of these constructors
unnecessary, because the `source` can be obtained from the
`DiagnosticReporter`.
Change-Id: I6a6a6964607ffb52a8332a6e618dc0c9a1e48c5c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/473442
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Observation 1: The legacy protocol answered completion requests via a
`server.resolveForCompletion` call, and the LSP protocol answered
completion requests via a `server.getResolvedUnit` call where the idea
is that resolving for completion requires less and is therefore faster.
Solution: Make the LSP protocol use `server.resolveForCompletion` too.
Observation 2: Completion requests often come right after change request
making timing important and the `await driver.applyPendingFileChanges()`
call done in the analysis server "pushed" the timing making the
`server.resolveForCompletion` actually finished after it had already
resolved the whole thing.
Solution: Don't do that - the driver adds it to the queue of work, the
work is done later in `performWork` called from
`AnalysisDriverScheduler._run` where `_applyPendingFileChanges` is
always called anyway (which is the call that completes the
`applyPendingFileChanges`call).
Observation 3: If there is no change yet to be processed, a
`resolveForCompletion` call is slower than a `getResolvedUnit` because
the resolved unit is cached (assuming it's a priority file) and the
`resolveForCompletion` call always parses the file again.
Solution: Respond to the `resolveForCompletion` call with the resolved
unit data if it's available in the cache. The cache is always cleared
when changes happen anyway.
Benchmarks on this stuff is a bit weird because it's timing related - so
while I'd say this is overall just better there are also runs where we
get "bad timing" and the runtimes are therefore not better. In an
attempt to clear it up I've run the benchmarks 25 times each, and
attempted to put the data in two different buckets as needed.
*lsp_type_in_big_file of size 16,000*
```
Fully done after last type (ms):
Difference at 95.0% confidence
-1419.2 +/- 322.074
-16.8866% +/- 3.83225%
(Student's t, pooled s = 566.238)
Whole typing time (ms):
Difference at 95.0% confidence
-1418.88 +/- 322.037
-13.9661% +/- 3.16983%
(Student's t, pooled s = 566.172)
Uncancelled completion response time (ms):
Difference at 95.0% confidence
-1590.48 +/- 682.753
-28.0194% +/- 12.028%
(Student's t, pooled s = 1200.35)
```
The `Uncancelled completion response time` has a big "+/-" so attempting
to "good and bad bucketize" it I get:
good bucket:
```
Difference at 95.0% confidence
-1929.8 +/- 347.712
-35.9528% +/- 6.47797%
(Student's t, pooled s = 543.261)
```
bad bucket (though truthfully there wasn't a clear cutoff before):
```
No difference proven at 95.0% confidence
```
which sort of makes sense: If the completion runs before a (new) change
starts processing we now `resolveForCompletion` instead which is faster,
but if completion runs after the change has started processing we
essentially - both before and after - do nothing (except wait for the
calculation to finish) because we just load the data from cache.
*lsp_type_in_big_file_ask_for_completion, 16,000*
```
Completion #1 (ms):
No difference proven at 95.0% confidence
```
Ehh. There's a clear cutoff in the now, so taking the 8 (how the cutoff
happens to be) fastest from each I get
```
Difference at 95.0% confidence
-1034.12 +/- 75.3617
-37.673% +/- 2.74542%
(Student's t, pooled s = 70.2673)
```
Moving on.
```
Completion #2 (ms):
Difference at 95.0% confidence
-1076.28 +/- 178.446
-37.2601% +/- 6.17769%
(Student's t, pooled s = 313.726)
```
here 2 in the "now" has bad timing, removing them from the statistics
gives
```
Difference at 95.0% confidence
-1182.39 +/- 104.543
-40.9334% +/- 3.6192%
(Student's t, pooled s = 179.748)
```
Moving on.
```
Completion #3 (ms):
Difference at 95.0% confidence
-735.6 +/- 292.553
-25.6471% +/- 10.2%
(Student's t, pooled s = 514.336)
and removing the 8 bad ones:
Difference at 95.0% confidence
-1195.69 +/- 99.305
-41.6884% +/- 3.46232%
(Student's t, pooled s = 156.306)
```
Continuing like this:
```
Completion #4 (ms):
Difference at 95.0% confidence
-948.4 +/- 265.704
-32.9887% +/- 9.24214%
(Student's t, pooled s = 467.134)
and removing the 5 bad ones:
Difference at 95.0% confidence
-1252.87 +/- 86.5425
-43.5793% +/- 3.01026%
(Student's t, pooled s = 143.022)
Completion #5 (ms):
Difference at 95.0% confidence
-1067.88 +/- 199.266
-37.0011% +/- 6.90439%
(Student's t, pooled s = 350.329)
and removing the 3 bad ones:
Difference at 95.0% confidence
-1236.76 +/- 69.5776
-42.8527% +/- 2.4108%
(Student's t, pooled s = 118.18)
```
Moving on to the "Completion without change" I realize just now that the
benchmark for the first entry is broken - it doesn't wait until the
previous change has been processed, meaning that in the 3 cases where we
got bad timing in "Completion #5 (ms)" we see about the same result as
before, but in the 22 other cases we see bad results because it has to
wait until the previous change has been processed. For the remaining
(2-5) there is no virtually change which makes sense because both before
and now it just fetches the resolved unit from cache.
*legacy_type_in_big_file_ask_for_completion*
```
Completion #1 (ms):
Difference at 95.0% confidence
-1817.08 +/- 60.6836
-48.0638% +/- 1.60515%
(Student's t, pooled s = 106.688)
Completion #2 (ms):
Difference at 95.0% confidence
-2208.56 +/- 48.4844
-55.4647% +/- 1.21761%
(Student's t, pooled s = 85.2403)
Completion #3 (ms):
Difference at 95.0% confidence
-2159.68 +/- 69.4145
-53.0717% +/- 1.70578%
(Student's t, pooled s = 122.037)
Completion #4 (ms):
Difference at 95.0% confidence
-2264.44 +/- 73.2112
-53.5455% +/- 1.73117%
(Student's t, pooled s = 128.712)
Completion #5 (ms):
Difference at 95.0% confidence
-2147.4 +/- 68.5023
-50.9254% +/- 1.62453%
(Student's t, pooled s = 120.434)
```
The first "Completion without change" suffers from the same as before
and I will skip it here.
```
Completion without change #2 (ms):
Difference at 95.0% confidence
-416.28 +/- 28.2024
-61.3512% +/- 4.15646%
(Student's t, pooled s = 49.5826)
Completion without change #3 (ms):
Difference at 95.0% confidence
-687.24 +/- 21.9848
-92.6499% +/- 2.96387%
(Student's t, pooled s = 38.6514)
Completion without change #4 (ms):
Difference at 95.0% confidence
-637.32 +/- 26.482
-95.2703% +/- 3.95868%
(Student's t, pooled s = 46.5579)
Completion without change #5 (ms):
Difference at 95.0% confidence
-702.32 +/- 18.7277
-95.8301% +/- 2.55535%
(Student's t, pooled s = 32.925)
```
I don't know why there doesn't appear to be any timing related issues
here (maybe sending and receiving the entire big file (in legacy vs in
lsp where a small 'diff' is send) taking more time pushes the timing,
but I'm guessing) - nor do I know why now "Completion without change #2"
is slower (~250 ms) than the subsequent ones (~30 ms).
Change-Id: I4c21d658efccbcf197eedb69f466b2942b78c4b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/457364
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This change adds a new parameter `typeParametersInScope` to `DartEditBuilder.writeType`, and related methods to deprecate the previous `methodBeingCopied` parameter, which limited our ability to pass in some type parameters that are in scope.
Also fixes a TODO about testing if the `Record` feature is enabled or not.
And as a bonus, we now always write down type parameters by looking for the closest public type or falling back to `Object?` or `dynamic`.
Bug: https://github.com/dart-lang/sdk/issues/61186
Change-Id: I7a7b1c62adb81aecfdf61ebbaa0d95f1d33bced8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/442282
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Auto-Submit: Felipe Morschel <git@fmorschel.dev>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Because ChangeBuilder can be used to modify multiple files, the EOL should come from an individual files edit builder and not from the overall ChangeBuilder.
This change renames `eol` on ChangeBuilder to `defaultEol` (since it's still useful to supply a custom default, for example when moving a class from an existing file to a new file, we want to default to the same EOL as the source file), and adds an EOL to file edit builders which is computed from the file content (and falls back to the default from ChangeBuilder if there are none).
My intention here is to not introduce any breaking changes (yet) so the ChangeBuilder constructor interface still has the "eol" named parameter (marked as deprecated) as well as the new `defaultEol` parameter.
I added a new `StringExtension` for `endOfLine` to both analyzer_plugin and analysis_server_plugin as we need this in both places (but I expect one of those projects will ultimately be dropped). This slightly differs from `CorrectionUtils.endOfLine` because it can return `null` which is a signal to fall back to the ChangeBuilder default instead of forcing its own default. (CorrectionUtils.endOfLine now uses that extension, although because of the null/default, it's likely that some - if not all - uses of this should migrate to the new extension and fall back to a default from a ChangeBuilder).
Change-Id: Ie2e6eaba78c96a1cbd79eba881e12bbeef724aa8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/443122
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Before an assist call on a file with non-windows lineendings would first
search through the entire file for Windows line endings, then search for
the non-Windows line endings. As such Windows line endings was faster.
This CL changes the semantics slightly from "Windows line endings if
any Windows line endings exist" to "whatever line endings are first
used" which should be good.
It speeds up assist calls with non-Windows line endings:
Statistics on 5 runs each:
```
Windows / 2000: No change.
Windows / 4000: No change.
Windows / 8000: No change.
Unix / 2000: 1000 assist calls: -46.9900% +/- 6.1715% (-0.88 +/- 0.12) (1.86 -> 0.99)
Unix / 4000: 1000 assist calls: -59.1561% +/- 6.3982% (-1.79 +/- 0.19) (3.03 -> 1.24)
Unix / 8000: 1000 assist calls: -63.8429% +/- 4.3083% (-3.68 +/- 0.25)(5.76 -> 2.08)
```
Change-Id: I61278ed4ad31b891e87951da0a6162fe37376888
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/437224
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Work towards #60635
In this change, we rework the AnalysisErrorListener deprecation to
better support users who have their own class that implements
AnalysisErrorListener. This change introduces a sealed supertype,
DiagnosticOrErrorListener, with the old implementation,
AnalysisErrorListener, and the new implementation, DiagnosticListener,
as its sole direct subclasses. Users who have implemented
AnalysisErrorListener should be able to instead implement
DiagnosticListener, and their class is an acceptable instance of
DiagnosticOrErrorListener, wherever that is needed.
In a breaking change we can drop AnalysisErrorListener and deprecate
DiagnosticOrErrorListener, and in the next breaking change, we can drop
DiagnosticOrErrorListener.
For reference, see the first API difference when deprecating AnalysisErrorListener and introducing DiagnosticListener: https://github.com/dart-lang/sdk/commit/903d77cc8229972a424941dcb7b7b79741e833eb#diff-dec15868961d7eadcd009f49d129bebcdb747aaa8dbfde5f5a08884e0cf11e32
Change-Id: I3ccf11d54b41fbca98d020d89978d250c16b4c04
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/436480
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Work towards https://github.com/dart-lang/sdk/issues/60635
I tried to keep this minimal but still sensible:
* Rename AnalysisErrorListener and all subtypes to use 'Diagnostic'.
* Rename all instantiations of such classes if they previously
contained the word 'error'.
* Rename `RecordingDiagnosticListener.errors` to `.diagnostics`.
* Rename some _testing_ instance members that had the word 'error'
to instead use 'diagnostic'.
Change-Id: I3948e27ba28ac2494092e04f4e1d201a20cc1135
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/433004
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
This allows us to traverse the children without needing to create lists.
It also allows us to iterate over lists of nodes more efficiently.
Tested against the script that computes fixes for 10,000 diagnostics,
I'm seeing the following result.
Before this CL the average time was 10692.4 ms.
After this CL the average time was 1900.0 ms.
That's just over an 82% improvement.
Change-Id: Ideff745288c8990e9948b22c5204d176af3cac6e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/426904
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Brian Wilkerson <brianwilkerson@google.com>
Sometimes - on responding to `edit.getFixes` requests, the method
`getTopLevelDeclarations` is called several times with the same
parameter and caching has then sometimes sped things up.
I then also noticed that it caused `reportAnalysisAnalytics` to be
called many times (e.g. 73 times for one `edit.getFixes` request)
each time spending time collecting data which - on passing to
`analyticsManager.analysisComplete` - was just thrown away because it
only saves the first one.
Combined these changes have been observed to reduce the response time
on `edit.getFixes` by up (down?) to ~20% (e.g. from 5 seconds
to 1 second).
In response to http://b/407797012 where more data is also available.
Unfortunately I haven't been able to reproduce this is a benchmark
(yet anyway). I don't know if it needs slower slower reads, an extreme
amount of files, several contexts or something else entirely.
Change-Id: Id214b0e1b7d8bbef7f6d956408f17fab45618b78
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/420323
Reviewed-by: Phil Quitslund <pquitslund@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
A few statements around the package read:
> Producers used in bulk fixes must not modify the FixKind during
> computation.
In fact, this `add_null_check` correction producer seems to be the
_only_ remaining producer that violates this statement. In order to
fix it, such that `fixKind` is set during constructor initialization,
we must do some poking around the AST.
So we convert `_computeTarget` to be static (and change it to also
compute and return any possible null-aware token, renaming it to
`_computeTargetAndNullAwareToken`). We also convert `_isNullAware` to
be static. Then the two public constructors are converted into
factory constructors that determine the fix kind, and store the
`_target` and `_nullAwareToken` in fields, so as to avoid re-computing
them during `compute()`.
Change-Id: Ic66319ef764dd9bd69f0e1059347acf174debb25
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/419420
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Samuel Rawlins <srawlins@google.com>
The `DartFixContext.librariesWithExtensions` utility function accepts
a `memberName` parameter but never did anything with it. This change
adds filtering in that function based on the member name.
This code depends on the set of exported extensions, which is provided
in an extension (`LibraryElementExtensions.exportedExtensions`) in the
analysis_server package, which must be moved to the analyzer package.
Change-Id: I06c8a9a0e11ceb70e75517b30c0b048321d6b024
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/418641
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
This updates `analyzer_use_new_elements` lint to stop reporting inside
deprecated declarations. So, we can deprecated and prepare for removal.
Apply deprecation dart/analysis/results.dart
Deprecate `ElementDeclarationResult`, add `FragmentDeclarationResult` instead.
Change `getFragmentDeclaration` (not yet published) return type to `FragmentDeclarationResult` to better correspond to the method name.
Rename `getElementDeclaration` from `AnalysisSessionHelper` to
`getFragmentDeclaration`, to better reflect on what it operates.
Change-Id: I2839c9a7e21ed2e5b34035fc8c3ec88fd23df1b5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/412840
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Konstantin Shcheglov <scheglov@google.com>
Konstantin Shcheglov