Jens Johansen
fa88e0c91f
This CL:
* Adds a benchmark of running `dart analyze` on a single small file.
* Adds a benchmark of running `dart analyze` on a single project.
* Adds a benchmark of running `dart analyze` on several projects.
* Adds a hidden flag to `dart analyze` so it reports ram usage:
run via `dart analyze --format=json --memory` and the memory
usage will be reported in the json output.
All the bencmarks run without and with cache for speed testing, and
without and with cache when measuring memory usage.
The idea of running this via `dart analyze` instead of running either
the script or the snapshot is to measure the "real world" speed which
could be different (although in practise it _does_ just run the
snapshot).
Future CL(s) should also add benchmarks for queries using the
language server.
Change-Id: Iad6d6d72c1a2ed18ab51d056b4914f8b6eb963e4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276100
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Analysis Server Benchmarks
How to run the benchmarks
To see a list of all available benchmarks, run:
dart benchmark/benchmarks.dart list
To run an individual benchmark, run:
dart benchmark/benchmarks.dart run <benchmark-id>
How they're tested
In order to make sure that our benchmarks don't regress in terms of their ability to run, we create one unit test per benchmark, and run those tests as part of our normal CI test suite.
To save time on the CI, we only run one iteration of each benchmark
(--repeat=1), and we run the benchmark on a smaller data set (--quick).
See test/benchmark_test.dart.
To add a new benchmark
Register the new benchmark in the main() method of benchmark/benchmarks.dart.
On the bots
Our benchmarks run on a continuous performance testing system. Currently, the benchmarks need to be manually registered ahead of time.