Commit Graph

31 Commits

Author SHA1 Message Date
Jens Johansen f169abcee0 [CFE] Make standard filesystem more synchronous by default; add async versions
This CL:
 * Makes the standard filesystem more synchronous by default.
   This makes reading files for normal compiles faster.
 * Adds a more asynchronous version of `exists` and `readAsBytes`.
   This potentially allows for reading files faster when in a context
   where the files can actually be read in parallel. The client have
   to make the choise though.
 * Skips `Uri.base.resolveUri` when the uri has a scheme. This makes it
   slightly faster in my tests, but does not remove any `.` and `..`.
   Having those when having a scheme seems sort of weird though.
   If this turns out to be a problem we can add it back.
 * Re-orders what the standard filesystem checks in `exists` to assume files
   which seems more likely to happen. This - for files - means less checks
   and should make checking for existance faster. (This was noticed by Siggi).

Using the same measurement procedure as in
https://dart-review.googlesource.com/c/sdk/+/85442
timing how long fasta spends on *reading* files when compiling dart2js
(when compiling via the VM) I get a good speedup.

Note that the measurements are in microseconds:

Before this CL (run like `out/ReleaseX64/dart pkg/compiler/bin/dart2js.dart 2>&1 | grep "Read file (total)" | tail -n 1` 10 times):
Read file (total): 94678
Read file (total): 94232
Read file (total): 91726
Read file (total): 77472
Read file (total): 95582
Read file (total): 88113
Read file (total): 89245
Read file (total): 91575
Read file (total): 96291
Read file (total): 95730

With this CL (run like `out/ReleaseX64/dart pkg/compiler/bin/dart2js.dart 2>&1 | grep "Read file (total)" | tail -n 1` 10 times):
Read file (total): 68379
Read file (total): 69320
Read file (total): 72930
Read file (total): 69692
Read file (total): 68685
Read file (total): 73548
Read file (total): 64649
Read file (total): 71951
Read file (total): 73486
Read file (total): 70621

Difference at 95.0% confidence
        -21138.3 +/- 4193
        -23.111% +/- 4.5843%
        (Student's t, pooled s = 4462.56)


Furthermore, using an internal benchmark (discussed in an email thread)
I get these numbers (note that the measurements are in milliseconds):

before:
4453
4249
4187
4190
4216

after:
2310
2379
2449
2467
2407

Difference at 95.0% confidence
        -1856.6 +/- 131.443
        -43.5924% +/- 3.08625%
        (Student's t, pooled s = 90.1257)

Using the asynchronous versions of exists and readAsBytes and checking for
existance and reading in parallel on the same internal benchmark it gets to
1137
1185
1067
1089
1189


For completion, comparing to using "raw" `File` (again using the internal benchmark):

sequential standard
2377
2419
2485
2373
2431

sequential io_sync
2279
2182
2237
2312
2324

io_sync faster:
        -150.2 +/- 76.3122
        -6.21432% +/- 3.15731%
        (Student's t, pooled s = 52.3245)

sequential standardasync
4865
4915
4753
5185
4760

sequential io_async
4746
4834
4989
4840
4891

No difference proven at 95.0% confidence

parallel standard
2526
2563
2652
2616
2685

parallel io_sync
2335
2184
2244
2341
2401

io_async faster
Difference at 95.0% confidence
        -307.4 +/- 111.037
        -11.785% +/- 4.25691%
        (Student's t, pooled s = 76.1341)

parallel standardasync
1137
1185
1067
1089
1189

parallel io_async
1105
1088
1145
1048
1113

No difference proven at 95.0% confidence

TEST=Existing tests.

Change-Id: I8ba56ab0768df8672bcdb693782d3f1eec86b683
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185101
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
2021-02-17 10:28:00 +00:00
Johnni Winther 34fb48bb8a [kernel,front_end] Migrate first wave of pkg/kernel and pkg/front_end
Migrates libraries dependent only on already migrated libraries.

Change-Id: I0e85ee8dbc2afce031b92e0009e71c206a55af28
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179502
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2021-01-18 15:40:21 +00:00
Sam Rawlins 090e234172 Remove unused dart:async imports
As of Dart 2.1, Future/Stream have been exported from dart:core.

Change-Id: Ia4d70d1ccb55ac117bafe630a24b4ee2f60ff993
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170126
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Samuel Rawlins <srawlins@google.com>
2020-11-03 18:49:54 +00:00
Johnni Winther 2c6f779a76 [cfe] Remove legacyMode
Change-Id: I9384631af94feaf1a345958ff23e6d35c2ba4049
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116202
Reviewed-by: Jens Johansen <jensj@google.com>
2019-09-13 07:38:25 +00:00
Jens Johansen 5601eabda6 [CFE] Spell check messages and dart code
This CL adds a simple spell-checker, adds a spell-checking-phase to
messages_test to spell-check our messages, adds a new test that
spell-check our dart-code and fixes a lot of spelling mistakes / typos.

Change-Id: I8943a5bd67970e9a8a775251ae0aa97799eab097
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112346
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2019-08-09 08:47:00 +00:00
Johnni Winther bcfe26cba0 [cfe] Fail early on unsupported evaluated constants
The many different entrypoints to the CFE have repeatedly shown problems
with targets that do not support unevaluated constants being used
without providing an environment.

Making the targets opt-in to supporting unevaluated constants will
give the users of CFE early notice that CFE is used inconsistently
(as opposed to the backends failing when they happen to see an
UnevaluatedConstant node that they don't support).

Change-Id: I98c80df3a551823598e413e2895f5649f22f7c22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/110561
Reviewed-by: Vijay Menon <vsm@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2019-07-26 16:57:51 +00:00
Peter von der Ahé 98987d2342 Talk about legacy mode, not strong mode in perf tools
Change-Id: I5a718b3db7905709caf92ca994784ee64a87b7b3
Reviewed-on: https://dart-review.googlesource.com/c/84410
Auto-Submit: Peter von der Ahé <ahe@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Peter von der Ahé <ahe@google.com>
2018-11-14 10:36:21 +00:00
Peter von der Ahé d4fac1684c Remove CompilerOptions.reportMessages
Change-Id: I6b3e2f44ef89ba6a234fab97e08f56a69247d13f
Reviewed-on: https://dart-review.googlesource.com/c/77081
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
2018-10-16 05:50:05 +00:00
Peter von der Ahé ee97806084 Replace strongMode with legacyMode in CompilerOptions
Also, remove --strong and --strong-mode flags (use --legacy-mode
flag to enable legacy mode).

Change-Id: I577ab5f755a2bee88f0f4d63915a6e852e792f06
Reviewed-on: https://dart-review.googlesource.com/c/79000
Commit-Queue: Jens Johansen <jensj@google.com>
Auto-Submit: Peter von der Ahé <ahe@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2018-10-10 12:04:26 +00:00
Peter von der Ahé 2a6b566ca2 Move byte store to package:analyzer
Change-Id: I1fc41cc0b2e2848565a5411d0512a4ca086ab659
Reviewed-on: https://dart-review.googlesource.com/c/77761
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Auto-Submit: Peter von der Ahé <ahe@google.com>
2018-10-04 09:15:54 +00:00
Peter von der Ahé 6fe272314d Tweak parameters of ProcessedOptions
Change-Id: Id7aa47d9119da3ab21a1130253b4735c160a2459
Reviewed-on: https://dart-review.googlesource.com/74740
Reviewed-by: Aske Simon Christensen <askesc@google.com>
2018-09-19 08:13:45 +00:00
Sigmund Cherem 9b44fc39c8 Deprecate chaseDependencies option.
The option is not used by any client at this time and removing it simplifies the
next stack of changes I'll be making on how we compose file systems.

I also don't believe we will necessarily add this back as an option, but instead
it might be that clients requriing hermeticity will provide a custom file system
that helps with that.

Change-Id: I401efb042920d234382b6c041b13d40ffae5c908
Reviewed-on: https://dart-review.googlesource.com/56462
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
2018-05-24 04:57:01 +00:00
Lasse R.H. Nielsen 973a1a0219 Remove uses of upper-case constants in remaining SDK code.
This includes Fasta, tools and observatory, so the checked-in SDK must
have the lower-case constants.

Change-Id: I8380ad041ad058f7d02ae19caccfecd434d13d75
Reviewed-on: https://dart-review.googlesource.com/50201
Commit-Queue: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
2018-04-16 16:11:57 +00:00
Peter von der Ahé e5c84938f2 Update names derived from Program
Change-Id: Ifdb072506fef909ccf5ad2e5270b1976ae61e3cc
Reviewed-on: https://dart-review.googlesource.com/31143
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
2018-03-16 09:33:07 +00:00
Peter von der Ahé 2cc21f515e Remove original and minimal IKG
Change-Id: Id845a025ae9823fd3e53aaac7b5a95aa9e2ad767
Reviewed-on: https://dart-review.googlesource.com/34380
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2018-01-15 15:02:20 +00:00
Aske Simon Christensen 9bdcc40f69 Rename PhysicalFileSystem to StandardFileSystem.
Preparation for data URI support. For clarity, this is just a rename.

Change-Id: Ibbd8da05d1a7106b5ec7c0ea9c1e13a95a89d1bd
Reviewed-on: https://dart-review.googlesource.com/33460
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2018-01-11 11:23:25 +00:00
Peter von der Ahé 8e127b4e53 Average incremental_perf benchmark over 8 loops
Closes https://github.com/dart-lang/sdk/issues/31675

Change-Id: Ida8d1b626cc4e3375f8f9f1d5b4626756ff16edc
Reviewed-on: https://dart-review.googlesource.com/30181
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Peter von der Ahé <ahe@google.com>
2017-12-19 08:52:52 +00:00
Peter von der Ahé 728a1770c4 Ensure edits of main are correctly detected
Change-Id: Ic7cf4f09c001d6a1d3b3a84fc3ead794f7530b09
Reviewed-on: https://dart-review.googlesource.com/28306
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2017-12-12 12:22:37 +00:00
Sigmund Cherem 6cab2dbc83 Report error messages in fasta benchmarks to stderr.
Our benchmark runner doesn't accept `stdout` larger than 64Kb, and some of our
benchmarks produce a lot of error messages. These error messages are whitelisted
while apps are updated to be strong-mode compliant, but the message list can be
long.

TBR=ahe@google.com,sortie@google.com
CC=​vegorov@google.com,kustermann@google.com

Change-Id: Ib2fe5812485357bf4ae0e61254321ba623519cd5
Reviewed-on: https://dart-review.googlesource.com/27500
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2017-12-08 04:12:19 +00:00
Sigmund Cherem b81d8a1c33 Ensure ikg-*-legacy also turn off inference.
The d2fe-fasta-legacy benchmark (which uses fasta_perf) was correctly disabling
both strong-mode and inference, but the d2fe-ikg-*-legacy benchmarks were only
disabling strong-mode and leaving inference enabled.

This change refactors the code to ensure that they are disabled consistently in
both.

Change-Id: Id7edc54209c41d27f735c66fc82024f5bf95fd09
Reviewed-on: https://dart-review.googlesource.com/26820
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2017-12-06 22:33:21 +00:00
Jens Johansen 22d0696d91 [CFE] Move front-end API under src/
This CL deprecates the front-end API and moves it into src/api_prototype.

For now all usages have been updated to point to the new location,
but they should be updated to use custom-client invocations instead (e.g.
one specific way for DDC, another for dart2js etc.)

Bug:
Change-Id: I9b4f41f6ebf55d42510fd35240d942d1dc7292d6
Reviewed-on: https://dart-review.googlesource.com/24822
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
2017-12-01 08:42:16 +00:00
Sigmund Cherem 32c306bca6 Add whitelist of error codes for fasta_perf and incremental_perf
Closes #31405

Change-Id: I272830b953590300b27a3b7ac41761936f35f454
Reviewed-on: https://dart-review.googlesource.com/22200
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
2017-11-20 22:08:47 +00:00
Sigmund Cherem 3b73de2060 Rename 'driver' to 'default' in incremental_perf.dart
This just makes the name more appropriate since now that the default incremental
generator is not using kernel-driver altogether.

Change-Id: Ic7357dbb4eaad482f20249dd13ba3a49a39322d3
Reviewed-on: https://dart-review.googlesource.com/22160
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2017-11-20 18:03:56 +00:00
Martin Kustermann 1d855b0067 Smaller changes to incremental_perf.dart, this is preparation for the flutter IKG perf benchmark
Amonst other things, it allows edits to reference package:<name> uris.

Change-Id: Iff90d5841901b5a003473e7086a266ccdcf94ad3
Reviewed-on: https://dart-review.googlesource.com/21540
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2017-11-17 10:34:12 +00:00
Sigmund Cherem 2a199ae689 Add flags to configure incremental_perf
This lets us benchmark separately:
- IKG implementations (the default based on kernel-driver or the minimal implementation)
- strong and non-strong mode

This also adds commands to try_benchmarks that highlight how these options will be used.

Change-Id: I5ce2c4563b7e79c33d78df7fd87be76f5d47e3f4
Reviewed-on: https://dart-review.googlesource.com/21320
Reviewed-by: Jonas Termansen <sortie@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2017-11-16 03:42:23 +00:00
Sigmund Cherem a49e945ac7 Ensure exit-code is non-zero when perf scripts encounter errors
Change-Id: I5f6e8e29e7e40547da7dfaf05b9b38e6d37dbbba
Reviewed-on: https://dart-review.googlesource.com/21001
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2017-11-15 18:12:04 +00:00
Sigmund Cherem be519c865f Use flag to configure the cache policy
We can separately pass this flag from individual benchmarks on the bots.

Change-Id: I541821976ad0538ee615718f0511e3154b983f2c
Reviewed-on: https://dart-review.googlesource.com/19764
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
2017-11-09 23:11:42 +00:00
Sigmund Cherem f776fc22ea Use the protected file bytestore in IKG benchmark.
Because by default we had the NullByteStore, we were practically recompiling the
entire app all the time.

Change-Id: I79ded9ab32d2518a35e392c95d7673b3abbb16ed
Reviewed-on: https://dart-review.googlesource.com/19762
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2017-11-09 21:36:08 +00:00
Martin Kustermann ff41877e04 Update incremental_perf.dart to use named changesets, add IKG benchmarks
Change-Id: I24d1e10bff363f021e201afbb996299f2c6a860d
Reviewed-on: https://dart-review.googlesource.com/13160
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2017-10-12 11:17:48 +00:00
Alexander Markov 9ee73fe322 [Kernel] Remove outdated and duplicated front-end Targets
This CL replaces outdated VmTarget and FlutterTarget with VmFastaTarget
and FlutterFastaTarget. 'Fasta' suffix is droped from target names.

The new FlutterTarget extends VmTarget, so they share more code.

Change-Id: Id79956698a889c9a49b8a67914f1f96a731407ab
Reviewed-on: https://dart-review.googlesource.com/9423
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2017-10-04 18:02:49 +00:00
Sigmund Cherem 655d60f2c5 Add incremental_perf: a script to measure performance of the incremental kernel
generator.

The script takes an input program Uri, and a file describing edits, and it mocks
applying the edits in multiple iterations.

incremental_perf_test illustrates how this can be used to do incremental
compiles of the dart2js source code.

Change-Id: I3518996a0faac34de8c6cef2f63468f4788b3c06
Reviewed-on: https://dart-review.googlesource.com/6026
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
2017-09-29 05:17:41 +00:00