Switch build to use Clang. This somewhat degrades FFI testing, but reduces build time by roughly half or one hour.
Cq-Include-Trybots: luci.dart.try:pkg-win-release-arm64-try,vm-aot-win-debug-arm64-try,vm-aot-win-release-arm64-try,vm-win-debug-arm64-try,vm-win-release-arm64-try
Change-Id: Ie267942e99934312b228cfadcd6d5b8477f855c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429421
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Add builder definitions for ASAN on Mac and Windows.
Combine reload and reload rollback into one builder.
Change-Id: If464f82d1d8c0560f4e7f1e341f59e5f29301a30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/427303
Reviewed-by: Alexander Aprelev <aam@google.com>
`package:native_assets_cli` has been split up and
`package:native_assets_builder` has been renamed.
Change-Id: Ic9d5c21bb3c7ecf7924fdee6b4281153ad37fac5
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-arm64-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-win-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/425504
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This change adds a build target (see utils/analysis_server/BUILD.gn)
called 'analysis_server_aot'. This new target is _not_ included in the
Dart SDK (the create_sdk target). It's "opt-in" "for development."
The name of the new output file matches that of other snapshots (see
the dartdevc snapshots).
Then we do special work in the plugin manager if "we are AOT." An
analysis server running as AOT cannot spawn Isolates from a dart
source files; we must first compile a dart source file to AOT as well,
then we can spawn an Isolate to that AOT file.
_Then_ when we run pub, we can no longer rely on using
`Platform.executable`. `dartaotruntime pub get` is not a thing. We
must instead find the `dart` tool on disk. To do that, we copy some
complex discovery code from dartdev.
Work towards https://github.com/dart-lang/sdk/issues/53402
Work towards https://github.com/dart-lang/sdk/issues/53576
Work towards https://github.com/dart-lang/sdk/issues/50498
Manually tested:
* [+] analysis_server JIT snapshot works in IDE.
* [+] analysis_server JIT snapshot works in IDE, with a legacy
plugin (custom_lint).
* [+] analysis_server JIT snapshot works at commandline.
* [+] analysis_server AOT snapshot works in IDE.
* [x] analysis_server AOT snapshot works in IDE, with a legacy
plugin (custom_lint) - BROKEN. Need similar work that is done
for new plugins.
* [x] analysis_server AOT snapshot works at commandline - BROKEN.
I think a fair bit of refactoring is required in dartdev
lib/src/analysis_server.dart to use `VmInteropHandler.run` or
similar.
Change-Id: I53173c716fa2a763331ef524a96304f62165810e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/417942
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
The hostasserts and non-hostasserts filesets do not differ significantly
and can be merged for ease of maintenance. The cost of copying extra
files is small compared to building, and we currently lack the
granularity to see meaningful savings in builds if we keep them separate.
Change-Id: If8e3dd1013f60b05dabb104c632abd7a1638fe1d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/417947
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Currently we have 3 different string types (JS Strings, OneByteString
and TwoByteString)s. There's some advantages to this, mainly that
if strings are used purely inside Dart we have more control over
optimizing them. But it does come with some issues
* Operations on mixture of strings are slow
* We get JS strings from outside (in DevTools e.g. websocket messages)
* Any kind of DOM interaction requires copying strings
* Regular expression matches can result in O(N*N) instead of O(N)
* Encoding of string literals/constants is terrible, high size overhead
* ...
Now that there's a standardized way to access JS strings (via
the `js-string` builtin spec) and this standard is finalized and
enabled in Chrome & Firefox it makes sense for us to switch to it.
It reduces app size:
* Smaller size: hello world -25%, flute -5.5%
* Faster startup
The performance changes are nuanced, some workloads will improve
significantly, some workloads will regress.
Improvements will come especially in cases where
strings are concatenated (due to JS not actually allocating new
strings in this case). That impacts e.g. string interpolations,
string buffer, json-to-string encoding, ...
Regressions will come especially for cases where we have to
construct strings from bytes (e.g. in utf8 decoder, utf8+json
decoder) - mainly due to having to go through an intermediary
`WasmArray<WasmI16>` to allocate strings. Also in cases where we
access individual char codes from the strings.
There's some follow-up improvements we can do, but it's better to
not iterate on this CL even longer but get it landed.
This CL will make the benchmarking system use
`--require-js-string-builtin` as well as most of test CI
(in `pkg/dart2wasm/tool/compile_benchmark`)
Though we run some configurations via overriding with
`--no-require-js-string-builtin`
(in `tools/bots/test_matrix.json`)
Issue https://github.com/flutter/flutter/issues/159400#issuecomment-2538593980
Issue https://github.com/dart-lang/sdk/issues/59699
TEST=ci
Change-Id: I238ac65efe092de569da870f23134f889ac929f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/392903
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The file will contain a JS expression that evaluates to a boolean.
If it (at runtime) evalutes to
* `true` it means that all required features are supported by the JS
environment and the dart2wasm-compiled app can be used
* `false` it means some features were not present in the JS
environment and the dart2wasm-compiled app shouldn't be used,
instead a dart2js fallback may be used
We introduce this mechanism to allow users, at compilation time, to tell
dart2wasm to take advantage of new spec features and allow the runtime
to self-detect whether they are available and fallback to dart2js if
not.
The first feature we introduce (already in this PR) is
`--require-js-string-builtin` that will tell dart2wasm it can assume the
`js-string` builtin is available (and emit corresponding `*.support.js`
code to detect it).
If the flag was passed, we take advantage of the `js-string` import
mechanism for string constants that doesn't require emitting them in the
mjs file (which significantly reduces code size and improves startup
time - compared with emitting JS strings in the mjs file).
We enable `--require-js-string-builtin` on one CI configuration for
testing that if we don't use any polyfill, the imports of the builtin
functions as well as magical utf8-encoded wasm imports work.
We also use a template mechanism to generate `*.mjs` as the code
becomes more readable (e.g. to conditionally include the js string
polyfill)
Issue https://github.com/dart-lang/sdk/issues/59951
Change-Id: Ic7e7818a2d5269095935022941352beeb9fed731
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/408781
Reviewed-by: Ömer Ağacan <omersa@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The optimized builders run with binaryen which by default strips static symbols
(in all -Ox modes). So we'll pass `--no-strip-wasm` to `dart compile
wasm` in order to keep the static symbols.
This makes stack traces have the symbols in them which will make tests
that rely on that more likely to work.
Change-Id: I9e0eab8c4ec50ef3a7208551d04a80e9ffa009e8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/406241
Reviewed-by: Ömer Ağacan <omersa@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Fixes https://ci.chromium.org/ui/p/dart/builders/ci.sandbox/pkg-win-release-arm64/3781.
With new split the steps are distributed roughly like this:
```
$ RBE=0 tools/build.py -mrelease -ax64 gen_snapshot
[982/982] LINK ./gen_snapshot
The build took 39.285 seconds
$ RBE=0 tools/build.py -mrelease -ax64 runtime/bin:dart runtime/bin:run_vm_tests
[1392/1392] LINK ./run_vm_tests
The build took 97.936 seconds
$ RBE=0 tools/build.py -mrelease -ax64 runtime
[1223/1223] ACTION //utils/kernel-service:frontend_server_aot_product(//build/toolchain/linux:clang_x64)
The build took 97.168 seconds
$ RBE=0 tools/build.py -mrelease -ax64 create_sdk ddc_stable_test
[402/402] COPY gen/dart2js_aot_product.dart.snapshot dart-sdk/bin/snapshots/dart2js_aot.dart.snapshot
The build took 100.373 seconds
```
Also add Ryan to the OWNERS_INFRA.
Change-Id: Ibbf1a13ccd33d7bcb366ac864b05e960e5d73b8b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/404201
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
The weekly bot currently fails because the weak suite no longer have
dart2js_platform_unsound.dill available. This CL fixes that.
Change-Id: I2a8e71b6b8ce1ec9a9aa3d9456245c217f8ac26d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/404102
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
DDC does not depend on the stack trace mapper application being built
with unsound null safety so the unsound .dill files are unused.
Add the unsound .dill files to the `dart2js_bot` target because they
are dependencies of the `dart2js-hostasserts-linux-unsound` and
`dart2js-unit-linux-x64-release` configurations.
Update build step of `front-end-linux-release-x64` configuraton to
build the dart2js_bot target instead of relying on the unsound
dart2js .dill files through the `ddc_stable_test` target.
Change-Id: Ia29740b60f93d1d0b4a978ff5e37080e537401a7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/403382
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
This reverts commit 74c5aa3a7a.
Reason for revert: Fix golem breakage by not changing the script dart_precompiled_runtime2
TEST=ci
Original change's description:
> Revert ""[SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory""
>
> This reverts commit f81a402aa1.
>
> Reason for revert: golem benchmarks are failing to run
>
> TEST=ci
>
> Original change's description:
> > "[SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory"
> >
> > Fixed golem breakage by temporarily copying dartaotruntime to dart_precompiled_runtime
> >
> > This reverts commit 75e6a748f7.
> >
> > TEST=ci
> >
Change-Id: I9efe40643c59bc617f6fb484b89b038deaffbb93
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/393941
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
This reverts commit f81a402aa1.
Reason for revert: golem benchmarks are failing to run
TEST=ci
Original change's description:
> "[SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory"
>
> Fixed golem breakage by temporarily copying dartaotruntime to dart_precompiled_runtime
>
> This reverts commit 75e6a748f7.
>
> TEST=ci
>
> Original change's description:
> > Revert "[SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory"
> >
> > This reverts commit 1b331d05c2.
> >
> > Reason for revert: golem builds are failing
> >
> > Original change's description:
> > > [SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory
> > >
> > > TEST=ci
> > >
>
> Change-Id: Id0f383eabb496c06c0acebc639c8e3b056ba82d0
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/393781
> Commit-Queue: Siva Annamalai <asiva@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
Change-Id: Iec494940412aa31dbefdc5280e35ae99e8cecb26
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/393764
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Fixed golem breakage by temporarily copying dartaotruntime to dart_precompiled_runtime
This reverts commit 75e6a748f7.
TEST=ci
Original change's description:
> Revert "[SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory"
>
> This reverts commit 1b331d05c2.
>
> Reason for revert: golem builds are failing
>
> Original change's description:
> > [SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory
> >
> > TEST=ci
> >
Change-Id: Id0f383eabb496c06c0acebc639c8e3b056ba82d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/393781
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Collects files from `package:async_helper` and `tests/language`
that are generally useful, so that all test-related helpers are
in `package:expect`.
Moves the two libraries from `package:async_helper` into `package:expect`,
and the `tests/language/static_type_helper.dart` file too.
Deprecates `async_minitest.dart`, to follow `minitest.dart`,
expecting the Flutter use of it to have been fixed to not break
on deprecation (I believe Flutter no longer breaks builds on deprecations at all).
Patch 1 is the actual change.
Patch 2+4+8 is changing all existing references to the files.
Patch 6 ignores deprecation in files still using `async_minitest.dart`.
3+5+7+9 are updating this text to make the numbers match.
Then it's just test-expectations and small tweaks from there.
Change-Id: I1b665135b5fef9b9a0c3b340ffe9daf874d0174c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/373120
Reviewed-by: Nate Bosch <nbosch@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Lasse Nielsen <lrn@google.com>
This allows tests that would fail in JS compatibility mode (e.g. tests that do a lot of string manipulation) to disable the mode for their own execution.
Also ensure dart2wasm-*-jscm-* builds do a "create_sdk" so that the dart2js platform dill is available for the test.
The alternative I considered was skipping the test but "--js-compatibility" is a dart2wasm-specific flag and the test infra is unaware of it so we can't skip the test via a status file.
Bug: https://github.com/dart-lang/sdk/issues/56858
Change-Id: I60179eaab2e4522acf1fe0b47c7c02ccb1ca957b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/388644
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
This is a followup to
https://dart-review.googlesource.com/c/sdk/+/386900.
Since manually sharding the co19 tests on the firefox configuration
by subdirectories there have been no infrastructure failures on the
ddc-linux-firefox bot.
This change reduces the number of shards per each subdirectory
shard as an experiment to see if this will cause one of them to start
causing failures.
Change-Id: I1d8e0440134ade0c4c0be91e7ad8278b7bcacb01
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/388068
Reviewed-by: Nate Bosch <nbosch@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
After these changes the tests pass 100% of the time on all expected configurations.
- Emit 'unittest-suite-wait-for-done' at the start of tests to allow for async code loading. Tests must therefore emit 'unittest-suite-done' when they complete. Calling `asyncStart` and `asyncEnd` helpers will achieve this.
- Fix test_runner runtime.
- Add file deps on necessary files (dart2js platform dill & flute) to dart2wasm_hostasserts config.
- Mark dart2js compilation test as slow.
Change-Id: Iee993deb3905ccd50068325a5c5fd0bf0512a513
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/386980
Commit-Queue: Nate Biggs <natebiggs@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is an initial step in the investigation to determine if there are
specific tests causing the infra failures on the ddc-linux-firefox
configuration. Tests will still be randomly distributed across shards
but now clustered with the other tests in the immediate sub-directory.
If we find only certain sub-directories cause the flakes then we can
investigate those further to pinpoint the issue.
Change-Id: Idfdc4a4aab0b9f6307703136dd505a27c01ec3f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/386900
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Alexander Markov