Refactor `JSStringImpl` to use the js-string-builtins API[1].
When the module `WebAssembly.String` is not available we define a
"polyfill" with the previous imports, so this change is backwards
compatible.
Also updates some of the methods to use avoid multiple `this.length`
calls (which do an FFI call), and use unchecked getters when possible.
A new library `dart:_error_utils` is introduced for faster range and
index checks when the length (or max value) is known to be positive
(e.g. when it's the length of a string).
For now only `JSStringImpl` and `operator []` and `operator []=` of
`JSArrayImpl` are updated to use the new range and index checks. Rest of
the libraries will be updated separately.
[1]: https://github.com/WebAssembly/js-string-builtins
CoreLibraryReviewExempt: dart2wasm specific library change.
Change-Id: I9436def0cfe59c631f6f4e15ea06cc18a47a738e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335043
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ömer Ağacan <omersa@google.com>
A recent change: https://dart-review.googlesource.com/c/sdk/+/333842
fixed a check in the test runner. The check fails when a flag
is provided to test.py but it is also implied by the configuration
name.
However, The web macos bots currently need to specify the arch flag,
even though it is implied by the named configuration. Without doing so
the sharding infrastructure seems to fail to assign the correct
architecture for the spawn bots.
This CL temporarily allows to continue specifying this flag to keep the
bots running. Ideally we should follow up with a infrastructure fix
to ensure this flag is not needed in the future.
This is being tracked in https://github.com/dart-lang/sdk/issues/53948
Change-Id: Ifba6a8924cbca9f3aca38933113abb168ff405d4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333705
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
When doing local testing of AOT configurations I need to
manually override use-elf property to shorten cycle time.
Currently this can only be done by editing test_matrix.json,
this CL adds a capability to achieve the same by defining
TEST_CONFIGURATION_OVERRIDES environment variable.
I also fix some incorrect checking code which never triggered before:
if you try (via CLI flag) to override an option which is already
defined by a selected named configuration you should get an error.
Change-Id: Ie9b7fa12bd441814ca662e7e820c4aa739471c22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333842
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Build one package with three elf_test_runner components: one for each of the binaries that the test harness invokes on the target. (Though currently only tests that use "dart" are setup.)
Create includable CML shards for the capabilites required by the AOT and JIT VMs.
Fold test_runner's fuchsia_cfv2.dart back into fuchsia.dart.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/38752
Change-Id: I7bd8d43d184cbcb11903c7aed77ce31fb30cb894
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332451
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zijie He <zijiehe@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Ensures there is no dependency on modifying the native JavaScript
Object prototype by preventing modifications when running browser
tests.
Change-Id: Ifb22018736f90b339b9c60bbab095fb3e1c483e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/320842
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
The first test being compiled on a batch compiler pays for the cost of
the batch compiler startup, meaning it is more likely to time out, e.g.
see https://github.com/dart-lang/sdk/issues/53672.
E.g via
`tools/test.py '--progress=status' --report --time --write-results --write-logs --clean-exit -ncfe-strong-linux service/next_...`
I get output like this:
```
0:00:37.713771 - fasta - fasta-none release_x64/service/next_through_simple_linear_test/service
0:00:37.682524 - fasta - fasta-none release_x64/service/next_through_for_loop_with_break_and_continue_test/dds
0:00:36.805734 - fasta - fasta-none release_x64/service/next_through_create_list_and_map_test/service
0:00:36.589329 - fasta - fasta-none release_x64/service/next_through_simple_linear_2_test/dds
0:00:36.586428 - fasta - fasta-none release_x64/service/next_through_simple_linear_2_test/service
0:00:36.518832 - fasta - fasta-none release_x64/service/next_through_for_each_loop_test/dds
0:00:36.407107 - fasta - fasta-none release_x64/service/next_through_create_list_and_map_test/dds
0:00:36.259945 - fasta - fasta-none release_x64/service/next_through_for_each_loop_test/service
0:00:36.211573 - fasta - fasta-none release_x64/service/next_through_call_on_field_in_class_test/service
0:00:36.189456 - fasta - fasta-none release_x64/service/next_through_call_on_field_in_class_test/dds
0:00:35.974007 - fasta - fasta-none release_x64/service/next_through_for_loop_with_break_and_continue_test/service
0:00:35.921138 - fasta - fasta-none release_x64/service/next_through_simple_linear_test/dds
0:00:09.690812 - fasta - fasta-none release_x64/service/next_through_multi_catch_test/service
0:00:09.562068 - fasta - fasta-none release_x64/service/async_next_test/service
0:00:09.530882 - fasta - fasta-none release_x64/service/next_through_operator_bracket_test/dds
0:00:09.407169 - fasta - fasta-none release_x64/service/next_through_function_expression_test/dds
0:00:09.366673 - fasta - fasta-none release_x64/service/next_through_multi_catch_test/dds
0:00:09.327895 - fasta - fasta-none release_x64/service/next_through_function_expression_test/service
0:00:09.303926 - fasta - fasta-none release_x64/service/next_through_await_for_test/service
0:00:09.236295 - fasta - fasta-none release_x64/service/next_through_simple_async_with_returns_test/dds
```
where the first 12 (the really slow ones) each is the first test being
run on that particular batch compiler, where they both pay for the
startup cost of the batch compiler (~12 seconds on my machine when
launching 12 simultaneously which is the default on my machine -- it's
probably more on the bots) and for the jit not having warmed up yet:
E.g. the first test compiles in ~22 seconds, the second in ~9 seconds,
the third in ~6 seconds on my machine (again with 12 simultaneous runs).
To make it less likely that the first compile times out this CL adds
some extra time allowance for the first run after the bulk compiler is
started. For now it's 30 seconds, but it can be adjusted if needed.
Longer term we should probably start a batch compilation test by
aot-compilng the batch compiler (which, at least currently,
doesn't support asserts
(https://github.com/dart-lang/sdk/issues/53343)
which is unfortunate).
I'll leave that for a future CL.
Bug: https://github.com/dart-lang/sdk/issues/53672
Change-Id: Id3427223c46c1e7a34b401097a3f7d5f1321ad93
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332220
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Though it's still failing, the test-script integration has pretty much
finished.
```=== 298 tests passed, 40 failed ===```
Errors are mainly
```
Exhausted heap space, trying to allocate 32 bytes.
../../runtime/vm/object.cc: 2854: error: Out of memory.
```
This change requires https://crrev.com/c/4913284 and
`"fuchsia_sdk_version": "version:15.20231007.2.1",`
in DEPS.
Bug: #38752
Change-Id: I92a387f4289fce7d05d84e483560729301541c1b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330025
Commit-Queue: Zijie He <zijiehe@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
This logic is responsible for creating a readable deobfuscated stack
from ddc and dart2js JavaScript stack traces. The logic was removing the
query parameter of URLs, but accidentally matched multi-line strings and
applied the regexp in invalid contexts (like the error message itself).
Excluding `\n` from the regexp pattern prevents this issue.
Example input (obfuscated error):
```
window.onerror called:
http://127.0.0.1:35679/root_build/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:5082:
Uncaught Error
Error: Expected a value of type 'FutureOr<FileSystem>?', but got one of type 'LegacyJavaScriptObject'
at Object.throw_ [as throw] (http://127.0.0.1:35679/root_build/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:5082:11)
at Object.castError (http://127.0.0.1:35679/root_build/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:5041:15)
at Object.cast [as as] (http://127.0.0.1:35679/root_build/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:5385:19)
at dart.NullableType.new.as (http://127.0.0.1:35679/root_build/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:7243:60)
at _AsyncCompleter.new.complete (http://127.0.0.1:35679/root_build/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:31216:28)
at http://127.0.0.1:35679/root_build/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:99073:19
```
Example deobfuscation before (see truncated error and first frame was dropped accidentally):
```
window.onerror called:
out/ReleaseX64/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:5082:
Uncaught Error
Error: Expected a value of type 'FutureOr<FileSystem>:5082:11)
at Object.castError ../../../../../../dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 127:3
at Object.cast ../../../../../../dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/operations.dart 818:12
at dart.NullableType.new.as ../../../../../../dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/types.dart 425:9
at _AsyncCompleter.new.complete ../../../../../../dart-sdk/lib/async/future_impl.dart 42:31
at <fn> ../../../../../../dart-sdk/lib/html/dart2js/html_dart2js.dart 33424:16
```
but after (error and first frame recovered):
```
window.onerror called:
out/ReleaseX64/gen/utils/ddc/stable/sdk/amd/dart_sdk.js:5082:
Uncaught Error
Error: Expected a value of type 'FutureOr<FileSystem>?', but got one of type 'LegacyJavaScriptObject'
at Object.throw_ out/ReleaseX64/gen/utils/ddc/stable/sdk/amd/dart_sdk.js 5082:11
at Object.castError ../../../../../../dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 127:3
at Object.cast ../../../../../../dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/operations.dart 818:12
at dart.NullableType.new.as ../../../../../../dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/types.dart 425:9
at _AsyncCompleter.new.complete ../../../../../../dart-sdk/lib/async/future_impl.dart 42:31
at <fn> ../../../../../../dart-sdk/lib/html/dart2js/html_dart2js.dart 33424:16
```
Change-Id: Id9f37efe9bdacc63aae3b24ebe0602a8ea7cf8fd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/328765
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
This change updates the FuchsiaEmulatorCFv2 to start the daemon and
emulator before executing the tests.
Meanwhile, instead of "dummy", this change uses "echo" as the dummy
command to allow
`FUCHSIA_CFV2=1 tools/test.py --progress=status
-nvm-fuchsia-release-x64 ffi`
running.
This change requires
`"fuchsia_sdk_version": "version:15.20230909.2.1",`
in DEPS to function, which isn't included due to the CFv1
implementation currently running.
Bug: #38752
Change-Id: I5011b6ff39b73955002431650d973d6adcfa6246
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/328766
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zijie He <zijiehe@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Timer events and perhaps other events are throttled when a Chrome tab
is in the background or not visible. This may be causing flaky timeouts
in Dart web tests that rely on many timers. In local testing, these
flags completely eliminated the problem.
IntensiveWakeUpThrottling is a web standard feature, and
TimerThrottlingForBackgroundTabs is a Blink-specific feature.
Bug: https://dart-review.googlesource.com/c/sdk/+/324203
Change-Id: I38904015b043768689be3a05bea5a50ad32df434
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325780
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This change created a CFv2 build target and provided two
implementations of the FuchsiaEmulator, so that I can continually work
on both CFv2 build and test script integration independently.
The v2 target can be built with
```
tools/build.py --os fuchsia --mode release fuchsia_ffi_test_package_cfv2
```
And v2 FuchsiaEmulator implementation can be triggered with
FUCHSIA_CFV2 environment variable.
Bug: #38752
Change-Id: I31936a2ca967fbfeb2bc5628e2f005aef6762687
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322583
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zijie He <zijiehe@google.com>
The with_envs.py works with the test script by providing the required
environment variables.
This change in theory does nothing since the `product download` is in
12.20230413.0.1, but `run` is removed before that,
https://fxrev.dev/831319. So the DEPS cannot really download the images.
The issue will be addressed in a coming change.
Bug: #38752
Change-Id: Ib5f3d1d619921651208fa442201ba607a99e54d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319867
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Zijie He <zijiehe@google.com>
- Increase the size of the FFI trampoline template.
- Increase alignment of ELF binaries generated by the VM.
- Pass flags for binaries generated by the NDK's linker increase alignment.
- Runtime allocation already either rounds up to the runtime-queried page size or assumes it is no more than 512k.
TEST=ci, readelf
Bug: go/16k-app-guide
Change-Id: I4fdb2a47534a6a24875e205c3a357b5415ca18b8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321302
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
- Updates test infra to use the assets built by the new "ddc" build
targets.
- The old "dartdevc" targets are still present to avoid breaking
golem when this change lands.
- Old targets are still used when packaging assets for the SDK.
- Golem and the SDK packaging will be updated in following changes.
Change-Id: I1926e0c86833c812f5ed8355d7cd4df8740d79ee
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/315224
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Previously, --suppress-core-dump option was inserted into the beginning
of the command line and was effectively passed to fssh, which
doesn't understand this option:
./third_party/fuchsia/sdk/linux/tools/x64/fssh --suppress-core-dump run fuchsia-pkg://fuchsia.com/dart_ffi_test_release#meta/fuchsia_ffi_test_component.cmx --sound-null-safety -Dtest_runner.configuration=vm-fuchsia-release-x64 --ignore-unrecognized-flags --packages=/pkg/data/.dart_tool/package_config.json --disable-dart-dev /pkg/data/tests/ffi/callback_unwind_error_test.dart
Instead, it should be passed as the first argument after 'run ...cmx':
./third_party/fuchsia/sdk/linux/tools/x64/fssh run fuchsia-pkg://fuchsia.com/dart_ffi_test_release#meta/fuchsia_ffi_test_component.cmx --suppress-core-dump --sound-null-safety -Dtest_runner.configuration=vm-fuchsia-release-x64 --ignore-unrecognized-flags --packages=/pkg/data/.dart_tool/package_config.json --disable-dart-dev /pkg/data/tests/ffi/callback_unwind_error_test.dart
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-x64-try
Change-Id: Id33a4c54478b872889dcdd7034e9dae6336a3ae8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316620
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: William Hesse <whesse@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
- Introduces a new output subdirectory `gen/utils/ddc` where the
new build targets output files. This is intended to make the
migration to the new assets easier since landing this change
shouldn't immediately break any dependencies.
- Enables building the canary and stable assets at the same time.
- Changes more names in our workflow from "dartdevc" to "ddc".
- Updates the ddc-canary-linux-chrome and
ddc-canary-linux-chrome-unsound configurations to use the
new assets.
The new outputs appear in directories that are more consistently
named, (no more sound vs kernel confusion). They are named in
preparation for the eventual deletion of the unsound targets
without any lingering cruft to be cleaned up in the sound
directories.
The new structure is:
```
gen/utils/ddc/
|- canary
| |- pkg
| |- sdk
|- canary_unsound
| |- pkg
| |- sdk
|- stable
| |- pkg
| |- sdk
|- stable-unsound
|- pkg
|- sdk
```
The 'pkg' and 'sdk' directories all contain outputs with the same
files names, each compiled in the respective build modes
(sound/unsound and stable/canary).
Change-Id: I66822ebf03bba487b6d359a8e0aa818b9e7b6bef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313081
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
This reverts commit f827eb3a78.
Reason for revert: The tests being run, and the results being compared to, are changing dramatically, causing all sorts of test results going from [status] -> Skipped and from New Test -> [status].
Something is really going wrong.
Original change's description:
> Balance tests equally across shards.
>
> The sharded test runner invocations are now passed the previous results
> which contains the test timing, which are used to simulate how long each
> shard would take to run. The shards are now balanced as evenly as
> possible on a test level, taking multiple cores into account.
>
> Sharded tests are now run starting with the slowest test first, such
> that extremely long running tests finish as early as possible. This
> behavior ensures the cores are saturated and can be padded with fast
> tests near the end, rather than waiting for a few slow tests to complete
> while the rest of the system is idle.
>
> The algorithm works very well whenever it's able to accurately predict
> the time to run shards. In a number of cases, the model doesn't quite
> reflect reality and the data, which makes it fairly imperfect but still
> reasonably good. I think a second order feedback loop might kick in once
> it reorders the tests across shards and the test timing data reflects
> the new test timings.
>
> Multitests are no longer always sent to the same shard, since the data
> isn't available at the moment, and the change as-is speeds up the test
> running considerably.
>
> The front end unit test suites currently ignore the feature as there are
> no benefits yet to improving those quick shards.
>
> Upgrade the language version to 3.0.0 so patterns can be used and fix
> a mixin not being a mixin.
>
> Fixes: b/291585137
> Change-Id: I3cc1b1d96038d5b46e836b091e299097717c226c
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314081
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
Change-Id: I233e4bfa6d6ecf0cea4f97c1e47f1635f7b9040c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/315060
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: William Hesse <whesse@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
The sharded test runner invocations are now passed the previous results
which contains the test timing, which are used to simulate how long each
shard would take to run. The shards are now balanced as evenly as
possible on a test level, taking multiple cores into account.
Sharded tests are now run starting with the slowest test first, such
that extremely long running tests finish as early as possible. This
behavior ensures the cores are saturated and can be padded with fast
tests near the end, rather than waiting for a few slow tests to complete
while the rest of the system is idle.
The algorithm works very well whenever it's able to accurately predict
the time to run shards. In a number of cases, the model doesn't quite
reflect reality and the data, which makes it fairly imperfect but still
reasonably good. I think a second order feedback loop might kick in once
it reorders the tests across shards and the test timing data reflects
the new test timings.
Multitests are no longer always sent to the same shard, since the data
isn't available at the moment, and the change as-is speeds up the test
running considerably.
The front end unit test suites currently ignore the feature as there are
no benefits yet to improving those quick shards.
Upgrade the language version to 3.0.0 so patterns can be used and fix
a mixin not being a mixin.
Fixes: b/291585137
Change-Id: I3cc1b1d96038d5b46e836b091e299097717c226c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314081
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
Ben Konyi