Listing each possible output directory separately doesn't reduce the amount copied to shards as there are no other files in "out" to skip.
Change-Id: Ie55bb68d3ffe670d48eba5b03af2bc8ce7ce0dcc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231337
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Currently:
* "ddc nnbd weak tests" is split into 5 shards.
* "ddc nnbd weak co19 tests" is split into 2 shards.
* "ddc nnbd strong tests" is not spit into shards.
* "ddc nnbd strong co19 tests" is not spit into shards.
From a recent run taking a total of 45 minutes the timings
were like this:
```
ddc nnbd weak tests shard 1: 02:57
ddc nnbd weak tests shard 2: 01:56
ddc nnbd weak tests shard 3: 02:16
ddc nnbd weak tests shard 4: 01:49
ddc nnbd weak tests shard 5: 02:31
```
(i.e. a combined total time of 11:39 if run one after another)
```
ddc nnbd weak co19 tests shard 1: 08:45
ddc nnbd weak co19 tests shard 2: 09:48
```
(i.e. a combined total time of 18:33 if run one after another)
```
ddc nnbd strong tests: 09:51
```
```
ddc nnbd strong co19 tests: 20:44
```
Here I try to change the sharding "robin hood style" so that
"ddc nnbd weak tests shard" which seems pretty fast gets fewer,
but "ddc nnbd strong co19 tests" gets sharded.
Hopefully the result will be something like
ddc nnbd weak tests shard 1: 06:00
ddc nnbd weak tests shard 2: 06:00
ddc nnbd weak co19 tests shard 1: 09:00
ddc nnbd weak co19 tests shard 2: 09:00
ddc nnbd strong co19 tests shard 1: 7:00
ddc nnbd strong co19 tests shard 2: 7:00
ddc nnbd strong co19 tests shard 3: 7:00
ddc nnbd strong tests: 10:00
Which - with the same number of shards used - would hopefully cut off
~20 minutes (as the shards run concurrently)
Update: Total runtime was ~21 minutes, and runtimes seems pretty
agreeing with the expected.
Change-Id: Ie699e5687ab9a0d49cd197a4064bc8007d1dd806
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231700
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
The dart2js script under the SDK will be deprecated soon, we are
switching all tests and benchmarks to use the supported `dart
compile js` entrypoint.
Change-Id: Ibd68aaddeb29fbae79ec11093446f233eb13c25e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229061
Reviewed-by: Jonas Termansen <sortie@google.com>
This change uses the DART_EXPERIMENTAL_BUILD environment variable to
detect whether the current build happens to be experimental and
shouldn't have side effects.
Bug: b/215113476
Change-Id: Id5442d80d69309e320077f053d8af75b3aab670a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/228862
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.
The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.
Quirks:
- RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
- There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
- The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
- For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
- Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
- There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
- The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
- We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.
Unimplemented:
- non-trivial FFI cases
- Compressed pointers - No intention to implement.
- Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
- BigInt intrinsics
TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
The snippets analyze step in [flutter]/dev/bots/analyze.dart now uses our built SDK when analyzing the code snippets. The explicit duplicate command to analyze the snippets with our SDK can now be removed.
The temp directory created in the script is not in the builder cache
on swarming, so it is automatically deleted when the builder finishes.
So we remove the cleanup step, which was trapping signals. That trap
seems to have been responsible for these failures and exit code confusion.
Bug: https://github.com/dart-lang/sdk/issues/47881
Change-Id: I49406954af54a0fba1ceec230c3551f8ad14e8e9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/226685
Reviewed-by: Paul Berry <paulberry@google.com>
This change bakes binary search table which maps PC ranges
to corresponding stack maps and Code objects (if still present
in the snapshot) into RO data section of the snapshot - instead
of constructing it at load time.
This allows to considerably reduce amount of work done when
loading Code cluster for programs which have majority of their
Code objects discarded (i.e. in DWARF stack traces mode): as
we no longer write / read any information for discarded Code
objects.
This CL also changes program visitor to deduplicate Code objects
if their instructions are deduplicated in AOT mode. Only a single
Code object can be choose as a representative for the given
PC range so it does not make sense to write multiple Code objects
into the snapshot which refer to the same Instructions.
The overall improvement is hard to quantify but ReadProgramSnapshot
shows the following improvement when starting a large
Flutter application on a slow Android device:
before 223.55±59.94 (192.02 .. 391.74) ms
after 178.06±47.03 (151.31 .. 291.34) ms
This CL packs CompressedStackMaps next to the binary search table
itself allowing us to address them via offsets instead of
pointers.
Snapshot sizes are actually affected positively by this change. On
the same large Flutter application I see
DWARF stack traces on: -1.34% total SO size
DWARF stack traces off: -1.63% total SO size
Issue https://github.com/dart-lang/sdk/issues/46116
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ic997045a33daa81ec68df462a0792915885df66b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220766
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
The results are now updated by the builder, so the builder status
does not need to poll the database for the results to finish processing.
The script is also migrated to null safety.
Change-Id: I7913f7fb0a8eedc074a4b79eb75c1c2813525c17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221780
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
This change refactors `_asExternalTypedData` into multiple functions,
which are data type specific, e.g `_asExternalTypedDataInt8`.
These functions are implemented as recognized methods.
Argument checks have previously been performed in the removed
runtime entry implementation of `_asExternalTypedData.` These are
now handled in Dart.
TEST=tests/ffi/external_typed_data_test.dart
Closes https://github.com/dart-lang/sdk/issues/39843
Closes https://github.com/dart-lang/sdk/pull/47780https://github.com/dart-lang/sdk/pull/47780
GitOrigin-RevId: 65763bddd06c633583b4adbf920c32c5a7a3720f
Change-Id: I703b350d89004ebe36e70eaa44dda1d9ebb050c7
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,app-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221360
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
The dart-archive bucket default ACL has been changed to make all
uploaded objects public. Therefore, we no longer need to specify an ACL
on upload. This change enables the scripts to work with uniform bucket
ACLs.
The bucket cannot be switched to uniform ACLs until this change reaches
the stable branch or it won't be possible to release Dart.
This change removes the uses of the gsutil -a public-read option from
the release scripts.
Change-Id: I27a76b9849771ddc380576ffe962926ebfbf4fc6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221341
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Every AOT user out there has been using bare instructions mode
and continuing to maintaining non-bare instructions mode simply
adds costs (both in terms of time spent making changes to work
in a mode that is not used and CI resources spent on testing it).
This change removes FLAG_use_bare_instructions and changes the code
to assume that FLAG_precompiled_mode implies bare instructions.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,app-kernel-linux-release-x64-try,app-kernel-linux-debug-x64-try
Change-Id: I5032b13bfcb613f79865f2cfa139cca8d1b42556
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220964
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The python running on Mac M1 test builders is now native ARM64,
which makes the compilation and test use the xcodebuild/ReleaseARM64
directory, not the xcodebuild/ReleaseXARM64.
Add the xcodebuild/[mode]ARM64 build directories to sharding filesets.
Change-Id: Ide074dab3d4645f1776e83dc62c02f5ded630a9e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/218923
Reviewed-by: Daco Harkes <dacoharkes@google.com>
* Remove all uses of `--dart2js-batch`.
* Enable batch mode for dart2js by default.
* Rename `--noBatch` to `--no-batch`.
* Make `--no-batch` affect dart2js configurations.
Change-Id: I33bdb5cfe8a82ccfc6ce9e43843f1b7a34ff1fbf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217006
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
William Hesse