Listing each possible output directory separately doesn't reduce the amount copied to shards as there are no other files in "out" to skip.
Change-Id: Ie55bb68d3ffe670d48eba5b03af2bc8ce7ce0dcc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231337
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Currently:
* "ddc nnbd weak tests" is split into 5 shards.
* "ddc nnbd weak co19 tests" is split into 2 shards.
* "ddc nnbd strong tests" is not spit into shards.
* "ddc nnbd strong co19 tests" is not spit into shards.
From a recent run taking a total of 45 minutes the timings
were like this:
```
ddc nnbd weak tests shard 1: 02:57
ddc nnbd weak tests shard 2: 01:56
ddc nnbd weak tests shard 3: 02:16
ddc nnbd weak tests shard 4: 01:49
ddc nnbd weak tests shard 5: 02:31
```
(i.e. a combined total time of 11:39 if run one after another)
```
ddc nnbd weak co19 tests shard 1: 08:45
ddc nnbd weak co19 tests shard 2: 09:48
```
(i.e. a combined total time of 18:33 if run one after another)
```
ddc nnbd strong tests: 09:51
```
```
ddc nnbd strong co19 tests: 20:44
```
Here I try to change the sharding "robin hood style" so that
"ddc nnbd weak tests shard" which seems pretty fast gets fewer,
but "ddc nnbd strong co19 tests" gets sharded.
Hopefully the result will be something like
ddc nnbd weak tests shard 1: 06:00
ddc nnbd weak tests shard 2: 06:00
ddc nnbd weak co19 tests shard 1: 09:00
ddc nnbd weak co19 tests shard 2: 09:00
ddc nnbd strong co19 tests shard 1: 7:00
ddc nnbd strong co19 tests shard 2: 7:00
ddc nnbd strong co19 tests shard 3: 7:00
ddc nnbd strong tests: 10:00
Which - with the same number of shards used - would hopefully cut off
~20 minutes (as the shards run concurrently)
Update: Total runtime was ~21 minutes, and runtimes seems pretty
agreeing with the expected.
Change-Id: Ie699e5687ab9a0d49cd197a4064bc8007d1dd806
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231700
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.
The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.
Quirks:
- RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
- There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
- The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
- For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
- Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
- There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
- The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
- We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.
Unimplemented:
- non-trivial FFI cases
- Compressed pointers - No intention to implement.
- Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
- BigInt intrinsics
TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This change bakes binary search table which maps PC ranges
to corresponding stack maps and Code objects (if still present
in the snapshot) into RO data section of the snapshot - instead
of constructing it at load time.
This allows to considerably reduce amount of work done when
loading Code cluster for programs which have majority of their
Code objects discarded (i.e. in DWARF stack traces mode): as
we no longer write / read any information for discarded Code
objects.
This CL also changes program visitor to deduplicate Code objects
if their instructions are deduplicated in AOT mode. Only a single
Code object can be choose as a representative for the given
PC range so it does not make sense to write multiple Code objects
into the snapshot which refer to the same Instructions.
The overall improvement is hard to quantify but ReadProgramSnapshot
shows the following improvement when starting a large
Flutter application on a slow Android device:
before 223.55±59.94 (192.02 .. 391.74) ms
after 178.06±47.03 (151.31 .. 291.34) ms
This CL packs CompressedStackMaps next to the binary search table
itself allowing us to address them via offsets instead of
pointers.
Snapshot sizes are actually affected positively by this change. On
the same large Flutter application I see
DWARF stack traces on: -1.34% total SO size
DWARF stack traces off: -1.63% total SO size
Issue https://github.com/dart-lang/sdk/issues/46116
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ic997045a33daa81ec68df462a0792915885df66b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220766
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Every AOT user out there has been using bare instructions mode
and continuing to maintaining non-bare instructions mode simply
adds costs (both in terms of time spent making changes to work
in a mode that is not used and CI resources spent on testing it).
This change removes FLAG_use_bare_instructions and changes the code
to assume that FLAG_precompiled_mode implies bare instructions.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,app-kernel-linux-release-x64-try,app-kernel-linux-debug-x64-try
Change-Id: I5032b13bfcb613f79865f2cfa139cca8d1b42556
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220964
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The python running on Mac M1 test builders is now native ARM64,
which makes the compilation and test use the xcodebuild/ReleaseARM64
directory, not the xcodebuild/ReleaseXARM64.
Add the xcodebuild/[mode]ARM64 build directories to sharding filesets.
Change-Id: Ide074dab3d4645f1776e83dc62c02f5ded630a9e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/218923
Reviewed-by: Daco Harkes <dacoharkes@google.com>
* Remove all uses of `--dart2js-batch`.
* Enable batch mode for dart2js by default.
* Rename `--noBatch` to `--no-batch`.
* Make `--no-batch` affect dart2js configurations.
Change-Id: I33bdb5cfe8a82ccfc6ce9e43843f1b7a34ff1fbf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217006
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
* This reverts commit c04a71e455.
* Fix pkg/compiler pubspec.yaml to work with package_deps script and
`dart pub get`.
Fixes https://github.com/dart-lang/sdk/issues/46877.
Reason for revert: Duplicate dart2js_info has been removed from DEPS.
TEST=dart pub get in pkg/compiler.
Original change's description:
> [infra] Temporarily remove package_deps.dart from the test matrix
>
> It currently fails because pkg/dart2js_info isn't wired up, yet.
>
> Change-Id: I10a2d43385fe3d900a62fa7438ed37b9e366dffb
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/209902
> Reviewed-by: William Hesse <whesse@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: I35e642d1adb931fff665d48f7f51c31c070ff180
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210680
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
macOS arm64 builds only create the x64 gen_snapshot by default because
they are cross-compiled in Rosetta. Adding the gen_snapshot target
ensures that we have a gen_snapshot for arm64 as well.
These bots require clang which currently isn't provisioned on shards, so
they are running on the main bot.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try
Change-Id: I842f37019a65dfb30e522d2d9bead5e62bb3ff79
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204505
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-linux-debug-try,pkg-mac-release-try,pkg-win-release-try
Change-Id: Iccd49bf2ef18250e4b1c755594d3382eff38cee5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204504
Reviewed-by: Jonas Termansen <sortie@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
* Add fallbacks to package smith to match the build recipes.
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,dart-sdk-win-try
Change-Id: I2d9da9e02119227bf59d4c200c886df4afc87487
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/202963
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Aske Simon Christensen