dartfuzz has been taking ~35 min to finish the build and upload steps even with 100% RBE cache hit rate, causing it to fail to finish before the first wave of nightly builders trigger.
Change-Id: I932facd77bc08f4d7236ae8c760013406a69fb0e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507267
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This lets bloaty attribute the data symbol to its source file, allowing the binary size visualization to nest it appropriately.
TEST=runtime/tools/binary_size
Change-Id: I683ad6729d2fe2e1730dd3be70c6cce15bdf0003
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/486524
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Drop libprotobuf dependency and update compile_perfetto_protos
to rely on externally provided protoc and protozero_plugin.
This is done because protoc does not build with our current
toolchain and fixing this build requires rolling new protobuf,
protobuf_gn and perfetto. Which also pulls more external dependencies
like abseil.
Regenerate Perfetto *.pbzero{.cc,.h}, *.dart files
TEST=ci
Change-Id: I11a32aa07e810b48ac12502e526bbaee6a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/482781
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL tries to address version skew issues which occur in internal
monorepo which we created by checking in *.pbzero.h files into
runtime/vm/protos/*. These files refer to protozero internal headers,
which in the internal monorepo come from a shared Perfetto dependency
which does not necessarily match the version we pinned in our DEPS and
used to generated these files.
This version skew caused problems for the Perfetto team, see
https://github.com/google/perfetto/pull/3195
To resolve this change layout of our repo to allow us to depend on the
shared Perfetto entirely - including its version of *.pbzero.h headers.
We change include paths in the VM source to
#include "third_party/perfetto/protos/..."
which match location of these headers in the internal monorepo.
To make the code compile in the normal SDK checkout we shift location of
Perfetto and checked in code:
* Perfetto source moves from third_party/perfetto to
third_party/perfetto/src.
* Generated code moves from runtime/vm/protos into
third_party/perfetto/protos.
Note: experiments show that *.pbzero.h files tree-shake very well and
you pay only for things that you use. That means depending on *.pbzero.h
files generated from full message definitions rather than our manually
tree-shaken versions does not actually increase the size of the VM
binaries. We could switch our external build to do the same but this
would make protoc a build time dependency, which seems excessive.
TEST=ci
Change-Id: I724b9d9304be83302a5d83c2337154b36a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/454260
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Also add a '--dry-run' parameter as running it as sudo doesn't work
for me (it times out when it tries to build because of the RBE stuff I
think).
Now I can run something like
```
pkg/vm/tool/precompiler2 --dwarf-stack-traces --generate-probe-points path/to/file.dart path/to/file.aot
out/ReleaseX64/dart-sdk/bin/dart runtime/tools/profiling/bin/set_uprobe.dart alloc AllocationProbePoint path/to/file.aot --dry-run | sudo tee "/sys/kernel/tracing/uprobe_events"
cp out/ReleaseX64/dartaotruntime out/ReleaseX64/dart-sdk/bin/dartaotruntimeNotProduct
sudo perf record -g -e uprobes:alloc out/ReleaseX64/dart-sdk/bin/dartaotruntimeNotProduct path/to/file.aot <args>
sudo chmod 0755 perf.data
```
Change-Id: I189c2c08ec9cef5e61698e4fb9c9444494d20815
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/450040
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
- split the Dart CLI tool out of the VM into it's own embedder which
runs in AOT mode. The pure Dart VM executable is called 'dartvm' and
has no Dart CLI functionality in it
- the Dart CLI executable parses the CLI commands and invokes the rest
of the AOT tools in the same process, for the 'run' and 'test'
commands it execs a process which runs 'dartvm' to run
- 'dart hello.dart' execs the 'dartvm' process and runs 'hello.dart'
- the Dart CLI is not generated for ia32 as we are not shipping a
Dart SDK for ia32 anymore (support to execute the 'dartvm' for ia32
architecture is retained)
- the Dart CLI tool is not built in the internal Dart SDK builds
TEST=ci
Some performance improvement numbers
'dart format pkg/dartdev' goes from 1.17 secs to 0.22 secs
'dart doc pkg/dartdev' goes from 100.2 secs to 66.6 secs
'dart fix pkg/dartdev' goes from 19.3 secs to 14.5 secs
Change-Id: I66984a26cb2ab014b34dc1873f1f3d2884e13518
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/364202
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
The name has always been annoying because it did not add strong typing. And now there aren't variants of the VM platform to distinguish.
Leave a copy at the old name to not immediately break illegal uses.
TEST=ci
Cq-Include-Trybots: luci.dart.try:flutter-analyze-try,flutter-frontend-try,flutter-linux-try
Change-Id: Ie76fa7f16940aa1ba8d582eb5197f0ae55dc8938
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429828
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Make the size tools less dependent on quirks mode.
- Add explicit language attribute so Chrome stops offering to translate.
- Add zip_size to look at the Dart SDK and Flutter APKs.
Change-Id: I0017bd68145344e2427644fcccd13f9b9f307218
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/428064
Reviewed-by: Brian Quinlan <bquinlan@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This is a reland of commit 7ae1d75869
TEST=manual, no watchOS SDK on bots yet.
Original change's description:
> watchOS target os
>
> - add support for --os=watchos | --os=watchos_simulator
> - do not depend on perfetto if dart_support_perfetto is false
> - change the default value for dart_support_perfetto to !is_watchos
>
> To minimize changes and to avoid modifying third-party zlib BUILD file, is_watchos implies is_ios for now.
>
> TEST=manual, no watchOS SDK on bots yet.
>
> Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
> Change-Id: If2130068ef546162a07a9ba53f94b11ff25fb565
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415021
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
Change-Id: I2b50f7eb4da173627aeb4833b88a08712b278d9f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/425523
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit 7ae1d75869.
Reason for revert: breaks flutter build
Original change's description:
> watchOS target os
>
> - add support for --os=watchos | --os=watchos_simulator
> - do not depend on perfetto if dart_support_perfetto is false
> - change the default value for dart_support_perfetto to !is_watchos
>
> To minimize changes and to avoid modifying third-party zlib BUILD file, is_watchos implies is_ios for now.
>
> TEST=manual, no watchOS SDK on bots yet.
>
> Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
> Change-Id: If2130068ef546162a07a9ba53f94b11ff25fb565
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415021
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: I4bede9ede04ac3c533b8ff7ae2f1adbd84eb2397
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/425262
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Auto-Submit: Ivan Inozemtsev <iinozemtsev@google.com>
- add support for --os=watchos | --os=watchos_simulator
- do not depend on perfetto if dart_support_perfetto is false
- change the default value for dart_support_perfetto to !is_watchos
To minimize changes and to avoid modifying third-party zlib BUILD file, is_watchos implies is_ios for now.
TEST=manual, no watchOS SDK on bots yet.
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
Change-Id: If2130068ef546162a07a9ba53f94b11ff25fb565
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415021
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
There were `pubspec.yaml` files in `tools/` that were not
included in the global package config.
That means that their imports were irrelevant, which could
be misleading. And if any tool would look at the
`pubspec.yaml` file, it might be inconsistent with
the actual package resolution.
Makes every `pubspec.yaml` file either be included by the
`tools/generate_package_config.dart` script, or deletes them
if they seem to be stale and unused.
(Compare vs. `git ls-files '**/pubspec.yaml'`.)
Excepted `tools/dart2js/sourceMapViewer/pubspec.yaml`.
The entire directory might be stale. The dependencies of
that pubspec are not SDK dependencies otherwise,
the pubspec has no SDK min-version, which is now a requirement,
and the README refers to a *packages directory*.
Keeping as-is and filing issue to have owners take a look.
Added `lib/` directory to `tools/` to avoid `tools/bots/`
being inside the package URI root of the `tools/` package,
which would cause its `../../pkg/...` import to fail.
Makes every `pubspec.yaml` file use a `^...` SDK constraint instead
of the longer `>= ... < ...` format.
Tested: No new tests, goal is to keep running the same way
Change-Id: I688e463fe985fc4de43550a1f4c7ff350536cffc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/414020
Commit-Queue: Lasse Nielsen <lrn@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Sigurd Meldgaard <sigurdm@google.com>
Reviewed-by: Brian Quinlan <bquinlan@google.com>
_BoolArrayList needs to extend FixedLengthListMixin instead of
UnmodifiableListMixin since it implements `operator []=`.
Also contains some other doc fixes that were brought up post-submit in
https://dart-review.googlesource.com/c/sdk/+/414821.
TEST=tests/ffi/array_primitive_elements_generated_test.dart
CoreLibraryReviewExempt: Dart VM only.
Bug: https://github.com/dart-lang/sdk/issues/45508
Change-Id: I7635152283aec51a4f38e06fdfc7da84e93e55ab
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-arm64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-mac-debug-simarm64_arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-arm64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-arm64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-arm64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/417100
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Michael Goderbauer <goderbauer@google.com>
For the arrays of the Int/Uint/Float/Double types this has been
implemented straight in the ffi_patch.dart file and `elements` returns a
typed-data list view on the original data backing the array.
The Bool and Pointer arrays return helper classes from their `elements`
getter. These helper classes implement the `List` interface utilizing
the `operator []` implementation of the underlying arrays. This is also
implemented directly in the ffi_patch.dart file.
The `elements` getter for array arrays is rewritten in the CFE
(use_sites.dart) to instantiate a helper class to which the size of a
single element of the array is passed (this information is not available
in ffi_patch.dart, hence the rewrite in the CFE). The helper class
implements the `List` interface and performs some pointer arithmetic
with the provided element size to implement its methods.
The `elements` getters for struct and union arrays are also rewritten in
the CFE to instantiate a helper class to which a constructor tearoff for
instantiating the underlying struct/union type is passed. The helper
class also implements the `List` interface and uses the provided
constructor tearoff to create instantiated structs/unions of the
elements in the list.
Last, but not least, the `elements` getter for abi-specific integer
arrays is also rewritten in the CFI to instantiate a helper class to
which a closure is provided to load abi specific integers from the
underlying array data structure.
TEST=tests/ffi/array_compound_elements_test.dart
TEST=tests/ffi/array_primitive_elements_generated_test.dart
CoreLibraryReviewExempt: Dart VM only.
Bug: https://github.com/dart-lang/sdk/issues/45508
Change-Id: I211a42174057c39632c6a363d4e8e6fa8e94e801
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-arm64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-mac-debug-simarm64_arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-arm64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-arm64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-arm64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/414821
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Ryan Macnak