When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL introduces new embedding APIs for supporting heap sample
profiling. A registered sampling callback is invoked approximately every
N bytes based on an exponential distribution, providing information
about the isolate group the allocation occurred in, the user visible
name of the allocated object type, a weak persistent handle to the
allocated object, and the size of the allocation.
Sampling is triggered using artificial TLAB boundaries to cause
allocations to be sampled to take the allocation slow path where the
registered callback can be invoked with the allocation information.
Only new space allocations are currently traced, with old space
allocation support to be added in a future CL.
TEST=Dart_HeapSampling
Change-Id: I22bcdeec6e823bc1ab44898d4c596fbed7169fa1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264520
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Some large applications are bumping up against the 16-bit limit.
The object header changes from hash:32,cid:16,size:8,gc:8 to hash:32,cid:20,size:4,gc:8.
Because the CID field is no longer a power of two, it takes two instructions to load instead of one, increasing code size. The alternative change to cid:32,hash:24,gc:8 was considered and rejected because the reduction in hash bits very noticeably affects performance in applications with large data sets.
TEST=ci
Bug: b/255934984
Change-Id: I5fa36ba48a6852359d994393cf80c6a761c5d84c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/269120
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
* Avoid calling memmove(dst, nullptr, 0) as this is flagged by UBSAN.
* Avoid hitting a bug[1] in the linker: LLD's identical code folding
(ICF) happens to replace RecordCoverageInstr::DebugName() with
DispatchTable::LargestSmallOffset() because they happen to contain
the same machine code, ICF fails to accomodate that DebugName also
contains a relocation to constant string. To avoid this we simply
eliminate LargestSmallOffset and replace it with a constant. Same for
OriginElement.
TEST=manually tested previously failing tests
[1]: reported https://github.com/llvm/llvm-project/issues/57693
Change-Id: I38637df6475c7670081b7af0a2de75ca37f6f07c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258801
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
* Merge ClassTable and SharedClassTable back together;
* Simplify handling of multiple arrays growing in sync;
* Refactor how reload deals with ClassTable.
The last change is the most important because it makes it
much easier to reason about the code. We move away from
copying bits and pieces of the class table and shared
class table into reload contexts.
Having two class table fields in the isolate group makes
it easier to reason about. One field contains program
class table (one modified by kernel loader and accessed
by various program structure cid lookups) and heap
walk class table (used by GC visitors). Normally these
two fields point to the same class table, but during
hot reload we temporary split them apart: original
class table is kept as a heap walk class table, while
program class table is replaced by a clone and updated
by reload.
If reload succeeds we drop original class table and
set program class table as heap walk one.
If reload fails we drop the program class table and
restore original one from heap walk table.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: I8b66259fcc474dea7dd2af063e4772df99be06c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258361
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This option dumps offsets of Dart fields and sizes of Dart instances
into the specified file in JSON format.
Could be useful for debugging or memory usage investigations.
Typical output:
{"class":"_JsonUtf8Parser","size":80,"fields":[{"field":"emptyChunk","static":true},{"field":"decoder","offset":56},{"field":"chunk","offset":64},{"field":"chunkEnd","offset":72}]}
TEST=runtime/tests/vm/dart/print_object_layout_test.dart
Fixes https://github.com/dart-lang/sdk/issues/47892
Change-Id: I0b2f7ed9e4991e5fcd18517d5a06aa826c6d7125
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/223662
Reviewed-by: Dan Field <dnfield@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup` (or to another place).
Part of that is `Isolate::reload_context()`. In order to avoid name
conflict of what a `reload_context()` is, we separate it into
`reload_context()` and `program_reload_context()` where the former
contains GC information and the latter contains information about
changes in program structure.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I7e1b732ce8b34c8842718699d618be542bd59ff6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177863
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Right now we assign class ids to top-level classes, abstract classes as
well as concrete classes. All of them have allocated from a 16-bit pool
of ids. The VM FATAL()s once it hits that limit.
Customers who run very large programs (significant amount of generated
code) on the Dart VM have started to hit this 16-bit class limit.
Concrete classes can have instances in the heap. Our current heap layout
only allows 16-bit class ids to be encoded in the header word. To avoid
increasing the size of heap objects or shrinking the size of the identity
hash code to 16-bit we keep class ids in object headers to be 16-bit.
Abstract classes cannot have instances in the heap. Though their class
ids are encoded in type objects. Furthermore we sort classes in
AOT/AppJIT mode to perform fast class-id range checks. To avoid impacting
this optimization we treat abstract classes the same way as concrete
classes.
Top-level classes cannot have instances in the heap. Their class ids are
only used in the runtime code, for example for hot-reload as well as
part of the service protocol.
=> We can allocate class ids outside the 16-bit range for top-level
classes, thereby freeing a significant amount of space in the 16-bit
range.
This CL does exactly that: We change classid_t to be int32_t. The
ClassLayout::id_ can now be assigned ids outside 16-bit range for
top-level classes. To do this we keep dart classes and top level classes
as separate arrays in the ClassTable.
Issue https://github.com/dart-lang/sdk/issues/42533
See also b/160229360
Change-Id: I6710a644e7b0ab2d4f4c792bef8e1f91cb117421
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153607
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit 922ea3e9b6 in patchset 1, fix for assertion triggered in https://ci.chromium.org/b/8883214567628884960 in patchset 2, fix for deadlock around symbols table mutex in patchset 4.
Original commit description:
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I655e337198214c9dfacbe76f7852b941b5a7e910
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143462
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Original revert in patchset 1, fix for deadlock issue in patchset 2: avoid reentrant calls to RunWithStoppedMutator.
Further review comments addressed in successive patchsets.
This reverts commit 8ef508ba36.
Original commit description:
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Change-Id: I6a0279d9812020ad7a5c2b7851980b6a29b95b9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143327
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I64c86525f4ef9cb30567a49a106bfe700355942b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136780
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Using memmove/memset bypasses non-trivial assignment. In particular, it changes relaxed atomic operations into non-atomic operations.
Fixes -Werror=class-memaccess errors in gcc.
Change-Id: Ic4589fe7a564a268a578a856058c2d2183961b46
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142549
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL:
* Moves [Heap]/[SharedClassTable] from [Isolate] to [IsolateGroup], which
will make all isolates in the group use the same heap. The GC will use
the shared class table for object size information.
* Adds support for entering/leaving an isolate group as a helper thread
(e.g. via [Thread::EnterIsolateGroupAsHelper]). The current active
isolate group can be accessed via TLS `IsolateGroup::Current()` or
`Thread::isolate_group_`. When entering as a helper thread there will be
no current isolate.
* Changes the GC to use the above mechanism and ensures GC works without
a currently active isolate. The GC will use information purely available via
[IsolateGroup]. The GC will iterate all isolates within an isolate
group e.g. for scanning roots.
* Makes spawning of new isolates start in their own isolate group.
Once the isolate is fully functional it's heap will be merged into
the original isolate group
* Moves ApiState, containing persistent and weak persistent handles,
from [Isolate] to [IsolateGroup], plus adds appropriate locking.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Ia8e1d8aa78750e8400864200f4825395a182c004
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126646
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit 9eb531bde4.
Reason for revert: Bots are red. Some tests are failing.
https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-obfuscate-linux-release-x64/6039https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-android-release-arm_x64/957
Original change's description:
> [vm] Add support for real unboxed floating point fields in AOT
>
> Non-nullable floating point fields (double, Float32x4, Float64x2)
> are fully unboxed in their classes.
>
> A bitmap for each class was added to the shared class table in order to keep
> track of the pointers of the classes. Since all classes in Flutter Gallery
> have less than 64 fields, the bitmap is represented by a 64 bit integer and
> fields whose offset is more than 64 words are not unboxed.
>
> The instance sizes and field offsets might change between target and host
> in cross-compilation, since the number of words used to store unboxed fields
> may differ.
>
> dart-aot Xeon
>
> SplayLatency -4.62%
> SplayHarderLatency -4.17%
> NavierStokes -2.20%
> Tracer 8.72%
> ParticleSystemPaint 2.90%
> NBodySIMD 8.35%
> NBody 25.59%
>
> With hack TFA to make doubles in Rect/Offset/Size classes in flutter non-nullable:
>
> flutter arm-v8:
>
> gallery total size: -1%
>
> matrix_utils_transform_rect_perspective -16.70% (less is better)
> matrix_utils_transform_rect_affine -31.82% (less is better)
> matrix_utils_transform_point_perspective -24.90% (less is better)
> matrix_utils_transform_point_affine) -27.26% (less is better)
> rrect_contains_bench -4.719% (less is better)
>
> Change-Id: I9ae09c9c3167d99f9efd071a92937aa51093fd1d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131824
> Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Samir Jindel <sjindel@google.com>
TBR=kustermann@google.com,rmacnak@google.com,sjindel@google.com,victoragnez@google.com
Change-Id: Ic73858f6adb7f55c4129d4f46ff4731b378cb634
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134020
Reviewed-by: Zichang Guo <zichangguo@google.com>
Commit-Queue: Zichang Guo <zichangguo@google.com>
Non-nullable floating point fields (double, Float32x4, Float64x2)
are fully unboxed in their classes.
A bitmap for each class was added to the shared class table in order to keep
track of the pointers of the classes. Since all classes in Flutter Gallery
have less than 64 fields, the bitmap is represented by a 64 bit integer and
fields whose offset is more than 64 words are not unboxed.
The instance sizes and field offsets might change between target and host
in cross-compilation, since the number of words used to store unboxed fields
may differ.
dart-aot Xeon
SplayLatency -4.62%
SplayHarderLatency -4.17%
NavierStokes -2.20%
Tracer 8.72%
ParticleSystemPaint 2.90%
NBodySIMD 8.35%
NBody 25.59%
With hack TFA to make doubles in Rect/Offset/Size classes in flutter non-nullable:
flutter arm-v8:
gallery total size: -1%
matrix_utils_transform_rect_perspective -16.70% (less is better)
matrix_utils_transform_rect_affine -31.82% (less is better)
matrix_utils_transform_point_perspective -24.90% (less is better)
matrix_utils_transform_point_affine) -27.26% (less is better)
rrect_contains_bench -4.719% (less is better)
Change-Id: I9ae09c9c3167d99f9efd071a92937aa51093fd1d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131824
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
Similar to the split of ClassTable into ClassTable/SharedClassTable,
this CL splits up the IsolateReloadContext into:
* IsolateGroupReloadContext: Consists of reload-related information across all
isolates. The [Reload()] method is split up in phases that are
performed on all isolates before the next phase is started.
=> This allows each isolate to add reasons for rolling back, if no
reasons are found the reload will be accepted atomically.
* IsolateReloadContext: Constists of reload-related information for a
particular isolate (e.g. mappings of old to new classes)
The assumption is that all isolates have the same source (and therefore
the same libraries). For certain things, e.g. discovering which libraries
changed, it is necessary to examine the object store. We use the first
isolate in a group (but could use any of them) to do so, since the
isolate group does not have this information atm.
This is a preparation CL for supporting hot-reloading multiple isolates
within one isolate group. Though the support in this CL stays at having
only a single isolate in a group.
=> This CL turns off FLAG_enable_isolate_groups in JIT mode.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I7f4d536d4f5ab4a2a73fb0c7618ba967c9b77234
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123254
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
This CL moves heap related information (namely instance sizes and
allocation stats) out of the [ClassTable] into a [SharedClassTable].
Both classes are always in sync (i.e. they have the same number of entries).
This CL also changes GC related code to start using the size information
from the new [SharedClassTable].
In a futher step we will move the heap as well as this shared class
table out of the [Isolate] and into [IsolateGroup].
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Id54a89c9251ad3bbc13e60d32dc4f7bcc7f1d805
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116064
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Until now it was possible to register classes with a default size (16 bytes on 64-bit)
and later on change the size for the cid.
This CL changes this to ensure the size information in the class table
for a given cid is either 0 or the final instance size (and adds an
ASSERT for it)
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I94c61c6a1566c13dec7b9eb80c9ae0dadf0e6b6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/115861
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Ryan Macnak