The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Moving these constants from global namespace to Page class prevents
possible conflicts with other kPageSize/kPageMask/kBlockSize constants
and allows us to export value of Page::kPageMask constant into runtime
offsets via runtime/vm/compiler/runtime_offsets_list.h.
TEST=ci
Change-Id: I4357f4644f0f5f7beba50184231fef55dd86b124
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/476147
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Arbitrary ObjectPtr can be a non-heap object (Smi), so methods
ObjectPtr::Is* which test class id should account for Smi.
Changed ObjectPtr::GetClassId to account for Smi similarly to
Object::GetClassId. Added unsafe ObjectPtr::GetClassIdOfHeapObject
which can be used when caller knows the heap nature of the object.
This change also fixes Integer::GetInt64Value which was relying on ObjectPtr::IsSmi.
TEST=vm/cc/Smi, vm/cc/Mint
Change-Id: I1391600e2acedc7b2a8f35c814df113ec9ba8698
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/380280
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Since null safety, all static fields with initializers are implicitly
late. This change cleans up transition_sentinel which was used in
the detection of cyclic initialization of legacy static fields.
TEST=ci
Change-Id: I6a990dc8ba030f5bd40eb0b86706cbfb0f725e33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/373520
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change removes support for external strings from the VM along with
Dart_NewExternalLatin1String, Dart_NewExternalUTF16String and
Dart_IsExternalString Dart C API functions.
External strings are not used by the VM nor any known embedder, but
Dart VM was paying the maintenance and performance price for
the external string implementation classes.
TEST=ci
Change-Id: I094cd2d2b7ec0840e9f09e1ca9e5a7acd4e78c28
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358760
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
We are no longer using Dart VM in a setting where this
matters as a security measure and it just complicates
portability for no benefit.
There is an indication that it is causing problems when
running Linux build of Dart VM under Docker on Mac OS X.
Fixes https://github.com/dart-lang/sdk/issues/54446
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try
Change-Id: I11bdaa8faebaca1df6fd59097049bdaea9cb8e12
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/344581
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This API can be used by app developers that want to diagnose high
memory consumption by dumping state of Dart VM heap at specific
points in their apps code and later analyzing those dumps offline
(e.g. using the tool in `<sdk>/runtime/tools/heapsnapshot`).
TEST=Existing test suite.
Change-Id: I04de6ba564f7b9eaf8b59d41fc4b3ec35ce8d2e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279972
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Michael Thomsen <mit@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Jacob Richman <jacobr@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Allows embedders to programmatically request heap snapshot from a
running isolate group instead of relying on a vm-service or hidden
internal Dart APIs
Additionally we allow to include snapshot writing functionality into
PRODUCT builds using DART_ENABLE_HEAP_SNAPSHOT_WRITER define.
TEST=vm/cc/DartAPI_WriteHeapSnapshot
Bug: b/259115846
Change-Id: Ic3ef76e5fb9adcf8f23a1959f5238742b64ecde2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273181
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
_InternalLinkedHashMap => _Map
_InternalImmutableLinkedHashMap => _ConstMap
_InternalLinkedHashSet => _Set
_InternalImmutableLinkedHashSet => _ConstSet
This makes things nicer to read in places that display implementation names, such as stack traces, debuggers, profilers and inspectors.
TEST=ci
Change-Id: Iec851c80ea2086cbe79934565dbf35f04809a836
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266303
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
In the heap snapshot writer, Pass1Visitor assigns object ids both to
heap objects and to Smis, and Pass2Visitor uses that information when
writing the heap snapshot. However, their logic for visiting slots in an
object with unboxed fields varies slightly. Pass1Visitor does not visit
each individual slot, but Pass2Visitor does. This means that SmiPtrs
that are only reachable via an object with unboxed fields are not added
to the writer in Pass1Visitor, and thus have an object id of 0 during
Pass2Visitor. This CL fixes this difference in visiting behavior.
TEST=vm/dart_2/heap_snapshot_regress_49711_test
vm/dart_2/heap_snapshot_test
Fixes: https://github.com/dart-lang/sdk/issues/50341
Change-Id: Id238771af18ee098f39e6d1cc80e021dc8e117fe
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/267580
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
- Prep work for non-moving promotion.
- The generations still have different alignment offsets.
- Old-space pages now participate in the page cache.
- Fix boolean/null bit tricks to assert the right requirements on the alignment of a page's first object.
TEST=ci
Change-Id: I4369d8c6af73228e162c226d411914868bafed33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260401
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
In AOT we can have unboxed fields that will not be visited by GC
visitors and as such are also not reported as <obj>.references when
producing heap snapshots.
Though the heapsnapshot's <class>.fields will contain entries
for unboxed fields, which causes the fields index calculation to be
incorrect.
This can cause various confusing things, e.g. incorrectly caluclated
retaining paths, since the <class>.fields.index doesn't match the
index in <obj>.references.
To fix this this CL will change the visiting code to emit dummy
references for unboxed fields.
The written test uncovered a few other issues:
- report type_arguments and native_fields in <class>.fields
=> the visitors already visit those fields
- fix `Class::next_{host,target}_field_offset` for classes
with native fields
=> this will also shrink size of `NativeWrapperClass` subclasses
- ensure classes with vm-defined layout are finalized
=> we assume objects should only exist in heap if their class
is finalized
- fix `FfiNativeFunction` type_argument offset
- fixes for object fields list
=> add missing entries, fix existing entries
Fixes https://github.com/dart-lang/sdk/issues/49711
TEST=vm/dart{,_2}/heap_snapshot_regress_49711_test
Change-Id: I8e25680b2c0c8c49caafbb8da57b0b6419e89c4f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255814
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Heap snapshots currently produced don't visit isolate stacks. As such
analyzing such snapshots may lead one to conclude there is a lot of
garbage while objects are actually reachable.
=> This CL makes us visit isolate stacks when building heap snapshots.
Furthermore we add a new `VMInternals.writeHeapSnapshotToFile` helper
that can be used to programmatically write snapshots and can be handy
for internal use at times. (We also use this helper in a test)
TEST=vm/dart{,_2}/heap_snapshot_test
Change-Id: I976544b7f6d20863764af9a40bf1ffb3c319bbce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253785
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This prevents objects, such as InstructionSection, that have no ordinary references from being labeled as "Uncollected Garbage".
TEST=examine heap snapshot in an AOT build with bare-instructions enabled
Bug: b/200288075
Change-Id: If576d7f9eabb80afaca1b600507c3a9629beeafd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215006
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This CL introduces immutable maps and sets in the VM backend but does
not yet target them from the frontend. The changes are tested by unit
tests constructing these immutable maps and sets.
This CL introduces immutable variants of the hash map and set in
compact_hash.dart and recognizes them in the VM. The immutable ones
use a different mixin with a different recognized method for accessing
members.
* Data list is an immutable list with a different cid. (Otherwise the
optimizer notices that immutable and mutable lists cannot be equal.)
* Index is a nullable mutable typed data. (Otherwise optimizer removes
necessary null checks.)
* Index should use a store-release barrier when written to.
Multiple isolates might lazily compute the index for const sets and
maps. This is fine because all identityHashCodes and hashCodes are
guaranteed to be race-free. The later isolates will override the index
pointer with an identical index.
This CL does not introduce support for using these immutable maps and
sets in AOT (clustered_snapshot) and in messages to other isolates
(message_snapshot) because that is harder to test with unit tests. That
will be added in the follow-up CL.
Design doc: go/dart-vm-const-maps
TEST=runtime/vm/object_test.cc
Bug: https://github.com/dart-lang/sdk/issues/45908
Change-Id: I4042179c15e8b37692d3255655351c01c7124991
Cq-Include-Trybots: luci.dart.try:analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210860
Reviewed-by: Martin Kustermann <kustermann@google.com>
Fix labels for sentinel, transition_sentinel, ImmutableArray, LinkedHashMap and LinkedHashSet.
Display unlimited children when look at successors, as the worst case is the size of the field table. For predecessors and dominators, the worst case is the whole heap.
TEST=ci
Change-Id: I6fcfdfb0833d58c9ac4f586b823244d817aeba27
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/209841
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
* IsInternalOnlyId -> IsInternalOnlyClassId
* Removes constexpr predicate ClassComesBeforeOtherInternalOnlyClasses()
used in only two locations and inlines the returned condition.
* Change IsErrorClassId to check <= kUnwindErrorCid instead of
<= kLastInternalOnlyCid.
TEST=Refactoring, so existing tests.
Change-Id: Ib7dfc89aba16d52733de05687f0ca7055c16e7a3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210126
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
* Adds IsInternalOnlyCid, which replaces the old `cid < kInstanceCid`
check.
* Adds kFirstInternalOnlyCid and kLastInternalOnlyCid, which should be
used in places that loop over the internal-only cids.
* Adds ObjectComesBeforeOtherInternalOnlyClasses(), which can be used
with COMPILE_ASSERT in places that assume kObjectCid immediately
precedes the other internal-only cids.
* Replaces assumption in IsErrorId() that Instance immediately follows
the internal-only cids with an assumption that the current last error
cid is the same as kLastInternalOnlyCid.
* Separates out CLASS_LIST_NO_OBJECT_NOR_STRING_... into two pieces to
make it clearer where to add new internal-only cids and where to add
new Instance cids:
* CLASS_LIST_INTERNAL_ONLY, which contains class ids for real
classes that are VM internal, except for Object.
* CLASS_LIST_INSTANCE_SINGLETONS, which contains class ids for
subclasses of Instance that are not part of a separately handled
group of cids.
TEST=Refactoring, so existing tests.
Change-Id: I9905d5928a592164d58de1f121b9ce669f2b3ce1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210122
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
`_CompactLinkedHashSet` now extends `_HashVMBase`. The class hierarchy
is organized as mixins similar to LinkedHashMap to accomodate for the
other Sets not extending `_HashVMBase`.
Also, rearranges some code so that introducing ImmutableHashMap and
ImmutableHashSet is easier.
1) snapshot.h and snapshot.cc now have a MapReadFrom, MapWriteTo,
SetReadFrom, and SetWriteTo to facilitate code sharing between
mutable and immutable implementations similar to ArrayReadFrom and
ArrayWriteTo.
2) Macros for CLASS_LIST_MAPS and CLASS_LIST_SETS to facilitate
treating mutable and immutable implementations with the same handle.
Also similar to Array.
Clustered snapshots for HashMaps is currently dead code. This CL makes
it explicit by marking these as unreachable. Immutable maps and sets
will end up in the clustered snapshot in follow up CLs.
Bug: https://github.com/dart-lang/sdk/issues/36077
Bug: https://github.com/dart-lang/sdk/issues/45908
TEST=runtime/vm/object_test.cc
TEST=tests/**_test.dart on many bots
Change-Id: If3cc5ebb3138535aeb0d5e06d9da3d1c9fb2deb2
Cq-Include-Trybots: luci.dart.try:analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206222
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Adjusts the counting phase to use the same visitor as the writing phase.
TEST=deliberately omitting checks from reader for backward compatibility
Change-Id: I8a08c1e9549fd4137714097a868541b09f160157
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205245
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Usually the identity hashcode is implemented by returning if present and
otherwise generating a random number, setting the hash code and
returning it.
This works fine in single-mutator environments, though when multiple
mutators might race to install identity hash codes we should ensure only
one of them wins and all others will obtain whatever hash code has been
set by the winner.
This will matter once we start sharing more objects across isolates.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=vm/cc/AsmIntrinsifier_SetHashIfNotSetYet
Change-Id: Ie760ca9658e6ec0640255361544d6822b07574e2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201827
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
We might interrupt Object._objectHashCode(obj) helper to serve a
vm-service request between checking for unset hash code and setting
a new hash code. While serving this request we might set the hash
code of [obj] (e.g. when generating heap snapshot or in
Instance::PrintJSONImpl). Later Object._objectHashCode will call
Object_setHash, which (on 64-bit platforms) assumes that
hash field in the object header is set to 0 and uses bitwise-or
to initialize it. This leads to a mismatch between hash code that
the first invocation of _objectHashCode will return and the
value stored in the header (because we OR non-zero value set
by vm-service with a value which _objectHashCode intended to
use as a hash).
This CL changes Object_setHash to avoid overwriting or mangling
the hash value if it was already set.
We also fix hash code generation in vm-service to ensure that we
only generate values which are valid Smis because the rest of the
code expects that.
TEST=pkg/front_end/test/incremental_compiler_leak_test.dart
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm64-try
Change-Id: Ica913af8bc1cfef0ad60a9e7504531ee4de53015
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197400
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
While deserializing AOT snapshot, Code objects which do not contain
valuable information besides entry point and stack maps are discarded
and not allocated on the heap (they are replaced with
StubCode::UnknownDartCode()).
PC -> Code/CompressedStackMaps lookup is implemented using a separate
table (InstructionsTable).
Flutter gallery in release-sizeopt mode:
Heap size of snapshot objects: arm -26.89%, arm64 -27.68%
Large Flutter application in release mode with --dwarf-stack-traces:
Heap size of snapshot objects: -24.3%.
Discarded Code objects: 72.5% of all Code objects.
Issue: https://github.com/dart-lang/sdk/issues/44852.
TEST=existing tests; "--dwarf_stack_traces --no-retain_function_objects
--no-retain_code_objects" mode is enabled for a few tests.
Change-Id: I5fe3e283630c8e8f4442319d5dcae38d174dd0d8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189560
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Ryan Macnak