This reverts commit 922ea3e9b6 in patchset 1, fix for assertion triggered in https://ci.chromium.org/b/8883214567628884960 in patchset 2, fix for deadlock around symbols table mutex in patchset 4.
Original commit description:
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I655e337198214c9dfacbe76f7852b941b5a7e910
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143462
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Original revert in patchset 1, fix for deadlock issue in patchset 2: avoid reentrant calls to RunWithStoppedMutator.
Further review comments addressed in successive patchsets.
This reverts commit 8ef508ba36.
Original commit description:
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Change-Id: I6a0279d9812020ad7a5c2b7851980b6a29b95b9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143327
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I64c86525f4ef9cb30567a49a106bfe700355942b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136780
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
If lookup succeeds, then the DWARF stack frame uses the symbol
name and offset instead of the dynamic library name. The
native_stack_traces package (once updated) can translate these
stack frames into symbolic stack frames even if given a partial
stack trace without the header.
Also, consistently use uword instead of a mix of uintptr_t and uword.
Change-Id: Ida9ba4726a306fb77b141cb9c6ad635a12a320ec
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140282
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Whenever the VM sends a request to evaluate an expression, it sends over
the list of alive dills (in order) and a sequence number (number of
reloads).
This can then either be
a) ignored if there's already a compiler around that has compiled
everything itself (e.g. if launched from source with observatory
enabled).
b) used to find out that we need to create a new compiler especially
for expression compilation. Either because we don't have any old
compiler or because a reload or GC changed the data available.
c) used to find out that the compiler we created previously is still in
a good state.
Tests for expression compilation has been added, and with the "framework"
added it should be easy to add new ones.
This CL leaves a few things to possibly be improved later:
* It doesn't support if one tries to do a mixed mode, where one
starts from source and reloads from dill. The compiler will not be
updated and the expression compilation will work on a old view of the
world.
* If - in any of the reloads - the main entry point changes library,
the initialization might not work correctly.
Fixes#34736.
Change-Id: Icae009c581e69b810ee19d2a5e2d0a803fd9f7c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135683
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
In the early days of AOT, we performed compilation and then ran results in the same process and isolate. This bit was used to track which phase we were in. Today, these always happen in separate processes and separate binaries, distinguished by the macro DART_PRECOMPILED_RUNTIME.
Change-Id: Idd97908f2e692321a6f5aba8d5738625d42f16e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134568
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This CL:
* Moves [Heap]/[SharedClassTable] from [Isolate] to [IsolateGroup], which
will make all isolates in the group use the same heap. The GC will use
the shared class table for object size information.
* Adds support for entering/leaving an isolate group as a helper thread
(e.g. via [Thread::EnterIsolateGroupAsHelper]). The current active
isolate group can be accessed via TLS `IsolateGroup::Current()` or
`Thread::isolate_group_`. When entering as a helper thread there will be
no current isolate.
* Changes the GC to use the above mechanism and ensures GC works without
a currently active isolate. The GC will use information purely available via
[IsolateGroup]. The GC will iterate all isolates within an isolate
group e.g. for scanning roots.
* Makes spawning of new isolates start in their own isolate group.
Once the isolate is fully functional it's heap will be merged into
the original isolate group
* Moves ApiState, containing persistent and weak persistent handles,
from [Isolate] to [IsolateGroup], plus adds appropriate locking.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Ia8e1d8aa78750e8400864200f4825395a182c004
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126646
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Tests involving breakpoints, stepping, evaluation or coverage are expected to fail.
Tests involving introspection, heap analysis, extensions, timeline events or profiling are expected to pass.
- Fix crash when `debugger` is called.
- Fix crash when metrics are queried.
- Replace unnecessary use of `eval` with `invoke`, allowing more tests to run on AOT, AppJIT and simulators.
Bug: https://github.com/dart-lang/sdk/issues/40274
Bug: https://github.com/dart-lang/sdk/issues/40275
Change-Id: I3023af7cbfda745238c487d9a3a3a99e56a30244
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132942
Reviewed-by: Ben Konyi <bkonyi@google.com>
When creating a KernelProgramInfo, we create several logical views into the kernel buffer. These are fresh ExternalTypedDatas, rather than proper TypedDataViews, so they do not automically keep the original ExternalTypedData alive. Create an explicit reference to the orginal ExternalTypedData in the KernelProgramInfo. When creating snapshots, this reference is ignored/null'd and the views are turned into copies, effectively dropping the parts of the original buffer that do not have views.
Fixes a leak with reload and a use-after-free with eval.
Bug: https://github.com/dart-lang/sdk/issues/33973
Bug: https://github.com/dart-lang/sdk/issues/39610
Change-Id: I09d3830133314ccbaa0341d904127c2b6925c4ec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126825
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Similar to the split of ClassTable into ClassTable/SharedClassTable,
this CL splits up the IsolateReloadContext into:
* IsolateGroupReloadContext: Consists of reload-related information across all
isolates. The [Reload()] method is split up in phases that are
performed on all isolates before the next phase is started.
=> This allows each isolate to add reasons for rolling back, if no
reasons are found the reload will be accepted atomically.
* IsolateReloadContext: Constists of reload-related information for a
particular isolate (e.g. mappings of old to new classes)
The assumption is that all isolates have the same source (and therefore
the same libraries). For certain things, e.g. discovering which libraries
changed, it is necessary to examine the object store. We use the first
isolate in a group (but could use any of them) to do so, since the
isolate group does not have this information atm.
This is a preparation CL for supporting hot-reloading multiple isolates
within one isolate group. Though the support in this CL stays at having
only a single isolate in a group.
=> This CL turns off FLAG_enable_isolate_groups in JIT mode.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I7f4d536d4f5ab4a2a73fb0c7618ba967c9b77234
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123254
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Disabling the profiler will **not** release the sample buffer on Linux,
MacOS, and other platforms which utilize signals for profiling as
there's no way to ensure all signals have been receieved after the
thread interrupter has been shutdown.
Change-Id: Ifb3230fdb95e9bc9147cbd4b85d1a84e976ce119
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123554
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Print more information after detecting a fatal context mismatch in the debugger.
When generating the line start of a script, first resolve scripts marked for
'lazy lookup' before looking up dart:* scripts.
Delete redundant ifdef/endif pair.
Change-Id: Icd7bd44bf8367fbc306b298246783e22e526be38
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117380
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
This function has been unused for a long time.
Also, computing fingerprints for bytecode function causes bytecode to be loaded, which can lead to an old enum class being initialized with enum values for the new enum class, causing a forwarding loop when migrating enum values.
Change-Id: Iad9b2bb11aaf161d66f7752cb99cc4b4fc9fb310
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116442
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This CL moves heap related information (namely instance sizes and
allocation stats) out of the [ClassTable] into a [SharedClassTable].
Both classes are always in sync (i.e. they have the same number of entries).
This CL also changes GC related code to start using the size information
from the new [SharedClassTable].
In a futher step we will move the heap as well as this shared class
table out of the [Isolate] and into [IsolateGroup].
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Id54a89c9251ad3bbc13e60d32dc4f7bcc7f1d805
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116064
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This replaces _requestHeapSnapshot and _getObjectByAddresses, which relied on the invalid assumption that objects don't move and the heap hasn't mutated since the initial snapshot.
Updates Observatory to the new format. Adds treemap visualization of the dominator tree. Adds the ability to save and load heap snapshots to files. Removes analysis grouping references by source and target class. Removes ability to navigate from a pixel in the fragmentation view to inspector. Removes ability to query the top retaining instance from an ordinary class browser.
Change-Id: Ia7781c05d43bf3ec149f8b4ecab803b37c3ee981
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112181
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Previously, KernelIsolate::CompileExpressionToKernel was called
with is_static == false and klass == nullptr, which doesn't make sense:
this call requested to compile expression inside instance method context but
without enclosing class. This results in a malformed kernel AST and crash
in bytecode generator.
Fixes vm/cc/LinkedHashMap and vm/cc/Service_* tests in bytecode mode.
Change-Id: I121b0cb0e149d378d6b12e9861ef941513e1e199
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112024
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Allows for VM service clients to determine information about the system
the service is running on. Needed for some DevTools functionality.
Change-Id: I530984816e6cc1e4ca378454882219085e2d9de8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/111361
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit ab6aeaa106.
Revert "[vm/compiler] Speed up the compiler part which deals with kernel reading up in DEBUG mode"
This reverts commit b316210d94.
Reason for revert: regression of snapshot sizes (DNO-599).
Original change's description:
> [vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers
>
> Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
> objects were referencing ExternalTypedData objects pointing into the middle of
> c-allocated memory. This made it impossible for the GC to determine when the last
> reference is gone.
>
> This CL ensures that the actual buffers are *always* made available via
> ExternalTypedData and any inner pointers into it are created via TypedDataViews.
>
> The embedder guarantees to the free kernel buffers it has provided to:
> - Dart_CreateIsolateFromKernel
> - Dart_LoadScriptFromKernel
> - Dart_LoadLibraryFromKernel
> - Dart_SetDartLibrarySourcesKernel
> on isolate shutdown.
>
> All other kernel buffers will get a finalizer attached, which ensures the
> kernel buffers get freed by the GC once they are no longer referenced:
> - Kernel blobs for expression evaluation
> - Kernel blobs for Hot-Reload
> - Kernel blobs for cc tests
>
> Fixes https://github.com/dart-lang/sdk/issues/33973
> Fixes https://github.com/dart-lang/sdk/issues/36857
> Issue https://github.com/dart-lang/sdk/issues/37030
>
> Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: I49715d2400f4a5c8806b7d6a2912b7258f671a0a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104343
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
objects were referencing ExternalTypedData objects pointing into the middle of
c-allocated memory. This made it impossible for the GC to determine when the last
reference is gone.
This CL ensures that the actual buffers are *always* made available via
ExternalTypedData and any inner pointers into it are created via TypedDataViews.
The embedder guarantees to the free kernel buffers it has provided to:
- Dart_CreateIsolateFromKernel
- Dart_LoadScriptFromKernel
- Dart_LoadLibraryFromKernel
- Dart_SetDartLibrarySourcesKernel
on isolate shutdown.
All other kernel buffers will get a finalizer attached, which ensures the
kernel buffers get freed by the GC once they are no longer referenced:
- Kernel blobs for expression evaluation
- Kernel blobs for Hot-Reload
- Kernel blobs for cc tests
Fixes https://github.com/dart-lang/sdk/issues/33973
Fixes https://github.com/dart-lang/sdk/issues/36857
Issue https://github.com/dart-lang/sdk/issues/37030
Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Ryan Macnak