When creating a KernelProgramInfo, we create several logical views into the kernel buffer. These are fresh ExternalTypedDatas, rather than proper TypedDataViews, so they do not automically keep the original ExternalTypedData alive. Create an explicit reference to the orginal ExternalTypedData in the KernelProgramInfo. When creating snapshots, this reference is ignored/null'd and the views are turned into copies, effectively dropping the parts of the original buffer that do not have views.
Fixes a leak with reload and a use-after-free with eval.
Bug: https://github.com/dart-lang/sdk/issues/33973
Bug: https://github.com/dart-lang/sdk/issues/39610
Change-Id: I09d3830133314ccbaa0341d904127c2b6925c4ec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126825
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is done by collecting all frames on the current stack until an async/async* frame is hit
which has yielded before (i.e. is not in sync-async case).
From there on it finds the closure of the async/async* frame and starts traversing the listeners:
while (closure != null) {
yield_index = closure.context[Context::kAsyncJumpVarIndex]
pc = closure.function.code.pc_descriptors.LookupPcFromYieldIndex(yield_index);
<emit pc in frame>
closure = closure.context[Context::kAsyncCompleterVarIndex]._future._resultOrListeners.callback;
}
Bug: https://github.com/dart-lang/sdk/issues/37668
Change-Id: I97030d22e529bf8a74ecd30a8a2589cfe57ad330
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122644
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Right now `Script::yield_positions()` is an array mapping function start
token positions to arrays. Those arrays contain token positions and are
indexed by yield index.
=> This `Script::yield_positions()` is not available in AOT mode.
The fast async stack implementation will need to be able to find out
where an async closure was suspended. It does so by looking at the
":await_jump_var", which contains the yield index. It then needs to
associate the yield index to token position of the yield.
This CL adds an entry into the PcDescriptors for every yield in a
async/async* function and removes `Script::yield_positions()`.
The entry will associate the yield index with the token position.
Flutter gallery total size impact for flutter-release:
- armv7: +0.016%
- armv8: +0.045%
Issue https://github.com/dart-lang/sdk/issues/37668
Change-Id: I0b2ce41e85d8f5d590201bf2fb091578d7379890
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125408
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This paves the way for removing the old constant evaluator from the VM,
by no longer supporting versions without front-end constant evaluation.
For the front-end test that tests old kernel versions on the newest VM,
the dill files for versions before 29 are deleted.
Version 29 is from 2019-08-16: eae781c73e
Change-Id: Icb472bffc764d6e37ed1349a0cb4ff281e0e1b21
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123739
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
VM creates Script objects with kKernelTag only, so this CL cleans up
all uses of script tags and Script::kind() along with code
which is no longer reachable.
Change-Id: Ia765e7757264aa614e18eddd16d21937f855c129
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123889
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Consume the extension flag setting for functions and the extension flag and is late
flags for fields when reading kernel file and set bit in Function/Field objects.
Change-Id: I661d30d53e817d968151d2b95474a6daf918de13
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118000
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Introduced a new code path to load declaration of a referenced class when
loading bytecode, if referenced class is from the same binary. In such case
it's possible to reuse current BytecodeReaderHelper.
Also, AlternativeReadingScope is simplified to save/restore only reader
offset (which is needed in most cases), while saving/restoring data is
still available in AlternativeReadingScopeWithNewData.
This change fixes stack overflow while reading deep class hierarchies in the
tests co19_2/Language/Types/Interface_Types/subtype_t27 and
co19_2/Language/Types/Interface_Types/subtype_t28 with bytecode in
vm-kernel-linux-debug-ia32 configuration.
Change-Id: Iffb1678286c6926926fab465f70251992d451241
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117204
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
* Set parent for TypeParameters created in createExpressionEvaluationComponent.
* Resolve references from bytecode to an expression evaluation class
using real class. Expression evaluation class could be referenced
from type parameters.
* Avoid registering expression evaluation classes in hierarchy. They
shouldn't be discovered through direct subclasses or implementors,
as expression evaluation classes don't have cids.
* Add :function_type_arguments_var local variable to bytecode local
variables info, so expression evaluation can find function type
arguments which are in scope.
Change-Id: I515431a989b4d02af4ecf90579931dd59d3acadd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114182
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This is the beginning of allowing us to - wherever we process kernel
data - make decisions based on the kernel version. Said another way,
we can actually start to change the kernel format without breaking
backwards compatibility in the VM.
Change-Id: Ied7b3d9fe72976c4f3c0895d02b6ab70e0f5dba9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/111642
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Rationale:
Previously all constants were read from the constants table,
whether they were needed (at that point or at all) or not. This
CL replaces that eager reading with a lazy reading that makes
the raw bytes of the constants table available through
KernelProgramInfo and attaches a hashmap to KernelProgramInfo,
which maps offsets into the constant table to evaluated constants.
The maps starts empty and on a miss, the constant is evaluated,
but no sooner than when it it needed.
https://github.com/dart-lang/sdk/issues/36220
Change-Id: Ief4df3d70c950c4beb61d896632ce06cfd12d525
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99088
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Size of a large application:
Before: 26628600
After: 21480120 (-19.3%)
Size of snapshots:
isolate_snapshot_framework.bin
Before: 9322496
After: 6782976 (-27.2%)
isolate_snapshot_product_framework.bin
Before: 9166848
After: 6602752 (-27.9%)
Regressions in tests:
1) Test language_2/type_alias_equality_test/04 fails similarly to default mode, as VM
does not implement comparison of function types according to the specification.
Previously this test was passing as function types were canonicalized in bytecode,
which was not always correct. This CL fixes the problem with canonicalization of
function types in bytecode and the test starts failing again.
2) Tests standalone_2/entrypoints_verification_test, standalone_2/io/test_extension_test,
standalone_2/io/test_extension_fail_test fail as native extensions are not supported
in bytecode yet. These tests start passing after df5e7aac17,
which switched bytecode tests to kernel service (on x64), because kernel service doesn't
drop ASTs. This CL switches from reading AST library declarations to bytecode even if
AST is not removed, so tests fail again.
Change-Id: I8b7ba44bfa49d0b1599b2509553ff7c831a4e244
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104700
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change renames Class::is_cycle_free to Class::is_declaration_loaded
and under the hood replaces cycle_free and type_finalized bits with
class loading state enum.
Also, added new assertions to check is_declaration_loaded().
Change-Id: Ib43e12731d0dc782e273be8e55912494e102cd79
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104920
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This reverts commit ab6aeaa106.
Revert "[vm/compiler] Speed up the compiler part which deals with kernel reading up in DEBUG mode"
This reverts commit b316210d94.
Reason for revert: regression of snapshot sizes (DNO-599).
Original change's description:
> [vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers
>
> Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
> objects were referencing ExternalTypedData objects pointing into the middle of
> c-allocated memory. This made it impossible for the GC to determine when the last
> reference is gone.
>
> This CL ensures that the actual buffers are *always* made available via
> ExternalTypedData and any inner pointers into it are created via TypedDataViews.
>
> The embedder guarantees to the free kernel buffers it has provided to:
> - Dart_CreateIsolateFromKernel
> - Dart_LoadScriptFromKernel
> - Dart_LoadLibraryFromKernel
> - Dart_SetDartLibrarySourcesKernel
> on isolate shutdown.
>
> All other kernel buffers will get a finalizer attached, which ensures the
> kernel buffers get freed by the GC once they are no longer referenced:
> - Kernel blobs for expression evaluation
> - Kernel blobs for Hot-Reload
> - Kernel blobs for cc tests
>
> Fixes https://github.com/dart-lang/sdk/issues/33973
> Fixes https://github.com/dart-lang/sdk/issues/36857
> Issue https://github.com/dart-lang/sdk/issues/37030
>
> Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: I49715d2400f4a5c8806b7d6a2912b7258f671a0a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104343
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
In DEBUG mode during AOT compilation MetadataHelper::SetMetadataMappings
may spend considerable time verifying that node offsets are sorted
in kernel metadata. The problem is that there are a lot of MetadataHelpers
created (several per each compiled function).
This change moves verification of kernel metadata mappings to
the kernel loader, so it is performed only once per kernel program.
AOT gen_snapshot time on a small test:
Before: 11.680s
After: 9.041s
Change-Id: I1c0a676b0ed28fc5cfa756ca60159f914190fac0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103486
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Aart Bik <ajcbik@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
objects were referencing ExternalTypedData objects pointing into the middle of
c-allocated memory. This made it impossible for the GC to determine when the last
reference is gone.
This CL ensures that the actual buffers are *always* made available via
ExternalTypedData and any inner pointers into it are created via TypedDataViews.
The embedder guarantees to the free kernel buffers it has provided to:
- Dart_CreateIsolateFromKernel
- Dart_LoadScriptFromKernel
- Dart_LoadLibraryFromKernel
- Dart_SetDartLibrarySourcesKernel
on isolate shutdown.
All other kernel buffers will get a finalizer attached, which ensures the
kernel buffers get freed by the GC once they are no longer referenced:
- Kernel blobs for expression evaluation
- Kernel blobs for Hot-Reload
- Kernel blobs for cc tests
Fixes https://github.com/dart-lang/sdk/issues/33973
Fixes https://github.com/dart-lang/sdk/issues/36857
Issue https://github.com/dart-lang/sdk/issues/37030
Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This for instance allows the VM to include all patches
(e.g. the previously missing `runtime/lib/date_patch.dart`)
in what's returned from `LoadedScripts()`.
Over time this also allows for `LoadedScripts()` to be simplified and
become faster (it's currently something like O(m + m*n) where m is the
number of results in DictionaryIterator(*this) and n is the number of
returned scripts).
Change-Id: I4b5a6a0fe666b774fc0987d099ed02e81ac97b43
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98660
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
By indexing all sources in a concatenated dill file and then using that
index to find the sources and line starts we can fix missing information
(leading to crashes when collecting coverage) in for instance circular
instances.
This reverts commit 57321c1590 and adds a
fix.
Change-Id: I7a91801fb318cad0218cf3101d6a15f1ec929175
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98006
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Ryan Macnak