Original revert in patchset 1, fix for deadlock issue in patchset 2: avoid reentrant calls to RunWithStoppedMutator.
Further review comments addressed in successive patchsets.
This reverts commit 8ef508ba36.
Original commit description:
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Change-Id: I6a0279d9812020ad7a5c2b7851980b6a29b95b9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143327
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I64c86525f4ef9cb30567a49a106bfe700355942b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136780
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Instead of each Visitor being a distinct type, now FunctionVisitors
are subclasses of ClassVisitors (since we visit each class to visit
its methods) and CodeVisitors a subclass of FunctionVisitors (since
we visit each function to visit its code). Thus, we replace the
different ProgramVisitor::Visit<type> methods with a single
ProgramVisitor::WalkProgram method that walks the program, looking
for applicable objects.
Change-Id: If93638ec2eccebb8b726fad9b8ea46f94f8c099b
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/141604
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Nullability of left hand side in type test was checked too early, i.e. before
handling FutureOr on the right hand side.
The following tests began passing (in strong mode):
standalone/io/addlatexhash_test was fixed (RuntimeError -> Pass)
standalone/io/file_output_stream_test was fixed (RuntimeError -> Pass)
standalone/io/file_stream_test was fixed (RuntimeError -> Pass)
standalone/io/file_system_async_links_test was fixed (RuntimeError -> Pass)
standalone/io/file_system_links_test was fixed (RuntimeError -> Pass)
standalone/io/file_write_only_test was fixed (RuntimeError -> Pass)
standalone/io/pipe_server_test/0 was fixed (RuntimeError -> Pass)
standalone/io/pipe_server_test/1 was fixed (RuntimeError -> Pass)
standalone/io/pipe_server_test/2 was fixed (RuntimeError -> Pass)
standalone/io/pipe_server_test/3 was fixed (RuntimeError -> Pass)
standalone/io/pipe_server_test/4 was fixed (RuntimeError -> Pass)
standalone/io/pipe_server_test/5 was fixed (RuntimeError -> Pass)
standalone/io/stream_pipe_test/0 was fixed (RuntimeError -> Pass)
standalone/io/stream_pipe_test/1 was fixed (RuntimeError -> Pass)
standalone/io/stream_pipe_test/2 was fixed (RuntimeError -> Pass)
standalone/io/stream_pipe_test/3 was fixed (RuntimeError -> Pass)
Change-Id: I855328f101382cf5f1af9119b396aa9d0cb14e40
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140776
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Static methods and constructors may receive unboxed parameters.
This change adds a bitmap to the RawFunction object indicating
which parameters are unboxed and their types. Also, the stack frame
bitmap was updated to include new bits for those parameters.
The ArgumentsDescriptor object was updated to store not only the number
of arguments, but also their size, since 32-bit platforms would use two
words for each int64 and double value.
In addition, static methods that return non-nullable integers
and doubles may return these values unboxed.
Dart AOT x64:
TypedData.Uint8ClampedListViewVarBench +11.73%
DeltaBlue +9.32%
Meteor (Intel Xeon) +4.64%
Flutter release android-armv8:
velocity_tracker_bench -2.32% (less is better)
flutter_gallery_readonlydata_size +0.18%
flutter_gallery_total_size +0.13%
flutter_gallery_instructions_size +0.12%
Issue https://github.com/dart-lang/sdk/issues/33549
Change-Id: I97e310f8f30213920af2333fbdedf193d03db5c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135313
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
On 1/27/20, the nnbd specification changed weak and strong mode instance checks
to make them behave uniformly across legacy and opted-in libraries.
Therefore, it is not necessary anymore to propagate the library mode in
generated code to the runtime.
Change-Id: I42d3ddc6e9a921899aeac21be6374c7893a6d27c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138111
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
When moving the newly spawned isolate from the spawnee group to the main
group we have to ensure to update the cached
`Isolate::mutator_thread_->isolate_group_` pointer (which we do not
reset when descheduling mutator threads (for other threads we do)).
This will fix the ASAN crash in this test:
tools/test.py -n dartk-asan-linux-release-x64 lib_2/isolate/mandel_isolate_test
Fixes https://github.com/dart-lang/sdk/issues/40756
Change-Id: I5a897098106e6748cf4e58bd5d2799ce28b130a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136971
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Originally I had the permutation map on IsolateGroup where the copy was
needed due to creating the temporary spawn group. Though it moved to
IsolateGroupSource now which is shared even with the temporary spawn
group.
This fixes flakiness in the app-jit tests:
tools/test.py -n app_jitk-linux-release-x64 standalone_2/typed_array_test/0
Fixes https://github.com/dart-lang/sdk/issues/40750
Change-Id: I0611fc98d03950b400ef70fa5127eba9d00742bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137304
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
In the early days of AOT, we performed compilation and then ran results in the same process and isolate. This bit was used to track which phase we were in. Today, these always happen in separate processes and separate binaries, distinguished by the macro DART_PRECOMPILED_RUNTIME.
Change-Id: Idd97908f2e692321a6f5aba8d5738625d42f16e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134568
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This CL:
* Moves [Heap]/[SharedClassTable] from [Isolate] to [IsolateGroup], which
will make all isolates in the group use the same heap. The GC will use
the shared class table for object size information.
* Adds support for entering/leaving an isolate group as a helper thread
(e.g. via [Thread::EnterIsolateGroupAsHelper]). The current active
isolate group can be accessed via TLS `IsolateGroup::Current()` or
`Thread::isolate_group_`. When entering as a helper thread there will be
no current isolate.
* Changes the GC to use the above mechanism and ensures GC works without
a currently active isolate. The GC will use information purely available via
[IsolateGroup]. The GC will iterate all isolates within an isolate
group e.g. for scanning roots.
* Makes spawning of new isolates start in their own isolate group.
Once the isolate is fully functional it's heap will be merged into
the original isolate group
* Moves ApiState, containing persistent and weak persistent handles,
from [Isolate] to [IsolateGroup], plus adds appropriate locking.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Ia8e1d8aa78750e8400864200f4825395a182c004
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126646
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Original change is in Patchset 1. I erroneously assumed DART_PRECOMPILER
was only used AOT compilation (it is also used for JIT snapshots).
Change-Id: I77f6c463db7f2b482c9b098f353dc28f0676ea0d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135900
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
Added the following methods to the VM embedding API:
* Dart_GetNonNullableType
* Dart_GetNullableType
* Dart_TypeToNonNullable
* Dart_TypeToNullable
* Dart_IsLegacyType
* Dart_IsNonNullableType
* Dart_IsNullableType
Change-Id: I7de1a99179c4d16a0e6a040bb209de18db379436
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135484
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
When these were originally added, we thought we would be able to use them in place of ifdefs and rely on the compiler optimizations and linker GC to remove things. This turned out not to reliably remove what we wanted removed, so we ended up with the ifdefs anyway.
Change-Id: I62e74d60d92b18a688b9dffaf77b1440c10a07ff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134402
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
This scope can sneak objects larger than kAllocatablePageSize directly into non-large heap pages.
This scope also holds a heap lock outside of the GC, which is likely to create issues when multiple isolates run on the same heap.
Change-Id: Ic56277c83ffccfe2e1396ed773dd30b8d172bbba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134040
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Instead of making the ELF object handle stripping and keep track of two
possible output stream, just revert ELF objects to only output to a
single stream. Now, layers that use ELF, like the ImageWriters, handle
the stripping by deciding whether or not to add certain sections or
static symbols to the generated ELF, and we create separate DWARF
objects for each unstripped ELF object.
We also avoid duplication of writing segment and section table entries
by just creating special Sections that correspond to reserved entries,
the program table header entries for itself, etc, and then move the
section and segment entry writing into Section itself. We also lift
a lot of the same calculations used in subclass construction to the
Section level, and replace the old fields with four different kind
of object members:
* Fields that are known at construction time and thus now const
(e.g., section_type)
* Fields that are not known at construction time that have reasonable
defaults for most instances and so need not be changed from their
default value (e.g., section_link)
* Fields that are not known at construction time which must be set once
(and only once) before use. These are now accessed via getters/setters
that check whether or not the field has been set (e.g., section_name)
* Fields that are calculated from the contents of the particular
subclass, which are now just methods (e.g., FileSize())
We also change the snapshot profile writer test to test the size of
internally stripped ELF output on all platforms without running it
through an external stripping utility (previously untested), and test
assembled or externally stripped output on more platforms where we
support those. Instead of depending on pkg/vm/tool/precompiler2 for
assembling (which always uses gcc), I've added assembleSnapshot and
stripSnapshot to use_flag_test_helper.dart, which mimics the way the
test_runner determines the executable and flags to use.
Change-Id: I4ade45c5caffb443abb3f5c567343504c8085f89
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/129083
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
NNBDMode does not reflect the semantics (legacy or nnbd) of type tests anymore.
Instead, the semantics are derived from the value of the strong mode flag.
This required one function to be specialized, namely the implementation of
'null is Type' in weak mode in an opted-in library, which still requires
nnbd semantics although run in weak mode.
Relevant changes are in object.h and object.cc:
- methods NNBD_NullIsInstanceOf and NNBD_IsTopType are new.
- methods IsNullType, IsTopType, and IsNeverType are modified.
Change-Id: I36cd43d93d2cfabd110cbcc6b26487a583bb089d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130444
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This does not yet include checking of required named parameters.
Also, inlined type testing code, type testing stubs, and type propagation are
not yet updated.
Change-Id: If6d8996635474e349d28b7fd207d724838d08881
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/129922
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- Removes the load port functionality from the service isolate, this was
used to load sources in Dart 1 and is pretty much dead code in Dart 2
- Moves resolution of URIs and Package URI (based on package map) to the
individual isolates instead of sending a request to the service isolate
- Setups the package map lazily as it is needed only if resolveURI is
called in Dart code, it is not needed for loading/compiling sources
(this is now done in the front end).
- Removed Dart_ServiceWaitForLoadPort() methods from the Dart C API as
it is not used anywhere.
Change-Id: I6c3704bc431bdcd49bf074eb58bee1ed492ccccb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/129742
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Previously when --save-debugging-info=<...> was used when not
compiling to ELF, we generated the requested snapshot format
and the separate debugging information as two separate snapshot
generation passes. However, we want to avoid this, to remove the
assumption that instruction snapshot serialization must be
deterministic.
Fixes https://github.com/dart-lang/sdk/issues/39803.
Also allows --strip to be used with --snapshot-kind=app-aot-assembly,
so that DWARF sections are not output in the generated assembly code.
Fixes https://github.com/dart-lang/sdk/issues/37560.
Change-Id: Ibdf1f5c771bae21870af252b38e64250fde53603
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128735
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Removes an unnecessary change to the names of type testing stubs that
allowed name collisions to happen.
Old commit message:
The flag can be used when creating AOT snapshots. The resulting file can be
used with package:vm/dwarf/convert.dart to convert DWARF-based stack traces
to stack traces with function, file, and line number information.
Currently the saved file will be an ELF file with DWARF debugging information,
but this is subject to change in the future. To avoid being affected by any
changes in format, read the DWARF information from the file using
Dwarf.fromFile() in package:vm/dwarf/dwarf.dart.
Also adds --dwarf-stack-traces to the VM global flag list, so its value at
compilation will be read out of snapshots by the precompiled runtime.
Fixes https://github.com/dart-lang/sdk/issues/39512, which was due to
a missing compiler::target:: prefix for Instructions::HeaderSize() in
BlobImageWriter::WriteText().
Exposes the information suggested in
https://github.com/dart-lang/sdk/issues/39490 so that package:vm/dwarf
can appropriately convert absolute PC addresses to the correct virtual
address for a given DWARF line number program.
Bug: https://github.com/dart-lang/sdk/issues/35851
Change-Id: I6723449dceb0b89c054a1f1e70e7acd7749baf14
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128730
Reviewed-by: Martin Kustermann <kustermann@google.com>
REASON: the first commit in the chain contains a change that breaks internal
Flutter roll due to duplicated symbols for type testing stubs in assembly
snapshots.
Revert "[vm/compiler] Add --save-debugging-info flag to gen_snapshot."
This reverts commit c2b6c2e1db.
Revert "[vm] De-obfuscate function and file names in DWARF sections."
This reverts commit 4b8fd3c412.
Revert "[vm] Add docs for DWARF stack traces and related tools/libraries."
This reverts commit 5543e5ceb8.
Revert "[vm] Weaken dwarf_stack_trace expected stack trace."
This reverts commit 5824d3ed53.
Change-Id: I95ecd16fe493f61ae761dabfdd91a6ec0c2f0ca3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128776
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
The flag can be used when creating AOT snapshots. The resulting file can be
used with package:vm/dwarf/convert.dart to convert DWARF-based stack traces
to stack traces with function, file, and line number information.
Currently the saved file will be an ELF file with DWARF debugging information,
but this is subject to change in the future. To avoid being affected by any
changes in format, read the DWARF information from the file using
Dwarf.fromFile() in package:vm/dwarf/dwarf.dart.
Also adds --dwarf-stack-traces to the VM global flag list, so its value at
compilation will be read out of snapshots by the precompiled runtime.
Fixes https://github.com/dart-lang/sdk/issues/39512, which was due to
a missing compiler::target:: prefix for Instructions::HeaderSize() in
BlobImageWriter::WriteText().
Exposes the information suggested in
https://github.com/dart-lang/sdk/issues/39490 so that package:vm/dwarf
can appropriately convert absolute PC addresses to the correct virtual
address for a given DWARF line number program.
Bug: https://github.com/dart-lang/sdk/issues/35851
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-mac-release-simarm_x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: I80d900fd9e5f1e2399ad3f2fd25c85131a7ea43c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121857
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Runtime functions do not yet implement the NNBD semantics.
In a next change, generated code will pass NNBD mode to the runtime.
Declare one new VM flags: --strong-non-nullable-type-checks
Change-Id: I050a468e4a6b665ce46c345bafba04a947cf7cb0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124105
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Similar to the split of ClassTable into ClassTable/SharedClassTable,
this CL splits up the IsolateReloadContext into:
* IsolateGroupReloadContext: Consists of reload-related information across all
isolates. The [Reload()] method is split up in phases that are
performed on all isolates before the next phase is started.
=> This allows each isolate to add reasons for rolling back, if no
reasons are found the reload will be accepted atomically.
* IsolateReloadContext: Constists of reload-related information for a
particular isolate (e.g. mappings of old to new classes)
The assumption is that all isolates have the same source (and therefore
the same libraries). For certain things, e.g. discovering which libraries
changed, it is necessary to examine the object store. We use the first
isolate in a group (but could use any of them) to do so, since the
isolate group does not have this information atm.
This is a preparation CL for supporting hot-reloading multiple isolates
within one isolate group. Though the support in this CL stays at having
only a single isolate in a group.
=> This CL turns off FLAG_enable_isolate_groups in JIT mode.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I7f4d536d4f5ab4a2a73fb0c7618ba967c9b77234
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123254
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Right now a message handler is taking care of detecting idle timeouts
and notifying the isolate (or rather it's heap) that a compaction can
start.
Once we move the heap to the isolate group, we should only notify the
heap that we're idle if *all* isolates are idle. As a preparatory step
we move the idle timer functionality into it's own class and make the
isolate own it.
When the heap gets moved from isolate to isolate group, we'll also move
the idle time handler.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: If9b244baf0425ef343f909ee3a8a1147f966845b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116200
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
The lower-case "value()" metrics getter can only be used to obtain
metrics which are explicitly set by the VM.
The metrics we expose also include ones which are computed on-the-fly.
Those have to be accessed via "Value()".
Change-Id: I283c3c7c660d47c383b880b4bff6357a4dba9c01
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123252
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Alexander Aprelev