(Assignment of libraries to loading units is already done in the kernel generation step.)
After compiling and before serializing, we walk the program and for each Code we assign its Instructions, CodeSourceMap and CompressedStackMap to the loading unit of that Code's defining library. Deduplication may cause Instructions, CodeSourceMaps and CompressedStackMaps to belong to more than one loading unit; in this case the objects are assigned to the root loading unit. Later they can be more precisely assigned to the dominating loading unit.
All objects except some Instructions, CodeSourceMaps and CompressedStackMaps belong to the root loading unit's snapshot. This snapshot is written like an unsplit snapshot, except that when serializing Code, we will write a reference to a stub or null when the Code's Instructions, CodeSourceMap or CompressedStackMap belongs to a non-root loading unit. The snapshots of non-root loading units contain these deferred objects and references to the corresponding Code objects to patch. The types of objects we defer (Instructions, CodeSourceMaps and CompressedStackMaps) usually represent 70+% of the snapshot size.
Bare instructions mode must be disabled when splitting because we cannot have PC-relative calls between loading units. Later we can re-enable this for calls within loading units.
Broken: Compactor probably crashes we can now have an unbounded number of image pages and the compactor assumes a fixed number.
Embedder's guide:
At compile-time, gen_snapshot should be passed --loading_unit_manifest with a path, which will enable splitting and output a mapping from loading unit ids to snapshot output paths.
At runtime, sometime during isolate startup, an embedder should call Dart_SetDeferredLoadHandler, probably near an existing call to Dart_SetLibraryTagHandler. The callback is given a loading unit id, and should eventually call Dart_DeferredLoadComplete[Error].
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ib597eb87c8cd634416d5ee1f00629c5550aebb00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152427
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This reverts commit bc4cb123a8.
Reason for revert: breaks crossword simarm tests https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-linux-debug-simarm_x64/2619
Original change's description:
> Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
>
> This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.
>
> By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:
>
> VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
> Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
> Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/42092)
>
> This reverts commit 3849b5061c.
>
> Change-Id: I4fd3ba33840771a9f284d733c4a25fac6cde64ca
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154706
> Reviewed-by: Siva Annamalai <asiva@google.com>
TBR=bkonyi@google.com,asiva@google.com
Change-Id: I649d94c668417f2edbfd7039fa5c876e10dc32fe
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154800
Reviewed-by: Alexander Aprelev <aam@google.com>
This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.
By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:
VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/42092)
This reverts commit 3849b5061c.
Change-Id: I4fd3ba33840771a9f284d733c4a25fac6cde64ca
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154706
Reviewed-by: Siva Annamalai <asiva@google.com>
This reverts commit 556f0fb5c8 and fixes merge conflict.
Revert "[vm] Perform non-covariant checks when dynamically invoking callables." as a collateral.
This reverts commit 7a6287bb92.
Revert "[vm] Use AreValidArguments so that names are checked as well." as a collateral
This reverts commit 90c683018f.
Change-Id: Iee2c5e91c39d19dca7a1e640c103443c95e6c4f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154705
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
If inline bump-pointer allocation fails, allocation stubs must check whether the allocation result from the runtime is old and ensure it is in the remembered set if so to make write barrier elimination safe. CreateArrayInstr would directly call the runtime entry if inline bump-pointer allocation failed without performing the remembered check. This CL changes the slow path to call the AllocateArray stub instead, matching every other allocation slow path.
Bug: https://github.com/dart-lang/sdk/issues/42489
Change-Id: Ic502e43683607e1d0f85154f0e3a8c3ea11deefa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154623
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL adds support for unboxing of return values of getters and
therefore completes the work on unboxing support for normal members
(closures are still missing).
As with existing unboxing support for methods and setters, we utilize
TFA information to proof a getter will always return non-nullable
int/double.
We then make such a getter return unboxed int/double. If there are
dynamic calls to the getter a dyn:get:* forwarder will be created
which performs boxing of the return value.
Overall this reduces RX by eliminating BoxInt64 instructions. It
sometimes increases metadata due to more dyn:get:* function objects.
=> On our main size benchmark targets show no significant change.
Issue https://github.com/dart-lang/sdk/issues/40876
Change-Id: If7450ef7e5e3fc9c2e0eaa6b86ffa817699a7e17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154329
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This reverts commit ffe258d2d4.
Reason for revert: Failures on bots
Original change's description:
> [ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process
>
> This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.
>
> By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:
>
> VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
> Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
> Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/41978)
>
> Change-Id: I1c6b1425831b691ad20284716aa80f817dbaf607
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152588
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TBR=bkonyi@google.com,rmacnak@google.com,asiva@google.com
Change-Id: Idb1d24a4524bdc3ccfb199a82710f3c0d9db539a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154702
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.
By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:
VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/41978)
Change-Id: I1c6b1425831b691ad20284716aa80f817dbaf607
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152588
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
The VM only does this when the callable function does not expect dynamic
invocations. Otherwise, performing the checks would be redundant, as the
function body already contains the appropriate non-covariant checks.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-dartkb-linux-release-x64-try
Bug: https://github.com/dart-lang/sdk/issues/40813
Change-Id: I27b5f4067247c6b1b76ba57951a1d91455c2eeb5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154463
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This change only affects compilation when running in non-precompiled
mode with --no-lazy-dispatchers enabled.
Instead of always compiling in non-covariant checks, even for closures
not called dynamically, remove the non-covariant checks from the closure
and instead do the non-covariant checks for dynamic calls during the
NoSuchMethodForCallStub fallback by calling
Function::DoArgumentTypesMatch.
Adds two overloads for Function::DoArgumentTypesMatch, one which takes a
function type argument vector and one which takes neither an
instantiator type argument vector or a function type argument vector.
For the versions that are not explicitly passed a type argument vector,
an appropriate one is constructed using the arguments. If there is not
enough information in the arguments, then we fall back to assuming the
empty type argument vector for the instantiator case and instantiating
to bounds in the function type argument case.
Fixes Function::DoArgumentTypesMatch to handle generic functions and to
check arguments appropriately according to the active null safety mode.
For generic functions, the provided or resulting function type vector
has non-covariant checks performed against the type parameter bounds.
This change uncovered one test that was incorrectly passing in strong
mode, see https://github.com/dart-lang/sdk/issues/42688 for details.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-dartkb-linux-release-x64-try
Change-Id: I5030a46b356778448b51a243f16406eacf1c04bf
Bug: https://github.com/dart-lang/sdk/issues/40813
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153202
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
We already make the callsites use dyn:* selectors for normal method
calls as well as setters, this is doing the same thing for getters.
Once callsites use dyn:get:* various pieces in runtime need to be
adjusted to accomodate for that (e.g. NSM handling, etc).
A follow-up CL will then start actually generating dyn:* getters in
certain situations.
Issue https://github.com/dart-lang/sdk/issues/40876
Change-Id: If219603bc0b8eb119edd08b211a8897d21ec0fb6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154320
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
CFE adds signature members to classes based on the NNBD mode of
an enclosing library, so identical anonymous mixin applications
in libraries with different NNBD modes could end up being different.
This change fixes mixin deduplication transformation to take
NNBD mode of anonymous mixin applications into account.
Fixes https://github.com/dart-lang/sdk/issues/42656
Change-Id: Ib83ae55f8193177012fa1a592fecf381ca8fdbff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154164
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The ICU headers in os_fuchsia.cc have been included with their path
relative to the dart root directory. They should instead be included
relative to the paths mentioned in `-I` directives of the current
compiler invocation.
This allows downstreams that have a different directory organization to
build dart sdk even if they don't exactly replicate the directory
structure of dart sdk.
Change-Id: Iec79f261aee5ccc7b74448978fc69244e260400a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154163
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Filip Filmar <fmil@google.com>
The precompiler has a notion of when a function should get its code
generated, when it should retain a functions types. This information
is used to prune function objects after AOT compilation.
This CL introduces the same pruning logic which we do for regular
functions as well as closure functions to invocation dispatchers.
Change-Id: I5c8a75f0cc822c71e876b8e2c9648d9976745895
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154002
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Even though this field is never written into snapshot dropping it
eagerly allows to compact away a lot of entries in the symbols
table, which are otherwise retained.
On the large internal application this gives the following improvement:
Breakdown by object type:
+--------------------------+--------------+---------+
| Type | Size (Bytes) | Percent |
+--------------------------+--------------+---------+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| ArtificialRoot | -7 | 0.01% |
| OneByteString | -1603 | 1.26% |
| Array | -3206 | 2.51% |
| (RO) _OneByteString | -122688 | 96.22% |
+--------------------------+--------------+---------+
Total: -127504 bytes
-0.39% of old size
Change-Id: Ib6993e3036ff6b36515a17ee7f60175e1188c8fe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153768
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Right now we assign class ids to top-level classes, abstract classes as
well as concrete classes. All of them have allocated from a 16-bit pool
of ids. The VM FATAL()s once it hits that limit.
Customers who run very large programs (significant amount of generated
code) on the Dart VM have started to hit this 16-bit class limit.
Concrete classes can have instances in the heap. Our current heap layout
only allows 16-bit class ids to be encoded in the header word. To avoid
increasing the size of heap objects or shrinking the size of the identity
hash code to 16-bit we keep class ids in object headers to be 16-bit.
Abstract classes cannot have instances in the heap. Though their class
ids are encoded in type objects. Furthermore we sort classes in
AOT/AppJIT mode to perform fast class-id range checks. To avoid impacting
this optimization we treat abstract classes the same way as concrete
classes.
Top-level classes cannot have instances in the heap. Their class ids are
only used in the runtime code, for example for hot-reload as well as
part of the service protocol.
=> We can allocate class ids outside the 16-bit range for top-level
classes, thereby freeing a significant amount of space in the 16-bit
range.
This CL does exactly that: We change classid_t to be int32_t. The
ClassLayout::id_ can now be assigned ids outside 16-bit range for
top-level classes. To do this we keep dart classes and top level classes
as separate arrays in the ClassTable.
Issue https://github.com/dart-lang/sdk/issues/42533
See also b/160229360
Change-Id: I6710a644e7b0ab2d4f4c792bef8e1f91cb117421
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153607
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Previous version
(landed via https://dart-review.googlesource.com/c/sdk/+/135683)
only worked if the dill was loaded via
Dart_LoadScriptFromKernel where script_kernel_size was set,
and when the platform dill could be loaded from a file called the right
thing.
This CL makes it work for Dart_LoadLibraryFromKernel too and allows
the VM to send the platform along (explicitly or implicitly) and only
tries to load the platform from a file with the right name if no
platform is given in any of the inputs.
This should fix http://b/148776866
Change-Id: I62317400a932b7dcd9e126a5a88907d507f9d658
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153609
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This is a reland of 16f09f20b3
The apparent break of internal tests was not caused by this change.
Original change's description:
> [vm] Replaces fuchsia.deprecatedtimezone
>
> (prior attempt was rolled back as it caused downstream tests to time
> out. See prior attempt at: See:
> https://dart-review.googlesource.com/c/sdk/+/149206)
>
> The FIDL library fuchsia.deprecatedtimezone is going away. There are
> different and better ways to obtain the same functionality. This change
> removes the dependency on fuchsia.deprecatedtimezone from the Dart SDK.
>
> Adds inspect metrics that allow whitebox testing of the runners. Here's
> a sample `fx iquery` excerpt from a running device, showing both a dart
> and a flutter runner exposing the same OS diagnostic metrics.
>
> ```
> /hub/c/dart_jit_runner.cmx/70981/out/diagnostics:
> /hub/c/dart_jit_runner.cmx/70981/out/diagnostics#os:
> dst_status = 0
> get_profile_status = 0
> timezone_content_status = 0
> tz_data_close_status = 0
> tz_data_status = 0
> /hub/c/flutter_jit_runner.cmx/29567/out/diagnostics:
> /hub/c/flutter_jit_runner.cmx/29567/out/diagnostics#os:
> dst_status = 0
> get_profile_status = 0
> timezone_content_status = 0
> tz_data_close_status = 0
> tz_data_status = 0
> ```
>
> Under nominal operation, all of the above values should be equal to 0.
> Nonzero values indicate an error.
>
> This functionality is guarded by Fuchsia integration tests at
> //src/tests/intl.
>
> Tested:
> (compile locally for Fuchsia and deploy)
> fx test //src/tests/intl
>
> See:
> - https://github.com/dart-lang/sdk/issues/42245
> - https://github.com/dart-lang/sdk/issues/39650
>
> Fixes#39650
>
> Change-Id: I97f6e17e57000f6eec71246aee670bca65b7e1d1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150662
> Commit-Queue: Filip Filmar <fmil@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: I5da6b0f481af0eb42c3b5e74c920588ac2ef5be9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151862
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Filip Filmar <fmil@google.com>
Loading unit metadata is expanded to create an object representing each loading unit, and each library to reference the loading unit it is a part of.
To be used at compile-time to help assign objects to snapshots.
To be used at runtime to track which units have been loading and the base objects for cross-snapshot references.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Icd3fa95ffd5949bd8f7f84a6f750c4c515de0dba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151865
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
List.generate could return both growable and fixed-size lists.
This change specializes invocations of List.generate when value of
'growable' argument is known (constant or omitted), so it becomes
possible to infer actual type returned by the factory.
This becomes more important with null safety as List.generate is used
more often to initialize lists of non-nullable elements.
Migrated NNBD benchmarks in AOT mode on x64:
Sudoku +11%
DartMicroBenchMM.{Min,Max}Lib +11-13%
DartMicroBenchMM.{Min,Max}Code +19-27%
ForInGeneratedLoop +19%
ForEachLoop +85%
ForInLoop +64%
ForLoop +680%
This change also includes test for inferred types of various List
constructors.
Change-Id: I801231b0a70e3aa8fb14ec9fe749f1dd420b1b9c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153388
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Ben Konyi