Fix the following cases of incorrect optimizations:
* In TFA, when anlyzing a dynamic call with unknown receiver type,
do not assume that all possible targets can be computed at compile
time (if there can be dynamically loaded classes).
* In the AOT, disable optimizations for dynamic calls with unique
selectors (if there can be dynamically loaded classes).
TEST=pkg/dynamic_modules/test/data/dyn_call_from_host
Change-Id: I39d620aae3c116de03d4a2a3fd61864d88c48c8e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/493960
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This is a small tweak to allow using `-r jit` when iterating with the
precompiled test runner snapshot. The `resolvedExecutable` was pointing
to the aotruntime rather than the dart binary. This makes the choice of
using the dart binary unambiguous.
TEST=local manual testing, target is only used for local iteration
Change-Id: I988b7597b68d7c16ec806774795109197e9a6996
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/493163
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
In AOT mode, when call site is transitioned from monomorphic state
(with receiver cid1) to polymorphic (with receiver cid2),
there is an optimization which checks if all _allocated_ classes
in the class id range cid1..cid2 have the same dispatch target.
If so, a specialized SingleTargetCall stub is used.
The problem is that 'allocated' bit is only set during precompilation,
and dynamically loaded classes were not considered as valid receiver
classes by this optimization.
As a result, the following situation could happen:
cid1 < cid3 < cid2,
cid1 dispatches to target1
cid2 dispatches to target1
cid3 should dispatch to target2, but it is still in range cid1..cid2 and
SingleTargetCall stub would incorrectly dispatch it to target1.
The fix is to treat all dynamically loaded classes as allocated when
checking for single target optimization.
TEST=pkg/dynamic_modules/test/data/single_target_cid_range_dispatch
Fixes b/493677699
Change-Id: I42e407385a5d9b0a4a1017f713b47587c3c6a818
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/488920
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This CL adds a new `can-be-used-as-type` section in the dynamic
interface to expose the type of a class or extension type, without
making them callable or exposing their members.
Details:
* add the new section to the documentation
* add support in the TFA transformation to annotate kernel nodes
* add support to the validator to check for proper usage from dynamic modules
* add CFE, transformation, and end2end tests.
The CL ignores DDC and Dart2wasm at the moment and we intend to approve
any failures. It's possible that DDC works due to its modular nature.
BUG=b/488343137
TEST=cfe, annotator, and e2e tests
Change-Id: I900b22b85950f8ac379eabaa2ef58f075e18519c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/486340
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This introduces an `extension` yaml key, similar to `extension_type` to
the dynamic interface.
The validator uses the key to find extensions. Test coverage added
similar to the previous change.
BUG=b/488345908
TEST=pkg/dynamic_modules/test/data and pkg/front_end/testcases/general/dynamic_modules
Change-Id: I4c802663dfee9bf216c0b3dc0f0a0198b8a1fbaa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/484024
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This introduces an `extension_type` yaml key to the dynamic interface to
be able to directly refer to extension type declarations.
The validator then uses the key to search differently for the
corresponding API.
BUG=b/483895271
TEST=updated extension_type2 to explicitly use this new API.
Change-Id: I641b7491e47c031e13271825a115e75fb798c0ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/480780
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
This adds a target for running pkg/dynamic_modules/test/runner/main.dart from
an aot snapshot, useful when repeating runs for local iteration.
I'm not including it with other build targets or using it in the
test_matrix because each bot only runs this script once, so it doesn't
provide much savings there.
Change-Id: I20fe50a82d2a59aca6a0e697658b26b5b7da97c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/480744
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Closures are all invoked dynamically when dynamic modules are enabled so we can skip most of the closure representation logic. However, we still need to handle instantiation of type parameters.
Previously we had only one representation for all generic functions. However, the instantiation logic is dependent on the number of type parameters. So we update this to have one representation per type parameter count.
Fixes: https://github.com/dart-lang/sdk/issues/62592
Change-Id: I5ec1468e1f366ffe4db9b339f777e7de98efe12c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/479800
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
In addition to ConcreteMixinStub also unwrap ConcreteForwardingStub
and MemberSignature synthetic stubs for the purposes of validating
callable and can-be-overridden members.
TEST=pkg/dynamic_modules/test/data/mixin_covariant_override
Fixes b/475956113
Change-Id: Ib394f863b0fea877668cf0a49610748f0e7bf784
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/473420
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
When mixin is applied, its instance members are cloned and copies
of the members would override original members.
This change fixes bugs and improved usability of extendable mixins with
private members:
- All instance members of extendable mixins and mixin classes are
marked as can-be-overridden to allow overriding by cloned members
when mixin is applied.
- TFA no longer takes privacy into account, as private members of
mixins can be overridden by their clones in other libraries.
- Dynamic module validator always unwraps cloned members to originals
before verifying if overriding is allowed. This eliminates overriding
errors between clone(s) and original members of mixins.
TEST=pkg/dynamic_modules/test/data/mixin_private_member1,
pkg/dynamic_modules/test/data/mixin_private_member2
Fixes b/470461203
Fixes b/469094721
Change-Id: If369c42c58e5ea4d707be83b1e7078ee9a8cc3ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/470803
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
When creating an instance of a class or calling a factory constructor,
dynamic module may pass a vector of instantator type arguments which
includes types mentioned in supertypes along with types specified
at the call site.
So, for every exposed generative constructor and factory all the types
mentioned in the supertypes of their declaring classes should be
annotated as implicitly callable in order to prevent their tree-shaking.
TEST=pkg/dynamic_modules/test/data/super_type_param
Fixes b/470446652
Change-Id: I6abe570261a8f23f8022cb77110c3a5ec40ccbe8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/470781
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
All libraries exported via dynamic_interface.yaml (except 'dart:'
libraries) are now always compiled, as if they were specified
using --source flag on the gen_kernel / frontend_server command line.
This makes sure that exported libraries would be available for
dynamic modules even if they are not used in the host app.
TEST=pkg/dynamic_modules/test
Fixes b/452833638
Change-Id: I513a75bea76a18a3591613ecd68e7307e66a7c24
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/458586
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
If a member is not considered "callable" from submodules, user code
should not be able to invoke it. However, dynamic module subclasses may
"inherit" the member. Virtual calls in the main module may then try to
invoke the inherited member.
We need to support updating the selector row in the dispatch table with
the class ID of the new subclass. This is similar to an overridable
function except that the slot will be filled with an imported member
from the main module (not a newly defined member).
See https://dart-review.googlesource.com/c/sdk/+/457441 for more
context.
Bug: https://github.com/dart-lang/sdk/issues/61813
Change-Id: I4162931747149a7356909ec62084375fc766b0cc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/458160
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Instance method dispatch uses boxed calling conventions if receiver
has a dynamically loaded class, even if method is not exported through
dynamic interface. So we need to disable unboxing and register calling
conventions for instance methods of classes with potential
dynamically loaded subtypes (implicitly or explicitly extendable).
Bug: b/454825012
TEST=pkg/dynamic_modules/test/data/extend_class4
Change-Id: I8256d72fa6fab82cdbe6f018094cbe20f429c220
Cq-Include-Trybots: luci.dart.try:vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/457441
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
It is not safe to eliminate duplicate mixin applications if
they are exposed through dynamic interface (as dynamic modules can
reference such mixin applications or their members).
TEST=pkg/dynamic_modules/test/data/mixin_deduplication
Bug: b/449661748
Change-Id: I56b167d5a4745d03dbf3a41a0ce2e4811a0e0960
Cq-Include-Trybots: luci.dart.try:vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/453560
Reviewed-by: Nate Biggs <natebiggs@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Private class _Enum is an implementation detail of Dart enums, so
we should not require users to explicitly expose it through dynamic
interface in order to use enums in the dynamic modules.
This change adds pragmas to make all public members of _Enum callable
and allows _Enum.toString() to be overridden in dynamic modules.
TEST=pkg/dynamic_modules/test/data/enum
CoreLibraryReviewExempt: no API changes, only pragmas.
Bug: b/448593948
Change-Id: I37040d4912d6da38fbf568c11ae0834a110d3a46
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/452880
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
The name has always been annoying because it did not add strong typing. And now there aren't variants of the VM platform to distinguish.
Leave a copy at the old name to not immediately break illegal uses.
TEST=ci
Cq-Include-Trybots: luci.dart.try:flutter-analyze-try,flutter-frontend-try,flutter-linux-try
Change-Id: Ie76fa7f16940aa1ba8d582eb5197f0ae55dc8938
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429828
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Adding the whole dart:core as callable has certain size overhead and
doing this by default prevents experimentation and fine-grained
control.
Bug: b/419727041
Change-Id: I7b24b3011ca07351dddf5a9e09292b5ecc6bde34
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/431580
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
These mixin constructors are now marked callable by the dynamic interface annotator: https://dart-review.googlesource.com/c/sdk/+/430002
However, they are not directly invokeable and trying to compile the constructor entrypoints causes dart2wasm to crash. So we explicitly skip them when considering dynamic callable constructors.
Note: This is blocking internal rolls as code patterns like this show up in our internal codebase.
Change-Id: I76383aec07bc53a0dbd1581c0577cba370d956f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430801
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Auto-Submit: Nate Biggs <natebiggs@google.com>
- For direct calls in selector branch, ensure 'if' branches have correct inputs.
- For overrideable selectors, the receiver type cannot be known in the main module so use 'top' type. Technically we could do better if selectors tracked interfaces they targeted. Then we could take the LUB of all those classes. But this would be a significant refactor for a small benefit.
- Type checks on classes defined in the main module should use the class ID ranges from the main module rather than those from the dynamic submodule. This only applies to non-dynamic module extendable types (otherwise we'd use the RTT checks). So we know the class can only exist in one of the range sets, not both.
I discovered (1) from running Flutter which led me to create this test which uncovered (2) and (3).
Change-Id: I80f39835f66aa7cf0cff527341e3f4a948a7a0cb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430360
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
When const constructor is exposed through a dynamic interface,
dynamic module can create constants with that const constructor and
it can reference everything used in that constructor including default
values of parameters and field initializers. So dynamic interface
annotator should visit bodies of const constructors and add all
references to implicit uses.
Annotator should also visit initializers of instance fields of
classes which can be used in a constant (as they participate in
the constant evaluation too). Kernel trim tool should not remove such
field initializers.
TEST=pkg/dynamic_modules/test/data/const_constructor
Fixes b/418928636
Change-Id: Ie216f3a4257fe5a905b83af8c60dfe8a5b774ecf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430002
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
When applying a mixin during dynamic module compilation, fields and
methods are copied from kernel AST of the mixin.
So field initializers within mixins should be preserved while trimming
host app kernel file, in order to be available for the dynamic module
compilation.
TEST=pkg/dynamic_modules/test/data/mixin_field
Fixes b/418729276
Change-Id: I00c4392989d552e24dc8c4b4b60c1667de234f98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429680
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Mixin transformation copies members of a mixin into a mixin
application class. Copied members should still behave similarly to
original members in terms of overriding.
When validating a dynamic module, overriding a copy of mixin
member should be allowed if original member can be overridden.
When annotating AST with dynamic interface pragmas, can-be-overridden
pragmas should apply to all copies of mixin members.
TEST=pkg/dynamic_modules/test/data/override_mixin_method
Fixes b/418681054
Change-Id: I038ff2133ef6f0c288a47cec7300c6ac4b23cbab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429423
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This change requires some modifications to the dart2wasm flags for
dynamic module support. Previously, we determined that dynamic module
support was needed if `--dynamic-main-module` was passed. We determined
that the main module was being compiled if `--dynamic-interface-uri` was
also passed, and a submodule (AKA "dynamic module") otherwise. This
design prevented us from passing the interface specification to the
dynamic module validator when submodules were being compiled.
Instead, we now build with dynamic support when the
`--dynamic-module-type` flag is passed. Allowable values are `main` and
`submodule`. Both the main module URI and interface URI are currently
required for both dynamic module types. (The main module URI could be
made optional if we generated a default filename like we do for dynamic
module metadata.)
Dynamic interface validation is enabled by default and is controlled by
the `--validate-dynamic-modules` flag. This flag can be negated via
`--validate-dynamic-modules=false` or `--no-validate-dynamic-modules`.
Change-Id: I3165c3a8255205a61c3ccfe546f5d436472ed0d9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/425154
Reviewed-by: Nate Biggs <natebiggs@google.com>
Reviewed-by: Ömer Ağacan <omersa@google.com>
Only enforce that a library must be 3.0 or higher if it
contains extendable elements. Otherwise, we can ignore legacy
mixin classes and trim the entire library.
TESTED=apply_mixin case in dynamic_modules tests.
Change-Id: I1378c6d4678305ae634b4259e7eb28a48327d645
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/425187
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Prior to this we were using the dispatch table to get the class IDs in order to register the callable references via `recordClassTargetUse`. However, the dispatch table only contains concrete classes.
A callable member may exist on an abstract class with no concrete subclasses. In this case the dispatch table would not contain any classes and the code for the member would never get generated. This is problematic when the abstract class (or a subclass) is dynamic module extendable. Then the callable member could be inherited onto a subclass in the dynamic module.
Also makes sure to mark members that are callable AND overrideable as invoked so that the updateable dispatch mechanism registers the member.
Change-Id: Icc079f76ab300aa761a47c8fbe24b8d9583d0da7
Fixes: https://github.com/dart-lang/sdk/issues/60602
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423260
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Running with dynamic modules means that closure invocation shapes cannot be statically known. A closure can flow between any modules and can then be invoked with a shape that's unknown to the module that defines the closure.
Given this, our 2 options are to generate code for every invocation shape or invoke the closure as if we don't know it's shape (dynamic invocation).
The former would scale exponentially relative to the number of named parameters so is not practical.
The latter generates slower, bigger code but is the more practical of the 2.
Change-Id: I5e3613d23662e5b2213fdaa725642678fe26e43a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/421920
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ömer Ağacan <omersa@google.com>
In order to simplify creating dynamic interface yaml files,
the following is added to dynamic interface by default:
callable: dart:core, pragma._
extendable: Object
TEST=pkg/dynamic_modules/test
CoreLibraryReviewExempt: no API changes, only adding pragmas
Change-Id: I925532c4c024ebbcf4972b00be0e0a080d5878fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/422024
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Instead of validating the erasure of an extension type,
validate that extension type declaration and type arguments
are specified as callable.
TEST=pkg/dynamic_modules/test/data/extension_type2
Fixes b/411433443
Change-Id: I063f0a622abd0d75594c4095573b29a7888f5afb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423882
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Dynamic module validation now accepts reference to classes and members
from 'dart:*' libraries if they are annotated with
dyn-module:language-impl:* pragmas. This matches the behavior of
dynamic interface annotator.
This is a follow-up to
https://dart-review.googlesource.com/c/sdk/+/418663.
Also, when checking for possible dynamic overrides in TFA,
treat _enumToString from dart:core as a public name because
front-end desugars enums into classes which override
private _enumToString method in a different library.
TEST=pkg/dynamic_modules/test/data/enum
Fixes b/395992622
Change-Id: I45ed9855f806224a510fa4e56962607de3371eb0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423820
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
We compare the signatures from the main module and the dynamic module. If the dynamic module has more parameters these must be from an override with extra optional paramters so we can provide the default value.
Change-Id: I818187fde3e38812dcdae0a14bbeefec4ee91e51
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/416460
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Introduce a helper library to trim components based on what we believe
it is needed for modular bytecode compilation. The script is configured
to accept a set of entry points, so unreachable libraries can be removed
entirely. The contents of the retained libraries is trimmed to remove
method bodies, constructor bodies, and initializers, except for where
they may be needed.
In the near future, this should be expanded to:
* include proper unit testing in the CFE
* review whether additional trimming operations can be made
* consider an explicit representation of trimmed content, to help the
CFE recover when assumptions are not met (e.g. sentinel markers
to establish whether a value has been trimmed)
* CFE produces trimmed data directly if needed, without having to first
produce the full dill.
Tests that specifically stress that we don't over-trim include:
apply_mixin (requires preserving method bodies), const_body (requires
preserving initializers).
TEST=existing and new e2e dynamic module aot tests.
b/394936876
Change-Id: I26db8385bdfe1664b2aea234ec8bb896c7c21230
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/418702
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Nate Biggs