In AOT mode, when call site is transitioned from monomorphic state
(with receiver cid1) to polymorphic (with receiver cid2),
there is an optimization which checks if all _allocated_ classes
in the class id range cid1..cid2 have the same dispatch target.
If so, a specialized SingleTargetCall stub is used.
The problem is that 'allocated' bit is only set during precompilation,
and dynamically loaded classes were not considered as valid receiver
classes by this optimization.
As a result, the following situation could happen:
cid1 < cid3 < cid2,
cid1 dispatches to target1
cid2 dispatches to target1
cid3 should dispatch to target2, but it is still in range cid1..cid2 and
SingleTargetCall stub would incorrectly dispatch it to target1.
The fix is to treat all dynamically loaded classes as allocated when
checking for single target optimization.
TEST=pkg/dynamic_modules/test/data/single_target_cid_range_dispatch
Fixes b/493677699
Change-Id: I42e407385a5d9b0a4a1017f713b47587c3c6a818
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/488920
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This CL adds a new `can-be-used-as-type` section in the dynamic
interface to expose the type of a class or extension type, without
making them callable or exposing their members.
Details:
* add the new section to the documentation
* add support in the TFA transformation to annotate kernel nodes
* add support to the validator to check for proper usage from dynamic modules
* add CFE, transformation, and end2end tests.
The CL ignores DDC and Dart2wasm at the moment and we intend to approve
any failures. It's possible that DDC works due to its modular nature.
BUG=b/488343137
TEST=cfe, annotator, and e2e tests
Change-Id: I900b22b85950f8ac379eabaa2ef58f075e18519c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/486340
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This introduces an `extension` yaml key, similar to `extension_type` to
the dynamic interface.
The validator uses the key to find extensions. Test coverage added
similar to the previous change.
BUG=b/488345908
TEST=pkg/dynamic_modules/test/data and pkg/front_end/testcases/general/dynamic_modules
Change-Id: I4c802663dfee9bf216c0b3dc0f0a0198b8a1fbaa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/484024
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This introduces an `extension_type` yaml key to the dynamic interface to
be able to directly refer to extension type declarations.
The validator then uses the key to search differently for the
corresponding API.
BUG=b/483895271
TEST=updated extension_type2 to explicitly use this new API.
Change-Id: I641b7491e47c031e13271825a115e75fb798c0ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/480780
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Closures are all invoked dynamically when dynamic modules are enabled so we can skip most of the closure representation logic. However, we still need to handle instantiation of type parameters.
Previously we had only one representation for all generic functions. However, the instantiation logic is dependent on the number of type parameters. So we update this to have one representation per type parameter count.
Fixes: https://github.com/dart-lang/sdk/issues/62592
Change-Id: I5ec1468e1f366ffe4db9b339f777e7de98efe12c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/479800
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
In addition to ConcreteMixinStub also unwrap ConcreteForwardingStub
and MemberSignature synthetic stubs for the purposes of validating
callable and can-be-overridden members.
TEST=pkg/dynamic_modules/test/data/mixin_covariant_override
Fixes b/475956113
Change-Id: Ib394f863b0fea877668cf0a49610748f0e7bf784
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/473420
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
When mixin is applied, its instance members are cloned and copies
of the members would override original members.
This change fixes bugs and improved usability of extendable mixins with
private members:
- All instance members of extendable mixins and mixin classes are
marked as can-be-overridden to allow overriding by cloned members
when mixin is applied.
- TFA no longer takes privacy into account, as private members of
mixins can be overridden by their clones in other libraries.
- Dynamic module validator always unwraps cloned members to originals
before verifying if overriding is allowed. This eliminates overriding
errors between clone(s) and original members of mixins.
TEST=pkg/dynamic_modules/test/data/mixin_private_member1,
pkg/dynamic_modules/test/data/mixin_private_member2
Fixes b/470461203
Fixes b/469094721
Change-Id: If369c42c58e5ea4d707be83b1e7078ee9a8cc3ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/470803
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
When creating an instance of a class or calling a factory constructor,
dynamic module may pass a vector of instantator type arguments which
includes types mentioned in supertypes along with types specified
at the call site.
So, for every exposed generative constructor and factory all the types
mentioned in the supertypes of their declaring classes should be
annotated as implicitly callable in order to prevent their tree-shaking.
TEST=pkg/dynamic_modules/test/data/super_type_param
Fixes b/470446652
Change-Id: I6abe570261a8f23f8022cb77110c3a5ec40ccbe8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/470781
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
All libraries exported via dynamic_interface.yaml (except 'dart:'
libraries) are now always compiled, as if they were specified
using --source flag on the gen_kernel / frontend_server command line.
This makes sure that exported libraries would be available for
dynamic modules even if they are not used in the host app.
TEST=pkg/dynamic_modules/test
Fixes b/452833638
Change-Id: I513a75bea76a18a3591613ecd68e7307e66a7c24
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/458586
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Instance method dispatch uses boxed calling conventions if receiver
has a dynamically loaded class, even if method is not exported through
dynamic interface. So we need to disable unboxing and register calling
conventions for instance methods of classes with potential
dynamically loaded subtypes (implicitly or explicitly extendable).
Bug: b/454825012
TEST=pkg/dynamic_modules/test/data/extend_class4
Change-Id: I8256d72fa6fab82cdbe6f018094cbe20f429c220
Cq-Include-Trybots: luci.dart.try:vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/457441
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
It is not safe to eliminate duplicate mixin applications if
they are exposed through dynamic interface (as dynamic modules can
reference such mixin applications or their members).
TEST=pkg/dynamic_modules/test/data/mixin_deduplication
Bug: b/449661748
Change-Id: I56b167d5a4745d03dbf3a41a0ce2e4811a0e0960
Cq-Include-Trybots: luci.dart.try:vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/453560
Reviewed-by: Nate Biggs <natebiggs@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Private class _Enum is an implementation detail of Dart enums, so
we should not require users to explicitly expose it through dynamic
interface in order to use enums in the dynamic modules.
This change adds pragmas to make all public members of _Enum callable
and allows _Enum.toString() to be overridden in dynamic modules.
TEST=pkg/dynamic_modules/test/data/enum
CoreLibraryReviewExempt: no API changes, only pragmas.
Bug: b/448593948
Change-Id: I37040d4912d6da38fbf568c11ae0834a110d3a46
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/452880
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Adding the whole dart:core as callable has certain size overhead and
doing this by default prevents experimentation and fine-grained
control.
Bug: b/419727041
Change-Id: I7b24b3011ca07351dddf5a9e09292b5ecc6bde34
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/431580
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
These mixin constructors are now marked callable by the dynamic interface annotator: https://dart-review.googlesource.com/c/sdk/+/430002
However, they are not directly invokeable and trying to compile the constructor entrypoints causes dart2wasm to crash. So we explicitly skip them when considering dynamic callable constructors.
Note: This is blocking internal rolls as code patterns like this show up in our internal codebase.
Change-Id: I76383aec07bc53a0dbd1581c0577cba370d956f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430801
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Auto-Submit: Nate Biggs <natebiggs@google.com>
- For direct calls in selector branch, ensure 'if' branches have correct inputs.
- For overrideable selectors, the receiver type cannot be known in the main module so use 'top' type. Technically we could do better if selectors tracked interfaces they targeted. Then we could take the LUB of all those classes. But this would be a significant refactor for a small benefit.
- Type checks on classes defined in the main module should use the class ID ranges from the main module rather than those from the dynamic submodule. This only applies to non-dynamic module extendable types (otherwise we'd use the RTT checks). So we know the class can only exist in one of the range sets, not both.
I discovered (1) from running Flutter which led me to create this test which uncovered (2) and (3).
Change-Id: I80f39835f66aa7cf0cff527341e3f4a948a7a0cb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430360
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
When const constructor is exposed through a dynamic interface,
dynamic module can create constants with that const constructor and
it can reference everything used in that constructor including default
values of parameters and field initializers. So dynamic interface
annotator should visit bodies of const constructors and add all
references to implicit uses.
Annotator should also visit initializers of instance fields of
classes which can be used in a constant (as they participate in
the constant evaluation too). Kernel trim tool should not remove such
field initializers.
TEST=pkg/dynamic_modules/test/data/const_constructor
Fixes b/418928636
Change-Id: Ie216f3a4257fe5a905b83af8c60dfe8a5b774ecf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430002
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
When applying a mixin during dynamic module compilation, fields and
methods are copied from kernel AST of the mixin.
So field initializers within mixins should be preserved while trimming
host app kernel file, in order to be available for the dynamic module
compilation.
TEST=pkg/dynamic_modules/test/data/mixin_field
Fixes b/418729276
Change-Id: I00c4392989d552e24dc8c4b4b60c1667de234f98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429680
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Mixin transformation copies members of a mixin into a mixin
application class. Copied members should still behave similarly to
original members in terms of overriding.
When validating a dynamic module, overriding a copy of mixin
member should be allowed if original member can be overridden.
When annotating AST with dynamic interface pragmas, can-be-overridden
pragmas should apply to all copies of mixin members.
TEST=pkg/dynamic_modules/test/data/override_mixin_method
Fixes b/418681054
Change-Id: I038ff2133ef6f0c288a47cec7300c6ac4b23cbab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429423
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Only enforce that a library must be 3.0 or higher if it
contains extendable elements. Otherwise, we can ignore legacy
mixin classes and trim the entire library.
TESTED=apply_mixin case in dynamic_modules tests.
Change-Id: I1378c6d4678305ae634b4259e7eb28a48327d645
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/425187
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Prior to this we were using the dispatch table to get the class IDs in order to register the callable references via `recordClassTargetUse`. However, the dispatch table only contains concrete classes.
A callable member may exist on an abstract class with no concrete subclasses. In this case the dispatch table would not contain any classes and the code for the member would never get generated. This is problematic when the abstract class (or a subclass) is dynamic module extendable. Then the callable member could be inherited onto a subclass in the dynamic module.
Also makes sure to mark members that are callable AND overrideable as invoked so that the updateable dispatch mechanism registers the member.
Change-Id: Icc079f76ab300aa761a47c8fbe24b8d9583d0da7
Fixes: https://github.com/dart-lang/sdk/issues/60602
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423260
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Running with dynamic modules means that closure invocation shapes cannot be statically known. A closure can flow between any modules and can then be invoked with a shape that's unknown to the module that defines the closure.
Given this, our 2 options are to generate code for every invocation shape or invoke the closure as if we don't know it's shape (dynamic invocation).
The former would scale exponentially relative to the number of named parameters so is not practical.
The latter generates slower, bigger code but is the more practical of the 2.
Change-Id: I5e3613d23662e5b2213fdaa725642678fe26e43a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/421920
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ömer Ağacan <omersa@google.com>
In order to simplify creating dynamic interface yaml files,
the following is added to dynamic interface by default:
callable: dart:core, pragma._
extendable: Object
TEST=pkg/dynamic_modules/test
CoreLibraryReviewExempt: no API changes, only adding pragmas
Change-Id: I925532c4c024ebbcf4972b00be0e0a080d5878fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/422024
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Instead of validating the erasure of an extension type,
validate that extension type declaration and type arguments
are specified as callable.
TEST=pkg/dynamic_modules/test/data/extension_type2
Fixes b/411433443
Change-Id: I063f0a622abd0d75594c4095573b29a7888f5afb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423882
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Dynamic module validation now accepts reference to classes and members
from 'dart:*' libraries if they are annotated with
dyn-module:language-impl:* pragmas. This matches the behavior of
dynamic interface annotator.
This is a follow-up to
https://dart-review.googlesource.com/c/sdk/+/418663.
Also, when checking for possible dynamic overrides in TFA,
treat _enumToString from dart:core as a public name because
front-end desugars enums into classes which override
private _enumToString method in a different library.
TEST=pkg/dynamic_modules/test/data/enum
Fixes b/395992622
Change-Id: I45ed9855f806224a510fa4e56962607de3371eb0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423820
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
We compare the signatures from the main module and the dynamic module. If the dynamic module has more parameters these must be from an override with extra optional paramters so we can provide the default value.
Change-Id: I818187fde3e38812dcdae0a14bbeefec4ee91e51
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/416460
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Introduce a helper library to trim components based on what we believe
it is needed for modular bytecode compilation. The script is configured
to accept a set of entry points, so unreachable libraries can be removed
entirely. The contents of the retained libraries is trimmed to remove
method bodies, constructor bodies, and initializers, except for where
they may be needed.
In the near future, this should be expanded to:
* include proper unit testing in the CFE
* review whether additional trimming operations can be made
* consider an explicit representation of trimmed content, to help the
CFE recover when assumptions are not met (e.g. sentinel markers
to establish whether a value has been trimmed)
* CFE produces trimmed data directly if needed, without having to first
produce the full dill.
Tests that specifically stress that we don't over-trim include:
apply_mixin (requires preserving method bodies), const_body (requires
preserving initializers).
TEST=existing and new e2e dynamic module aot tests.
b/394936876
Change-Id: I26db8385bdfe1664b2aea234ec8bb896c7c21230
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/418702
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Missing from this implementation:
- Closure/dynamic calls with differing signatures
- Overrides with extra optional parameters
- Records with same shape defined in different dynamic modules
- Avoiding running TFA on dynamic module.
- Recompilation of only updateable functions from main module.
- Persist wasm def types from main module.
Testing is currently done locally via the dynamic_modules package test suite:
dart pkg/dynamic_modules/test/runner/main.dart --runtime=dart2wasm
Immediately after this lands we can introduce a new step to one of the wasm test matrix configurations that runs the above test suite (the VM has a similar configuration).
Change-Id: I3386d84be11b773842d45f4268a62a54c47e352b
Tested: Tested via new tests in dynamic_modules package. Tests run locally but will add to existing config.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/397721
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
All supertypes of classes marked as "dyn-module:extendable" are technically also extendable in dynamic modules since subtyping is transitive.
Also skip marking 'final' classes as dynamic module extendable.
Change-Id: I124eae56b78b98aba8712158fda87bbc8e652ed3
Tested: Ran tests for dynamic interface annotator.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/403960
Commit-Queue: Nate Biggs <natebiggs@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This includes:
* tests extending non-base class
* tests for applying mixins
* generalization of top-level test to include methods and gettters
* tests for annotating entities indirectly (reexport, hidden
implementation)
* test for using core libraries and core language features.
Change-Id: I2ff928c15794547368d2871f5b982685593ef662
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/396105
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Collects files from `package:async_helper` and `tests/language`
that are generally useful, so that all test-related helpers are
in `package:expect`.
Moves the two libraries from `package:async_helper` into `package:expect`,
and the `tests/language/static_type_helper.dart` file too.
Deprecates `async_minitest.dart`, to follow `minitest.dart`,
expecting the Flutter use of it to have been fixed to not break
on deprecation (I believe Flutter no longer breaks builds on deprecations at all).
Patch 1 is the actual change.
Patch 2+4+8 is changing all existing references to the files.
Patch 6 ignores deprecation in files still using `async_minitest.dart`.
3+5+7+9 are updating this text to make the numbers match.
Then it's just test-expectations and small tweaks from there.
Change-Id: I1b665135b5fef9b9a0c3b340ffe9daf874d0174c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/373120
Reviewed-by: Nate Bosch <nbosch@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Lasse Nielsen <lrn@google.com>
* Introduce steps to execute tests in AOT (using gen-kernel,
dart2bytecode, and aot compiler)
* Accept using filesystem-scheme to provide dynamic_interfaces.yaml input
* Accept empty dynamic_interfaces.yaml
* Make temporary test changes to get them running:
* include core types in dynamic_interfaces that we may want to have
by default
* rename entrypoint to `main`
Most existing test pass, the two tests that validate that a library
cannot be defined twice fail (expectation is to throw, bytecode ignores
the second definition).
To run locally:
```
./tool/build.py -m release --dart-dynamic-modules create_sdk
DART_CONFIGURATION=ReleaseX64 out/ReleaseX64/dart-sdk/bin/dart pkg/dynamic_modules/test/runner/main.dart -r aot
```
Tested: CL adds additional test coverage, currently ran manually, integrated in CI in child CL
Change-Id: I4868e765855d9951bff160c18b846aa628f5e0b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383928
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
This CL introduces a test runner and test cases under
`pkg/dynamic_modules/test/`. These tests are end-to-end tests that
specifically target the dynamic modules semantics. This includes:
* semantics expecations on the program evolutions (e.g. libraries
defined only once)
* individual cases around the dynamic interface (callable, extensible,
overrides).
We should continue to add more test as we discover coner cases worth validating
(e.g. mixins, constants, etc).
The suite is not yet integrated to the test matrix, my plan is to do so
after we have some initial targets set up.
Change-Id: I2a63946c36d99baacb6fb7edc01c4ef377ce20b6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/379921
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Alexander Markov