Dynamic module validation now accepts reference to classes and members
from 'dart:*' libraries if they are annotated with
dyn-module:language-impl:* pragmas. This matches the behavior of
dynamic interface annotator.
This is a follow-up to
https://dart-review.googlesource.com/c/sdk/+/418663.
Also, when checking for possible dynamic overrides in TFA,
treat _enumToString from dart:core as a public name because
front-end desugars enums into classes which override
private _enumToString method in a different library.
TEST=pkg/dynamic_modules/test/data/enum
Fixes b/395992622
Change-Id: I45ed9855f806224a510fa4e56962607de3371eb0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423820
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change adds pragmas which can be used in dart:* libraries to
specify classes and members which are used by the compiler
to implement certain language features. Compiler can insert
references to those classes and members when desugaring language
constructs, so dynamic module code can use these classes and members
implicitly even if they are private.
Previously, we required to specify these members in the
dynamic_interface.yaml before dynamic modules could use them.
However, this is cumbersome and error-prone.
Now, the following pragmas are recognized:
@pragma('dyn-module:language-impl:extendable')
@pragma('dyn-module:language-impl:can-be-overridden')
@pragma('dyn-module:language-impl:callable')
If dynamic interface yaml file is specified during compilation,
then classes/members annotated with these pragmas are treated as if
they are used in corresponding sections of dynamic_interface.yaml.
If dynamic interface yaml file is not specified, these pragmas
are ignored.
Bug: b/395992622
TEST=existing
CoreLibraryReviewExempt: no API changes, only adding pragmas.
Change-Id: I6606467fc5e5264a2565d7eb8e8baccc2eebea84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/418663
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
- Rename resource identifiers in the VM to usage recordings.
- Use package:record_use for serialization.
- Rename and use the experimental flag for this feature.
- Recognize tear-offs and top-level methods as well.
Next steps:
- Add constant instance recording.
- Expose API in package:native_assets_cli's link callback.
TEST=pkg/vm/test/transformations/record_use_test.dart
Change-Id: I8af3625165f78925ae943711245af93a239d1012
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383040
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Moritz Sümmermann <mosum@google.com>
* Introduce steps to execute tests in AOT (using gen-kernel,
dart2bytecode, and aot compiler)
* Accept using filesystem-scheme to provide dynamic_interfaces.yaml input
* Accept empty dynamic_interfaces.yaml
* Make temporary test changes to get them running:
* include core types in dynamic_interfaces that we may want to have
by default
* rename entrypoint to `main`
Most existing test pass, the two tests that validate that a library
cannot be defined twice fail (expectation is to throw, bytecode ignores
the second definition).
To run locally:
```
./tool/build.py -m release --dart-dynamic-modules create_sdk
DART_CONFIGURATION=ReleaseX64 out/ReleaseX64/dart-sdk/bin/dart pkg/dynamic_modules/test/runner/main.dart -r aot
```
Tested: CL adds additional test coverage, currently ran manually, integrated in CI in child CL
Change-Id: I4868e765855d9951bff160c18b846aa628f5e0b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383928
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
I would like to add an option to our BUILD
files to precompile gen_kernel and
compile_platform to speedup iteration cycles
when working on core library changes.
To make this robust I would like to use
depfile to track dependencies and know
when compile_platform should be
recompiled.
TEST=pkg/dartdev/test/commands/compile_test.dart
R=kustermann@google.com
Change-Id: Id674f7353342c8275a8a0c4a70e3f5eaeb7f05d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367023
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
The kernel_front_end will now allow passing in no Dart main program
when compiling a native-assets file.
The test now tests concatenating the dill files.
(A follow up CL will attempt the same for AOT dills.)
TEST=tests/ffi/native_assets/asset_relative_test.dart
Bug: https://github.com/dart-lang/sdk/issues/55377
Change-Id: Icbc6ab87d0015809f978966914064dbc6ab564a1
Cq-Include-Trybots: dart/try:vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362340
Reviewed-by: Tess Strickland <sstrickl@google.com>
The modular parts in package:vm/modular/** will only depend on
* package:kernel
* package:front_end
=> Only for auto-generated error code messages
=> Those error codes can be their own dart_library() in g3
Based on https://dart-review.googlesource.com/c/sdk/+/353980, with
changes:
* Exclude package:kernel from "no exports" CFE presubmit.
* Add package:vm/modular to CFE "allowed deps".
* Fix import in `front_end/tool/perf_common.dart` and various others.
Tested: no code changes, only moves.
Change-Id: I9a44ac6ee05478812a9c8af31f6c4bbcf44b7c42
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/354221
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
If a static field or getter is annotated with
@pragma("vm:platform-const-if", <cond>) and <cond> const evaluates
to true, then uses of the static field or getter are const evaluated
when a target operating system is available. If <cond> const evaluates
to any other value, then the annotation is ignored.
For example, when runtime-only code is guarded like the following,
using Flutter's kDebugMode constant, then debug mode Flutter programs
can alter the defaultTargetPlatform for testing, but in release mode,
the defaultTargetPlatform getter is const evaluated and code guarded
with defaultTargetPlatform checks can be eliminated if unreachable:
@pragma("vm:platform-const-if", !kDebugMode)
TargetPlatform get defaultTargetPlatform {
...
assert(() {
if (Platform.environment.containsKey('FLUTTER_TEST')) {
result = TestPlatform.android;
}
return true;
}());
if (kDebugMode &&
platform.debugDefaultTargetPlatformOverride != null) {
result = platform.debugDefaultTargetPlatformOverride;
}
...
}
TEST=pkg/vm/test/transformations/vm_constant_evaluator
Change-Id: I55b88502a908c56cf42a761dd06741f15c8a23d9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333220
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This is a reland of commit 262311772b
When running with sound null safety, reducing a logical expression to
the right hand side if the left hand side is constant and does not short
circuit is valid, as the right hand side is guaranteed not to evaluate
to null.
In other modes, however, the right hand side may evaluate to null.
Thus, in thos modes, we return the original node if the RHS is not a
constant boolean value, so that the operator can perform whatever null
checking is required.
Fixes: https://github.com/dart-lang/sdk/issues/54029
Original change's description:
> Reland "[pkg/vm] Handle switch statements in unreachable code eliminator."
>
> This is a reland of commit 92bf76d9e8
>
> In the original CL, the changes to the UCE assumed all
> SwitchStatements were exhaustive. Now if a SwitchStatement isn't
> explicitly or implicitly (via a default case) exhaustive and no case
> matches the tested constant, the SwitchStatement is properly removed.
>
> In addition, if a guaranteed to match case is found, more than one
> cases remain (thus the SwitchStatement is not removed or replaced with
> the single case's body), and the default case is removed, then the
> resulting SwitchStatement is marked as explicitly exhaustive, as this
> serves as a signal to backends that they do not need to handle the
> possibility of no case matching in the absence of a default case.
>
> Original change's description:
> > [pkg/vm] Handle switch statements in unreachable code eliminator.
> >
> > Namely, if the tested expression for a switch statement is constant,
> > then we can remove any constant cases where the constants differ,
> > and if all but a single case is removed, we can replace the switch
> > with the case body.
> >
> > If constant functions are not enabled, then getters annotated with
> > @pragma("vm:platform-const") are still evaluated with the constant
> > function evaluation machinery, but only those and no others (including
> > any functions called within an annotated getter). This way, functions
> > can be annotated with @pragma("vm:platform-const") without having to
> > rewrite them to be a single returned expression.
> >
> > TEST=pkg/vm/test/transformations/unreachable_code_elimination
> > pkg/vm/test/transformations/vm_constant_evaluator
> >
> > Issue: https://github.com/dart-lang/sdk/issues/50473
> > Issue: https://github.com/dart-lang/sdk/issues/31969
> > Change-Id: Ie290d2f1f469326238d66c3d9631f8e696685ff0
> > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332760
> > Commit-Queue: Tess Strickland <sstrickl@google.com>
> > Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
> > Reviewed-by: Alexander Markov <alexmarkov@google.com>
>
> TEST=pkg/vm/test/transformations/unreachable_code_elimination
> pkg/vm/test/transformations/vm_constant_evaluator
>
> Issue: https://github.com/dart-lang/sdk/issues/50473
> Issue: https://github.com/dart-lang/sdk/issues/31969
> Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try
> Change-Id: I557ca933808012e670e306f2d880221a0d7dd670
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/334224
> Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
TEST=pkg/vm/test/transformations/unreachable_code_elimination
pkg/vm/test/transformations/vm_constant_evaluator
Change-Id: Ia51b7c5f3b51f57a6a306551fe74b47e0cba3c23
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335828
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
vm.
This identifies all calls to static methods annotated with
`@ResourceIdentifier`, collects the constant arguments passed to the
method, and writes the results into a file.
The purpose of this feature is to be able to pass the recorded
information to packages in a post-compilation step, allowing them to
remove or modify assets based on the actual usage in the code prior to
bundling in the final application.
Tested: pkg/vm/test/transformations/resource_identifier_test.dart
Change-Id: I58cb313b66ee23c1d154dcc242547723a1ced359
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/329961
Commit-Queue: Moritz Sümmermann <mosum@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
The "additionalExports" are sorted in production code to give stable
output.
Before this CL it did `toString()` on the canonical names which is
O(n^2) because of continued string concatenation.
This CL changes it to sort on the name, going to the parent if it's the
same. This _does_ change the sorting, but should still be stable.
When compiling via the flutter frontend try bot the time spent on
sorting does from ~45 seconds to ~9 seconds (arguable still a lot, but
much better).
This will also speed up flutter testing because it will spend less time
waiting for the frontend.
TEST=Assuming existing tests.
Change-Id: If350b7c2ce49f00b2924732a7fa7eb51c38c8172
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322280
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
The VM can read native asset mappings from kernel.
Previous CLs already added support to embed native asset mappings for
one-shot compilation.
This CL adds support for adding native assets mappings to kernel files
created by the frontend_server with the incremental compiler.
The frontend_server accepts a `--native-assets=<uri>` at startup and
accepts a `native-assets <uri>` message on stdin as compilation
command.
The frontend_server caches the compiled native assets library.
When a `reset` command is sent to request a full dill from the
incremental compiler, the native assets mapping is taken from the
cache and added to the final dill file.
Split of DartSDK & flutter_tools prototype to land separately.
TEST=pkg/frontend_server/test/native_assets_test.dart
Bug: https://github.com/dart-lang/sdk/issues/49803
Bug: https://github.com/dart-lang/sdk/issues/50565
Change-Id: I6e15f177564b8a962e81261815e951e7c9525513
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/282101
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
* Add an `targetOS` argument to `pkg/vm`'s `compileToKernel`,
that contains the target operating system's name.
* Add a new `--target-os` command line argument for all binaries
that use `compileToKernel` for clients to provide the target
operating system, if known.
* Add a new`"vm:platform:const"` annotation to certain field and
getters in the Platform class.
This annotation is used to annotate static getters and fields with
initializers where the getter body or field initializer must evaluate
to a constant value if the target operating system is known. This
annotation may be used outside the Platform class and in user code.
For example, this annotation can be used on a static `String` field
that is initialized with one value if `Platform.isWindows` is true
and to a different value if `Platform.isWindows` is false.
Note: If the const functions experimental flag is disabled, then
any annotated static methods can only contain a single expression
whose value is returned. If it is enabled, then the static method
is evaluated as if it is a const function with the special
handling of annotated static fields and getters above.
* Create a VM constant evaluator that evaluates uses of static getters
and fields marked with the above annotations when a target operating
system is provided.
* Use the new VM constant evaluator in the unreachable code elimination
transformer.
TEST=pkg/vm/test/transformations/platform_use_transformer
pkg/vm/test/transformations/unreachable_code_elimination
Change-Id: Ie381de70486a767fd7b1d515fd9e6bb58c6bf090
Bug: https://github.com/dart-lang/sdk/issues/31969
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try
CoreLibraryReviewExempt: Just adding vm-specific annotations.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/274386
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Adds support for `asset: 'asset-id'` in `FfiNative`s.
Symbols in assets are resolved by
1. the provided [asset] (the bulk of this CL),
2. the native resolver set with `Dart_SetFfiNativeResolver` in
`dart_api.h` (old behavior), and
3. the current process.
Symbols in assets are resolved by the Dart VM through the mapping
provided in the `vm:ffi:native_assets` library:
```
@pragma('vm:ffi:native-assets', {
"linux_x64": {
"<asset1>": [
"absolute",
"<absolute-path>"
],
"<asset2>": [
"relative",
"<relative-path>"
],
"<asset3>": [
"system",
"<system-path>"
],
"<asset4>": [
"process",
],
"<asset5>": [
"executable",
],
}
})
library;
```
There are 5 asset path types. Symbols resolved in these asset types are
resolved as follows.
A. Absolute paths are dlopened, after which the symbol is looked up.
B. Relative paths are resolved relative to `Platform.script.uri`, and
dlopened, after which the symbol is looked up.
C. System paths are treated as absolute paths. (But we might explicitly
try to use `PATH` in the future.)
D. Symbols looked up in Process assets are looked up in the current
process.
E. Symbols looked up in Executable assets are looked up in the current
executable.
The native assets mapping can be provided in three ways.
X. In the invocation to `gen_kernel` with the `--native-assets`
argument. This uses the gen_kernel entry point for compiling
kernel.
Y. By placing a `native_assets.yaml` next to the
`package_config.json` used by the isolate group. This works for
`dart <source file>` and `Isolate.spawnUri(<dart source file>)`.
This uses the kernel_service entry point for compiling kernel.
Z. By sending a String containing the native assets mapping when
invoking the kernel_service directly from within the VM. Currently,
only used for unit tests.
TEST=tests/ffi/native_assets/asset_*_test.dart (X)
TEST=tests/ffi/native_assets/infer_native_assets_yaml_*.dart (Y)
TEST=runtime/vm/ffi/native_assets_test.cc (Z)
The library is synthesized from yaml in pkg:vm as AST.
Design doc: go/dart-native-assets
Bug: https://github.com/dart-lang/sdk/issues/49803
Change-Id: I8bf7000bfcc03b362948efd3baf168838948e6b9
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264842
Reviewed-by: Martin Kustermann <kustermann@google.com>
- Flip flag to make strong null safety the default
- Remove code that auto detects null safety mode from source files,
it is necessary to specify --no-strong-null-safety to opt out.
- Retains sniffing of AOT/JIT snapshots and kernel files to determine
null safety mode, the opt out has to be done when generating these
file.
TEST=ci
Change-Id: If2c9608eedb7c46d9c3cd85e261ee9640e0d28eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261140
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Weaken assertion in gen_snapshot requiring all libraries to have a loading unit as there can still be unreachable libraries:
- Google3 and Fuchsia will compile all the sources in a package to a single dill file, then present multiple input dill files to the AOT compilation. Since the set of libraries was derived from package membership instead of imports, many can be unreachable.
- When the root library's main comes from an export, the frontend's representation will incorrectly report the library containing main as the root library and the true root library may be unreachable from it.
Instead, assert only that surviving compiled code is assigned a loading unit.
TEST=gallery
Bug: https://github.com/flutter/gallery/issues/545
Bug: https://github.com/dart-lang/sdk/issues/49325
Bug: https://github.com/dart-lang/sdk/issues/41974
Bug: b/237016312
Change-Id: Ia52563a6f517308d041368be11dcc85270f19acc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249724
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
If a library contained an entry point but was not reachable from the root library, it was not assigned to any loading unit and caused a null dereference in gen_snapshot. This is not possible with the standalone embedder, but is possible in Flutter because it passes multiple sources to frontend_server. E.g., `--source dart_plugin_registrant.dart`.
TEST=gallery
Bug: https://github.com/flutter/gallery/issues/545
Change-Id: I9c67f0e39f7509094ee873610d80851a702a0cf2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249640
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This change includes kernel-related changes:
* --compact-async option is added to kernel compilers (front-end
server and gen_kernel). This option disables desugaring of async
and await on kernel AST. Note that 'await for' is still desugared.
* File offset of the 'await' is now written for AwaitExpression nodes
in the kernel binaries (will be used for async stack traces).
* Async/async*/sync* functions and AwaitExpression nodes are supported
in TFA.
Design doc: go/compact-async-await.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I4233086b7434bc48347f4220645b0be5f9133456
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241842
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change adds --source option to kernel compilers (front-end server
and gen_kernel). This option instructs compiler to include extra source
files into compilation even if they are not referenced from main
library.
TEST=Added test cases to pkg/frontend_server/test/frontend_server_test.dart
and pkg/vm/test/kernel_front_end_test.dart
Fixes https://github.com/dart-lang/sdk/issues/48144
Change-Id: If3d71538751e9ccfa8c82a5685d810cf811e021c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231334
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The VM patch files contain many annotations that guide the TFA and the
VM compilation pipeline. Most of these are not relevant to dart2wasm,
and having the TFA look at these when running inside dart2wasm leads to
sub-optimal, and sometimes even unsound, results.
In order to enable sharing of patch files between the VM and dart2wasm,
the annotation parser is given a target flag to control which
annotations it should recognize and how to parse them. The annotation
parser thus becomes an abstraction layer between the concrete
annotations in the patch files and the parsed annotations that the TFA
sees.
Additionally, external members used by dart2wasm don't always have
external names. To support this, the signature shaker must skip all
external members, not just the ones with an external name. This makes no
difference for the VM, since all of its external members either have an
external name or are implemented in a patch file, making them no longer
external.
TEST=ci + upcoming dart2wasm CL
Change-Id: Id425dd1ccc5560721034ae83c8cdc1593801a4e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231102
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Alexander Markov