- Rename resource identifiers in the VM to usage recordings.
- Use package:record_use for serialization.
- Rename and use the experimental flag for this feature.
- Recognize tear-offs and top-level methods as well.
Next steps:
- Add constant instance recording.
- Expose API in package:native_assets_cli's link callback.
TEST=pkg/vm/test/transformations/record_use_test.dart
Change-Id: I8af3625165f78925ae943711245af93a239d1012
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383040
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Moritz Sümmermann <mosum@google.com>
I would like to add an option to our BUILD
files to precompile gen_kernel and
compile_platform to speedup iteration cycles
when working on core library changes.
To make this robust I would like to use
depfile to track dependencies and know
when compile_platform should be
recompiled.
TEST=pkg/dartdev/test/commands/compile_test.dart
R=kustermann@google.com
Change-Id: Id674f7353342c8275a8a0c4a70e3f5eaeb7f05d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367023
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Adding a linking script support, see go/resource-shaking. The goal is to run a `link.dart` script after kernel compilation complimentary to `build.dart` running before.
Change-Id: Iadc8648ae5fa2e823b6541c5bc08617bb860a017
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-arm64-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-arm64-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338380
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Moritz Sümmermann <mosum@google.com>
vm.
This identifies all calls to static methods annotated with
`@ResourceIdentifier`, collects the constant arguments passed to the
method, and writes the results into a file.
The purpose of this feature is to be able to pass the recorded
information to packages in a post-compilation step, allowing them to
remove or modify assets based on the actual usage in the code prior to
bundling in the final application.
Tested: pkg/vm/test/transformations/resource_identifier_test.dart
Change-Id: I58cb313b66ee23c1d154dcc242547723a1ced359
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/329961
Commit-Queue: Moritz Sümmermann <mosum@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
This CL introduces native assets suport for `dart run` and introduces
`dart build` which is similar to `dart compile` but outputs a folder
instead to that native assets can be bundled with an executable.
Change-Id: Ib6cfb95539f0adee46c99e531e440928c3f72f2b
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/267340
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
* Add an `targetOS` argument to `pkg/vm`'s `compileToKernel`,
that contains the target operating system's name.
* Add a new `--target-os` command line argument for all binaries
that use `compileToKernel` for clients to provide the target
operating system, if known.
* Add a new`"vm:platform:const"` annotation to certain field and
getters in the Platform class.
This annotation is used to annotate static getters and fields with
initializers where the getter body or field initializer must evaluate
to a constant value if the target operating system is known. This
annotation may be used outside the Platform class and in user code.
For example, this annotation can be used on a static `String` field
that is initialized with one value if `Platform.isWindows` is true
and to a different value if `Platform.isWindows` is false.
Note: If the const functions experimental flag is disabled, then
any annotated static methods can only contain a single expression
whose value is returned. If it is enabled, then the static method
is evaluated as if it is a const function with the special
handling of annotated static fields and getters above.
* Create a VM constant evaluator that evaluates uses of static getters
and fields marked with the above annotations when a target operating
system is provided.
* Use the new VM constant evaluator in the unreachable code elimination
transformer.
TEST=pkg/vm/test/transformations/platform_use_transformer
pkg/vm/test/transformations/unreachable_code_elimination
Change-Id: Ie381de70486a767fd7b1d515fd9e6bb58c6bf090
Bug: https://github.com/dart-lang/sdk/issues/31969
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try
CoreLibraryReviewExempt: Just adding vm-specific annotations.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/274386
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Now that 9df6656aa9 has appropriately propagated and the build rules in
google3 have been updated, re-add checks to ensure that the empty
section used to pad the original header exists and that the new header
is smaller than the old header.
Bug: https://github.com/dart-lang/sdk/issues/49783
Change-Id: I5b8b31bf9edf4814686eb7928457a9a7d53c0727
Cq-Include-Trybots: luci.dart.try:dart-sdk-mac-try,dart-sdk-mac-arm64-try,pkg-mac-release-arm64-try,pkg-mac-release-try,vm-kernel-mac-release-arm64-try,vm-kernel-mac-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/275622
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- Flip flag to make strong null safety the default
- Remove code that auto detects null safety mode from source files,
it is necessary to specify --no-strong-null-safety to opt out.
- Retains sniffing of AOT/JIT snapshots and kernel files to determine
null safety mode, the opt out has to be done when generating these
file.
TEST=ci
Change-Id: If2c9608eedb7c46d9c3cd85e261ee9640e0d28eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261140
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
To create a Dart standalone executable on MacOS, we modify the
dartaotruntime executable to add the snapshot contents, and the
VM looks into the executable on disk to find the snapshot to load.
Previously, we did this by adding a new 64-bit segment load command
with a single section, where the section's file offset and size
describes the inserted snapshot. This meant the Mach-O header size increased by 152 bytes.
Originally, this wasn't an issue as there was plenty of padding, but
later clang updates removed most of this padding, and so writing the
new header actually overwrote the initial contents of the first section
in the file, which happens to be the __text section. In addition, since
the first section's offset was now declared to be within the header,
utilities that strictly validated the Mach-O format, like codesign,
would report errors.
This CL changes it so that we actually reserve space in the
dartaotruntime header using the -add_empty_section flag to the linker.
In addition, we change from using a segment load command to using a
(40 byte) note load command. This is because a segment load command
specifies that the contents should be loaded in memory, but we don't
use that loaded version. Instead, the VM reloads it from the executable
on disk so it can appropriately mmap the different parts of the
snapshot. A note section instead just declares a section of the
executable as arbitrary data that the owner can read from the file
and use as desired, which is semantically closer to our current usage.
This CL also adds a test to pkg/dartdev/test/commands/compile_test to
ensure that corrupting a random part of the snapshot in the executable
causes signature verification to fail.
This CL also reverts CL 256208, thus relanding the clang changes
starting from June that originally raised awareness of the issue by
greatly reduced the amount of padding after the load commands.
TEST=pkg/dartdev/test/commands/compile_test
Bug: https://github.com/dart-lang/sdk/issues/49783
Change-Id: Iee554d87b0eabaecd7a534ca4e4facfefbce6385
Cq-Include-Trybots: luci.dart.try:analyzer-mac-release-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260108
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
* Move parsing functions to static methods of the classes that
represent the data being parsed.
* Move calculations for adjusting offsets in load commands into the
corresponding classes.
* Make function to insert a new payload segment an instance method
of MachOFile.
* Remove unnecessary abstractions and definitions for MachO load
commands that we don't need to parse.
These refactorings are being done to make later changes easier to
review, by first lifting the initial refactorings out into a
separate CL.
Issue: https://github.com/dart-lang/sdk/issues/49783
Change-Id: I133eb368cbb9ee0d8e4f3998ba1a0bbe8555b8aa
Cq-Include-Trybots: luci.dart.try:analyzer-mac-release-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256821
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit 161f8fd942.
Reason for revert: breaks google3 (b/243899439)
Original change's description:
> [pkg/dart2native] Add negative padding check to MachO writer.
>
> This catches an inadvertent partial overwrite of the first section
> in the `dartaotruntime` executable that happens when there's not
> enough padding after the headers. The first section is a text section,
> so unless there are no calls to code in this overwritten portion, there
> are no runtime failures when this happens.
>
> However, having a MachO section which has a file offset within the
> headers is checked by MachO verification code within the `codesign`
> utility when run with the `-f` flag, and thus caused failures in
> certain tests and builds. Note that we only use that flag when
> `codesign` does not have the `linker-signed` option, which is for
> MacOS versions prior to 12.0.
>
> Example failure with this change when this case occurs (e.g., prior to
> the recent clang revert):
> ```
> $ xcodebuild/ReleaseARM64/dart-sdk/bin/dart compile exe -o test test.dart
> Info: Compiling with sound null safety
> Error: AOT compilation failed
> FormatException: The MachO header overlaps with the first 120 bytes of the section contents
> ```
>
> Change-Id: Ib27db910777f61b90f162f7a0bcfa4ba6592a5a0
> Bug: https://github.com/dart-lang/sdk/issues/49783
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256264
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
TBR=kustermann@google.com,dacoharkes@google.com,sstrickl@google.com
Change-Id: Iccf27ffab6c6b6272e366b77c5092b346145e263
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/49783
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256460
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Emmanuel Pellereau <emmanuelp@google.com>
Reviewed-by: Emmanuel Pellereau <emmanuelp@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This catches an inadvertent partial overwrite of the first section
in the `dartaotruntime` executable that happens when there's not
enough padding after the headers. The first section is a text section,
so unless there are no calls to code in this overwritten portion, there
are no runtime failures when this happens.
However, having a MachO section which has a file offset within the
headers is checked by MachO verification code within the `codesign`
utility when run with the `-f` flag, and thus caused failures in
certain tests and builds. Note that we only use that flag when
`codesign` does not have the `linker-signed` option, which is for
MacOS versions prior to 12.0.
Example failure with this change when this case occurs (e.g., prior to
the recent clang revert):
```
$ xcodebuild/ReleaseARM64/dart-sdk/bin/dart compile exe -o test test.dart
Info: Compiling with sound null safety
Error: AOT compilation failed
FormatException: The MachO header overlaps with the first 120 bytes of the section contents
```
Change-Id: Ib27db910777f61b90f162f7a0bcfa4ba6592a5a0
Bug: https://github.com/dart-lang/sdk/issues/49783
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256264
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Instead of assuming that linker-signed is a valid option flag for
codesign, only attempt to use it on macOS 11.0 or later, where
signatures on linker-signed binaries need not be force overwritten.
Otherwise, fall back to a regular signature that must be force
overwritten.
Bug: https://github.com/dart-lang/sdk/issues/49010
Cq-Include-Trybots: luci.dart.try:pkg-mac-release-arm64-try,pkg-mac-release-try
Change-Id: Ia2bff2f501dbe37b3269a2a2f16ce9a29026664e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/245000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This CL implements a slightly altered compilation process for `dart
compile` on MacOS. Instead of the traditional approach of
concatenating a dart snapshot with the dart runtime, this new
implementation uses a new MachO segment to store the snapshot. Doing
so ensure users can properly sign the resulting binaries.
The dart runtime has also been updated to look for the snapshots in
this new segment.
There are new unit tests ensuring the dart runtime can correctly
identify MachO-formatted executables and an end-to-end test ensuring
`dart compile` produces code that has the new segment and produces
code that can be signed.
TEST=pkg/dartdev/test/commands/compile_test
Change-Id: Ibdce80455e02bc2b6fd345d80be8d75f989a9c28
Cq-Include-Trybots: luci.dart.try:analyzer-mac-release-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try,vm-kernel-mac-debug-x64-try,vm-kernel-mac-product-x64-try,vm-kernel-mac-release-arm64-try,vm-kernel-mac-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/228080
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
* Add team "groups" in tools/OWNERS_<group name>.
* Add top-level OWNERS as a fallback.
* Add OWNERS for all top-level directories.
* Add OWNERS to all packages.
For additional background information see go/dart-sdk-owners.
TEST=No op until code-owners is enabled.
Bug: b/200915407
Change-Id: I7fe6116cc599c749cd50ca16151d6d6a801d99d7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229147
Reviewed-by: Jonas Termansen <sortie@google.com>
Moritz