- Introduced Dart_SetObfuscationMap to restore obfuscation maps before AOT precompilation.
- Added Dart_AotPatchInstallOptions structure for AOT patch installation options.
- Implemented Dart_AotPatchingEnabled to check if compact AOT patching is supported.
- Created Dart_SetAotPatchKeyCallback for AES key resolution during AOT patch installation.
- Developed Dart_InstallAotPatch for validating and installing encrypted AOT patches.
- Added Dart_FreeAotPatchPayload to free memory allocated for patch payloads.
- Updated runtime_args.gni to include dart_enable_aot_patching flag.
- Added tests for AOT patching functionality and ensured exported symbols include new APIs.
- Refactored existing code to accommodate new AOT patching features and improve error handling.
Put local actions in a common pool, especially
- Assembler (many run for boringssl)
- Linker (high memory use)
- Dart (high memory use)
This should limit the number of concurrent local actions to the number of harts available, avoiding the racing actions triggering out-of-memory failures.
Change-Id: Ifa168012d73472a3db349acc36545dd176385bbc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/510190
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Extend --check-clean (enabled on bots) to verify build commands don't include absolute paths.
The non-RBE GCC and MSVC builds still have absolute paths. GCC doesn't have -ffile-compilation-dir so it uses -fdebug-prefix-map, which is not fixable. The MSVC build has absolute path in the toolchain wrappers, which might be fixable.
TEST=ci
Change-Id: I3b984aaab7aefa7ff527f0a039ca42281224a09d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506505
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
On Mac and Linux, the makes the output of non-RBE, RBE local and RBE remote builds identical, and also independent of the build directories path. On Windows, non-RBE and RBE builds still disagree because paths are rewritten from \ to / to run on the Linux workers.
Bug: b/316893839
Change-Id: I5935785489c73445e4c6ca275020e6cf7464433d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506281
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Switch build to use Clang. This somewhat degrades FFI testing, but reduces build time by roughly half or one hour.
Cq-Include-Trybots: luci.dart.try:pkg-win-release-arm64-try,vm-aot-win-debug-arm64-try,vm-aot-win-release-arm64-try,vm-win-debug-arm64-try,vm-win-release-arm64-try
Change-Id: Ie267942e99934312b228cfadcd6d5b8477f855c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429421
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
This is a reland of commit 7ae1d75869
TEST=manual, no watchOS SDK on bots yet.
Original change's description:
> watchOS target os
>
> - add support for --os=watchos | --os=watchos_simulator
> - do not depend on perfetto if dart_support_perfetto is false
> - change the default value for dart_support_perfetto to !is_watchos
>
> To minimize changes and to avoid modifying third-party zlib BUILD file, is_watchos implies is_ios for now.
>
> TEST=manual, no watchOS SDK on bots yet.
>
> Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
> Change-Id: If2130068ef546162a07a9ba53f94b11ff25fb565
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415021
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
Change-Id: I2b50f7eb4da173627aeb4833b88a08712b278d9f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/425523
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit 7ae1d75869.
Reason for revert: breaks flutter build
Original change's description:
> watchOS target os
>
> - add support for --os=watchos | --os=watchos_simulator
> - do not depend on perfetto if dart_support_perfetto is false
> - change the default value for dart_support_perfetto to !is_watchos
>
> To minimize changes and to avoid modifying third-party zlib BUILD file, is_watchos implies is_ios for now.
>
> TEST=manual, no watchOS SDK on bots yet.
>
> Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
> Change-Id: If2130068ef546162a07a9ba53f94b11ff25fb565
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415021
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: I4bede9ede04ac3c533b8ff7ae2f1adbd84eb2397
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/425262
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Auto-Submit: Ivan Inozemtsev <iinozemtsev@google.com>
- add support for --os=watchos | --os=watchos_simulator
- do not depend on perfetto if dart_support_perfetto is false
- change the default value for dart_support_perfetto to !is_watchos
To minimize changes and to avoid modifying third-party zlib BUILD file, is_watchos implies is_ios for now.
TEST=manual, no watchOS SDK on bots yet.
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try
Change-Id: If2130068ef546162a07a9ba53f94b11ff25fb565
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415021
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Ninja may be run with a large --jobs value to run more C/C++ compilation actions in parallel on remote workers. The dart and gen_snapshot actions during the build should in this case remain limited to the number of cores on the machine to avoid a coincident memory usage that is too high.
When running in the Dart tree with RBE, limit this by changing the RBE exec_strategy away from racing. Otherwise, use a pool to limit the concurrency of this action type.
TEST=build.py --no-rbe -j1000
Bug: https://github.com/flutter/flutter/issues/164591
Change-Id: Ia5e5615d0067b34d49871f26ead074cb891baf70
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423562
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
1. Added `toolchain_suite` template and switched toolchain definitions
to it, so now each toolchain definition defines a regular and
`_shared` toolchain, with different values for `is_shared_library`
argument.
2. Changed the logic of choosing between `fPIC` and `fPIE` based on
`is_shared_library` arg.
3. Changed `shared_library` template to define a group, depending on
the same target in a shared toolchain, and a copy of an output
artifact.
4. Made a copy of `runtime:dart_shared_lib` config and add it
automatically to all shared libraries.
5. Removed `runtime:dart_shared_lib` config and `DART_SHARED_LIB`
define from existing `shared_library` targets.
This CL should be a no-op refactoring to enable
https://dart-review.googlesource.com/c/sdk/+/394102.
TEST=ci
Change-Id: I6c044254d8e74b6b3ddadbf784e58d5f641fbcf3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/392661
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
- `build/mac/find_sdk.py` can search for iPhone and Watch SDKs and their simulators
- `tools/build.py` supports `--os=ios` and `-os=ios_simulator` now. Treating simulator as a separate os to minimize changes and avoid an additional dimension for configs.
- `vm-mac-(release|debug)-arm64-try` tryjobs make sure a shared library builds successfully for ios.
TEST=ci
Change-Id: I76358ec8fd33752260bf0b8462da22a13cd7562e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381623
Auto-Submit: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This is a reland of commit b3253d9c8b
Fix the gn rules so the RBE offloading only happens in the Dart build
and the conflicting RBE configuration is not included otherwise. This
ensures this change can roll into Flutter while we design a better
solution to compose the projects' RBE usages.
Original change's description:
> [infra] RBE all the things.
>
> This change offloads all Dart actions during the build to RBE using
> the new rewrapper_dart.py script that understands all the different
> command line invocations of Dart programs and translates it into the
> appropriate rewrapper invocation, with a full list of input and output
> files and no absolute paths.
>
> The dart actions are all considered to be expensive RBE compilation
> steps that may be much slower on RBE than locally due to the low
> parallelism of the build during these stages and the comperatively
> slower bots in the RBE pools. The build uses the racing strategy by
> default for these compilations, such that cache hits are still used
> if available, and otherwise a local build is transparently used if it
> happens to be faster. The bots will use the remote strategy by default
> unlike developers, such that the rewrapper_dart.py script does not fail.
>
> rewrapper_dart.py contains a tiny dart import proprocessor written
> in python and a big argument parser that understands every command
> invoked during the build and the semantic meaning of every option.
>
> The absolute paths used during the Dart SDK build is not solved in
> this changelist, which just works around them initially, but these
> will be fixed in follow up changes with the appropriate teams now
> that this change proves they are not needed.
>
> Bug: b/333595242
> Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
Bug: b/333595242
Change-Id: I43bec2e7fcad375e972ca447aa13ae65b0ce09ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362661
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
This reverts commit b3253d9c8b.
Reason for revert: Flutter build is broken, it also has a GN variable rbi_dir, and dart_action.gni now imports two conflicting definitions.
Original change's description:
> [infra] RBE all the things.
>
> This change offloads all Dart actions during the build to RBE using
> the new rewrapper_dart.py script that understands all the different
> command line invocations of Dart programs and translates it into the
> appropriate rewrapper invocation, with a full list of input and output
> files and no absolute paths.
>
> The dart actions are all considered to be expensive RBE compilation
> steps that may be much slower on RBE than locally due to the low
> parallelism of the build during these stages and the comperatively
> slower bots in the RBE pools. The build uses the racing strategy by
> default for these compilations, such that cache hits are still used
> if available, and otherwise a local build is transparently used if it
> happens to be faster. The bots will use the remote strategy by default
> unlike developers, such that the rewrapper_dart.py script does not fail.
>
> rewrapper_dart.py contains a tiny dart import proprocessor written
> in python and a big argument parser that understands every command
> invoked during the build and the semantic meaning of every option.
>
> The absolute paths used during the Dart SDK build is not solved in
> this changelist, which just works around them initially, but these
> will be fixed in follow up changes with the appropriate teams now
> that this change proves they are not needed.
>
> Bug: b/333595242
> Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
Bug: b/333595242
Change-Id: I56c9df11e672e7bd096f2a6e7d0243dccd0396c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362385
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
This change offloads all Dart actions during the build to RBE using
the new rewrapper_dart.py script that understands all the different
command line invocations of Dart programs and translates it into the
appropriate rewrapper invocation, with a full list of input and output
files and no absolute paths.
The dart actions are all considered to be expensive RBE compilation
steps that may be much slower on RBE than locally due to the low
parallelism of the build during these stages and the comperatively
slower bots in the RBE pools. The build uses the racing strategy by
default for these compilations, such that cache hits are still used
if available, and otherwise a local build is transparently used if it
happens to be faster. The bots will use the remote strategy by default
unlike developers, such that the rewrapper_dart.py script does not fail.
rewrapper_dart.py contains a tiny dart import proprocessor written
in python and a big argument parser that understands every command
invoked during the build and the semantic meaning of every option.
The absolute paths used during the Dart SDK build is not solved in
this changelist, which just works around them initially, but these
will be fixed in follow up changes with the appropriate teams now
that this change proves they are not needed.
Bug: b/333595242
Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
Automatically detect the appropriate configuration file if RBE=1 is
set to request RBE on all projects, or if DART_RBE=1 is set to request
RBE for Dart only, or otherwise respect the explicit configuration file.
Automatically set the server_address location to the build directory
if it has not already been set. This is unfortunately not supported on
Windows due to the environment variable being set during build but it
needed to be set during gn.
Retain the older configuration files during the transitory period.
Bug: b/296994239
Fixes: b/320876546
Change-Id: I62d1fbfed35248477731cceda3f7267c605c4969
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355400
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Place the windows toolchain in the build root so it can be sent to RBE.
Use relative paths in debug symbols when building with Windows clang,
which is needed for remotely building with RBE.
Set the executable bit when invoking clang-cl remotely as it has not
been set on Windows RBE clients.
Stop including .rst files during build commands as @options-from-file
is not supported by rewrapper at this time.
Forward RBE_cfg into the msvc build environment and rewrite the
INCLUDE, LIB, and LIBPATH environment variables to use relative paths
instead of absolute paths.
Don't download reclient on windows-arm64 where there is no cipd
package and the windows-arm64 python lies and says it's x64 because
they actually packaged a x64 python.
Tune the Windows -j performance to match the historic bot performance
as most casual Windows machines won't handle -j 1000 well.
Bug: b/296994239
Change-Id: I2bd1b4478fca796f0a326c062a5f478f16c2c6ef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/343440
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
The sysroot needs to be passed as a relative path inside the exec root
since RBE will fail to compile with an absolute path.
Bug: b/296994239
Change-Id: Idc0e155368daaf904ac4650bd8668b8955d09e28
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338002
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
This can be controlled via codesigning_identity GN arg.
For example, setting codesigning_identity="-"
would produce ad-hoc signed binaries.
This CL also includes changes in vm/cc tests which are needed
for tests to be green when running with hardened runtime.
Issue https://github.com/dart-lang/sdk/issues/53928
Tested: enabled ad-hoc signing and tested on bots.
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-mac-debug-x64-try
Change-Id: I3c3a6265c62b2904e43a326b7d8223bcfd393577
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333401
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
We were exporting all symbols to the dynamic table so that they could be looked up using `dladdr` for the profiler and backtracer. The symbols include our statically-linked libcxx, which can create trouble when another DSO has a different version of libcxx. Now we export only the VM embedding API functions (`Dart_*`) and use a specially produced table to do the symbolization.
TEST=runtime/tests/vm/dart/exported_symbols_test.dart
TEST=runtime/tests/vm/dart/symbolized_crash_test.dart
Bug: https://github.com/dart-lang/sdk/issues/53267
Change-Id: I2ee494fba86f67127ba0f6f402622f01a4662207
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/323702
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This is a reland of commit b19091385a
Fixed the calculation of the paths of binaryen source files so that they
work properly when building as a dependency (inside the flutter engine
repo).
Original change's description:
> Reland "[dart2wasm] Include wasm-opt in the shipped dart-sdk."
>
> This is a reland of commit 6271d26baf
>
> The change broke the gcc riscv64 builders, due to a `unused-variable`
> warning coming from binaryen sources. That warning has now been
> disabled for binaryen targets.
>
> Original change's description:
> > [dart2wasm] Include wasm-opt in the shipped dart-sdk.
> >
> > Change-Id: Ib224f0b92fa28019ad3cf67d7ba2bef5c31b92ef
> > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280840
> > Reviewed-by: Martin Kustermann <kustermann@google.com>
> > Commit-Queue: Jackson Gardner <jacksongardner@google.com>
> > Reviewed-by: Aske Simon Christensen <askesc@google.com>
>
> Change-Id: I76bf4ad5d1f6a8116631df4b91cbc5a52cac4f31
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283240
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Jackson Gardner <jacksongardner@google.com>
Change-Id: I5cc269daebc5e2068c084bb72b2bd7cb58ba6fbe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283721
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Jackson Gardner <jacksongardner@google.com>
Tony