This is an initial cut of a tool for automatically creating shims for
C/C++ programs using the entry point annotations in Dart code. The tool
has two required arguments:
* the .dill file containing the kernel representation of the Dart code
* the base path of the header and implementation files to create
With a base path of 'dir/name', the header file is created as
'dir/name.h' and the implementation file is created as 'dir/name.cc'.
In addition, 'dir/name.h' is used as a basis for creating a
#ifndef/#define/#endif header guard around the header contents.
The created shims are specific to a single package, either
* a user-specified package, provided via '-p'/'--package', or
* the package of the main method
If the user does not specify a package and there is no main method
in the .dill file, the tool fails.
Each shim takes the following arguments in order when applicable:
* the isolate in which to perform the requested operation,
* the instantiated type of the generic class (for invoking, setting, or
getting constructors, static methods, and static fields)
* the instance (for invoking, getting, or setting instance methods and
fields)
* the type arguments (for retrieving nullable or non-nullable
instantiated types of a generic class)
* the arguments (for invoking, setting, or getting constructors,
methods, and fields)
The generated shims:
* cache the package library, types for non-generic classes, and
types for generic classes instantiated with default type arguments.
The cached persistent handles are cleared if any of the methods
are called with a different isolate from the one used to populate
the cache.
* automatically handle conversions between C int64_t <=> Dart int
and C double <=> Dart double.
By default, shims are not created for allocation or initializing
uninitialized instances. To create such shims, use the '-u' command
line argument.
Currently, shims are not created for methods that take optional
or named arguments. To report an error if a shim cannot be created
for any entry points, use the '-e' command line argument.
TEST=tests/standalone/embedder_samples_test
Change-Id: Ibdf3b52d900ba98038528178485f295c5868ac9d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/410500
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This is a reland of commit 103ffd6041
Parent CL adds "aarch64" as a `uname -m` possibility for ARM64.
TEST=manual testing
Original change's description:
> [pkg/vm] Handle cross compilation in dart_precompiled_runtime2.
>
> When cross compiling, use qemu-<arch> to run dartaotruntime for
> the target architecture.
>
> TEST=manual testing with native X64 and cross-compiling ARM64 on X64.
>
> Change-Id: I6ce4e2cfa3108108fe05c7a1fcd2c400a8a47492
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/404600
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I4407f64e804fbc19d05440ed961a67b5e1c2c047
Cq-Include-Trybots: luci.dart.try:benchmark-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/405761
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This is a reland of commit 11a338000b
Adds "aarch64" as a `uname -m` possibility for ARM64.
TEST=manual testing
Original change's description:
> [pkg/vm] Handle cross compilation in the precompiler2 script.
>
> When cross compiling, gen_snapshot for the host architecture is in
> ${BUILD_DIR}/clang_<arch>/gen_snapshot.
>
> TEST=manual testing with native X64 and cross-compiling ARM64 on X64.
>
> Change-Id: I9f6af045675612651c659683081aca3e65acad1b
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/404421
> Reviewed-by: Slava Egorov <vegorov@google.com>
Change-Id: I4fb4812dfa96ba582ca3bedbfd68d58ce2a4ab5b
Cq-Include-Trybots: luci.dart.try:benchmark-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/405780
Reviewed-by: Slava Egorov <vegorov@google.com>
Revert changes to precompiler2 and dart_precompiled_runtime.
Reason for revert: Golem is broken because host_arch helper does not handle aarch64 as a possibility (which is what we get from uname on these machines).
Reverted changes: /q/submissionid:404600
Change-Id: I88cbaba2a369d6deeca0866ef28dc220ca4bf4d6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/405641
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Revert changes to precompiler2 and dart_precompiled_runtime.
Reason for revert: Golem is broken because host_arch helper does not handle aarch64 as a possibility (which is what we get from uname on these machines).
Reverted changes: /q/submissionid:404600
Change-Id: I4a78ed1e4853de1c12b3341da15c0f068bf2839a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/405640
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
When cross compiling, use qemu-<arch> to run dartaotruntime for
the target architecture.
TEST=manual testing with native X64 and cross-compiling ARM64 on X64.
Change-Id: I6ce4e2cfa3108108fe05c7a1fcd2c400a8a47492
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/404600
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
When cross compiling, gen_snapshot for the host architecture is in
${BUILD_DIR}/clang_<arch>/gen_snapshot.
TEST=manual testing with native X64 and cross-compiling ARM64 on X64.
Change-Id: I9f6af045675612651c659683081aca3e65acad1b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/404421
Reviewed-by: Slava Egorov <vegorov@google.com>
This reverts commit f81a402aa1.
Reason for revert: golem benchmarks are failing to run
TEST=ci
Original change's description:
> "[SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory"
>
> Fixed golem breakage by temporarily copying dartaotruntime to dart_precompiled_runtime
>
> This reverts commit 75e6a748f7.
>
> TEST=ci
>
> Original change's description:
> > Revert "[SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory"
> >
> > This reverts commit 1b331d05c2.
> >
> > Reason for revert: golem builds are failing
> >
> > Original change's description:
> > > [SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory
> > >
> > > TEST=ci
> > >
>
> Change-Id: Id0f383eabb496c06c0acebc639c8e3b056ba82d0
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/393781
> Commit-Queue: Siva Annamalai <asiva@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
Change-Id: Iec494940412aa31dbefdc5280e35ae99e8cecb26
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/393764
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Fixed golem breakage by temporarily copying dartaotruntime to dart_precompiled_runtime
This reverts commit 75e6a748f7.
TEST=ci
Original change's description:
> Revert "[SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory"
>
> This reverts commit 1b331d05c2.
>
> Reason for revert: golem builds are failing
>
> Original change's description:
> > [SDK/VM] - Rename dart_precompiled_runtime to dartaotruntime, ensures we have a uniform name for the executable between the build directories and the SDK directory
> >
> > TEST=ci
> >
Change-Id: Id0f383eabb496c06c0acebc639c8e3b056ba82d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/393781
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When iterating on core library changes or changes in the AOT compiler
many seconds are wasted waiting on gen_kernel/compile_platform to
parse Dart code. This happens because we are running these tools
from sources on prebuilt Dart SDK.
This CL allows SDK developer to opt-in into AOT compiling these
tools by adding `precompile_tools=true` to their DART_GN_ARGS.
AOT compilation is performed using prebuilt SDK - so these
executables do not need to be recompiled if core libraries or
VM changes reducing iteration cycles.
pkg/vm/tool/precompiler2 is tweaked to detect when DART_GN_ARGS
contains `precompile_tools=true` and use precompiled
gen_kernel.exe instead of running it from source.
Using precompiled compile_platform takes vm_platform_strong.dill
build from 20 seconds to 3 seconds.
Using precompiled gen_kernel takes small benchmark build from
~10 seconds to 2 seconds.
This relands 5cda2a871c with fixes
for Flutter build.
TEST=manually tested
Change-Id: I552861c80c152890655e41baaf6ea3fb3b03a57e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367961
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This reverts commit 5cda2a871c.
Reason for revert: broke Flutter build.
Original change's description:
> [tools] Allow precompiling gen_kernel and compile_platform
>
> When iterating on core library changes or changes in the AOT compiler
> many seconds are wasted waiting on gen_kernel/compile_platform to
> parse Dart code. This happens because we are running these tools
> from sources on prebuilt Dart SDK.
>
> This CL allows SDK developer to opt-in into AOT compiling these
> tools by adding `precompile_tools=true` to their DART_GN_ARGS.
>
> AOT compilation is performed using prebuilt SDK - so these
> executables do not need to be recompiled if core libraries or
> VM changes reducing iteration cycles.
>
> pkg/vm/tool/precompiler2 is tweaked to detect when DART_GN_ARGS
> contains `precompile_tools=true` and use precompiled
> gen_kernel.exe instead of running it from source.
>
> Using precompiled compile_platform takes vm_platform_strong.dill
> build from 20 seconds to 3 seconds.
>
> Using precompiled gen_kernel takes small benchmark build from
> ~10 seconds to 2 seconds.
>
> TEST=manually tested
>
> Change-Id: Ieec6ad4e1081023d140eb744f0a3cd0c754414ca
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367940
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Id3e4eb44d33516f31c165d9a1e55911e8d356e7f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367960
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
When iterating on core library changes or changes in the AOT compiler
many seconds are wasted waiting on gen_kernel/compile_platform to
parse Dart code. This happens because we are running these tools
from sources on prebuilt Dart SDK.
This CL allows SDK developer to opt-in into AOT compiling these
tools by adding `precompile_tools=true` to their DART_GN_ARGS.
AOT compilation is performed using prebuilt SDK - so these
executables do not need to be recompiled if core libraries or
VM changes reducing iteration cycles.
pkg/vm/tool/precompiler2 is tweaked to detect when DART_GN_ARGS
contains `precompile_tools=true` and use precompiled
gen_kernel.exe instead of running it from source.
Using precompiled compile_platform takes vm_platform_strong.dill
build from 20 seconds to 3 seconds.
Using precompiled gen_kernel takes small benchmark build from
~10 seconds to 2 seconds.
TEST=manually tested
Change-Id: Ieec6ad4e1081023d140eb744f0a3cd0c754414ca
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367940
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Additional flags can be passed to the kernel compilation step via the
DART_VM_FLAGS environment variable, but there's no way to specify
additional flags to be passed to the gen_snapshot step. This CL changes
that by also passing the contents of the GEN_SNAPSHOT_FLAGS environment
variable to that step, so the tool can be invoked like:
DART_CONFIGURATION=ReleaseX64 \
GEN_SNAPSHOT_FLAGS="--redirect-isolate-log-to=tmp/graphs.log \
--print-flow-graph-optimized" \
pkg/vm/tool/precompiler2 ...
TEST=manual use
Change-Id: Iadc6a888c60ec306799885e49a39568b124a1c10
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/345660
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Functions returning non-nullable records of 2 fields can now return
them as separate values on 2 registers. This postpones and sometimes
eliminates creation of record instances. As other unboxing
optimizations of return values, this optimization is only
performed in AOT.
Pair of values for an unboxed record are represented using
kPairOfTagged representation, which can be now used for the input of
Return and for the output of StaticCall/InstanceCall
PolymorphicInstanceCall/DispatchTableCall instructions.
In order to combine separate values for Return, a new MakePair
instruction is added. Extracting separate values from the result of
a call is implemented using existing ExtractNthOutput instruction.
Benchmarks (AOT mode):
MultipleReturns.Forwarded.Record +45-57%
MultipleReturns.Forwarded.RecordNamed +43-57%
MultipleReturns.NotInlined.Record +58-79%
MultipleReturns.NotInlined.RecordNamed +53-76%
TEST=runtime/tests/vm/dart/records_return_value_unboxing_il_test.dart
TEST=benchmarks/MultipleReturns/dart/MultipleReturns.dart
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I7117b19a134c1db0ba5117a1ef093867f9ba20e2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/269100
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
* Add support for dumping flow graphs in JSON (to make it machine
readable);
* Add support for controlling which functions are dumped through
a pragma annotation:
`@pragma('vm:testing:print-flow-graph'[, "passes filter"])`
* Replace simple matching DSL with programmatic matching
* Support obfuscated builds
Fixes https://github.com/dart-lang/sdk/issues/47340
TEST=vm/dart{,_2}/aot_prefer_equality_comparison_il_test
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ie067ba451d311e6019a8c3a88c012231b0c50eb5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/219240
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Our current unit testing infrastructure does not make it possible to
test AOT compilation pipeline end-to-end, because it does not run TFA
when generating Kernel.
This makes it challenging to write regression tests for certain issues.
Instead we extend test runner with a capability to perform IL matching
when running AOT tests.
runtime/docs/infa/il_tests.md provides details on how to write such tests.
TEST=manually
Change-Id: I6f5220b814f4a5d8c053efacd3711df495dea404
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/214961
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This change introduces handling of protobufs while doing type flow
analysis. Metadata in protobuf message classes is updated dynamically
according to the set of called accessors, invalidating and rebuilding
TFA summaries as needed.
Previously, protobuf-aware tree shaker required the 2nd run of TFA
in order to do the actual tree-shaking after protobuf messages are
pruned. This significantly increases compilation time.
AOT compilation time of a large app (--from-dill): 274s -> 152s
New tree shaker is available in kernel compilers under the flag
--protobuf-tree-shaker-v2.
Issue https://github.com/dart-lang/sdk/issues/42442
Fixes https://github.com/dart-lang/sdk/issues/40785
Change-Id: I4347896737b9b0f7407b845e614dda9ba7621921
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152100
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
New option can be used instead of --aot to produce a tree-shaken
minimal .dill file which can run on the VM in JIT mode.
Can be used along with --no-link-platform and --gen-bytecode to further
decrease size of the .dill file.
Change-Id: I35e8d81263a5e223decf160d79322c06cfd27321
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140740
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Code references other objects via object pools. In bare instructions mode
individual objects pool are merged together into a global object pool,
which means that we no longer can clearly see which objects are
referenced by each individual code object.
This CL adds logic into snapshot writer to include these implicit
references into the snapshot profile. We achieve this by keeping
individual object pools around (but avoiding writing them into the
snapshot itself).
R=sjindel@google.com
Change-Id: I35b48fcfc26c0a8c0b66593139fa7ac8e627da6c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136626
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
Alexander Markov