As number of bytecode generation options grows, it becomes cumbersome
to add them and propagate from the place where they are parsed to
the place where they are used. In order to make it easier to
change and add new bytecode generation options, this CL introduces
BytecodeOptions class which consolidates all options for bytecode
generation. Also, command line options --emit-bytecode-*** are gathered
into a single multi-option --bytecode-options=opt1,opt2,...
Also, unused --use-future-bytecode-format option is cleaned up. If needed,
it could be easily re-introduced in the new BytecodeOptions.
Change-Id: I637bf28ceb4233ead2562afe7ad51c69a99f2d60
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106965
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Size of a large application:
Before: 26628600
After: 21480120 (-19.3%)
Size of snapshots:
isolate_snapshot_framework.bin
Before: 9322496
After: 6782976 (-27.2%)
isolate_snapshot_product_framework.bin
Before: 9166848
After: 6602752 (-27.9%)
Regressions in tests:
1) Test language_2/type_alias_equality_test/04 fails similarly to default mode, as VM
does not implement comparison of function types according to the specification.
Previously this test was passing as function types were canonicalized in bytecode,
which was not always correct. This CL fixes the problem with canonicalization of
function types in bytecode and the test starts failing again.
2) Tests standalone_2/entrypoints_verification_test, standalone_2/io/test_extension_test,
standalone_2/io/test_extension_fail_test fail as native extensions are not supported
in bytecode yet. These tests start passing after df5e7aac17,
which switched bytecode tests to kernel service (on x64), because kernel service doesn't
drop ASTs. This CL switches from reading AST library declarations to bytecode even if
AST is not removed, so tests fail again.
Change-Id: I8b7ba44bfa49d0b1599b2509553ff7c831a4e244
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104700
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Pkg/vm tests were adjusted after constant evaluation was enabled in the front-end.
As constant evaluation was reverted (4b7d256a06),
we also need to revert corresponding changes in pkg/vm unit tests.
Revert "[vm/bytecode] Update expectations for bytecode generator unit test"
This reverts commit 734c50cbf2.
Revert "[vm/aot/tfa] Update TFA tests after front-end starts evaluating constants"
This reverts commit 25cb45c02f.
Change-Id: I2fcc30571bec2dd904925724ddb7105567c7ecda
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99782
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Usually TFA takes kernel AST with all constants evaluated.
However, in order to avoid calling constant evaluator pass in TFA unit tests
there is a special implementation of pragma annotation parser (for testing)
which accepts annotations which are not ConstantExpressions.
After CFE starts evaluating all constants (constant-update-2018 was enabled in
e9ca6a5335), TFA unit tests need be
switched to use ConstantPragmaAnnotationParser.
Also, empty environment defines are passed to CFE to avoid unevaluated constants
and unit test expectations are updated.
Closes https://github.com/dart-lang/sdk/issues/36602
Change-Id: I2e7b0647e065cca129449b4531c6278c1d98061d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99269
Reviewed-by: Samir Jindel <sjindel@google.com>
This CL includes:
* A dart script that - given a flutter checkout - finds and compiles
tests to ensure we actually can.
* A shell script that checks out flutter, compiles the platform
(to cope with any changes to the kernel format) and runs the above
script.
Change-Id: I02806a375943361c3c111d9aa12a76d745391ca3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98140
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Previously, a valid web socket connection would use the following URI:
`ws://127.0.0.1/ws`
Now, by default, the VM service requires a connection to be made with a
URI similar to the following:
`ws://127.0.0.1:8181/Ug_U0QVsqFs=/ws`
where `Ug_U0QVsqFs` is an authentication code generated and shared by the
service.
This behavior can be disabled with the `--disable-service-auth-codes`
flag.
Change-Id: I288aac58e3ba9d35dca8071f3f7e7a073896c271
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98433
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
The following analyzer hints are fixed:
hint • This function has a return type of 'String', but doesn't end with a return statement at pkg/vm/bin/compare_sizes.dart:140:3 • missing_return
hint • This function has a return type of 'String', but doesn't end with a return statement at pkg/vm/bin/compare_sizes.dart:198:3 • missing_return
Change-Id: Ia34006eb023c0b3f9b60f8f38dc248cfa1c87a30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98983
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
By indexing all sources in a concatenated dill file and then using that
index to find the sources and line starts we can fix missing information
(leading to crashes when collecting coverage) in for instance circular
instances.
This reverts commit 57321c1590 and adds a
fix.
Change-Id: I7a91801fb318cad0218cf3101d6a15f1ec929175
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98006
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This reverts commit a7ab99dd84.
Reason for revert: <INSERT REASONING HERE>
Original change's description:
> [vm/kernel] Index all source and line starts before using them
>
> By indexing all sources in a concatenated dill file and then using that
> index to find the sources and line starts we can fix missing information
> (leading to crashes when collecting coverage) in for instance circular
> instances.
>
> Change-Id: I454cff57f84f21de72f6486e1cecf15d1389a498
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96901
> Commit-Queue: Jens Johansen <jensj@google.com>
> Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
TBR=vegorov@google.com,jensj@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Iea75c9dd56e74fa0e98c90ea6841ed66b39d40cc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97941
Reviewed-by: Keerti Parthasarathy <keertip@google.com>
Commit-Queue: Keerti Parthasarathy <keertip@google.com>
This change replaces kernel AST declarations of fields and functions
with bytecode declarations.
Size of dilp files is reduced by 11-12%.
Startup latency:
Time to the first full frame: 1.945s -> 1.687s
FinalizeClass: 554ms -> 277ms
FinishClassLoading: 296ms -> 156ms
There are following regressions in bytecode mode, which will be fixed
in future:
* dart:mirrors are not supported yet (implementation of mirrors relies
on reading kernel AST in certain cases).
As the result, lib_2/mirrors/* tests fail.
* native extensions are not supported yet (annotations on libraries
and classes in AST are cleaned up as they could reference members
which are now removed from AST).
As the result, standalone_2/entrypoints_verification_test test fails.
* language_2/spread_collections/const_error_test/* tests fail
due to https://github.com/dart-lang/sdk/issues/36286.
Change-Id: I5130f401fd7b84038b136136e7ccc1a6e51b6cea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97561
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
By indexing all sources in a concatenated dill file and then using that
index to find the sources and line starts we can fix missing information
(leading to crashes when collecting coverage) in for instance circular
instances.
Change-Id: I454cff57f84f21de72f6486e1cecf15d1389a498
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96901
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This CL adds a test where:
* Two libraries depend on each other and both have mixins where
the body comes from the other file.
* The two-library component is then split into two one-library
components and loaded by the VM via the "multiple kernels"
functionality (basically as a concatenated dill file).
(Notice, that each individual component has a reference
(but no line starts or source), to the other library in its
uriToSource table. This currently causes the VM to let (at least)
one script have an empty source and and an empty line starts table.)
* The test then checks that it via the service protocol can find
a) a non-null, non-empty "tokenPosTable" for each script,
b) a translation between token positions and line-column via the
tokenPosTable for each token position the VM specifies as a
hit, a miss etc.
Said another way, it basically checks if we can collect coverage.
The test is currently skipped because it fails and we can't -
to my knowledge - mark it as such.
Change-Id: I76a049bffdd554755372a19235bf0c270bad3764
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96103
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This CL removes the sources from uriToSource for libraries
the incremental compiler doesn't keep around.
This includes:
* Explicitly invalidated libraries, also if in a package.
* Transitively invalidated libraries, also if in a package.
* Libraries that are now no longer referenced in the resulting whole
program, unless it's in a package.
* Libraries from packages that are implicitly invalidated by an
updated .packages file (e.g. if the .packages file no longer
reference it or reference a new version of it).
This does *NOT* include:
* Libraries from packages that are not invalidated and that are still
mentioned in .packages, even if the actual libraries are no longer
referenced in the resulting whole program. The reason is that these
libraries are kept around in the incremental compiler.
Bug: 36197
Change-Id: I0ed41567ab54828585326ffd4dcb3722980bf874
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97201
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
If main script is specified as a 'package:' URI, kernel compiler
should not try to convert it to 'package:' URI using --packages file.
Bug: FL-186
Change-Id: I65e8825470ca08b5dd1ae5bd30cf4f5699d6d03c
Reviewed-on: https://dart-review.googlesource.com/c/93365
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
* Type arguments are moved to global object table (and de-duplicated).
* Finalize non-recursive generic types at bytecode generation time.
* Constants are moved to global object table (and de-duplicated).
* ICData creation is avoided for direct calls.
* Interface calls reference target member instead of selector name.
Size of a dilp files: 17868K => 15896K (-11%).
Change-Id: I0c9bf338137a0fae6ed90ab6b125ed2b24a1a8ad
Reviewed-on: https://dart-review.googlesource.com/c/91108
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Validating the class hierarchy requires subtype tests. Previously,
Fasta has worked around this by computing the hierarchy twice, as
well as visiting the hierarchy several times. In order to improve
performance of modular compilation, we're working on changing this.
To enable creating a TypeEnvironment without a hierarchy, we're
removing the field from the API.
Change-Id: I7c9f88d5c6495aaa8223eaeff7d2dfd5669e9e95
Reviewed-on: https://dart-review.googlesource.com/c/88823
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- Also allow using the unchecked entry-point for invocations of generic functions where
there are no bounds or the bounds don't require dynamic checks.
Change-Id: I6ca1ebec777ecf2989c4fb77425d65d542d5adf2
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/87181
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
In Kernel's TargetFlags, this flag still defaulted to false. Flip the
default to true and update all client code in the SDK. The
expectation is that many of the places that now pass false explicitly
really just want the default, but that will be verified separately and
then the flag will be removed.
Change-Id: I2a38eb53f280f21f59bb1d2e88c42516f827fd39
Reviewed-on: https://dart-review.googlesource.com/c/85448
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Kevin Millikin <kmillikin@google.com>
This is a major revamp of bytecode metadata format. Now bytecode has
its own serialization mechanisms.
This CL adds 'bytecode component' metadata, which contains bytecode
object table and string table. All references from bytecode (constant
pools) to libraries, classes, members, types and strings now have a new
format. References to frequently used objects are represented as indices
in object table, while rarely used objects are written inline.
This allows VM to cache frequently used objects while reading bytecode.
Representation of strings is aligned with VM - string characters are
stored in separate pools of one-byte and two-byte strings. This allows
VM to avoid UTF-8 decoding and extra copying.
Closure declarations are now explicit. Type parameters no longer require
enslosing scopes when reading/writing them.
Benchmarks:
GenKernelKernelReadAllBytecode (Intel Core i5) +29.84%
GenKernelKernelReadAllBytecode (Intel Xeon) +28.74%
Change-Id: I4b80009733a8f8c038264af74f97c4e094b9e311
Reviewed-on: https://dart-review.googlesource.com/c/85469
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
'package:'-scheme uris are important with compiler enforcing unique identification of files present in packages(that can be referenced either via 'package:'-uri or via 'file:'-uri).
Change-Id: Ib3006dadbdd1dfc5fe1df800cae3f95bcbb37063
Reviewed-on: https://dart-review.googlesource.com/c/80361
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
--unsafe-package-serialization saves the frontend_server from having to
re-serialize everything all the time.
Before this CL, though, things could go wrong:
If you had a situation where you had previously compiled a file that
depend on a package A, and where (some of) package A depends on
package B this would happen:
- All of package A was serialized together.
- All of package B was serialized together.
When later, you compile something that depend less on package A - namely
on only on parts that does not depend on package B, but is included in
the previously serialized package A, the following would happen:
- The new (non-package) libraries would be serialized.
- Package A serialization would be reused.
This is basically fine: Running the app would be fine, everything it
actually depend on is there.
However, if the VM is forced to compile everything it now also compiles
stuff that was included - but really unused - from package A - namely
also the libraries that depend on package B --- which is not included.
This CL changes the last part by also including package B.
The result is that even more unused libraries are included, but that
the VM can survive a forceful compile.
Note that the --unsafe-package-serialization is only used for tests,
so the "including even more unused" part is probably not a big deal,
and all in all there's still a big speed advantage to doing this.
Change-Id: Iac06ba6f40c2caaacce641c5853e9491496dce53
Reviewed-on: https://dart-review.googlesource.com/c/78541
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Also, convert package:vm to use onDiagnostic.
We're consolidating all dependencies on package:front_end in one file per tool.
The idea is, if you want to depend on something in the package:front_end, you modify pkg/front_end/lib/src/api_unstable/vm.dart and consult with the front-end team.
These consolidated files will help us when designing a public API in the future.
Change-Id: I66bafdd0ae29605fd12f8d6c589dbc761e5c7a97
Reviewed-on: https://dart-review.googlesource.com/c/77581
Auto-Submit: Peter von der Ahé <ahe@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Peter von der Ahé <ahe@google.com>
Entry points JSON files are deprecated (replaced with pragmas).
So TFA should not require them. As a sanity check, it will complain
if an entry points JSON file is not empty.
Change-Id: Ie64121ab4cdd0c330f81a8cb71a671e1df82f136
Reviewed-on: https://dart-review.googlesource.com/75423
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This is required for tracking type arguments, since field initializers can
reference type variables from the enclosing class.
Also a few drive-by spelling fixes.
Change-Id: I91e33f7d8c61e288ccb2d8e18b33f4e67ac22a1b
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-release-x64-try, vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/74666
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
We need to block interrupts while evaluating pragmas to prevert reentrant class finalization.
Original revision is in patchset 0.
Change-Id: I872cec4eaf4ca85567c9657c458ed39c8b2e30de
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-release-x64-try, vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/73160
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Alexander Markov