During constant evaluation, unused arguments to a const constructor are
thrown away after evaluation, since their values do not affect the
resulting instance constant. If such an unused argument ends up
unevaluated, any errors that would arise in the final evaluation are
not reported.
This CL adds space in the Kernel AST for saving these unevaluated
expressions so they can be checked during final constant evaluation.
Even though this is an incompatible change, no update is needed to the
VM code (except for the version bump), since the VM does not support
InstanceCreation nodes in the first place.
Change-Id: I4752562c1164efbba79eb018c15b07ed8354ce5f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105761
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Prior to this CL, we would issue a warning whenever a CanonicalNameError
was encountered.
This is in principal a good thing, but because we currently have no way
to detect if the sdk we get is the one we expect
(by any other measure than when it issues a CanonicalNameError) we often
issue these warnings for no "real reason" whenever, for instance,
the flutter sdk changes.
This CL splits the CanonicalNameError in two such that errors with
references to the sdk ("dart:" libraries) issue CanonicalNameSdkError
instead, an we then handle that differently. Namely we silently ignore
the error (i.e. don't issue a warning) and just don't initialize from
dill.
This should remedy the situation and be strictly better than to always
swallow CanonicalNameErrors.
Bug: 36032
Change-Id: Idbae0b5ee5b9843a5dbeb49b3c65ae25f5962e36
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105240
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
TL;DR: Unbind canonical names doesn't do what you think it does and
probably shouldn't (ever) be used. This CL stops using it in a few places.
Longer version:
When loading a dill file it:
- First loads the table of canonical names. These have no references yet.
- When a canonical name is asked for its reference it creates one if it
doesn't yet have one.
- When loading, for instance, a library, it asks for the reference.
When unbinding a canonical name:
- It removes itself (the canonical name) from the reference
- It removes the reference in itself
- Note: Whatever has a pointer to the reference keeps it, and the
reference points to whatever node it already pointed to.
This also means, that if we have a dill file that's split in two and:
- Load #1
- Load #2
that works fine, but if we
- Load #1
- Unbind canonical names
- Binds canonical names
- Load #2
stuff is not bound correctly (and an error is thrown).
And - the cause of this bug:
- Load #1
- Load #2
everything is fine
- Unbind canonical names
- Binds canonical names
- Load #2'
stuff is not bound correctly --- references points to stuff loaded as #2,
not as #2'. On top of being weird, wrong and confusing it also caused wrong
things to be but into the class hierarchy which ultimatly caused the crash.
This CL fixes it by not calling unbind and force loading of dill files
(at specific call sites) to create new libraries
(and in the process overwriting references ".node").
Revert "[dartdevc] Retry ddc incremental compile on crash"
This reverts commit ecdbdf00b8.
Revert "[kernel_worker] retry on failure"
This reverts commit 43eebea5a3.
Fixes#36644
Bug: #36644
Change-Id: Id8f548179e6a409b01f2ebfa3219f94cb64b1c05
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100380
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jenny Messerly <jmesserly@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This is used in constant evaluation to represent const constructor
invocations with unevaluated field values or asserts.
Change-Id: I1d2d60a18f967a4dd195b3b5895db9a126c47803
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98561
Reviewed-by: Kevin Millikin <kmillikin@google.com>
These arise when the constant evaluator partially evaluates collections
containing spreads or control-flow constructs with unevaluated
subexpressions. They are removed by the final constant evaluation.
Change-Id: Icdd155c4805cbcefe6aa4b45c2f85ec258e7bd36
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/95760
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Having a block as body instead of a list of expressions simplifies
code handling block expressions. Since it is always a block, it can
be left untagged in the serialized representation so the serialized
representation is not changed with this change.
Change-Id: I3c34f663041ff8cb4a530c2d6a148343e59c68fe
Reviewed-on: https://dart-review.googlesource.com/c/94223
Commit-Queue: Kevin Millikin <kmillikin@google.com>
Commit-Queue: Dmitry Stefantsov <dmitryas@google.com>
Auto-Submit: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Fasta can generate synthetic libraries.
Previously we didn't include them when serializing, but that leaves
references in the binary to libraries that doesn't exist which isn't
ideal.
This change includes them, and adds a flag to kernel Libraries such that
we know they are synthetic.
Change-Id: Ied25a21cd1f384d318347021bc7ec18dae3a4e05
Reviewed-on: https://dart-review.googlesource.com/c/91722
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
Add Kernel syntactic support for constants constructed by
bool.fromEnvironment, int.fromEnvironment, and String.fromEnvironment.
These values are not necessarily available at compile time. Because
constants can depend on these values, there is also an unevaluated constant
that represents an expression depending on environment values.
This syntax is not yet produced by the Fasta compiler.
Change-Id: Ie96ea7f60a7efcd35ac802b320a00f398d41232e
Reviewed-on: https://dart-review.googlesource.com/c/88827
Commit-Queue: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
References can be null --- it's written as the byte [0].
Thus in the null case it would before be [0] and now be [0].
In the non-null case it would before be [1, xyz] (1 for Tag.Something
and xyz for the actual, now positive, uint30). Now it would be [xyz].
Change-Id: Ibc08d3afb7275b0429a4d6c5e667fbd381121489
Reviewed-on: https://dart-review.googlesource.com/c/85394
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This is a major revamp of bytecode metadata format. Now bytecode has
its own serialization mechanisms.
This CL adds 'bytecode component' metadata, which contains bytecode
object table and string table. All references from bytecode (constant
pools) to libraries, classes, members, types and strings now have a new
format. References to frequently used objects are represented as indices
in object table, while rarely used objects are written inline.
This allows VM to cache frequently used objects while reading bytecode.
Representation of strings is aligned with VM - string characters are
stored in separate pools of one-byte and two-byte strings. This allows
VM to avoid UTF-8 decoding and extra copying.
Closure declarations are now explicit. Type parameters no longer require
enslosing scopes when reading/writing them.
Benchmarks:
GenKernelKernelReadAllBytecode (Intel Core i5) +29.84%
GenKernelKernelReadAllBytecode (Intel Xeon) +28.74%
Change-Id: I4b80009733a8f8c038264af74f97c4e094b9e311
Reviewed-on: https://dart-review.googlesource.com/c/85469
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
For all other "debugPath.add"s we've wrapped it in assert to avoid
doing it in the general case.
One apparently have slipped through.
Change-Id: Ib0dfc4a671428993e393d57a75db6612785e37b4
Reviewed-on: https://dart-review.googlesource.com/c/85161
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Before this CL we could get a dill file out of compiling
tests/language_2/import_nonexisting_dart_uri_test.dart
but we couldn't load it back in again.
Change-Id: I05b6b8d6368c7ed03dbb798535ea03e0493df780
Reviewed-on: https://dart-review.googlesource.com/c/79220
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Previously, we only stored a reference to the originating Typedef declaration
and is thus loosing the originating type arguments. We need these type arguments
in various places, including diagnostics and bounds checks.
Note: this CL maintains backwards compatibility wrt to the Dart API of
FunctionType. I'll update the API in a follow-up CL.
Change-Id: I896ce3c3b4522d542a82c5036f8a17ee098143dc
Reviewed-on: https://dart-review.googlesource.com/c/81367
Commit-Queue: Peter von der Ahé <ahe@google.com>
Auto-Submit: Peter von der Ahé <ahe@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
* Ensure that we have a .dill file and it has the correct version before
we start decoding it in dump.dart.
* Throw errors, not strings. Ensure that they have a useful
toString().
* Do not print usage except when the command is invoked
incorrectly (wrong number of arguments). Once we get into the
command it's less likely that it's been invoked incorrectly and more
likely that something else has gone wrong.
Because these utilities are invoked from other scripts (like the
fasta command), printing their usage for problems other than
invoking them correctly doesn't match the way that they were invoked
and it's confusing.
Change-Id: I7832383594d2b3719a0a7a7392ba4685717a79d2
Reviewed-on: https://dart-review.googlesource.com/c/78206
Commit-Queue: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
The canonicalization of symbols needs to take library privacy into
account. Since the Symbol class itself only has a [_name] field but does
not reference in which library the symbol came from, the [_name] must be
mangled.
Mangling is done by backends and so we make a new [SymbolConstant] which
the backends can desugar by doing appropriate mangling and construction
of a [Symbol] instance.
Fixes https://github.com/dart-lang/sdk/issues/34396
Change-Id: I2e13288483c35f875d39eefdb73677b2cc03527a
Reviewed-on: https://dart-review.googlesource.com/74360
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The canonicalization of symbols needs to take library privacy into
account. Since the Symbol class itself only has a [_name] field but does
not reference in which library the symbol came from, the [_name] must be
mangled.
Mangling is done by backends and so we make a new [SymbolConstant] which
the backends can desugar by doing appropriate mangling and construction
of a [Symbol] instance.
Fixes https://github.com/dart-lang/sdk/issues/34396
Change-Id: I5ddb5331ce79a0b942807929d4b8f1050a9899e7
Reviewed-on: https://dart-review.googlesource.com/73883
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Use positions already available in for instance AsExpressions,
also adds new offsets in the kernel format to
SuperInitializer and RedirectingInitializer.
Bug: #33216
Change-Id: I542967ddc6ec782e6513d62fce038a49239e1622
Reviewed-on: https://dart-review.googlesource.com/63883
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Prior to this change we only had fileOffset which points to the name.
This CL adds a field - startFileOffset - that points to the start
(e.g. 'static', 'factory', the return type or any presiding annotation
(including '@')), as well as fixing `Function::GetSource` in the VM to
fix a crash when using mirrors to ask for the source of a function or
constructor.
The field is added to procedures, constructors and classes.
The latter changes the location of classes with annotations to be
the position of the annotation. This is on-par with --no-preview-dart-2.
The change in regards to procedures and constructors mean, that both
the location and the source (from .location and .source on a
MethodMirror) will change to include any annotations. This is a
different behavior than --no-preview-dart-2.
Closes#33271.
Change-Id: I90f1232c5ec2d01e60e0bab070d44c37232b2730
Reviewed-on: https://dart-review.googlesource.com/60560
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
This CL adds an option to the dill loader (i.e. BinaryBuilder) to
enable performing a check of all canonical names. The theory is,
that if a dill is self-contained "every" (1) canonical name should
contain a reference to a node.
This check is (currently) disabled by default, but enabled
excplicitly in the incremental compiler when initializing from dill.
If the check fails a warning will be issued (and the dill wont be
used).
On my machine the check seems to take ~15 ms in all tested cases
(sdk, flutter test).
(1): Except for extra layer of URI of a library if the qualified
name is private.
Closes#32449.
Change-Id: I65c8fba647be3d5ba47f7ee16caef78cb2cfae07
Reviewed-on: https://dart-review.googlesource.com/51462
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Kernel mixin transformation desugars mixin applications into normal
classes. Mixed-in type is pulled into interfaces list.
However, dart:mirrors needs to know the original mixed-in type of
a mixin application.
This change solves this problem by propagating a 'isTransformedMixinApplication'
attribute of a class through kernel AST, kernel binary and VM objects
into dart:mirrors implementation.
Fixes: https://github.com/dart-lang/sdk/issues/33240
Change-Id: I98ca69294e1ad445402a5ca91d90c30447aabcb2
Reviewed-on: https://dart-review.googlesource.com/56721
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Summary:
The constants table contains a concatenation of the variable-length encodings of
each constant used in the serialized component. Previously, we referenced the
constants in the code by index into the constants table, which requires reading
potentially the entire constants table to dereference. Now, we refer to the
constant by the binary offset into the constants table, which allows us to peek
into it in constant time.
Test Plan:
The constants table is used by all the precompiler modes, so the new encoding
will inherit the coverage of the existing encoding.
Results from debug and release precompiler bots are available under the
"cl-linux-try" button.
Change-Id: I4782bc0035d0b0cbd5bd6b9d086561ea601286c8
Reviewed-on: https://dart-review.googlesource.com/55470
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL updates the frontend_server to allow specifying what dill
file to initialize from.
Normally, it tries to initialize from the same dill file as is the
output. This is fine in many cases (e.g. flutter run always says it
should output to build/app.dill): The first time it compiles
everything, subsequent times (across restarts) it only recompiles
what's needed.
When running tests, however, the output is in a temporary directory,
so it doesn't work across restarts. The startup time for tests is
always rather high, because it always has to recompile everything.
This CL updates the frontend_server to allow specifying what dill
file to initialize from. Flutter can thus save the compiled test
file in a centralized location (e.g. build/test.dill) and load from
that across restarts.
Plumbing this into flutter yields the following result, when running
a single test (the one automatically created when creating a new
flutter project):
Before: ~5.5 seconds
Now, first run: ~5.5 second (unchanged)
Now, subsequent runs: ~3.4 seconds
Approximate breakdown:
Startup cost (e.g. if there's no test directory): ~ 800 ms
Starting up the actual frontend server: ~150 ms
Reading source from old dill, loading all relevant files
and invalidating source: ~ 300 ms
Compiling everything (or, rather, nothing as it's all
from the dill at this point): ~ 650 ms
Serializing: ~ 250 ms
Copying files: ~ 50 ms
Running the actual test: ~1100 ms
Total: ~3300 ms
`time flutter test --local-engine=host_debug --preview-dart-2`
says ~3.4 seconds...
To compare, --no-preview-dart-2 takes ~2.2 seconds,
so dart2 still has a (significantly) higher startup cost.
Moves the needle on https://github.com/flutter/flutter/issues/15767.
Change-Id: I36a2d05bc76e0193d369df42eb3b9f08416dd78f
Reviewed-on: https://dart-review.googlesource.com/51820
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Metadata is no longer written ahead of all nodes. Instead, metadata for
each node is written in the same context as the node itself (into a separate
buffer). This allows metadata to contain (serialize) arbitrary nodes
(for example, arbitrary DartTypes) and use serialization context of parent
nodes (such as declared type parameters).
However, with this change metadata looses the ability to reference
arbitrary AST nodes. This ability was overly restricted and had no
practical uses. (It was not possible to reference nodes which are not
reachable from root Component. As a consequence, it was not possible to
write references to arbitrary DartTypes.)
This change aligns the serialization capabilities of metadata with
how kernel AST nodes are serialized.
Change-Id: I027299a33b599b62572eccd4aa7083ad1dd2b3b3
Reviewed-on: https://dart-review.googlesource.com/54481
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Second try. First try was reverted because a previously empty URL (the
default 'always there' one) is now a null url instead, which wasn't
handled properly in the frontend_server.
See 47e9039512 for original details.
This reverts commit 4c9b712052.
Change-Id: I717eb927cbe29b6388e72f7e270428d234d44e4d
Reviewed-on: https://dart-review.googlesource.com/53580
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
This is the second try. First commit failed because another reading of
the binary was added after the change was originally made and before
actually landing it (and wasn't noticed when rebasing).
This reverts commit b298fc6d8f.
See 6e2536f585 for more information.
Change-Id: Ia0e7f8921de2cec8088654fa24950df13d846237
Reviewed-on: https://dart-review.googlesource.com/53560
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Change kernel serialization so that
a) We don't UTF8 encode strings up front, but only as needed
(if nothing else it makes it more obvious where the cost of stuff is)
b) Don't call toString on URIs to save them in a StringIndexer, only to
parse the string later to get a URI back again.
This shaves a bit of the serialization time.
Timings from running `flutter test` in flutter/packages/flutter:
Without change:
```
02:14 +2438 ~18: All tests passed!
02:12 +2438 ~18: All tests passed!
02:14 +2438 ~18: All tests passed!
```
with change:
```
02:10 +2438 ~18: All tests passed!
02:11 +2438 ~18: All tests passed!
02:09 +2438 ~18: All tests passed!
```
=> -2.5% +/- 1.83615%
Change-Id: I03e651bbef23445fb1901452a4efcc732789a168
Reviewed-on: https://dart-review.googlesource.com/51302
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Previously doubles was saved as strings in the string table,
with a DoubleLiteral holding a StringReference.
This can cause overhead in both computation time
(converting the double to and from string) as well as size
(e.g. a single usage of the previously unused double 1000000.42
would use (at least)
* 10 bytes for the characters
* 1 byte for the size
* 1 byte for the reference to the string
whereas saving it as a double would simply save the 8 bytes.
On the other hand the string table doesn't contain duplicates so
many usages of the same double will use more space.
The SDK dill file size decreases slightly (< 1 KB).
On a Dart file with 1M different doubles (0.42, 1.42, ..., 999999.42)
added to a list:
Before:
compile and write via fasta (non-strong-mode): 0:12.18
Reading (via dart, eager): 2500-2600 ms
Writing (to null sink) (after reading): 1600-1800 ms
Output dill file (via fasta compile): ~62 MB
After:
compile and write via fasta (non-strong-mode): 0:11.76
Reading (via dart, eager): 2050-2350 ms
Writing (to null sink) (after reading): 400-550 ms
Output dill file (via fasta compile): ~54 MB
Running the dill file is ~the same time, but "Maximum resident set size
(kbytes)" (from /usr/bin/time -v) decreases with ~4%.
On the other side, if it's 1M of the same doubles (0.42), while
compiling is ~the same speed, the output dill goes from 43MB to 50MB.
Surprisingly the "Maximum resident set size (kbytes)" still decreases
though (~3%).
Running flutter test in flutter/packages/flutter:
Before:
```
02:33 +2425 ~18: All tests passed!
02:28 +2425 ~18: All tests passed!
02:28 +2425 ~18: All tests passed!
```
After:
```
02:12 +2425 ~18: All tests passed!
02:11 +2425 ~18: All tests passed!
02:12 +2425 ~18: All tests passed!
```
So that's -12.0267% +/- 3.15253%
File size of a dill file of an arbitrary test using flutter is reduced by ~44 KB (~0.3%).
Change-Id: I64151376cde1dae6f0d02b3d96991bc432a994ae
Reviewed-on: https://dart-review.googlesource.com/41660
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Make all lists returned by ast_from_binary growable so the ast is more consistent with the ast generated by parsing dart code which generally contains growable lists as opposed to fixed-length lists.
This fixes a bug in the Flutter --track-widget-creation transform
where it would fail on an AST if it was deserialized from a binary file
rather than parsed from source code.
Change-Id: I4e6b199ed98df1b72d8d6844829341975a2c29c9
Reviewed-on: https://dart-review.googlesource.com/51780
Commit-Queue: Jacob Richman <jacobr@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
It complicates the intermediate language, none of the back ends are
using it, and it's not something that we want transformation writers
and code generators to deal with.
Change-Id: Ic79f7935dd8619bd233346bb25947e864f38a104
Reviewed-on: https://dart-review.googlesource.com/50440
Commit-Queue: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
Aske Simon Christensen