The CL is a step towards have a more restricted and wellstructured
handled of references and canonical names.
The CL moves Reference to canonical_name.dart and makes
CanonicalName.reference private, and replaces CanonicalName.getReference
with a 'reference' getter.
It also removes NamedNode.canonicalName, Field.getterCanonicalName and
Field.setterCanonicalName so that these can only be accessed through the
corresponding reference. This is to reduce the reliance on the
canonical names which, ideally, should only be part of serialization and
deserialization.
TEST=existing
Change-Id: I955fb7d52d4e112d8741f7c12dcf38b74ae0c91a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190442
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
This removes the @fields and @=fields canonical name
encodings that allowed for encoding of conflicting members
and didn't support field<->getter/setter conversion
between dills or between outline and full dill.
TEST=existing tests+add aot expectation tests
Change-Id: I119b0c95f90e456356146cdc2d9241de4c1b4fff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186680
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
This reverts commit 3892e95547.
Reason for revert: Breaks Flutter web
Original change's description:
> [cfe] Encode field references as @getters and @setters
>
> This removes the @fields and @=fields canonical name
> encodings that allowed for encoding of conflicting members
> and didn't support field<->getter/setter conversion
> between dills or between outline and full dill.
>
> TEST=existing tests
>
> Change-Id: Id15e58ad4d1847d2c98a688705e5945196146c6d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184783
> Commit-Queue: Johnni Winther <johnniwinther@google.com>
> Reviewed-by: Jens Johansen <jensj@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
Change-Id: I744e284b16e097fa0833c5bdf1bc7653f13bdf63
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186147
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
This removes the @fields and @=fields canonical name
encodings that allowed for encoding of conflicting members
and didn't support field<->getter/setter conversion
between dills or between outline and full dill.
TEST=existing tests
Change-Id: Id15e58ad4d1847d2c98a688705e5945196146c6d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184783
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL completes the migration of the first wave of
interdependent libraries in package:kernel, including ast.dart.
In order to ensure non-nullability on AST properties, the Transformer
has been split in 2 variants: Transformer which doesn't support
removal of nodes and RemovingTransformer which supports removal where
allowed by the context using 'removal sentinels'.
Start reviewing Transformer and RemovingTransformer in visitors.dart
since many of the changes are caused by the changes here.
Included in the migration are the mixin_deduplication.dart and
unreachable_code_elimination.dart since these needed porting to
the RemovingTransformer which was aided by opting in the libraries
which only depended on ast.dart.
TEST=existing
Change-Id: I9e63b985bd24896c25edd4ee51e37770187bcc17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184786
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Changes fields to be either mutable or immutable by construction.
This ensure that we don't create setter references for fields that
cannot be assigned to and is a prerequisite for replacing @fields/
@fields= canonical names with @getters/@setters.
TEST=existing expectation tests and verification
Change-Id: I70b9a504ee6f221b7c334ac02620feb0d5f7ae01
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176665
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This CL adds stub for each concrete mixed in member into mixin
applications. The body of the stub calls the original member via
a super call.
This addition means that resolution of super access now use the
stub and not the original declaration as the target, which means
that it will be correct even when mixed in members are cloned. For
this reason, dart2js no longer needs to perform its own super
member resolution.
When a super call targets a mixin super stub (after cloning) it
can be optimized away by redirecting the call to the `stubTarget`
of the call. This optimization is performed in dart2js.
Since dart2js now uses the correct super target, its runtime
mixin application needs to avoid overriding members already
declared in the mixin application. These members are the
forwarding super stubs which ensure that correct runtime types
when members with covariant parameters are implemented.
Change-Id: Iab71ffcc400aa6a683987bc20b9553a263ebc8e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176526
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This CL gives the option to load classes lazily (on by default),
meaning that fields, procedures, constructors and redirecting
factory constructors of classes are not initially read,
but instead read when (or if) needed.
The idea being that many classes aren’t needed, and that
spending time on reading them is thus just a waste of time.
This is especially true in modular compilation where a lot
of modules needs to be given as input as the compilation at
hand might technically rely on it, but in practice only very
few of the classes are actually needed (or very few classes
actually need their members).
The below numbers are comparing a VM without this change and without
the lazy class hierarchy member lists change with
b31566b297 (the base change for those
two changes).
For running a simple hello-world script, on my machine 500 times
and doing statistics on it I can measure the following differences:
With verbose, from kernel_service startup to compile complete:
Difference at 95.0% confidence
-17.8 +/- 0.555817
-24.3455% +/- 0.760206%
(Student's t, pooled s = 4.48379)
With verbose, total runtime measure by `time`
Difference at 95.0% confidence
-19.346 +/- 0.905217
-12.0333% +/- 0.563051%
(Student's t, pooled s = 7.30242)
Without verbose, total runtime measure by `time`
Difference at 95.0% confidence
-17.862 +/- 0.842905
-12.0635% +/- 0.569276%
(Student's t, pooled s = 6.79974)
For running a number of modular compilations I’ve recorded an
improvement in actual runtime in the order of 20-37% depending
on the circumstance. In a real build system, though, the
difference seems to be more in the order of around 5%.
Change-Id: Id329bcf2b01d12c12d7a49f2b8abacd9c2447f05
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/115703
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
TL;DR: Unbind canonical names doesn't do what you think it does and
probably shouldn't (ever) be used. This CL stops using it in a few places.
Longer version:
When loading a dill file it:
- First loads the table of canonical names. These have no references yet.
- When a canonical name is asked for its reference it creates one if it
doesn't yet have one.
- When loading, for instance, a library, it asks for the reference.
When unbinding a canonical name:
- It removes itself (the canonical name) from the reference
- It removes the reference in itself
- Note: Whatever has a pointer to the reference keeps it, and the
reference points to whatever node it already pointed to.
This also means, that if we have a dill file that's split in two and:
- Load #1
- Load #2
that works fine, but if we
- Load #1
- Unbind canonical names
- Binds canonical names
- Load #2
stuff is not bound correctly (and an error is thrown).
And - the cause of this bug:
- Load #1
- Load #2
everything is fine
- Unbind canonical names
- Binds canonical names
- Load #2'
stuff is not bound correctly --- references points to stuff loaded as #2,
not as #2'. On top of being weird, wrong and confusing it also caused wrong
things to be but into the class hierarchy which ultimatly caused the crash.
This CL fixes it by not calling unbind and force loading of dill files
(at specific call sites) to create new libraries
(and in the process overwriting references ".node").
Revert "[dartdevc] Retry ddc incremental compile on crash"
This reverts commit ecdbdf00b8.
Revert "[kernel_worker] retry on failure"
This reverts commit 43eebea5a3.
Fixes#36644
Bug: #36644
Change-Id: Id8f548179e6a409b01f2ebfa3219f94cb64b1c05
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100380
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jenny Messerly <jmesserly@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This CL adds an option to the dill loader (i.e. BinaryBuilder) to
enable performing a check of all canonical names. The theory is,
that if a dill is self-contained "every" (1) canonical name should
contain a reference to a node.
This check is (currently) disabled by default, but enabled
excplicitly in the incremental compiler when initializing from dill.
If the check fails a warning will be issued (and the dill wont be
used).
On my machine the check seems to take ~15 ms in all tested cases
(sdk, flutter test).
(1): Except for extra layer of URI of a library if the qualified
name is private.
Closes#32449.
Change-Id: I65c8fba647be3d5ba47f7ee16caef78cb2cfae07
Reviewed-on: https://dart-review.googlesource.com/51462
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
New type of AST nodes RedirectingFactoryConstructor is added to Kernel.
The goal is to keep the information about redirecting factory
constructors in the class for the linking phase.
Change-Id: I7477c617fa7fd4935ceb23098ec7c056d1f286ca
Reviewed-on: https://dart-review.googlesource.com/14740
Commit-Queue: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
Before: ownership of some or all of the libraries from one Kernel
program (P1) would be transferred to another program (P2). All the
canonical names in P1 would be unbound from their references and the
canonical names would eventually be recreated for the libraries that
were transferred.
After: when ownership of a library is transferred, the ownership of
the canonical name subtree rooted at the library's name is also
transferred. This allows the 1:1 relationship between Canonicalname
and Reference to be maintained which will enable lazy deserialization
of procedure bodies (because the mapping from CanonicalNames to
References in the corresponding link table will be persistent).
Change-Id: I98f975d6ba5804f975c30528a484756b39f09d2a
Reviewed-on: https://dart-review.googlesource.com/7549
Commit-Queue: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Currently serializing the ast for kernel is done in two passe:
1) Scan the program to find and index all strings. These are then
sorted based on frequency and assigned an id. All string-
references are refering to that id. As small numbers use less
space in the binary than big numbers, sorting the numbers by
frequency saves a certain amount of space.
In addition the string indexing is "hijacked" for the
"LimitedBinaryPrinter" to also perform some CanonicalName
re-indexing.
2) We then serialize the entire thing.
This CL gets rid of a pass by not indexing the strings up-front.
Whenever it is asked to serialize a string it adds it to the index
(if not already there). The serialization is otherwise the same.
This means that:
1) Strings are not sorted by frequency, i.e. the binary output size
can by bigger (numbers below).
2) The stringindex and canonical names are moved to the end of the
binary instead of the front. As we still need it up front for
deserialization some additional data is added to the
ProgramIndex.
3) The "hijacking" done in "LimitedBinaryPrinter" is replaced by
an alternative.
4) We don't spend time on walking the tree twice.
The cost is the binary size. Compiling helloworld with fasta,
as well as looking at outline.dill, platform.dill and
vmservice_io.dill reveals these numbers:
* helloworld.dill is 0.657248732% bigger (26573 bytes)
* outline.dill is 1.686911399% bigger (9395 bytes)
* platform.dill is 0.657062238% bigger (26565 bytes)
* vmservice_io.dill is 0.44991899% bigger (19147 bytes)
The cost does thus not appear to be very big.
The gain is the serialization time.
From 20 runs of an instrumented VM/serialization, running numbers
through calculations stolens from ministat
(https://www.freebsd.org/cgi/man.cgi?query=ministat) reveals the
following:
* Serialization time: -21.69% +/- 1.44%
* Total time spend in relevant parts of bootstrap_nocore.cc,
dart_api_impl.cc (Dart_LoadKernel), bootstrap_nocore.cc,
dart_api_impl.cc (LoadKernelProgram) as well as serialization:
-14.01% +/- 1.58%
From 5 runs of
"time python tools/test.py -m release -cdartk language -j6"
(again run through ministat calculations):
* real: -4.18% +/- 0.5%
* user: -4.2% +/- 0.29%
* sys: No difference at 95%
* user+sys: -3.3% +/- 0.36%
Change-Id: I1c220eac083496994f0a9f1e2a2445b3707c9a93
Reviewed-on: https://dart-review.googlesource.com/2880
Reviewed-by: Samir Jindel <sjindel@google.com>
Summary:
We use a modest set of annotations in ast.dart to describe how the Kernel AST
should be converted into Coq definitions.
We define a Kernel transformation that converts the kernel tree of ast.dart into
a valid Coq file containing the corresponding definitions.
Currently generating the Coq file is not done in the build system because
compiling it requires having Coq installed, and I don't want to introduce a
depedency on Coq into the build system.
Some parts of the AST are not represented because they don't significantly
contribute to the typing semantics:
- asserts
- typedefs
- most literals/basic types (excl. bool, which is needed for "is" tests)
- switch
- for-in
- parts
- yield/await
Test Plan:
Ran the output KernelSyntax.v file through "coqc".
Change-Id: Ic573163a017eaaf3759b741b9eec5ce3ce19225c
Reviewed-on: https://dart-review.googlesource.com/2960
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Commit-Queue: Dmitry Stefantsov <dmitryas@google.com>
Before: Canonical names contained pointers to the corresponding Kernel
tree which assumed that the whole tree was in memory whenever the
canonical names were.
Now: Canonical names do not contain these pointers. They were only
really used to perform name-based lookup in the VM's heap so the
canonical name itself is enough.
If we later find that we need to get from a canonical name to its
Kernel tree we can add an offset in the binary (for instance) to the
canonical name or in a separate mapping on the side.
BUG=
R=asgerf@google.com, jensj@google.com, vegorov@google.com
Committed: https://github.com/dart-lang/sdk/commit/ed77783cd32d55fdad61bf9bc749030847ba9384
Review-Url: https://codereview.chromium.org/2781893004 .
Before: Canonical names contained pointers to the corresponding Kernel
tree which assumed that the whole tree was in memory whenever the
canonical names were.
Now: Canonical names do not contain these pointers. They were only
really used to perform name-based lookup in the VM's heap so the
canonical name itself is enough.
If we later find that we need to get from a canonical name to its
Kernel tree we can add an offset in the binary (for instance) to the
canonical name or in a separate mapping on the side.
BUG=
R=asgerf@google.com, vegorov@google.com
Review-Url: https://codereview.chromium.org/2781893004 .
This adds a class CanonicalName that can represent a library, class,
or member. All references now go through a Reference object, which is
linked to both the AST node and its CanonicalName, so either can be
created first.
dartk now accepts multiple input files:
- If multiple dart files are given, they are all compiled.
- If multiple binaries are given, they are linked together.
Mixed dart and binary input is not supported by dartk.
dartk now has a flag --include-sdk which includes the entire SDK in
the output. This is so the SDK can be compiled alone and then linked.
Example of compiling separately and then linking:
dartk foo.dart -o foo.dill
dartk main.dart -o main.dill
dartk --include-sdk -o sdk.dill
dartk main.dill foo.dill sdk.dill --target=vm --link -o program.dill
dartk still has incredibly slow cold start due to the analyzer loading
the dart sdk, so this does not actually speed things up at the moment.
BUG=
R=ahe@google.com, kmillikin@google.com, kustermann@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2665723002 .
Johnni Winther