This adds more indexes to the kernel format so we know where classes
and procedures starts and stops. This allows for more random access.
E.g. one could now read the program index and jump directly to
library $i_1$, then read the library index and jump directly to class $i_2$,
read the class index and jump directly to procedure $i_3$.
The utilization (in this CL) is to not (always) read the procedure body
up front when loading kernel code on the dart side (ast_from_binary).
The observation is that - when running through the VM - almost none
of the bodies from the platform file are actually used.
This lowers the start-up cost which is noticeable for small programs
(e.g. hello world, or tests).
In this CL this is only done on the dart side and not on the C++ side,
that's for another CL.
Startup time:
dart2js: -1.84253% +/- 1.22157%
hello world: -11.1188% +/- 5.57892%
Running "time python tools/test.py -m release -cdartk language -j6":
real: -11.72% +/- 0.35%
user: -14.59% +/- 0.24%
sys: -12.71% +/- 0.61%
File size change (compiling with fasta to dill file incl. platform):
hello world: 0.88% (35,934 bytes).
dart2js: 0.97% (200,967 bytes).
Change-Id: I1f0ec121bc75bb17f11d3fade03da9815037d0bb
Reviewed-on: https://dart-review.googlesource.com/5262
Reviewed-by: Kevin Millikin <kmillikin@google.com>
This CL adds the ability to create a list of "forwarding nodes" for a
source class. A forwarding node is a data structure that will later
be resolved to either an explicitly declared member in the class or a
superclass, or to a forwarding stub. The idea is that we will create
the forwarding nodes at the time of outline building, and later,
during type inference, we will resolve each forwarding node as it is
encountered.
The reason we need to defer resolution of the forwarding nodes until
inference is because we may need to use the results of type inference
to determine which member a given forwarding node resolves to. For
example:
num f() => 1;
class A {
final x = 1; // Inferred type: int
}
class B {
final x = f(); // Inferred type: num
}
abstract class C implements A, B {}
We cannot determine at the time of building the outline for C whether
it inherits its x from A or B, because we need the results of type
inference to determine which of the two x's has a more specific type.
Note that some refactoring of ClassHierarchy was necessary in order to
allow the front end to maintain member lists in the same order used
internally by ClassHierarchy. This will let us avoid unnecessary
redundant sorting of methods.
Change-Id: Iee754957e0ad3b16c4b60608e17a4a7b0006dfb4
Reviewed-on: https://dart-review.googlesource.com/7851
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
This should take care of some or all flaky tests like:
TypePropagationTest_Kernel | test_forEach_async_inheritedStream
What was happening is that every tests adds /test.dart to AnalysisDriver,
which means that this file is scheduled for analysis at some point,
and then it also calls getResult() to get the resolved unit. When we
resolve the file for the first time, the ByteStore is empty, so we
build the corresponding Kernel file from scratch, and it has the offset.
But the second time we read the Kernel file from ByteStore. So, if we
manage to process the file as added first, and then as getResult(),
we fail because we cannot resolve the import directive. But if we
were not able to process the file as added, and just do getResult()
first (which also marks the file as added as ready), we succeed.
So, it was flaky.
R=ahe@google.com, kmillikin@google.com, paulberry@google.com, sigmund@google.com
Bug: https://github.com/dart-lang/sdk/issues/30863
Change-Id: I96151e3ebefcd212f2a7a1b2b22abb7d87ed4781
Reviewed-on: https://dart-review.googlesource.com/7782
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Konstantin Shcheglov <scheglov@google.com>
This will allow tests that use batch_util.dart to be run inside
google3, where package layout conventions are more strictly enforced
(files in test/ cannot import files in bin/ or vice versa).
Change-Id: I046b864bc3b1c4e78b984b0047b7567873146c52
Reviewed-on: https://dart-review.googlesource.com/7340
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Note that the parser used for unit tests in kernel didn't have a
notation for promoted bounds. I chose `T & A` to mean "type parameter
T with promoted bound A", following Leaf's convention (the rationale
is that semantically, a promoted bound functions like an intersection
type).
It was also necessary to change the test infrastructure so that we
could specify the (non-promoted) bounds of type parameters when
running a subtype test.
Fixes#30833.
Change-Id: Ic62722df63c5ed5288b23560745a1cd0869b63fb
Reviewed-on: https://dart-review.googlesource.com/7628
Reviewed-by: Peter von der Ahé <ahe@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
* The definition of the well-formedness property for
configurations is added.
* The theorem that states that the abstract CESK-machine can
make one transition step from any well-formed configuration
is defined and proven.
* Execution of a variable declaration statement is added to
the operational semantics formalization with all necessary
changes (one new eval configuration, three new transition
rules).
* Some auxiliary theorems are added. One of them states that
any runtime value has its method bodies in the function
environment.
Change-Id: I95f233a4db498ce0df76983d9e605c3c263100bb
Reviewed-on: https://dart-review.googlesource.com/7266
Reviewed-by: Samir Jindel <sjindel@google.com>
Before: ownership of some or all of the libraries from one Kernel
program (P1) would be transferred to another program (P2). All the
canonical names in P1 would be unbound from their references and the
canonical names would eventually be recreated for the libraries that
were transferred.
After: when ownership of a library is transferred, the ownership of
the canonical name subtree rooted at the library's name is also
transferred. This allows the 1:1 relationship between Canonicalname
and Reference to be maintained which will enable lazy deserialization
of procedure bodies (because the mapping from CanonicalNames to
References in the corresponding link table will be persistent).
Change-Id: I98f975d6ba5804f975c30528a484756b39f09d2a
Reviewed-on: https://dart-review.googlesource.com/7549
Commit-Queue: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
This reverts commit 9a8621b60a.
Revert "Rework getElement() in resynthesizer."
This reverts commit e4fa080f69.
Revert "Create (empty) initializers for parameters resynthesized from Kernel."
This reverts commit 8df6c79b9d.
Revert "Run NonErrorResolverTest in strong mode. Extract not strong tests."
This reverts commit 9bdda4b1d3.
Revert "Move TypeProvider creation into KernelResynthesizer and create loadLibrary functions."
This reverts commit c59eaf1788.
Revert "Return SimpleIdentifier or PrefixedIdentifier from _buildIdentifier()."
This reverts commit 6d0515f9ca.
Change-Id: I1099ca715ce6287ab56808b7cc3abe0589e939c1
Reviewed-on: https://dart-review.googlesource.com/7550
Reviewed-by: Peter von der Ahé <ahe@google.com>
This revision includes changes from:
- [kernel] Completion of consistency proofs for type system of first subset of kernel.
due to the suckiness of gerrit.
We generalize type equivalence to subtyping. The contravariant property of
function parameter types causes properties for the totality checker. To
cicumvent this, we define a well-ordered relation on pairs of dart types and
prove subtyping respects it. We develop new lemmas and tactics for managing
proofs involving subtyping, includinging factoring out the messy business of
dealing with its convoluted recursion scheme.
Bug:
Change-Id: I18936168006617874a82eefc983f1b2d4d8af5af
Reviewed-on: https://dart-review.googlesource.com/5861
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This CL changes devirtualization to detect corner case of a method
invocation which is resolved to a getter or a field. This kind of
invocation should call getter first, and then call 'call()' method on
the result of the getter, passing the rest of the arguments.
For simplicity, devirtualization of such method invocations is avoided.
Issue: https://github.com/dart-lang/sdk/issues/30480
Change-Id: Ibe6321931cbb5527a26b139c5f1ee9773a253629
Reviewed-on: https://dart-review.googlesource.com/5902
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Instead of using enums, we use booleans, and we change the terminology
as follows:
FormalSafety.semiSafe -> isGenericCovariantImpl
InterfaceSafety.semiTyped -> isGenericCovariantInterface
(The enum value FormalSafety.unsafe turned out to be redundant with
isCovariant, so it is no longer needed).
Similarly, the annotations in the front end tests are updated as follows:
@checkFormal=unsafe -> @covariance=explicit
@checkFormal=semiSafe -> @covariance=genericImpl
@checkInterface=semiTyped -> @covariance=genericInterface
Change-Id: Iafc0c5d3fc4e7608a2b8c52d8c29f293d9219995
Reviewed-on: https://dart-review.googlesource.com/5540
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
It's sometimes easier to read SML than text. This is the higher-order
interpreter, we have plans for a first-order version.
Bug:
Change-Id: Ic6bcc989e6a544889d0ff3eefac266bd54f9489b
Reviewed-on: https://dart-review.googlesource.com/5420
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
While writeByte(0) can be read with readUint() (and writeUInt30(0) can be
read correctly with readByte()) it's probably better to use
writeUInt30/readUInt as a pair and writeByte/readByte as a pair.
Change-Id: I3e638c1de0bd66b112cfa1370a54412e876dca5a
Reviewed-on: https://dart-review.googlesource.com/4720
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
I also added mutual induction schemes for types and expressions.
Some changes from the "Cleanup" revision are in here as well because Gerrit is terrible.
Bug:
Change-Id: I0859a6c1cba8179e0a64cc0455ab2a83fad8f26b
Reviewed-on: https://dart-review.googlesource.com/5300
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Summary:
Common datastructures used by both the Kernel AST definition and the object
model are factored into a shared module. A monad for partial computation is
defined to allow us to factor out termination proofs and syntactic validity
checks from the type checking and subtyping relations.
Test Plan:
Ran through coqc.
Bug:
Change-Id: I884666d7cc5b757d62541a46b868f8579a06f011
Reviewed-on: https://dart-review.googlesource.com/4700
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This CL adds a new target for kernel front-end, vm_precompiler. This is
an experimental target for new Dart VM precompiler pipeline which
will fully exploit strong mode type system and perform whole-program
optimizations.
As an example of such whole-program optimization, this CL adds
draft implementation of devirtualization of method invocations, which
uses single target computed by closed-world class hierarchy analysis.
Corresponding null checks (required for correctness) are not generated
yet.
Issue: https://github.com/dart-lang/sdk/issues/30480
Change-Id: I704cd16843a08f036a188b1188c80ee4dfbeab3b
Reviewed-on: https://dart-review.googlesource.com/3402
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Before this CL to read the source (and line endings etc) for a specific
file index, one had to read at least part of the data for all previous
file indexes (e.g. read all the line endings).
This CL introduces an index to the included sources meaning that we have
random access based on file id, i.e. can go to the data concerning a
specific file id in constant time.
Benchmarks run with "time python tools/test.py -m release -cdartk language -j6"
shows that - of 5 runs - the runtime has changed as follows:
real: -3.93% +/- 1.03%
user: -3.41% +/- 0.54%
sys: No difference at 95%
(statistics by math stolen from ministat)
So it is ~4% faster to run the language tests (with above command),
shaving approximately 9 seconds off the real runtime.
Change-Id: I9e60a16958356b16b3da0bf6c01ffc5619deb976
Reviewed-on: https://dart-review.googlesource.com/3180
Reviewed-by: Samir Jindel <sjindel@google.com>
Previously the VM couldn't handle external libraries, but that was
fixed in 2f49198520.
Part of the CL was reverted though because the compilatin was changed
to using an outline instead of the platform which doesn't work.
What does work though, is not including the external libraries in the
output.
This CL makes the following changes:
* Don't include external libraries in the output (by not setting all
libraries to be non-external).
* Only writes the sources actually used to the binary (i.e. whatever
libraries left out because they were external will not contribute
source code either).
* Cleanup of now unused code.
Timings (only run once though):
Without this CL (but with the CL it's based on):
$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[05:43 | 100% | + 3504 | - 0]
real 5m43.597s
user 33m48.152s
sys 9m34.140s
Only the "utils/kernel-service/kernel-service.dart" part of this CL:
$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[04:55 | 100% | + 3504 | - 0]
real 4m55.684s
user 29m54.360s
sys 8m7.408s
Entire CL:
$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[04:20 | 100% | + 3504 | - 0]
real 4m20.416s
user 27m17.320s
sys 6m53.472s
Change-Id: Ie9c5bfa958e558a5007784e821a0b58d417bae55
Reviewed-on: https://dart-review.googlesource.com/3161
Reviewed-by: Samir Jindel <sjindel@google.com>
Currently serializing the ast for kernel is done in two passe:
1) Scan the program to find and index all strings. These are then
sorted based on frequency and assigned an id. All string-
references are refering to that id. As small numbers use less
space in the binary than big numbers, sorting the numbers by
frequency saves a certain amount of space.
In addition the string indexing is "hijacked" for the
"LimitedBinaryPrinter" to also perform some CanonicalName
re-indexing.
2) We then serialize the entire thing.
This CL gets rid of a pass by not indexing the strings up-front.
Whenever it is asked to serialize a string it adds it to the index
(if not already there). The serialization is otherwise the same.
This means that:
1) Strings are not sorted by frequency, i.e. the binary output size
can by bigger (numbers below).
2) The stringindex and canonical names are moved to the end of the
binary instead of the front. As we still need it up front for
deserialization some additional data is added to the
ProgramIndex.
3) The "hijacking" done in "LimitedBinaryPrinter" is replaced by
an alternative.
4) We don't spend time on walking the tree twice.
The cost is the binary size. Compiling helloworld with fasta,
as well as looking at outline.dill, platform.dill and
vmservice_io.dill reveals these numbers:
* helloworld.dill is 0.657248732% bigger (26573 bytes)
* outline.dill is 1.686911399% bigger (9395 bytes)
* platform.dill is 0.657062238% bigger (26565 bytes)
* vmservice_io.dill is 0.44991899% bigger (19147 bytes)
The cost does thus not appear to be very big.
The gain is the serialization time.
From 20 runs of an instrumented VM/serialization, running numbers
through calculations stolens from ministat
(https://www.freebsd.org/cgi/man.cgi?query=ministat) reveals the
following:
* Serialization time: -21.69% +/- 1.44%
* Total time spend in relevant parts of bootstrap_nocore.cc,
dart_api_impl.cc (Dart_LoadKernel), bootstrap_nocore.cc,
dart_api_impl.cc (LoadKernelProgram) as well as serialization:
-14.01% +/- 1.58%
From 5 runs of
"time python tools/test.py -m release -cdartk language -j6"
(again run through ministat calculations):
* real: -4.18% +/- 0.5%
* user: -4.2% +/- 0.29%
* sys: No difference at 95%
* user+sys: -3.3% +/- 0.36%
Change-Id: I1c220eac083496994f0a9f1e2a2445b3707c9a93
Reviewed-on: https://dart-review.googlesource.com/2880
Reviewed-by: Samir Jindel <sjindel@google.com>
Summary:
We use a modest set of annotations in ast.dart to describe how the Kernel AST
should be converted into Coq definitions.
We define a Kernel transformation that converts the kernel tree of ast.dart into
a valid Coq file containing the corresponding definitions.
Currently generating the Coq file is not done in the build system because
compiling it requires having Coq installed, and I don't want to introduce a
depedency on Coq into the build system.
Some parts of the AST are not represented because they don't significantly
contribute to the typing semantics:
- asserts
- typedefs
- most literals/basic types (excl. bool, which is needed for "is" tests)
- switch
- for-in
- parts
- yield/await
Test Plan:
Ran the output KernelSyntax.v file through "coqc".
Change-Id: Ic573163a017eaaf3759b741b9eec5ce3ce19225c
Reviewed-on: https://dart-review.googlesource.com/2960
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Commit-Queue: Dmitry Stefantsov <dmitryas@google.com>
Supermixin support is not currently planned for the initial release of
Dart 2.0, so it seems unwise for front end and kernel to enable it.
Kernel didn't actually contain any usages of supermixin support; front
end contained one usage in a test.
Change-Id: I2b225f8d6c5f0ea32cb60d7636e043c89439548c
Reviewed-on: https://dart-review.googlesource.com/3020
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Summary:
Previously, we would create a wrapper function in the flowgraph around converted
closures, which would forward all the closure's arguments and unpack the context
argument before calling the real closure function.
Now, we perform the unpacking at the top of the real function to avoid having
any wrapper function.
Previously, captured parameters would still be appear live to the GC even if
they're updated, because after they are copied into the context, all updates to
them are done there.
Now, as in regular closures, we zero-out the parameter variable after copying
it's value into the context, avoiding potential memory leaks.
Test Plan:
Ran the closure conversion test suite.
Ran benchmarks on Golem -- all statistically significant regressions are gone.
BUG=
R=dmitryas@google.com, regis@google.com
Review-Url: https://codereview.chromium.org/3008923002 .
This proposal just describes the API to the AST classes; it does not
introduce any serialization/deserialization code, nor does it try to
represent the annotations in a compact way.
In the final implementation we will probably want to make use of
bitfields; e.g. the new fields VariableDeclaration.formalSafety and
VariableDeclaration.interfaceSafety should probably be replaced with
getters and setters that access bits in VariableDeclaration.flags.
R=dmitryas@google.com
Review-Url: https://codereview.chromium.org/3008853002 .
Jens Johansen