This CL adds two new kernel tools which can answer questions about
a dill file:
* count_breakdown.dart
Enumerates the different node types in the provided dill file and
counts them.
This has proved useful in the past to e.g. see how many doubles are
included in Flutter.
* size_breakdown.dart
Gives an overview of which parts of the dill file contributes how many
bytes.
This has proved useful in the past to see that the string table gets
huge when we have many doubles (because we currently save doubles in
kernel as strings).
It also adds error handling and usage information to existing tools.
Change-Id: Ie2ce9e4b8806d5975dba8f57445705a840be3474
Reviewed-on: https://dart-review.googlesource.com/42660
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Currently we produce 'partial dills' for the VM when compiling through
kernel, i.e. we only include the user-code and whatever canonical names
the user refers to. That's perfectly fine for the VM as it always loads
the platform.dill file too.
We cannot currently print it though.
(Technically we should be able to concatenate it with platform.dill and
print that).
This CL allows us to print these partial dills without concatenating
anything first.
Bug:
Change-Id: I7d5f12332e16623d02531ad46e2dc103222f59e9
Reviewed-on: https://dart-review.googlesource.com/19002
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
The introduced "constants" transformation can evaluate constant expressions. The
original use-sites of constant expressions are replaced by a new [ConstantExpression]
node, which points to a subclass of a new [Constant] class hierarchy. Constant
[Field]s and [VariableDeclarations]s will be removed, since all use-sites are
re-written.
The [Constant] class hierarchy is, similarly to the [DartType] class hierarchy, not
part of the AST tree (also has no parent pointer). The constants form a
DAG (directed acyclic graph).
There is no canonicalization requirement of the [Constant] objects referenced by the
AST (via [ConstantExpression]). Although it is beneficial to canonicalize them during
construction, since it reduces time spent in operator==/hashCode.
This CL furthermore adds support for a constant table in the binary format. Similarly
to [String]s, we canonicalize the constants before writing the table to the binary.
The constant table entries in the binary are written in a post-order way, to ensure
easy construction on the backend side.
The text format will be augmented with a "constants { ... }" section at the end,
which lists the constants in the same order as in the binary format.
The transformation can be used by those backends who choose to do so. It is not
enabled by default atm. It should therefore not affect analyzer, fasta or other
components.
Change-Id: I57cd9624fedcf537ab6870db76246149647bed21
Reviewed-on: https://dart-review.googlesource.com/14382
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
The kernel format has been designed so that one can concatenate several
dill files into one file and then load it. For instance the dart
function BinaryBuilder.readProgram supports this.
Currently a dill file contains one or more programs.
In the VM each of these programs are called either program or subprogram.
Technically a dill "program" isn't necessarily a program at all (e.g. it
could be missing a library).
This naming snafu should probably be cleaned up at some point, but that's
for another CL.
When loading a dill file via BinaryBuilder.readProgram what happens is
this:
- Each program in the dill file ends in 4 bytes that indicates the size
of the program.
- Reading the input from the end one can then read the size, skip back
that amount of bytes, if we have more data (i.e. there's another
program), read another size and so on, and continue until we have
accounted for all bytes in the input.
- We then read each program from the start, and basically overwrite any
library, class, procedure etc. we find.
The first main reference found is the one used though.
(Saying that we overwrite is not completely true, but when the library
is a non-external library that's basically what happens).
This CL introduces (some) support on the C++ side for the same thing.
So far the C++ side could only handle single-program-dills, and trying
to load anything else would probably crash the VM.
The support added is this:
- Assume the SDK (i.e. vm_platform.dill) is not a concatenated file
(error out if it is).
- For user provided input, loop over each contained program one-by-one,
for each individual one behave as normal.
- The way LibraryLoad is implemented (i.e. it skips if the library is
already loaded) this means that it currently would behave differently
than the dart version (i.e. the first one is used, not the last one).
For now it is assumed that that's not a problem.
- There is a possibly snafu if the same script is included several times.
This could probably mostly be remedied by not creating scripts up front,
but only as needed. By the "keep only one" (and fixing the above point,
probably by simply loading in the opposite order, i.e. last program
in the binary first) the (theoretical) problem would probably do away.
Note that we will have separate string tables, canonical name tables etc
per "sub program" and that there might be some duplication.
The implementation was tested as indicated below, but introduces no tests.
$ cat test_lib1.dart
import "test_lib2.dart" as lib2;
String lib1field = "lib #1 field!!";
main() {
foo();
lib2.foo();
print("From lib2: ${lib2.lib2field}");
}
foo() {
print("Hello, Foo, from test_lib1!");
var x = 42;
print(x);
}
$ cat test_lib2.dart
String lib2field = "Lib #2 field!!!!";
foo() {
print("Hello, Foo, from test_lib2!");
var y = 34;
print(y);
}
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart --packages=.packages --platform=out/ReleaseX64/vm_platform.dill test_lib1.dart
$ ls -lha test_lib1.dart.dill
[...] 4.2M Oct 26 14:42 test_lib1.dart.dill
$ dart pkg/kernel/bin/split.dart test_lib1.dart.dill
Wrote test_lib1.dart.dill.part1.dill
Wrote test_lib1.dart.dill.part2.dill
$ ls -lha test_lib1.dart.dill.part{1,2}.dill
[...] 811 Oct 26 14:42 test_lib1.dart.dill.part1.dill
[...] 582 Oct 26 14:42 test_lib1.dart.dill.part2.dill
$ cat test_lib1.dart.dill.part1.dill test_lib1.dart.dill.part2.dill > test_lib1.dart.dill.concat.dill
$ ls -lha test_lib1.dart.dill.concat.dill
[...] 1.4K Oct 26 14:44 test_lib1.dart.dill.concat.dill
$ out/ReleaseX64/dart --kernel-binaries=out/ReleaseX64 --packages=.packages test_lib1.dart.dill.concat.dill
Hello, Foo, from test_lib1!
42
Hello, Foo, from test_lib2!
34
From lib2: Lib #2 field!!!!
Change-Id: I233a033aa3042b202dd4708908a5be3089474588
Reviewed-on: https://dart-review.googlesource.com/16820
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This will allow it to be used by front_end unit tests.
Also plumb through the "ignoreSdk" boolean, since we will need to use
it when invoking the naive type checker from front_end unit tests.
Change-Id: I9ae50f48e24a5c45e93f4218e6b61f193f549e7a
Reviewed-on: https://dart-review.googlesource.com/16601
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
This change silences errors that were reported for Function.call callsites because
there is no valid interface target for them:
Function f;
f();
f(null);
f(null, null, 10);
f<bool, int>(11, true);
All of these call sites are valid, so we have to simply treat them as dynamic
invocations.
Bug: https://github.com/dart-lang/sdk/issues/31052
Change-Id: I13b5dc3ff69e7adede3040334556ff6653b50515
Reviewed-on: https://dart-review.googlesource.com/14524
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
This type checker can be used to find strong mode violations in the
Kernel files.
This will be used to work on cleaning strong mode violations in VM's patch files
in the absence of any other way to type-check them.
Bug:
Change-Id: Id7005f6312dafe04eb0e7b33d008934b62a1b726
Reviewed-on: https://dart-review.googlesource.com/11883
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
This will allow tests that use batch_util.dart to be run inside
google3, where package layout conventions are more strictly enforced
(files in test/ cannot import files in bin/ or vice versa).
Change-Id: I046b864bc3b1c4e78b984b0047b7567873146c52
Reviewed-on: https://dart-review.googlesource.com/7340
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Summary:
We use a modest set of annotations in ast.dart to describe how the Kernel AST
should be converted into Coq definitions.
We define a Kernel transformation that converts the kernel tree of ast.dart into
a valid Coq file containing the corresponding definitions.
Currently generating the Coq file is not done in the build system because
compiling it requires having Coq installed, and I don't want to introduce a
depedency on Coq into the build system.
Some parts of the AST are not represented because they don't significantly
contribute to the typing semantics:
- asserts
- typedefs
- most literals/basic types (excl. bool, which is needed for "is" tests)
- switch
- for-in
- parts
- yield/await
Test Plan:
Ran the output KernelSyntax.v file through "coqc".
Change-Id: Ic573163a017eaaf3759b741b9eec5ce3ce19225c
Reviewed-on: https://dart-review.googlesource.com/2960
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Commit-Queue: Dmitry Stefantsov <dmitryas@google.com>
"bin" is part of the public API of a package, private tools can go under the
`tool` folder. This allows us to use a dev_dependency on analyzer without
exposing a broken import publicly.
I've only moved `dartk` so far (which I expect will be removed once we have a
better API for front_end.
R=asgerf@google.com, kmillikin@google.com
Review-Url: https://codereview.chromium.org/2842643002 .
This CL moves many of the files from
pkg/front_end/lib/src/fasta/analyzer/ to pkg/analyzer/lib/src/fasta/.
It also moves two files from pkg/kernel/lib/analyzer/ to
pkg/analyzer/lib/src/kernel/.
This reduces the amount of circularity between analyzer and
front_end/kernel so that there are no files in front_end or kernel
which are both dependend upon by analyzer and depend upon analyzer. I
will clean up the remaining circularities in future CLs.
There should be no functional change.
R=ahe@google.com, asgerf@google.com, kmillikin@google.com, scheglov@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2756593004 .
All generic methods are equipped with one extra named parameter for
passing type arguments as a list of type values.
Additinally, this change forces strong mode usage for 'dartk' in
'reified_dart'. Strong mode is required for loader to not strip away
type arguments from generic methods. In future, a command line argument
may be implemented for that (e.g. --generic-methods), if generic method
support will land before the strong mode.
R=karlklose@google.com
Review-Url: https://codereview.chromium.org/2713163002 .
This adds a class CanonicalName that can represent a library, class,
or member. All references now go through a Reference object, which is
linked to both the AST node and its CanonicalName, so either can be
created first.
dartk now accepts multiple input files:
- If multiple dart files are given, they are all compiled.
- If multiple binaries are given, they are linked together.
Mixed dart and binary input is not supported by dartk.
dartk now has a flag --include-sdk which includes the entire SDK in
the output. This is so the SDK can be compiled alone and then linked.
Example of compiling separately and then linking:
dartk foo.dart -o foo.dill
dartk main.dart -o main.dill
dartk --include-sdk -o sdk.dill
dartk main.dill foo.dill sdk.dill --target=vm --link -o program.dill
dartk still has incredibly slow cold start due to the analyzer loading
the dart sdk, so this does not actually speed things up at the moment.
BUG=
R=ahe@google.com, kmillikin@google.com, kustermann@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2665723002 .
This CL adds the work done at https://github.com/dart-lang/reify to SDK.
The commit that is used for the merge is
a2066a68374d49de92ff75f5e1ffc36335fd9451 (Nov 23, 2016). The code is
adjusted to respect the changes of the kernel package in SDK since that
commit.
The reify transformation is run by specifying 'vmreify' target to
'dartk'. The transformation requires its runtime library to present in
the program being transformed. The library is found in its default
location in SDK checkout if 'dartk' is run from its default location in
SDK checkout. To preserve the library in the output, TreeShaker is
disabled in 'vmreify' target.
The "golden" set of tests is also copied from 'dart-lang/reify'
repository, and the appropriate test suite is defined for it.
The bash script 'bin/reified_dart' from 'dart-lang/reify' is rewritten
as Dart script 'pkg/kernel/bin/reified_dart.dart'. It requires path to
'dartk' and path to SDK. Those are taken from their default locations in
SDK if 'reified_dart.dart' is run from its default location in SDK.
The added files were formatted using 'dartfmt' with default settings.
Additionally, the files were checked with 'dartanalyzer --strong'. The
necessary changes were made to fix the error messages. There are some
'hint' and 'error' messages left for some .dart files from the added
test cases, but they reflect intentional errors or conventions in those
files.
R=asgerf@google.com, karlklose@google.com
Review-Url: https://codereview.chromium.org/2697873007 .
When a class from a different build unit is mixed in, the instance
members of the mixed-in class are retained in the external library
definition, so the mixin resolution pass can clone them.
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2669303002 .
Target-specific modular transformations have to be able to cope with
external libraries. Target-specific global transformations should be
optimizations, and not required for correctness, since a simple linker
may choose not to perform them.
Make mixin resolution modular by making it fail when a mixed-in class
comes from an external library. (We cannot mix in such a class because
we do not necessarily have all class members.)
R=asgerf@google.com
Review-Url: https://codereview.chromium.org/2671653003 .
Updates to kernel that sets more offsets (and introduces end offests
and 'debuggability') - but doesn't necessarily persist them,
i.e. the format doesn't change and no C++ changes are neccessary yet.
This is step #1 in introducing these things, next step(s) will be
persisting the new stuff and using it on the C++ side.
R=asgerf@google.com, kmillikin@google.com
Review-Url: https://codereview.chromium.org/2610133002 .
This is the result of:
- taking the diff of the branch closure_conversion to master in the kernel
repository
- updating the file paths
- applying the diff to the Dart SDK
- fixing conflicts between the changes to pkg/kernel in the Dart SDK and the master branch in the kernel repository
R=asgerf@google.com
Review-Url: https://codereview.chromium.org/2561723003 .
In kernel, library import URIs now support an "app" scheme as an
alternative to the "file" scheme, representing a path relative to
the application root.
dartk takes an --app-root flag giving the application root. If none
is given, file URIs are used instead.
The intention is that kernel binaries should not carry irrelevant
path information, such as the path to the home directory of the
user who compiled a given file.
It is not the intention that end-users should see an app URI.
Import paths are currently not shown to users at all, and if we need
to do this, they should be translated to file paths first.
In theory we could stick to file URIs with relative paths, but the Uri
class from dart:core makes this difficult, as certain operations on it
assume that file paths should be absolute.
Source mapping URIs are not yet affected by this change.
R=kmillikin@google.com
Committed: https://github.com/dart-lang/sdk/commit/60adb852ad706ecf9424c77d47fa05583d543def
Review URL: https://codereview.chromium.org/2532053005 .
Reverted: https://github.com/dart-lang/sdk/commit/bb540416f27c39d75ee7f243899939fc37a2cd03
In kernel, library import URIs now have the "app" scheme instead of
the "file" scheme, representing a path relative to the application root.
dartk takes an --app-root flag giving the application root. This
defaults to the current working directory.
The intention is that kernel binaries should not carry irrelevant
path information, such as the path to the home directory of the
user who compiled a given file.
It is not the intention that end-users should see an app URI.
Import paths are currently not shown to users at all, and if we need
to do this, they should be translated to file paths first.
In theory we could stick to file URIs with relative paths, but the Uri
class from dart:core makes this difficult, as certain operations on it
assume that file paths should be absolute.
Source mapping URIs are not yet affected by this change.
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2532053005 .
This replaces the old --sanity-check flag from dartk. Some files have
been renamed to avoid the wording "sanity check".
Compared to --sanity-check, the following checks have been added:
- variables are not referenced out of scope
- variables are not redeclared
- class type parameters are not referenced from static context
A unit test has been added to check that the verifier rejects certain
invalid ASTs.
BUG=
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2531873002 .
Peter von der Ahé