The kernel format has been designed so that one can concatenate several
dill files into one file and then load it. For instance the dart
function BinaryBuilder.readProgram supports this.
Currently a dill file contains one or more programs.
In the VM each of these programs are called either program or subprogram.
Technically a dill "program" isn't necessarily a program at all (e.g. it
could be missing a library).
This naming snafu should probably be cleaned up at some point, but that's
for another CL.
When loading a dill file via BinaryBuilder.readProgram what happens is
this:
- Each program in the dill file ends in 4 bytes that indicates the size
of the program.
- Reading the input from the end one can then read the size, skip back
that amount of bytes, if we have more data (i.e. there's another
program), read another size and so on, and continue until we have
accounted for all bytes in the input.
- We then read each program from the start, and basically overwrite any
library, class, procedure etc. we find.
The first main reference found is the one used though.
(Saying that we overwrite is not completely true, but when the library
is a non-external library that's basically what happens).
This CL introduces (some) support on the C++ side for the same thing.
So far the C++ side could only handle single-program-dills, and trying
to load anything else would probably crash the VM.
The support added is this:
- Assume the SDK (i.e. vm_platform.dill) is not a concatenated file
(error out if it is).
- For user provided input, loop over each contained program one-by-one,
for each individual one behave as normal.
- The way LibraryLoad is implemented (i.e. it skips if the library is
already loaded) this means that it currently would behave differently
than the dart version (i.e. the first one is used, not the last one).
For now it is assumed that that's not a problem.
- There is a possibly snafu if the same script is included several times.
This could probably mostly be remedied by not creating scripts up front,
but only as needed. By the "keep only one" (and fixing the above point,
probably by simply loading in the opposite order, i.e. last program
in the binary first) the (theoretical) problem would probably do away.
Note that we will have separate string tables, canonical name tables etc
per "sub program" and that there might be some duplication.
The implementation was tested as indicated below, but introduces no tests.
$ cat test_lib1.dart
import "test_lib2.dart" as lib2;
String lib1field = "lib #1 field!!";
main() {
foo();
lib2.foo();
print("From lib2: ${lib2.lib2field}");
}
foo() {
print("Hello, Foo, from test_lib1!");
var x = 42;
print(x);
}
$ cat test_lib2.dart
String lib2field = "Lib #2 field!!!!";
foo() {
print("Hello, Foo, from test_lib2!");
var y = 34;
print(y);
}
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart --packages=.packages --platform=out/ReleaseX64/vm_platform.dill test_lib1.dart
$ ls -lha test_lib1.dart.dill
[...] 4.2M Oct 26 14:42 test_lib1.dart.dill
$ dart pkg/kernel/bin/split.dart test_lib1.dart.dill
Wrote test_lib1.dart.dill.part1.dill
Wrote test_lib1.dart.dill.part2.dill
$ ls -lha test_lib1.dart.dill.part{1,2}.dill
[...] 811 Oct 26 14:42 test_lib1.dart.dill.part1.dill
[...] 582 Oct 26 14:42 test_lib1.dart.dill.part2.dill
$ cat test_lib1.dart.dill.part1.dill test_lib1.dart.dill.part2.dill > test_lib1.dart.dill.concat.dill
$ ls -lha test_lib1.dart.dill.concat.dill
[...] 1.4K Oct 26 14:44 test_lib1.dart.dill.concat.dill
$ out/ReleaseX64/dart --kernel-binaries=out/ReleaseX64 --packages=.packages test_lib1.dart.dill.concat.dill
Hello, Foo, from test_lib1!
42
Hello, Foo, from test_lib2!
34
From lib2: Lib #2 field!!!!
Change-Id: I233a033aa3042b202dd4708908a5be3089474588
Reviewed-on: https://dart-review.googlesource.com/16820
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This will allow it to be used by front_end unit tests.
Also plumb through the "ignoreSdk" boolean, since we will need to use
it when invoking the naive type checker from front_end unit tests.
Change-Id: I9ae50f48e24a5c45e93f4218e6b61f193f549e7a
Reviewed-on: https://dart-review.googlesource.com/16601
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
This change silences errors that were reported for Function.call callsites because
there is no valid interface target for them:
Function f;
f();
f(null);
f(null, null, 10);
f<bool, int>(11, true);
All of these call sites are valid, so we have to simply treat them as dynamic
invocations.
Bug: https://github.com/dart-lang/sdk/issues/31052
Change-Id: I13b5dc3ff69e7adede3040334556ff6653b50515
Reviewed-on: https://dart-review.googlesource.com/14524
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
This type checker can be used to find strong mode violations in the
Kernel files.
This will be used to work on cleaning strong mode violations in VM's patch files
in the absence of any other way to type-check them.
Bug:
Change-Id: Id7005f6312dafe04eb0e7b33d008934b62a1b726
Reviewed-on: https://dart-review.googlesource.com/11883
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
This will allow tests that use batch_util.dart to be run inside
google3, where package layout conventions are more strictly enforced
(files in test/ cannot import files in bin/ or vice versa).
Change-Id: I046b864bc3b1c4e78b984b0047b7567873146c52
Reviewed-on: https://dart-review.googlesource.com/7340
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Summary:
We use a modest set of annotations in ast.dart to describe how the Kernel AST
should be converted into Coq definitions.
We define a Kernel transformation that converts the kernel tree of ast.dart into
a valid Coq file containing the corresponding definitions.
Currently generating the Coq file is not done in the build system because
compiling it requires having Coq installed, and I don't want to introduce a
depedency on Coq into the build system.
Some parts of the AST are not represented because they don't significantly
contribute to the typing semantics:
- asserts
- typedefs
- most literals/basic types (excl. bool, which is needed for "is" tests)
- switch
- for-in
- parts
- yield/await
Test Plan:
Ran the output KernelSyntax.v file through "coqc".
Change-Id: Ic573163a017eaaf3759b741b9eec5ce3ce19225c
Reviewed-on: https://dart-review.googlesource.com/2960
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Commit-Queue: Dmitry Stefantsov <dmitryas@google.com>
"bin" is part of the public API of a package, private tools can go under the
`tool` folder. This allows us to use a dev_dependency on analyzer without
exposing a broken import publicly.
I've only moved `dartk` so far (which I expect will be removed once we have a
better API for front_end.
R=asgerf@google.com, kmillikin@google.com
Review-Url: https://codereview.chromium.org/2842643002 .
This CL moves many of the files from
pkg/front_end/lib/src/fasta/analyzer/ to pkg/analyzer/lib/src/fasta/.
It also moves two files from pkg/kernel/lib/analyzer/ to
pkg/analyzer/lib/src/kernel/.
This reduces the amount of circularity between analyzer and
front_end/kernel so that there are no files in front_end or kernel
which are both dependend upon by analyzer and depend upon analyzer. I
will clean up the remaining circularities in future CLs.
There should be no functional change.
R=ahe@google.com, asgerf@google.com, kmillikin@google.com, scheglov@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2756593004 .
All generic methods are equipped with one extra named parameter for
passing type arguments as a list of type values.
Additinally, this change forces strong mode usage for 'dartk' in
'reified_dart'. Strong mode is required for loader to not strip away
type arguments from generic methods. In future, a command line argument
may be implemented for that (e.g. --generic-methods), if generic method
support will land before the strong mode.
R=karlklose@google.com
Review-Url: https://codereview.chromium.org/2713163002 .
This adds a class CanonicalName that can represent a library, class,
or member. All references now go through a Reference object, which is
linked to both the AST node and its CanonicalName, so either can be
created first.
dartk now accepts multiple input files:
- If multiple dart files are given, they are all compiled.
- If multiple binaries are given, they are linked together.
Mixed dart and binary input is not supported by dartk.
dartk now has a flag --include-sdk which includes the entire SDK in
the output. This is so the SDK can be compiled alone and then linked.
Example of compiling separately and then linking:
dartk foo.dart -o foo.dill
dartk main.dart -o main.dill
dartk --include-sdk -o sdk.dill
dartk main.dill foo.dill sdk.dill --target=vm --link -o program.dill
dartk still has incredibly slow cold start due to the analyzer loading
the dart sdk, so this does not actually speed things up at the moment.
BUG=
R=ahe@google.com, kmillikin@google.com, kustermann@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2665723002 .
This CL adds the work done at https://github.com/dart-lang/reify to SDK.
The commit that is used for the merge is
a2066a68374d49de92ff75f5e1ffc36335fd9451 (Nov 23, 2016). The code is
adjusted to respect the changes of the kernel package in SDK since that
commit.
The reify transformation is run by specifying 'vmreify' target to
'dartk'. The transformation requires its runtime library to present in
the program being transformed. The library is found in its default
location in SDK checkout if 'dartk' is run from its default location in
SDK checkout. To preserve the library in the output, TreeShaker is
disabled in 'vmreify' target.
The "golden" set of tests is also copied from 'dart-lang/reify'
repository, and the appropriate test suite is defined for it.
The bash script 'bin/reified_dart' from 'dart-lang/reify' is rewritten
as Dart script 'pkg/kernel/bin/reified_dart.dart'. It requires path to
'dartk' and path to SDK. Those are taken from their default locations in
SDK if 'reified_dart.dart' is run from its default location in SDK.
The added files were formatted using 'dartfmt' with default settings.
Additionally, the files were checked with 'dartanalyzer --strong'. The
necessary changes were made to fix the error messages. There are some
'hint' and 'error' messages left for some .dart files from the added
test cases, but they reflect intentional errors or conventions in those
files.
R=asgerf@google.com, karlklose@google.com
Review-Url: https://codereview.chromium.org/2697873007 .
When a class from a different build unit is mixed in, the instance
members of the mixed-in class are retained in the external library
definition, so the mixin resolution pass can clone them.
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2669303002 .
Target-specific modular transformations have to be able to cope with
external libraries. Target-specific global transformations should be
optimizations, and not required for correctness, since a simple linker
may choose not to perform them.
Make mixin resolution modular by making it fail when a mixed-in class
comes from an external library. (We cannot mix in such a class because
we do not necessarily have all class members.)
R=asgerf@google.com
Review-Url: https://codereview.chromium.org/2671653003 .
Updates to kernel that sets more offsets (and introduces end offests
and 'debuggability') - but doesn't necessarily persist them,
i.e. the format doesn't change and no C++ changes are neccessary yet.
This is step #1 in introducing these things, next step(s) will be
persisting the new stuff and using it on the C++ side.
R=asgerf@google.com, kmillikin@google.com
Review-Url: https://codereview.chromium.org/2610133002 .
This is the result of:
- taking the diff of the branch closure_conversion to master in the kernel
repository
- updating the file paths
- applying the diff to the Dart SDK
- fixing conflicts between the changes to pkg/kernel in the Dart SDK and the master branch in the kernel repository
R=asgerf@google.com
Review-Url: https://codereview.chromium.org/2561723003 .
In kernel, library import URIs now support an "app" scheme as an
alternative to the "file" scheme, representing a path relative to
the application root.
dartk takes an --app-root flag giving the application root. If none
is given, file URIs are used instead.
The intention is that kernel binaries should not carry irrelevant
path information, such as the path to the home directory of the
user who compiled a given file.
It is not the intention that end-users should see an app URI.
Import paths are currently not shown to users at all, and if we need
to do this, they should be translated to file paths first.
In theory we could stick to file URIs with relative paths, but the Uri
class from dart:core makes this difficult, as certain operations on it
assume that file paths should be absolute.
Source mapping URIs are not yet affected by this change.
R=kmillikin@google.com
Committed: https://github.com/dart-lang/sdk/commit/60adb852ad706ecf9424c77d47fa05583d543def
Review URL: https://codereview.chromium.org/2532053005 .
Reverted: https://github.com/dart-lang/sdk/commit/bb540416f27c39d75ee7f243899939fc37a2cd03
In kernel, library import URIs now have the "app" scheme instead of
the "file" scheme, representing a path relative to the application root.
dartk takes an --app-root flag giving the application root. This
defaults to the current working directory.
The intention is that kernel binaries should not carry irrelevant
path information, such as the path to the home directory of the
user who compiled a given file.
It is not the intention that end-users should see an app URI.
Import paths are currently not shown to users at all, and if we need
to do this, they should be translated to file paths first.
In theory we could stick to file URIs with relative paths, but the Uri
class from dart:core makes this difficult, as certain operations on it
assume that file paths should be absolute.
Source mapping URIs are not yet affected by this change.
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2532053005 .
This replaces the old --sanity-check flag from dartk. Some files have
been renamed to avoid the wording "sanity check".
Compared to --sanity-check, the following checks have been added:
- variables are not referenced out of scope
- variables are not redeclared
- class type parameters are not referenced from static context
A unit test has been added to check that the verifier rejects certain
invalid ASTs.
BUG=
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2531873002 .
Sanity checks now run in a single pass rather than two, and
instead of building a set consisting of all members, it uses
a bit from the transformer flags to remember which members are
not orphaned.
As an additional check, it now checks that members are not
declared more than once.
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2529973002 .
This file is not strong-mode clean, but since a more recent version
of closure conversion still exists in a branch on the kernel github
repo, there is not much point in fixing this version of it.
BUG=
R=ahe@google.com
Review URL: https://codereview.chromium.org/2526793002 .
The concept of a binary library file no longer exists.
A kernel file can contain any number of libraries, and some of these
libraries can be "external". To reference a class or member from
another build, the class or member must be declared in an external
library.
Members in an external library contain all their type information,
but have no body.
Classes in an external library have their hierarchy information
present, but are not guaranteed to contain all their actual members.
The idea is that references themselves don't really cross module
boundaries, but rather refer to a local definition whose body is
contributed from elsewhere, much like 'external' members in Dart.
A modular backend such as DDC should be able to compile from one of
these kernel files without needing to load auxiliary information from
summaries or other kernel files.
For whole program transformations or backends, a linking step, which
is not yet implemented, must merge classes and members in external
libraries based on their name.
External libraries share the same IR and binary format as ordinary
libraries. Transformations that affect the interface for a member
or class should transform the external libraries alongside with the
internal ones, ideally without needing to treat them any different.
R=kmillikin@google.com
Review URL: https://chromereviews.googleplex.com/516847013 .
dartk and other command-line tools no longer rely on analyzer classes
like AnalysisContext or DartSdk, and instead use only the interface
defined in loader.dart.
The class AnalyzerLoader has been renamed to DartLoader.
The sharing of state in batch mode is now owned by loader.dart, so
dartk.dart does not deal with DartSdk and package_config.
Repository no longer has a package or SDK path. These concepts are
now specific to the Dart frontend. Consequently, loading a binary
file no longer relies on URI resolution.
The Repository class is now more clearly focused on linking symbolic
names to their IR objects, which for the time being is only
URI-to-library bindings, but should be extended to support linking
class and member references across different files and different
loaders.
BUG=
R=kmillikin@google.com
Review URL: https://chromereviews.googleplex.com/512907013 .
Paul Berry