Commit Graph

25 Commits

Author SHA1 Message Date
Mark Zhou 8b28475baa [dartdevc] Extending e2e eval tests to multiple module systems.
* Now tests both DDC + AMD module systems
* Porting more tests + purging them from the regular test
* Fixing a bug related to string-typed JS scope objects

Change-Id: I703d3760971ef8c14ef4157a71b364d8d3c22aaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194080
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Mark Zhou <markzipan@google.com>
2021-04-06 22:22:43 +00:00
Mark Zhou 9361234cd2 [dartdevc] Splitting SDK eval e2e tests.
Adding 'dartdevc_test' target to pkg-X builders.
Splits eval tests into sound and unsound suites.
Adding more robust event polling (instead of just waiting 500 ms).
Turns on sound mode for the ported tests.
Also ports a couple more tests.

Change-Id: Ia8e9095e1e3330ea1732a4fa4f14425777de5127
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193446
Reviewed-by: Anna Gringauze <annagrin@google.com>
Commit-Queue: Mark Zhou <markzipan@google.com>
2021-04-05 16:57:36 +00:00
Nicholas Shahan 38e5e9bc19 [ddc] Update pedantic version to 1.11.0
Apply new lints, cleanup violations, and ignore in `lib/src/js_ast` to
avoid adding additional diffs with the version it was forked from.

Change-Id: I1def25865bf195120c594223ebdc0f59139e75b7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192241
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
2021-03-24 17:35:32 +00:00
Anna Gringauze e2baf16ebd Support ddc module format in expression compilation in ddc.
Closes: https://github.com/dart-lang/sdk/issues/45303
Change-Id: I2b1eb9d149c03c138150d126c108ad86af8584c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191023
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
2021-03-15 23:58:44 +00:00
Anna Gringauze 14eeb7507f First version of incremental DDC mode for expression compilation
- Support evaluate() calls from VM service in expression compiler
  - emit all accessed symbols, types, constants, extension symbols,
    and imports as part of synthetic evaluation function
    - Note: this fixes missing symbol issues in evaluateInFrame()
      as well
  - update expression evaluation tests
  - fix expression compilation broken after hot reload

See widget inspector layout explorer, a result of evaluate() call:
https://drive.google.com/file/d/16UdSE5_V1ZRXAf2KeBxNwYNHMfo1RbnT/view?usp=sharing&resourcekey=0-HZcPm68VbsVzrZ672CApvA

Closes: https://github.com/dart-lang/sdk/issues/41480
Closes: https://github.com/dart-lang/sdk/issues/44979
Closes: https://github.com/dart-lang/sdk/issues/44713
Closes: https://github.com/dart-lang/sdk/issues/44933
Closes: https://github.com/dart-lang/sdk/issues/44813
Closes: https://github.com/dart-lang/sdk/issues/44686
Change-Id: I96c74578c51503adbc4bfe6d6e6112319addc959
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188400
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Mark Zhou <markzipan@google.com>
2021-03-09 17:23:05 +00:00
Anna Gringauze d6cf6b6bc9 Pass type parameters to expression evaluation function
- Pass type parameters to the synthetic expression evaluation
  function
- Add tests

Closes: https://github.com/dart-lang/sdk/issues/44755
Change-Id: Ifbc859b72c616d8f47a76ad9de2cc2aeef67e636
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181660
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
2021-01-29 01:24:05 +00:00
Mark Zhou f09a5d4243 [dartdevc] Avoid caching constants during eval.
Avoids masking the issue underlying #44713.

Change-Id: I107c1ea3aab18a97ac08e868741fa0929e2292b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181304
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Mark Zhou <markzipan@google.com>
2021-01-27 20:25:45 +00:00
Anna Gringauze b1b0a6ce14 Do not add variables to scope that appear after the breakpoint
During expression evaluation, we are collecting  all variables in
all scopes that contain the current breakpoint line, and adding them
as available in current scope, which makes variables declared below
the current breakpoint line declared but undefined in JavaScript.

There is one exception to this seen so far - a variable
declared on the current breakpoint line might appear as not declared
in JavaScript and cause expression evaluation not to work due to
JS compilation errors.

This change fixes the issue by not collecting variables that are
declared on or after the current breakpoint line, making them undeclared
in dart (which also is correct according to dart scoping rules).

Closes: https://github.com/flutter/flutter/issues/72094
Change-Id: I113b69531171e0348d44edb8db6dd08a599c9db3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177760
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Mark Zhou <markzipan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2021-01-07 23:55:59 +00:00
Mark Zhou bd30532d82 [dartdevc] Creating container entrypoints for expression eval.
The expression evaluator should now index/read object containers (types and symbols right now) via accessors like 'A$Eval'.

Fixes #44494

Change-Id: I31d203cfe59a6e2c03eeb22e8c86270c28b5e2fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176923
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Mark Zhou <markzipan@google.com>
2021-01-06 00:20:29 +00:00
Sam Rawlins 5ece7f3a6e dev_compiler: Remove unnecessary imports
Bug: https://github.com/dart-lang/sdk/issues/44569
Change-Id: Icad2908772eb42593c0149a99b8256d470490670
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177241
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Samuel Rawlins <srawlins@google.com>
2021-01-05 19:57:57 +00:00
Mark Zhou 2e97d91dcd [dartdevc] Fixing expression eval breaks from stepping update.
Followup on https://dart-review.googlesource.com/c/sdk/+/173800

This change forces certain top-level containers to be emitted (even if empty) per module. These modules also carry a self-referential element so that V8 does not optimize them away.

Change-Id: Ic02adb9c4e322c29ba092951fd912019931b3871
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175445
Reviewed-by: Anna Gringauze <annagrin@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Mark Zhou <markzipan@google.com>
2020-12-11 00:18:28 +00:00
Nicholas Shahan d0ee5dbe40 [ddc] Verify null safety modes of modules agree
Checks that the null safety mode of every module loaded matches the mode
of the SDK.

This relands https://dart-review.googlesource.com/c/sdk/+/173120 with
additional fix to correctly set the mode of a new component created
in the frontend_server.

Change-Id: I855b226f724ce0b6c83dbdc11ff6113853892dbb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174881
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Anna Gringauze <annagrin@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
2020-12-09 21:12:20 +00:00
Nicholas Shahan a9f3c66bd8 Revert "[ddc] Verify null safety modes of modules agree"
This reverts commit fda897a423.

Reason for revert: Breaks all the flutter web tests here:
https://ci.chromium.org/ui/p/dart/builders/ci.sandbox/flutter-engine-linux-web_tests/1089/overview

This indicates we still have a misconfiguration in flutter web dev 
builds where artifacts with different null safety modes are being 
mixed.

Original change's description:
> [ddc] Verify null safety modes of modules agree
>
> Checks that the null safety mode of every module loaded matches the mode
> of the SDK.
>
> Change-Id: I50543d10fbea22a61bd7f12d51d6e9c8f8286890
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173120
> Commit-Queue: Nicholas Shahan <nshahan@google.com>
> Reviewed-by: Mark Zhou <markzipan@google.com>

TBR=sigmund@google.com,nshahan@google.com,markzipan@google.com

Change-Id: I0d09efde53f268b353cb842286b11bb3f375e754
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173440
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
2020-11-21 17:42:25 +00:00
Nicholas Shahan fda897a423 [ddc] Verify null safety modes of modules agree
Checks that the null safety mode of every module loaded matches the mode
of the SDK.

Change-Id: I50543d10fbea22a61bd7f12d51d6e9c8f8286890
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173120
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Mark Zhou <markzipan@google.com>
2020-11-21 00:51:14 +00:00
Anna Gringauze c6c4661567 Enable fixed expression compilation tests, add more loop tests.
Tests are no longer failing since
https://github.com/dart-lang/sdk/issues/44235 is now fixed.

Change-Id: I2654233e4c62426e363eab518fdb5b041bc22724
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173160
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
2020-11-20 21:47:23 +00:00
Anna Gringauze 57cbfa73ed Add tests for expression compilation with sound null safety
Change-Id: I895b769ce203c2aed0cfcce9f1fca70f42dbb97c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172689
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
2020-11-19 01:24:41 +00:00
Anna Gringauze 9c54996510 Fix issues in expression evaluation in google3
- Define private symbols load without reloading the module
  and knowing module format
- Remove libraryName->moduleName map from ExpressionCompiler's
  compileExpressionToJs API as a result of above
- Make ddc tolerate null locations on kernel loaded from dill
  during assert insertions for nullability
- Disable asserts in expression compilation worker
  due to ddc reading source for assert statement compilation
- Remove unused test configuration for expression compiler worker
- Add and update expression compiler tests

Part of: https://github.com/dart-lang/webdev/issues/1174

Change-Id: Idf8f508308915a5487fa0e9810b5674140393fed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170028
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
2020-11-14 01:34:16 +00:00
Anna Gringauze 42dc617ee8 Emit constants during expression compilation to js
Closes: https://github.com/dart-lang/sdk/issues/43963
Change-Id: Ic8af046b2cf1506984b84ea66b2676b3cc0f1771
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/169580
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
2020-11-02 20:03:14 +00:00
Nicholas Shahan 2615e865e2 [ddc] Opt code in expression compiler test
Avoids breaks that appear when enabling the `nnbd` experiment
by default as seen in:
https://dart-review.googlesource.com/c/sdk/+/166790

Change-Id: I052853c42b772af9180d109f231215c00a5a8202
Fixes: https://github.com/dart-lang/sdk/issues/43843
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168742
Reviewed-by: Anna Gringauze <annagrin@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
2020-10-23 17:49:22 +00:00
Anna Gringauze f04bc85e4f Refined scope calculation in expression compiler
- make sure nested blocks scopes are visited
- make sure variable definitions do not leak beyond block scopes
- properly collect scopes for loops, if statements, constructors
- add calculation of fileEndOffsets for blocks
- save block file offsets to dill
  - update binary format version
  - change kernel readers and writers to read and write block offsets
  - change vm readers to read and block offsets for new version
- add missing fileOffsets and fileEndOffsets on functions for
  late fields
- add missing fileOffsets and fileEndOffsets on functions for
  extensions
- add errors on failures to find scope
- find libraries for private fields correctly
- add more expression compilation tests
- add test to verify fileOffsets and fileEndOffsets are set for
  SDK summary (will add full dill tests later)

Closes: https://github.com/dart-lang/sdk/issues/40278
Related: https://github.com/dart-lang/sdk/issues/34942

Change-Id: I5bc1bb645543045b689d8d61069ee77dc4ee9025
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/167541
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-10-20 01:44:52 +00:00
Nicholas Shahan f3962367a5 [ddc] Add language version to test files
Avoids breaks that appear when enabling the `nnbd` experiment
by default as seen in:
https://dart-review.googlesource.com/c/sdk/+/166790

Some of these synthetic files are created as packages but the
expression compiler wasn't designed to handle that information.
In the future we should support language versioning better in the
expression compiler or setup similar tests for opted in code since
in reality the language versioning might be handled by the dev
build system.

Fixes: https://github.com/dart-lang/sdk/issues/43843

Change-Id: Icfac40d1c9b47e75fb92d7672700e066ac372267
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168441
Reviewed-by: Anna Gringauze <annagrin@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
2020-10-20 00:48:08 +00:00
Anna Gringauze a8c727a61a Use types of parameters for expression compilation
- use parameter types passed to IncrementalCompiler.compileExpression
  for compilation of the synthetic function containing the expression
- updated expression compiler tests

Note: some expressions that previously compiled wouldn't now
Example:

** package:test/library.dart **

class TestLibraryClass {
  final int field;
  final int _field;
  TestLibraryClass(this.field, this._field);
}

** main.dart **

import 'package:test/library.dart';

void main() {
  var instance = TestLibraryClass(1, 2);
  print('$instance'); // Breakpoint here
}

Expression:

`instance._field`

Previously, the incremental compiler would compile the expression
to kernel due to the type of instance being Dynamic. That compilation
result made it impossible for dartdevc to generate correct code due
 to missing type information.

Now, the compilation will fail with "Getter isn't defined"
error. To properly evaluate private fields now, we would need
the CFE to support a special mode for expression compilation
where private fields from other libraries are allowed.

Closes: https://github.com/dart-lang/sdk/issues/43469
Change-Id: Id8015c5e334115a82049fb2d1725db54dd3ea3dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166925
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
2020-10-13 01:44:02 +00:00
Leaf Petersen e85bf740ef Opt pkg/dev_compiler out of null safety.
Change-Id: I62207a436605480b6981b8b00bf7b88d8589db0f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166980
Commit-Queue: Leaf Petersen <leafp@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
2020-10-12 23:25:42 +00:00
Anna Gringauze 1c8020cfc7 Emit module and add logging with timer to expression compilation
- Emit module before expression compilation to JavaScript so the
  ProgramCompiler has all types available.

- Add logging with ticker to expression compilation in expression
  compilation worker in DDC and in frontend server.

- Remove excessively verbose logging

Related:
https://github.com/dart-lang/webdev/issues/1006
https://github.com/dart-lang/webdev/issues/1106
Change-Id: Ia3a985f2d12d27f0bcc2c5d3923db137070e9cdd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162624
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
2020-09-22 02:17:12 +00:00
Anna Gringauze 34052bf2bb Add expression evaluation to ddc for google3
- Move expression evaluation to ddc in preparation for google3
- Added server to ddc to handle update and compileExpression requests
- Added tests
- Added 'experimental-output-compiled-kernel' option to ddc to generate
  full kernel files only for compiled libraries, and store with
  '.full.dill' extension
- Added AssetFileSystem to communicate to the asset server in the
  debugger
- Made expression_compiler_worker work with full kernel files,
  so removed invalidation of current file to improve performance
- Made expression_compiler_worker reuse already loaded imports
  to avoid reading them from source in the incremental compiler
- Updated tests to work with DDC (for simulating webdev)
- Disabled tests that work with bazel kernel worker for now
  as it does not generate full dill files yet
- Addressed code review comments from the prototype version:
  https://dart-review.googlesource.com/c/sdk/+/157005

Details:

Currently, in flutter tools, expression evaluation is supported via
expression compilation, which is done by the incremental compiler in
the frontend server. The same incremental compiler is used for initial
application compilation, incremental code compilation for hot reload,
and any number of expression compilation requests.

In google3, the apps are typically too large to be compiled as a whole
in memory by the frontend server. Build in google3 is currently done by
blaze, as a distributed build using a task dependency graph. Build tasks
output kernel outline files as an interface between components produced
by individual tasks.

We are proposing an implementation of the expression compilation in
google3 that is taking advantage of full kernel files produced by the
build (supporting build changes to follow). This change introduces a
small server based on dev_compiler, which can handle following requests:

- update: load full kernel for given modules (done on app start)
- compileExpression: compile expression in a given library and module
  (done when paused on a breakpoint)

Expression compilation uses previously loaded kernel files for the
application component and its dependencies to compile an expression.


Change-Id: Icf73868069faf3a2eb6d43ba78e459f8457e9e35
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160944
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
2020-09-03 03:05:13 +00:00