While working on the e2e tests, I noticed this test was taking a
long time. Turns out this was because it fully exercises an http
client retry logic using exponential backoff and up to 4 retries.
To reduce the costs, I changed the test to use controlled
environment instead. This still hits the same code-paths but
provides a faster turnaround by using at most 1 retry, and issuing
retries immediately.
With these changes, the test completes in 3s (instead of 28s).
I noticed that some of the tests to check an "unreliable" server
where non-deterministic (multiple parallel requests that sometimes
fail or succeed). The changes here also make it deterministic. While
this test seems to have some flakes due to other reasons, my hope is
that these changes will reduce the chances of this test being flaky
on the bots.
Change-Id: I8eb2a00c7a489ab4e5a46e2b5cdbee40d1ffb8d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356623
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Ideally we would write each source next to each test to make tests more
readable, but clustering all tests to use a common test file really
helps boost the runtime of the end2end tests now that we reuse the same
page between tests that share the same code.
This CL refactos the tests to mostly use one shared test file. Doing so
speeds up tests between 2x-4x depending on the test.
Change-Id: I9afd00c6a42226899969e3f605ac18e0d2b1a906
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356285
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Since we changed how the test driver sets up breakpoints, we now we
have the ability to reuse a page for multiple tests.
Before, the test harness would follow this sequence:
* on each group of tests, it would call `_initSource`, which sets up
a boostraper and code to execute on the browser. This code was
reused by all tests in the group.
* on each test, it would indirectly call `_loadScript`, which
navigated to brand new page for each test. If there were multiple
tests in a group, this would load the boostraper and code once per
test.
This CL keeps the sequence of operations the same, except, that
`_loadScript` will reuse the existing page if it already loaded the
boostraper and DDC code needed by the next test.
This greatly reduces the total running time of some of the
expresison compiler shards. On my local machine one such test took
24s before both this and the parent CL, but now takes about 5s.
Change-Id: I7774df9e83142b42efc4ad523f589758a2a4660c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356301
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Mark Zhou <markzipan@google.com>
Instead of pausing the application while loading, this
changes the bootstrap logic to not invoke main immediately.
Later the test harness uses an expression evaluation to
trigger the start of the application when it is ready for
it.
This change will enable the ability to reuse the same page
and script across multiple tests, which will save a lot of
test execution time (see related child CL).
Change-Id: Ifa1050ac777aabb3e21b69953e85c6578b3e0cc0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356281
Reviewed-by: Mark Zhou <markzipan@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
The data shows a lot of variation by machine. The slowest test samples
bring the p90 above 4s, which explains that we often hit the limit.
I expect that we'll continue seeing flakes, but less often as a result
of this change. That said, some of the e2e test are taking close to 4
minutes, so we may need additional sharding or switch our approach if
that ocurrs more often.
Change-Id: I47c0dbce6f766edf7c2845d26f11ae7c6372a7bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355980
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Multiple end-to-end tests are failing in the bots because some steps
during the test execution are timing out. We currently have a
timeout of 5 seconds for these steps, but it's unclear if the
timeout is because of conditions in the bot machines (e.g. they are
slow), one off exceptions, or something else.
To address the flakiness, I expect we will want to increase the timeout,
but before we do it may be valuable to start measuring what is the
current distribution of timing in the bots to figure out the best value
to use.
This CL adds logging for timing data for that purpose. I've added
tracking for some interesting steps in the expresson-compiler test
framework, including the top two steps that usually hit a timeout in
practice (find-script and pause-event-for-line).
Example output on my local machine:
```
init-source: 233ms (avg), 121ms (p50), 696ms (p90), 696ms (max), 7 (total)
find-script: 18ms (avg), 18ms (p50), 21ms (p90), 25ms (max), 52 (total)
load-script: 22ms (avg), 22ms (p50), 27ms (p90), 30ms (max), 52 (total)
set-breakpoint: 0ms (avg), 1ms (p50), 1ms (p90), 2ms (max), 52 (total)
pause-event-for-line: 422ms (avg), 410ms (p50), 440ms (p90), 658ms (max), 52 (total)
check-in-frame: 459ms (avg), 446ms (p50), 480ms (p90), 695ms (max), 51 (total)
```
Change-Id: I04734b9905da8e1b4c9d06451be65a2dee32ba1c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/352988
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
This CL:
* adds empty dart:_dart2js_only and dart:_ddc_only libraries for use in
conditional imports,
* updates pkg/dart2js_runtime_metrics to use the new libraries rather
than dart:_dart2js_runtime_metrics and dart:js,
* and removes some unnecessary libraries, including VM-specific
implementations in pkg/dart2js_runtime_metrics and the DDC
implementation of dart:_dart2js_runtime_metrics.
Change-Id: I9500aa303fa5ad8aba0e1d413f69957c268f3f11
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/350681
Reviewed-by: Sigmund Cherem <sigmund@google.com>
When landing https://dart-review.googlesource.com/c/sdk/+/342400
the ddc-mac builder (but not Linux and Windows) started failing.
I could reproduce on Linux though so I'm not sure why those builders
didn't fail (nor why it wasn't caught by the try bot), either way this
fixes the issues:
* Test update: E.g. a breakpoint at the end of the scope doesn't work
because it will not be inside the (wanted) scope.
* Test update: Evalating "this" now actually works.
* Scope finding update: DDC adds sourcemapping entries for the *end* of
things so e.g. the getter "c" will have offset and offset+1 added as
source mappings. When translating from javascript position to dart
position we might pick that and thus ask for the scope of offset+1,
but nothing will be found because no node has that offset.
Here I add a fall-back saying that if we get no results we ask again
for the nearest lower offset.
Change-Id: I7e4430d9954466494b514cf51d999358483c9f8c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/345501
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Anna Gringauze <annagrin@google.com>
This allows using the new scope finder and facilitates expression
compilation with and in extension types.
For now the script uri is optional (and only passable in the new json
input via package:frontend_server), and only if the script uri is passed
we'll use the new scope finder.
Flutter etc should be updated to pass the new data.
Change-Id: I36eed1ea76a825e63e4c5b9ea60daf18aee39f3d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/342400
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Anna Gringauze <annagrin@google.com>
Stop running expression evaluation and module symbols tests on legacy
code. These still run with unsound null safety, just on migrated code.
Change-Id: Ifb602c058ebcc3fbd8254e8c2f431f8b18146a64
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/337615
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Anna Gringauze <annagrin@google.com>
The @dart marker should only be used for released language versions,
rather than the upcoming release, since the upcoming language features
has not been finally assigned to a language version yet. The appropriate
experiments should be used instead to opt into upcoming features.
This change is part of the effort to make it possible to easily bump
tools/VERSION without unexpected issues whenever tests have to have
their pinned language versions updated as well.
Bug: b/304745266
Change-Id: If9c7f07bb8bf0482257d0476b4e7d64077e53b8d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330106
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Please prefer not to revert this change. Any tests that fail are release
blockers and need to be approved and urgently fixed forward. Dart 3.2
has already been cut and the version number must be updated. It is
acceptable for rolls to be blocked until the release blockers have been
resolved as P0 issues.
Remove needless version number comment from experimental_features to
remove duplicated information and simplify version updates.
Update dartdoc to a version that supports 3.3.
Change-Id: I1a19aa86e185c99e61374665f18cf24c498935a5
Tested: CQ
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/329588
Commit-Queue: Slava Egorov <vegorov@google.com>
Auto-Submit: Jonas Termansen <sortie@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Remove needless version number comment from experimental_features to
remove duplicated information and simplify version updates.
Tested: CQ
Change-Id: I864bfc44070136406e95fdaf4d83f491b2c95943
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/329460
Reviewed-by: Alexander Thomas <athom@google.com>
This splits visitors for (Tree)Visitor(1) into pure interfaces, mixins,
and a base class with the base implementation. This is a step towards
avoid having an accidental default implementation where a static error
would have been preferable.
This extract a ConstantReferenceVisitor(1) and its corresponding
DefaultMixin from the Visitor(1) interface.
TEST=existing
Change-Id: Ibc55bed9cff76581deaade91a10600c17fafc6dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325704
Reviewed-by: Mayank Patke <fishythefish@google.com>
Reviewed-by: Ömer Ağacan <omersa@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
This is a reland of commit a9fc9ffc4d
Additional changes:
- Set incremental mode on generic class table
- fixes expression evaluation failure discovered by dwds tests
(generic class table was not always defined in compiled expression)
- Allow expression evaluation while the app is running in e2e suite
- Add regression tests for the expression evaluation failure.
Original change's description:
> [ddc] Add module local caches for new types
>
> - Provides fast access for types that are used multiple times in the
> same module.
> - Enable the existing type table cache when running with new types.
> - Add a similar cache for instantiated generic classes. This cache
> is used in the current type system as well to help keep the
> difference between types and classes more clear.
>
> Issue: https://github.com/dart-lang/sdk/issues/48585
> Change-Id: I32103cf0c0bcf9b9771e789c0d04e63a4365a066
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321320
> Commit-Queue: Nicholas Shahan <nshahan@google.com>
> Reviewed-by: Mark Zhou <markzipan@google.com>
Change-Id: I9c31d1d07d7f9bb15645ac9aa6e91d35e8906e85
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/323501
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Incremental compiler only runs procedure transformations during expression compilation, as opposed to modular library transformations
that are run during initial compilation stage on libraries (including JS interop-related transformations).
In this CL:
- Add implementation of `performTransformationsOnProcedure` to dev compiler target
- runs JS interop-related transformations on a procedure.
- Add related expression evaluation tests
- expressions using JS interop and extension types.
Closes: https://github.com/dart-lang/sdk/issues/53048
Change-Id: I085920db9c3af4c680283c574087d8901c99dfcf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319585
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
- Allow running the tests with asserts enabled
- Allow running the tests in canary mode
- Unify common test code and remove duplication
- merge expression_compiler/setup_compiler_options.dart
into share_compiler_options.dart
- Merge TestCompiler code from expression evaluation and
expression compilation tests and move into
expression_compiler/test_compiler.dart
- Rename various TestDrivers so they have more descriptive names
- Remove 'golden' JS comparison tests from expression_compiler_test.dart
- replace by evaluation tests where needed
Closes: https://github.com/dart-lang/sdk/issues/53145
Change-Id: Ic797fa4ee9bfa6b858b924be9f9a53fd10ae1448
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/318080
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
- Updates test infra to use the assets built by the new "ddc" build
targets.
- The old "dartdevc" targets are still present to avoid breaking
golem when this change lands.
- Old targets are still used when packaging assets for the SDK.
- Golem and the SDK packaging will be updated in following changes.
Change-Id: I1926e0c86833c812f5ed8355d7cd4df8740d79ee
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/315224
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Make pulling both events from the stream more explicit and
ensure that different errors appear when each event doesn't
arrive in time.
There are several issues with the previous approach:
- `.skip(1)` actually returns a new stream that will skip the
first event.
- Calling `.timeout()` on the new stream returns yet another
stream with the timeout.
- It appeared like each event should have a timeout but the
ordering means both the skip and first event actually shared
a single timeout.
Change-Id: I60062634a35424455c03b6e1778cc56bbdeb1716
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294080
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
The intention of this change is to break apart the suites and should
not add or remove any tests in any configuration.
- Moved records and patterns tests into a Dart 3.0 specific suite.
- Broke the before/after null safety agnostic tests into two shards.
- Added new suites to run all of the same tests in all of the same
configs.
Change-Id: If2aea16f1f9e8ac01b1aab49a99d652813fe42fc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293384
Reviewed-by: Anna Gringauze <annagrin@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Sigmund Cherem