Commit Graph

455 Commits

Author SHA1 Message Date
Lasse R.H. Nielsen 973a1a0219 Remove uses of upper-case constants in remaining SDK code.
This includes Fasta, tools and observatory, so the checked-in SDK must
have the lower-case constants.

Change-Id: I8380ad041ad058f7d02ae19caccfecd434d13d75
Reviewed-on: https://dart-review.googlesource.com/50201
Commit-Queue: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
2018-04-16 16:11:57 +00:00
Martin Kustermann 52c884dc6a Add ExtraSlow status file marker
This is used (at least temporarily) for unblocking other P0 changes.

Change-Id: I002af7cfe6384f8813646d5c488a2cf2e58673a5
Reviewed-on: https://dart-review.googlesource.com/46541
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-03-14 15:49:09 +00:00
Peter von der Ahé 5882a0d13d Address comments from previous CL
Change-Id: If98db1948e21c43eed60c7e1ee18c9fff61be88c
Reviewed-on: https://dart-review.googlesource.com/40540
Commit-Queue: Peter von der Ahé <ahe@google.com>
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2018-02-12 18:25:11 +00:00
Peter von der Ahé bc7d1137fc Implement Fasta compiler configuration
Change-Id: Id17c35592838fd5ccc203b9e4ea0a86a29fe527e
Reviewed-on: https://dart-review.googlesource.com/36643
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2018-02-12 11:39:18 +00:00
Jens Johansen a37c34b5fb [test.py] Pass environment to dartdev[ck] through createCommand
Some tests are run via createCommand, some via
computeCompilationArtifact. This meant that some tests got an
environment and some didn't, resulting in the batch workers
getting restarted all the time.
This CL fixes the issue by passing the environment to createCommand as
well, meaning that the batch workers doesn't restart unnecessarily.

Before:

$ cls; xvfb-run -a '--server-args=-screen 0 1024x768x24' python -u ./tools/test.py -mrelease --checked --strong --use-sdk -rchrome -cdartdevk language_2
[03:09 | 100% | + 5171 | -    0]

$ cls; xvfb-run -a '--server-args=-screen 0 1024x768x24' python -u ./tools/test.py -mrelease --checked --strong --use-sdk -rchrome -cdartdevc language_2
[02:33 | 100% | + 5170 | -    0]


Now:

$ cls; xvfb-run -a '--server-args=-screen 0 1024x768x24' python -u ./tools/test.py -mrelease --checked --strong --use-sdk -rchrome -cdartdevk language_2
[01:41 | 100% | + 5171 | -    0]

$ cls; xvfb-run -a '--server-args=-screen 0 1024x768x24' python -u ./tools/test.py -mrelease --checked --strong --use-sdk -rchrome -cdartdevc language_2
[01:59 | 100% | + 5170 | -    0]


I.e. dartdevk tests are now almost twice as fast (on my machine).

Bug:
Change-Id: I15111436d3411b6ba85209950e4a5f20ee515539
Reviewed-on: https://dart-review.googlesource.com/29100
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2017-12-13 12:22:55 +00:00
Martin Kustermann 317c55fa2c Reland "Use batch mode compilation for -cdartkp --strong"
Preliminary testing shows, this increases performance by around 20%. The
main benefit is by re-using a warmed-up VM and not start one from
scratch for every compilation.

Going forward we can do more optimizations, e.g. reading the platform
dill file only once into memory (instead of repeatedly) ...

  => This requires us using the new state-full IKG compiler.

Issue https://github.com/dart-lang/sdk/issues/31585

Change-Id: I2d3448783fc118611baf4671187a897227a65a6c
Reviewed-on: https://dart-review.googlesource.com/28400
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2017-12-12 10:14:20 +00:00
Alan Knight af0b27480a Revert "Use batch mode compilation for -cdartkp --strong"
This reverts commit 8c38a415de.

Bug:
Change-Id: I1b159f2d2d10b51ee843a2ba9a61feb339152338
Reviewed-on: https://dart-review.googlesource.com/28362
Reviewed-by: Martin Kustermann <kustermann@google.com>
2017-12-11 18:22:49 +00:00
Martin Kustermann 8c38a415de Use batch mode compilation for -cdartkp --strong
Preliminary testing shows, this increases performance by around 20%. The
main benefit is by re-using a warmed-up VM and not start one from
scratch for every compilation.

Going forward we can do more optimizations, e.g. reading the platform
dill file only once into memory (instead of repeatedly) ...

  => This requires us using the new state-full IKG compiler.

Change-Id: I74db7dbb1aa79289d1045ef41f960215cf5b3b35
Reviewed-on: https://dart-review.googlesource.com/28240
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2017-12-11 15:55:31 +00:00
Bob Nystrom 1e6e72af79 Refactor some code around TestSuite.
I was working on adding support for treating RuntimeError as Pass if
runtime == none and I noticed a lot of dead or redundant code, so
cleaned it up some.

Change-Id: I0be451a7075d99bc07d1407883861fb06d777c02
Reviewed-on: https://dart-review.googlesource.com/22800
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2017-11-22 16:56:17 +00:00
Sigmund Cherem b69cc7c98b Sort expectations in test.dart --list
This makes it easier to compare the output of two different runs of test.py
--list.

Change-Id: Iabbbd4d6f60b9b7523e2773520a8b2d588c99ae3
Reviewed-on: https://dart-review.googlesource.com/21520
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2017-11-18 00:51:06 +00:00
Paul Berry 39fc40447e Fix subscription types in test_runner.dart.
This should fix the buildbot failure in vm-kernel-linux-release-x64-be.

Change-Id: I4078ff4eb677614684cf07dc70a50641d6926aa1
Reviewed-on: https://dart-review.googlesource.com/20905
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
2017-11-15 00:51:15 +00:00
Erik Ernst 40b6741778 Adds support for syntax error in tests to test.py.
This CL modifies the Dart source used from test.py such that it takes
`syntax error` into account as an expected outcome in test files (so
that we can have `//# 01: syntax error` with a similar meaning as
`//# 01: compile-time error`).

For all tools except the spec_parser, `syntax error` is the same
outcome as `compile-time error`; that is, nobody else will see the
difference.

For the spec_parser, `syntax error` is the outcome where parsing has
failed; `compile-time error` is taken to mean some other compile-time
error, i.e., the spec_parser is expected to _succeed_ when the
expected outcome is `compile-time error`.

Test files in language and language_2 have been adjusted to use the
outcome `syntax error` where appropriate.

The status files in language and language_2 for the spec_parser have
been adjusted such that they fit all the new `syntax error` outcomes
in test files.

Other status files have been adjusted in a few cases where tests were
corrected (because a compile-time error which was clearly not intended
to be a syntax error turned out to be caused by a typo, which means
that the actual compile-time error has never been tested).

The spec grammar Dart.g was adjusted in a few cases, when some bugs
were discovered. In particular, the treatment of Function has been
changed: It is now known by the parser that Function does not take
any type arguments. This makes no difference for developers, because
they cannot declare a type named Function anyway, but it means that
a number of tricky parsing issues were resolved.

Dart.g was also adjusted to allow `qualified` to contain three
identifiers, which is an old bug (preventing things like metadata on
the form `@p.C.myConst`).

Change-Id: Ie420887d45c882ef97c84143365219f8aa0d2933
Reviewed-on: https://dart-review.googlesource.com/18262
Commit-Queue: Erik Ernst <eernst@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
2017-11-06 08:56:09 +00:00
Jenny Messerly 384678e4b3 port DDC's code_generator to Kernel
Change-Id: I2b6a5f56a69f5d3a965aefc3f8d4550d5940ac5a
Reviewed-on: https://dart-review.googlesource.com/17980
Reviewed-by: Vijay Menon <vsm@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
2017-11-04 22:17:56 +00:00
Bob Nystrom ba2e4aff24 Some minor clean-up in test.dart.
- Fix fuzzy arrow errors.
- Default the Dart repo URI so it doesn't always need to be explicitly
  set.
- Run dartfmt on everything.
- Move some members out of the garbage bin "TestUtils" class:
  - Move the stuff around the Dart repo directory to a new Repository
    class.
  - Move absolute() into Path where it belongs.
  - Move workingDirectory in Path since it is a Path.
  - Delete the random number stuff since it was apparently unused.

Change-Id: I3dab3a4f1713b7a749e64b6776149d05a0ce1b69
Reviewed-on: https://dart-review.googlesource.com/14502
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2017-10-19 19:05:58 +00:00
Morten Krogh-Jespersen 2564d99b65 New limit for truncating stdoutput from a running test.
Bug:
Change-Id: I0ad4ea30ef0ca0b9e1f15de8dd0313be27bcff12
Reviewed-on: https://dart-review.googlesource.com/11445
Commit-Queue: Morten Krogh-jespersen <mkroghj@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2017-10-05 12:48:00 +00:00
Florian Loitsch 38bf70d7ac Use generic functions in zones.
Migrated from https://chromiumcodereview.appspot.com/2893893002/

Change-Id: I0bd6dc1438eb1e6762e7760a08b5a760b07d4b10
Reviewed-on: https://dart-review.googlesource.com/4942
Reviewed-by: Florian Loitsch <floitsch@google.com>
Commit-Queue: Florian Loitsch <floitsch@google.com>
2017-09-20 15:46:06 +00:00
Jennifer Messerly 62303b6711 enable batch mode for dartdevc tests, also fix status so ddc bots pass
R=rnystrom@google.com

Review-Url: https://codereview.chromium.org/2987393002 .
2017-08-03 14:39:45 -07:00
Bob Nystrom 796eb21071 Move the status file parser into its own package.
This required munging a bit of test.dart code too to tease out some
dependencies, but the changes are minor. I considered moving all of
test.dart out into a package and making the status file library a
public one that other packages in the repo could import but this seemed
like the less intrusive change.

R=bkonyi@google.com

Review-Url: https://codereview.chromium.org/2984203002 .
2017-07-25 15:29:59 -07:00
Bob Nystrom 012774c78e Remove Dartium support from test.dart.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2981223002 .
2017-07-20 12:46:50 -07:00
Bob Nystrom 82df163eca Refactor dependency_graph.dart.
It multiplexed a bunch of different events onto a single stream. But
every single listener on that stream immediately demuxed them and only
cared about a subset of the events.

So I made separate streams for each event. Other minor clean-ups too.

R=sigmund@google.com, whesse@google.com

Review-Url: https://codereview.chromium.org/2957703002 .
2017-06-26 17:07:37 -07:00
Bob Nystrom 556c0550bf Simplify CommandOutput and friends.
- Merge CommandOutputImpl and CommandOutput. There were no classes that
  implemented CommandOutput that didn't extend the Impl.
- Remove "Impl" from the other class names.
- Make stuff private when possible. Likewise final.
- Other tiny style changes.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2946783002 .
2017-06-23 12:13:31 -07:00
Bob Nystrom b452b39962 Basic support for dev_compiler in test.dart.
It can compile and run tests on Chrome. There are a lot of failing tests
that I (or the team) will need to triage, but I think at least basic
tests are working as expected.

There is code that could be cleaned up to more neatly factor how dart2js
and dartdevc are handled now that there are two separate compilers to
JS. There's also some redundant code between the path for testing
compile errors (enqueueStandardTest()) and the path for running a test
in the browser.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2947473002 .
2017-06-21 13:19:28 -07:00
Bob Nystrom a6ca718e98 Simplify Command classes.
- Get rid of separate CommandBuilder class and singleton pattern. It
  was being passed around explicitly even though half of the places
  that received a CommandBuilder as a parameter still directly called
  CommandBuilder.instance instead of using it.
- Get rid of Command caching. As far as I can tell, it makes no
  measurable difference in runtime performance or memory usage. Even
  with a large invocation of a lot of configurations and tests, the
  Command classes don't seem to be a significant use of memory.
- Shorten the factory names. "get" adds no value, and we know it
  returns a "Command" since it's on Command.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2933973002 .
2017-06-15 14:19:03 -07:00
Bob Nystrom 979026abef Don't allow "none" as a runtime for dartk.
Also, add a little sanity checking in TestCase.

Fix #29842.

R=ahe@google.com

Review-Url: https://codereview.chromium.org/2941603002 .
2017-06-14 10:31:09 -07:00
William Hesse e64ee95959 Move ie11 timeout handling code to BrowserControllerTestOutcome
Only content_shell testing uses the BrowserCommandOutputImpl class.
Rename the class to ContentShellCommandOutputImpl.

BUG=https://github.com/dart-lang/sdk/issues/28955
BUG=https://github.com/dart-lang/sdk/issues/29869
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2934243002 .
2017-06-14 15:50:41 +02:00
William Hesse d98d32b63c Report ie11 timeouts in debug log, not as errors (attempt 2)
BUG=https://github.com/dart-lang/sdk/issues/28955
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2938813002 .
2017-06-14 14:17:27 +02:00
Bob Nystrom 8bc4e89219 Remove unused record and replay functionality.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2920633002 .
2017-06-08 09:18:03 -07:00
Bob Nystrom 7e2d9ac85d Revert "Revert "Replace the configuration map with a typed object.""
This reverts commit 8bada4873a.

R=nweiz@google.com

Review-Url: https://codereview.chromium.org/2919573003 .
2017-05-31 15:39:23 -07:00
Bob Nystrom 8bada4873a Revert "Replace the configuration map with a typed object."
This reverts commit 06f75fe5cd.

Review-Url: https://codereview.chromium.org/2914893003 .
2017-05-31 15:08:21 -07:00
Bob Nystrom 06f75fe5cd Replace the configuration map with a typed object.
This is a pretty massive change but my hope is it will make
test.dart easier to maintain going forward. Instead of
passing around a stringly-typed Map<String, dynamic> all
throughout the program, we parse the options and then create
a Configuration object that has typed getters for all of the
various bits of configuration data.

This is a little tedious because it means declaring a new
option requires also declaring a corresponding field in the
Configuration class and passing it through the constructor.
I think it's worth it.

Also, enum-like configuration properties like architecture
and runtime now have their own classes as well. Moved a
bunch of stuff from TestUtil into those classes now that
there is an object to hang those methods off of.

In the process, I found a few typos in string literals where
the code wasn't correctly looking up a configuration
property.

Added more sanity checking and validation to status file
parsing. You will get an error at parse time if you try to
refer to a variable that isn't in the whitelist of known
variables. Also, you'll get an error if you try to compare
a variable to a value that it isn't expected to have.

Many other small-scale cleanups.

Aside from the status file validation, this should behave
mostly the same as current test.dart except that tests
may be enqueued in a slightly different order. The
rewritten code for expanding configurations iterates
through the architecture, runtime, etc. options in a
slightly different order.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2901923003 .
2017-05-31 14:47:31 -07:00
Bob Nystrom 7ce978bf91 Tighten types in test.dart even more.
This removes all implicit casts and many implicit uses of dynamic. It
adds a gratuitous number of explicit "as" casts and arguably makes the
code worse.

This is an interim step towards replacing the big configuration map
with an actual typed object. These "as" casts should help catch places
where the configuration object is being used and where the code will
need to be changed to use a new object.

R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2903703002 .
2017-05-30 14:02:24 -07:00
Martin Kustermann 22c7536d36 Reduce temporarily flakiness rate of dart2js-drt by re-running until they have been fixed
We should either remove dart2js-drt or use a stable version of
content_shell from upstream chromium (instead of our dartium-based one)
for dart2js-drt.

Issue #29655

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2891343002 .
2017-05-19 13:13:40 +02:00
Bob Nystrom 92267afc66 Refactor and clean up the status file parsing code. - Make the parser less error tolerant. The expression parser used to ignore any unrecognized tokens, which means a status like "RuntimeError CompileError" (not the missing comma) was parsed as simply "RuntimeError", which seems bad. Now it reports an error. Fixed a couple of status files that thought they were setting statuses that they weren't (!).
- Separate out parsing a status file from applying the environment to
  determine which sections are active. This makes it possible to, for
  example, generate expectation sets for multiple environments without
  having to reparse each time.

- Simplify expression parsing. Remove set expressions since they weren't
  used for anything useful. A test's expectations are a simple
  comma-separated list and don't need anything beyond that. Merge
  Scanner and Tokenizer since the latter was a glorified function.

- Make more names private so that it's clearer what's used outside of
  various libraries.

- Generally modernize the style.

- Add *lots* of documentation.

Again, there should be no behavioral changes. I ran:

  ./tools/test.py -m release,debug -c none,dart2js,dart2analyzer -r none,vm,d8 corelib

Before and after the change and verified that the output was the same
(aside from timing).

R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2891753003 .
2017-05-18 12:42:52 -07:00
Bob Nystrom c269ef53a5 Revert "Re-apply status file parser changes from 0b7728da1bef08c1c1e092005d9fd8c8bff5fa6c."
This reverts commit a881aed354.

TBR.

R=paulberry@google.com

Review-Url: https://codereview.chromium.org/2888213002 .
2017-05-17 13:45:40 -07:00
Bob Nystrom a881aed354 Re-apply status file parser changes from 0b7728da1b.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2880203004 .
2017-05-17 13:04:35 -07:00
Bob Nystrom 01e5b1b40c Revert "Refactor and clean up the status file parsing code."
This reverts commit 0b7728da1b.

R=nbosch@google.com

Review-Url: https://codereview.chromium.org/2885623002 .
2017-05-15 15:58:05 -07:00
Bob Nystrom 0b7728da1b Refactor and clean up the status file parsing code.
- Make the parser less error tolerant. The expression parser used to
  ignore any unrecognized tokens, which means a status like
  "RuntimeError CompileError" (not the missing comma) was parsed as
  simply "RuntimeError", which seems bad. Now it reports an error.
  Fixed a couple of status files that thought they were setting statuses
  that they weren't (!).

- Separate out parsing a status file from applying the environment to
  determine which sections are active. This makes it possible to, for
  example, generate expectation sets for multiple environments without
  having to reparse each time.

- Simplify expression parsing. Remove set expressions since they weren't
  used for anything useful. A test's expectations are a simple
  comma-separated list and don't need anything beyond that. Merge
  Scanner and Tokenizer since the latter was a glorified function.

- Make more names private so that it's clearer what's used outside of
  various libraries.

- Generally modernize the style.

- Add *lots* of documentation.

Again, there should be no behavioral changes. I ran:

  ./tools/test.py -m release,debug -c none,dart2js,dart2analyzer -r none,vm,d8 corelib

Before and after the change and verified that the output was the same
(aside from timing).

BUG=
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2875203005 .
2017-05-15 15:42:15 -07:00
Bob Nystrom e0180925f6 Tighten up a bunch of types in test.dart.
This doesn't get rid of all implicit dynamic and casts, but it covers
many of them.

Also did some minor modernization when I noticed it:

- Using "var" for local variables where inference does the right thing.
- camelCase for variable names.
- More collection literals.

There are (or should be!) zero behavioral changes.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2863253002 .
2017-05-15 15:26:40 -07:00
William Hesse 6f3d45f5a3 Check for infrastructure failure before reporting timeouts and crashes.
BUG=http://dartbug.com/28955

Review-Url: https://codereview.chromium.org/2881073002 .
2017-05-15 18:27:46 +02:00
William Hesse bff560292c Fix bug in test_runner.dart
This error was introduced in https://codereview.chromium.org/2875683002/
committed as d6ca1a5def.

BUG=
TBR=karlklose@google.com

Review-Url: https://codereview.chromium.org/2874193002 .
2017-05-11 13:39:46 +02:00
William Hesse d6ca1a5def Report IE11 timeouts in debug log, not as failing tests.
BUG=https://github.com/dart-lang/sdk/issues/28955
R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2875683002 .
2017-05-11 11:18:42 +02:00
Terry Lucas f27144d7b5 Roll 50: Updated for push to origin/master.
TBR=jacobr@google.com

Review-Url: https://codereview.chromium.org/2875773003 .
2017-05-10 18:16:51 -07:00
Bob Nystrom f5471fb948 Make test.dart strong mode clean!
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2855883008 .
2017-05-04 10:48:39 -07:00
Erik Corry aa6353b6da Dart SDK Spelling b, c, and d.
R=kmillikin@google.com
BUG=

Review-Url: https://codereview.chromium.org/2850783002 .
2017-05-01 08:28:10 +02:00
Bob Nystrom 5a03535766 Remove the "pkgbuild" step on the bots.
Those tests are flaky since they fail when AppEngine flakes out. Even
when they work, they don't provide much value.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2801143003 .
2017-04-24 13:28:16 -07:00
Jacob Richman 119b2d58ce Format all files under tools and utils directory.
BUG=
R=terry@google.com

Review-Url: https://codereview.chromium.org/2827793002 .
2017-04-20 09:08:31 -07:00
Ryan Macnak 25282fbf64 test.py: Push dart_precompiled_runtime and process_test once per device instead of once per test.
R=kustermann@google.com, vegorov@google.com

Review-Url: https://codereview.chromium.org/2750403006 .
2017-03-21 09:45:35 -07:00
Zach Anderson 7af47d5ea9 [test.dart]: Fix output encoding in case of non-utf8 test output
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2721683002 .
2017-02-28 07:50:28 -08:00
Vyacheslav Egorov b026aed3cf Remove dartk and DFE workers support from testing scripts and kernel-service.
We only support running -c dark and -c dartkp configuration through Kernel isolate and fasta in non-batch mode.

R=kustermann@google.com
BUG=

Review-Url: https://codereview.chromium.org/2721543003 .
2017-02-27 13:31:03 +01:00
Ben Konyi 2b81624bb2 Revert "Revert "[test.dart] Complain if there is non-utf8 formatted data in test output""
This reverts commit 7af8f320a7.

TBR=zra@google.com
BUG=

Review-Url: https://codereview.chromium.org/2710603009 .
2017-02-23 17:08:15 -08:00