Commit Graph

1865 Commits

Author SHA1 Message Date
Bob Nystrom 47985d6dbd Set up directories for migrated Dart 2.0 tests and migrate a couple. (#30149)
* Set up directories for migrated Dart 2.0 tests and migrate a couple.

- Create new "_2" directories where tests that have been validated as
  ready for Dart 2.0 will end up.
- Create empty status files for each directory.
- Add those directories to the set of default selectors you get when you
  run test.py. This gets the VM bots running them.
- Get the DDC bots running those suites.
- Move abstract_exact_selector_test over to dart2js since it's a
  dart2js-specific regression test and not an actual language test.
- Migrate corelib/apply_test.dart.
- Delete abstract_beats_arguments[2]_test.dart since that code is
  statically wrong in 2.0 and can't be run.

This doesn't get the dart2js bots running the new suites. I'll email
the relevant folks to get help with that.

* Resurrect abstract_beats_arguments_test.dart.

It usefully checked that an implementation reports an warning (now
error) if you construct an abstract class.

Also added support to test.dart to mark a test as expecting to
generate a compile error in the test itself. That way, the status file
reflects what is *wrong* about the current status, not what is *right*.

* Change static error syntax to match front_end notation.

* Migrate abstract_getter_test.
2017-07-14 10:27:37 -07:00
Bob Nystrom b1fdd84817 One more fix to Windows file path handling. (#30133) 2017-07-10 17:02:56 -07:00
Bob Nystrom 15f80a42d6 More Windows path whack-a-mole in test.dart. (#30132) 2017-07-10 16:19:25 -07:00
Bob Nystrom b1ae919390 Fix test.dart path arguments to DDC on Windows (hopefully). (#30130) 2017-07-10 14:47:37 -07:00
Bob Nystrom 7b17a3dd01 Use "|" as the separator between module path and name. (#30106)
* Use "|" as the separator between module path and name.

":" is already a path character in Windows. This is why the
dartdevc tests aren't working on Windows. Because they try to do:

-sC:/some/path/blah.dart:blah

And it splits at the first ":". Oops.

* Switch to "=" for separating module path from name.

It looks a little funny when used with "--summary=", but it works fine
and is very unlikely to be used for anything else on any platform
(since we already use it as the separator between arg name and value).
2017-07-07 17:13:43 -07:00
Bob Nystrom 5a3b8326e4 Handle spaces in paths in status files. (#30107)
Also triage a couple more dartdevc tests.
2017-07-07 16:49:44 -07:00
Erik Ernst 3b05eb9553 Add support to dart2js for option --enable-asserts.
R=johnniwinther@google.com, sigmund@google.com, sra@google.com

Review-Url: https://codereview.chromium.org/2879153005 .
2017-07-07 10:59:09 +02:00
Bob Nystrom 1bd907dc2e Run dartdevc tests on drt instead of Chrome on the bots.
Also, do another triage pass on the tests.

R=vsm@google.com

Review-Url: https://codereview.chromium.org/2964043004 .
2017-07-05 09:45:16 -07:00
Vyacheslav Egorov f167ad18da test.py: temporarily disable background optimizer in vm-dartk-debug config.
It is causing flakiness on bots.

Unmark tests marked as flaky by 45e4d848d2

See https://github.com/dart-lang/sdk/issues/30016.

BUG=
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2970733002 .
2017-07-03 17:05:12 +02:00
Bob Nystrom 0b2c9b048d Dynamically load packages for dartdevc tests in test.dart.
This involves a few pieces:

- Add support to DDC for specifying the module name associated with a
  given summary. This lets test.dart invoke DDC using summaries in the
  build directory outside of the directory containing the test itself.

- Add support to the build scripts for building the packages. This adds
  a new GN target that builds everything needed to run test.dart with
  dartdevc. In particular, it invokes build_pkgs.dart to compile the
  relevant packages to JS+summary so that the tests can use them.

  This requires some changes to build_pkgs.dart so it can output to a
  given directory.

- In test.dart, when tests are compiled with dartdevc, pass in the
  summaries for the packages so they don't get compiled in. Then, when
  the test is run, configure require.js with the right paths to their
  JS files so they can be loaded.

  I also removed a bunch of unneeded buildDir parameters being passed
  around the various CompilerConfiguration class methods now that they
  have direct access to the configuration.

Fix #29923.

R=vsm@google.com, whesse@google.com, zra@google.com

Review-Url: https://codereview.chromium.org/2955513002 .
2017-06-29 13:45:57 -07:00
Bob Nystrom 82df163eca Refactor dependency_graph.dart.
It multiplexed a bunch of different events onto a single stream. But
every single listener on that stream immediately demuxed them and only
cared about a subset of the events.

So I made separate streams for each event. Other minor clean-ups too.

R=sigmund@google.com, whesse@google.com

Review-Url: https://codereview.chromium.org/2957703002 .
2017-06-26 17:07:37 -07:00
Erik Corry 029b1cb948 Spelling fixes e to i.
R=kevmoo@google.com
BUG=

Review-Url: https://codereview.chromium.org/2957593002 .
2017-06-24 13:41:39 +02:00
Bob Nystrom dfb4f52b43 Simplify enqueueBrowserTest().
It's still big and monolithic, but it's not as big as it was before.

- Remove old unused Polymer stuff.
- Move some of the logic out into a helper function.
- Other small tweaks.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2949823003 .
2017-06-23 15:02:36 -07:00
Bob Nystrom 3144296c4f Remove unused PubCommand stuff.
The rest of the pub functionality was removed already, so I think this
is just a straggler.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2944163002 .
2017-06-23 12:55:46 -07:00
Bob Nystrom 556c0550bf Simplify CommandOutput and friends.
- Merge CommandOutputImpl and CommandOutput. There were no classes that
  implemented CommandOutput that didn't extend the Impl.
- Remove "Impl" from the other class names.
- Make stuff private when possible. Likewise final.
- Other tiny style changes.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2946783002 .
2017-06-23 12:13:31 -07:00
William Hesse 6482690781 Rebased simplify CompilerConfiguration
BUG=

Review-Url: https://codereview.chromium.org/2954113002 .
2017-06-23 14:10:03 +02:00
Zachary Anderson ac16656161 Remove MIPS support
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2858623002 .
2017-06-22 08:49:22 -07:00
William Hesse d8ec6450d0 Fix compiler==none case in browser testing (Dartium case)
Error was introduced in https://codereview.chromium.org/2947473002/

BUG=
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2950173004 .
2017-06-22 11:30:23 +02:00
Bob Nystrom b452b39962 Basic support for dev_compiler in test.dart.
It can compile and run tests on Chrome. There are a lot of failing tests
that I (or the team) will need to triage, but I think at least basic
tests are working as expected.

There is code that could be cleaned up to more neatly factor how dart2js
and dartdevc are handled now that there are two separate compilers to
JS. There's also some redundant code between the path for testing
compile errors (enqueueStandardTest()) and the path for running a test
in the browser.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2947473002 .
2017-06-21 13:19:28 -07:00
Siva Chandra bac83e0973 Replace the --platform vm_option with --kernel-binaries option.
This new flag value specifies the directory in which the VM should look
up the platform.dill file. A future change will require the VM to load
another kernel binary from disk. This binary will also live in the same
directory that --kernel-binaries specifies. This way, we avoid adding a
different flag for each of the different binaries.

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2933203004 .
2017-06-16 12:22:31 -07:00
William Hesse f99509ad42 Fix bad merge in refactoring
The land of https://codereview.chromium.org/2933973002 merged badly
with https://codereview.chromium.org/2934243002/

BUG=https://github.com/dart-lang/sdk/issues/28955
R=sortie@google.com

Review-Url: https://codereview.chromium.org/2938383002 .
2017-06-16 10:10:54 +02:00
Bob Nystrom a6ca718e98 Simplify Command classes.
- Get rid of separate CommandBuilder class and singleton pattern. It
  was being passed around explicitly even though half of the places
  that received a CommandBuilder as a parameter still directly called
  CommandBuilder.instance instead of using it.
- Get rid of Command caching. As far as I can tell, it makes no
  measurable difference in runtime performance or memory usage. Even
  with a large invocation of a lot of configurations and tests, the
  Command classes don't seem to be a significant use of memory.
- Shorten the factory names. "get" adds no value, and we know it
  returns a "Command" since it's on Command.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2933973002 .
2017-06-15 14:19:03 -07:00
Bob Nystrom 979026abef Don't allow "none" as a runtime for dartk.
Also, add a little sanity checking in TestCase.

Fix #29842.

R=ahe@google.com

Review-Url: https://codereview.chromium.org/2941603002 .
2017-06-14 10:31:09 -07:00
William Hesse e64ee95959 Move ie11 timeout handling code to BrowserControllerTestOutcome
Only content_shell testing uses the BrowserCommandOutputImpl class.
Rename the class to ContentShellCommandOutputImpl.

BUG=https://github.com/dart-lang/sdk/issues/28955
BUG=https://github.com/dart-lang/sdk/issues/29869
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2934243002 .
2017-06-14 15:50:41 +02:00
William Hesse d98d32b63c Report ie11 timeouts in debug log, not as errors (attempt 2)
BUG=https://github.com/dart-lang/sdk/issues/28955
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2938813002 .
2017-06-14 14:17:27 +02:00
Bob Nystrom 571839ed90 Remove negated variables now that we support "!".
R=lrn@google.com

Review-Url: https://codereview.chromium.org/2924183002 .
2017-06-12 15:50:59 -07:00
Bob Nystrom 19cb0cafb7 Simplify test_suite.dart.
- Get rid of baseCommand, which is only ever an empty list.
- Move some utility classes out of the giant test_suite.dart.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2915843003 .
2017-06-09 16:46:49 -07:00
Bob Nystrom 8bc4e89219 Remove unused record and replay functionality.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2920633002 .
2017-06-08 09:18:03 -07:00
Ryan Macnak b69b3e58f0 Update test.py's --arch=all to match that of build.py.
R=zra@google.com

Review-Url: https://codereview.chromium.org/2918403002 .
2017-06-05 13:07:30 -07:00
Alexander Aprelev 6573c74301 Reenable vm tests with -cdartk DFE parser.
This was accidentally disabled in 7e2d9ac85d.

BUG=#28264
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2920733004 .
2017-06-02 09:34:11 -07:00
Lasse R.H. Nielsen 5720e3556c Add negation to single-identifier tests in status files.
Optimize some regegps too.

Fixes #29756

BUG= http://dartbug.com/29756
R=rnystrom@google.com

Review-Url: https://codereview.chromium.org/2913963002 .
2017-06-02 10:38:14 +02:00
Bob Nystrom db06324975 Remove temp code to ignore "--failure-summary".
Now that the BuildBots aren't passing it, we don't need to accept it.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2916123002 .
2017-06-01 15:45:55 -07:00
William Hesse e161ad12ac Allow test.py to run dartdoc tests. Run dartdoc tests on pub bots.
BUG=
R=jcollins@google.com

Review-Url: https://codereview.chromium.org/2914233002 .
2017-06-01 18:53:04 +02:00
Bob Nystrom a9d4afc077 Handle "ff" as a runtime name in status files.
R=efortuna@google.com

Review-Url: https://codereview.chromium.org/2917843002 .
2017-05-31 17:49:22 -07:00
Bob Nystrom 7e2d9ac85d Revert "Revert "Replace the configuration map with a typed object.""
This reverts commit 8bada4873a.

R=nweiz@google.com

Review-Url: https://codereview.chromium.org/2919573003 .
2017-05-31 15:39:23 -07:00
Bob Nystrom 8bada4873a Revert "Replace the configuration map with a typed object."
This reverts commit 06f75fe5cd.

Review-Url: https://codereview.chromium.org/2914893003 .
2017-05-31 15:08:21 -07:00
Bob Nystrom 06f75fe5cd Replace the configuration map with a typed object.
This is a pretty massive change but my hope is it will make
test.dart easier to maintain going forward. Instead of
passing around a stringly-typed Map<String, dynamic> all
throughout the program, we parse the options and then create
a Configuration object that has typed getters for all of the
various bits of configuration data.

This is a little tedious because it means declaring a new
option requires also declaring a corresponding field in the
Configuration class and passing it through the constructor.
I think it's worth it.

Also, enum-like configuration properties like architecture
and runtime now have their own classes as well. Moved a
bunch of stuff from TestUtil into those classes now that
there is an object to hang those methods off of.

In the process, I found a few typos in string literals where
the code wasn't correctly looking up a configuration
property.

Added more sanity checking and validation to status file
parsing. You will get an error at parse time if you try to
refer to a variable that isn't in the whitelist of known
variables. Also, you'll get an error if you try to compare
a variable to a value that it isn't expected to have.

Many other small-scale cleanups.

Aside from the status file validation, this should behave
mostly the same as current test.dart except that tests
may be enqueued in a slightly different order. The
rewritten code for expanding configurations iterates
through the architecture, runtime, etc. options in a
slightly different order.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2901923003 .
2017-05-31 14:47:31 -07:00
Bob Nystrom 7ce978bf91 Tighten types in test.dart even more.
This removes all implicit casts and many implicit uses of dynamic. It
adds a gratuitous number of explicit "as" casts and arguably makes the
code worse.

This is an interim step towards replacing the big configuration map
with an actual typed object. These "as" casts should help catch places
where the configuration object is being used and where the code will
need to be changed to use a new object.

R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2903703002 .
2017-05-30 14:02:24 -07:00
Bob Nystrom c296f75d10 Revert "Revert "Revert "Revert "Refactor test option parsing code.""""
This reverts commit 02547656cc.

R=efortuna@google.com

Review-Url: https://codereview.chromium.org/2904313002 .
2017-05-26 15:04:55 -07:00
Bob Nystrom 02547656cc Revert "Revert "Revert "Refactor test option parsing code."""
This reverts commit 79af418430.

Review-Url: https://codereview.chromium.org/2912563002 .
2017-05-26 14:46:04 -07:00
Bob Nystrom 79af418430 Revert "Revert "Refactor test option parsing code.""
This reverts commit d03835783c.

R=efortuna@google.com

Review-Url: https://codereview.chromium.org/2908833002 .
2017-05-26 14:18:13 -07:00
Bob Nystrom d03835783c Revert "Refactor test option parsing code."
This reverts commit 9c2dff8ebc.

R=nweiz@google.com

Review-Url: https://codereview.chromium.org/2909723002 .
2017-05-26 13:51:26 -07:00
Bob Nystrom 9c2dff8ebc Refactor test option parsing code.
This doesn’t touch the resulting configuration map, which is what this
is leading up to, but it cleans up the specification and parsing of the
options and removes a lot of redundancy.

Behavior should be the same as it was before, except that all options
now allow both their underscore-separated and hyphen-separated names.
So this is now valid:

test.py --hot_reload --append-logs

Where before, you use to have to do --hot-reload (OK) and
--append_logs (bad, since almost all other options expect hyphens).

R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2902023002 .
2017-05-26 13:36:44 -07:00
Alexander Aprelev 16c0db1022 Revert "Revert "With this change 'tools/test.py -cdartk vm' will run vm tests with Dart Frontend parser.""
This reverts commit e0cdea811a.

Update status of vm tests, that don't work with DFE yet.

BUG=https://github.com/dart-lang/sdk/issues/28264
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2895153005 .
2017-05-24 20:16:48 -07:00
Alexander Aprelev e0cdea811a Revert "With this change 'tools/test.py -cdartk vm' will run vm tests with Dart Frontend parser."
This reverts commit f62a2a9562.

Commit above [expectedly] broke runtime/vm tests since it uses DFE
instead of vm parser, but I didn't realize we actually test -cdartk
configuration. This has to be resubmitted with status file update to
mark 235 vm tests failing with DFE.

TBR=siva
BUG=

Review-Url: https://codereview.chromium.org/2901733002 .
2017-05-22 19:42:53 -07:00
Alexander Aprelev f62a2a9562 With this change 'tools/test.py -cdartk vm' will run vm tests with Dart Frontend parser.
BUG=https://github.com/dart-lang/sdk/issues/28264
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2881833003 .
2017-05-22 17:24:38 -07:00
Martin Kustermann 22c7536d36 Reduce temporarily flakiness rate of dart2js-drt by re-running until they have been fixed
We should either remove dart2js-drt or use a stable version of
content_shell from upstream chromium (instead of our dartium-based one)
for dart2js-drt.

Issue #29655

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2891343002 .
2017-05-19 13:13:40 +02:00
Bob Nystrom 92267afc66 Refactor and clean up the status file parsing code. - Make the parser less error tolerant. The expression parser used to ignore any unrecognized tokens, which means a status like "RuntimeError CompileError" (not the missing comma) was parsed as simply "RuntimeError", which seems bad. Now it reports an error. Fixed a couple of status files that thought they were setting statuses that they weren't (!).
- Separate out parsing a status file from applying the environment to
  determine which sections are active. This makes it possible to, for
  example, generate expectation sets for multiple environments without
  having to reparse each time.

- Simplify expression parsing. Remove set expressions since they weren't
  used for anything useful. A test's expectations are a simple
  comma-separated list and don't need anything beyond that. Merge
  Scanner and Tokenizer since the latter was a glorified function.

- Make more names private so that it's clearer what's used outside of
  various libraries.

- Generally modernize the style.

- Add *lots* of documentation.

Again, there should be no behavioral changes. I ran:

  ./tools/test.py -m release,debug -c none,dart2js,dart2analyzer -r none,vm,d8 corelib

Before and after the change and verified that the output was the same
(aside from timing).

R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2891753003 .
2017-05-18 12:42:52 -07:00
Bob Nystrom c269ef53a5 Revert "Re-apply status file parser changes from 0b7728da1bef08c1c1e092005d9fd8c8bff5fa6c."
This reverts commit a881aed354.

TBR.

R=paulberry@google.com

Review-Url: https://codereview.chromium.org/2888213002 .
2017-05-17 13:45:40 -07:00
Bob Nystrom a881aed354 Re-apply status file parser changes from 0b7728da1b.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2880203004 .
2017-05-17 13:04:35 -07:00