Required a new flag for test.py; passing tests may have multiple test
cases and to collect that we need the stdout. So you can now pass
--print_passing_stdout to get that.
Parse the result, doing some dances to handle
MM:SS +PP -FF Some tests failed
vs
MM:SS +PP All tests passed!
With even an extra line-ending in the latter case.
Bug:
Change-Id: Ic0b5591b4867508edd3f23a61a7764aa88762be6
Reviewed-on: https://dart-review.googlesource.com/24701
Commit-Queue: Mike Fairhurst <mfairhurst@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
If you pass "-c" or "-r", but not both, choose a smart default for the
other instead of simply yelling at the user and telling them they
forgot something.
Change-Id: Ic47a20fa8f137a8f6488f16cf883839999e2225b
Reviewed-on: https://dart-review.googlesource.com/16905
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
- Fix fuzzy arrow errors.
- Default the Dart repo URI so it doesn't always need to be explicitly
set.
- Run dartfmt on everything.
- Move some members out of the garbage bin "TestUtils" class:
- Move the stuff around the Dart repo directory to a new Repository
class.
- Move absolute() into Path where it belongs.
- Move workingDirectory in Path since it is a Path.
- Delete the random number stuff since it was apparently unused.
Change-Id: I3dab3a4f1713b7a749e64b6776149d05a0ce1b69
Reviewed-on: https://dart-review.googlesource.com/14502
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
This CL modifies tools/test.py such that it can run the spec parser
(after doing `make parser` in tools/spec_parser, and assuming that the
ANTLR 3 library is available at /usr/share/java/antlr3-runtime.jar)
with a command line like
`tools/test.py -c spec_parser -r none language/callable_test`
It also changes status files to have a name which follows the expected
patterns (e.g., `language/language_spec_parser.status`). Finally, it
adds/changes many entries in status files, such that parsing of the
directories `language` and `language_2` run successfully.
Change-Id: I82a22e32ac4fecd23ac0d4434bcac08f75dd8ffe
Reviewed-on: https://dart-review.googlesource.com/12680
Commit-Queue: Erik Ernst <eernst@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
This will help other tools to use the same status files at test.py
uses.
Bug:
Change-Id: Ib01d0c8368089a35682d0d79e257b8df01e54de6
Reviewed-on: https://dart-review.googlesource.com/3941
Reviewed-by: William Hesse <whesse@google.com>
This allows us to collect stats about language-style tests as follows:
./tools/test.py -m release -c dart2analyzer -r none --preview-dart-2 --report \
language language_2 corelib corelib_2
As of this morning, the command above has 87% of the tests passing.
Change-Id: I0f84211d5f33659d5923601267367be90c578107
Reviewed-on: https://dart-review.googlesource.com/3381
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Dan Rubel <danrubel@google.com>
Revert "Precompiler doesn't report Dart 2.0 static type errors yet."
This reverts commit 7af2d86ed3.
Revert "Move absstract_exact_selector back into the language tests. (#30169)"
This reverts commit 452d2d0840.
Revert "Set up directories for migrated Dart 2.0 tests and migrate a couple. (#30149)"
This reverts commit 47985d6dbd.
R=zra@google.com
BUG:dartbug.com/30171
Review-Url: https://codereview.chromium.org/2979073002 .
* Set up directories for migrated Dart 2.0 tests and migrate a couple.
- Create new "_2" directories where tests that have been validated as
ready for Dart 2.0 will end up.
- Create empty status files for each directory.
- Add those directories to the set of default selectors you get when you
run test.py. This gets the VM bots running them.
- Get the DDC bots running those suites.
- Move abstract_exact_selector_test over to dart2js since it's a
dart2js-specific regression test and not an actual language test.
- Migrate corelib/apply_test.dart.
- Delete abstract_beats_arguments[2]_test.dart since that code is
statically wrong in 2.0 and can't be run.
This doesn't get the dart2js bots running the new suites. I'll email
the relevant folks to get help with that.
* Resurrect abstract_beats_arguments_test.dart.
It usefully checked that an implementation reports an warning (now
error) if you construct an abstract class.
Also added support to test.dart to mark a test as expecting to
generate a compile error in the test itself. That way, the status file
reflects what is *wrong* about the current status, not what is *right*.
* Change static error syntax to match front_end notation.
* Migrate abstract_getter_test.
This is a pretty massive change but my hope is it will make
test.dart easier to maintain going forward. Instead of
passing around a stringly-typed Map<String, dynamic> all
throughout the program, we parse the options and then create
a Configuration object that has typed getters for all of the
various bits of configuration data.
This is a little tedious because it means declaring a new
option requires also declaring a corresponding field in the
Configuration class and passing it through the constructor.
I think it's worth it.
Also, enum-like configuration properties like architecture
and runtime now have their own classes as well. Moved a
bunch of stuff from TestUtil into those classes now that
there is an object to hang those methods off of.
In the process, I found a few typos in string literals where
the code wasn't correctly looking up a configuration
property.
Added more sanity checking and validation to status file
parsing. You will get an error at parse time if you try to
refer to a variable that isn't in the whitelist of known
variables. Also, you'll get an error if you try to compare
a variable to a value that it isn't expected to have.
Many other small-scale cleanups.
Aside from the status file validation, this should behave
mostly the same as current test.dart except that tests
may be enqueued in a slightly different order. The
rewritten code for expanding configurations iterates
through the architecture, runtime, etc. options in a
slightly different order.
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2901923003 .
This removes all implicit casts and many implicit uses of dynamic. It
adds a gratuitous number of explicit "as" casts and arguably makes the
code worse.
This is an interim step towards replacing the big configuration map
with an actual typed object. These "as" casts should help catch places
where the configuration object is being used and where the code will
need to be changed to use a new object.
R=sigmund@google.com
Review-Url: https://codereview.chromium.org/2903703002 .
This doesn’t touch the resulting configuration map, which is what this
is leading up to, but it cleans up the specification and parsing of the
options and removes a lot of redundancy.
Behavior should be the same as it was before, except that all options
now allow both their underscore-separated and hyphen-separated names.
So this is now valid:
test.py --hot_reload --append-logs
Where before, you use to have to do --hot-reload (OK) and
--append_logs (bad, since almost all other options expect hyphens).
R=sigmund@google.com
Review-Url: https://codereview.chromium.org/2902023002 .
Alexander Thomas