This includes Fasta, tools and observatory, so the checked-in SDK must
have the lower-case constants.
Change-Id: I8380ad041ad058f7d02ae19caccfecd434d13d75
Reviewed-on: https://dart-review.googlesource.com/50201
Commit-Queue: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Required a new flag for test.py; passing tests may have multiple test
cases and to collect that we need the stdout. So you can now pass
--print_passing_stdout to get that.
Parse the result, doing some dances to handle
MM:SS +PP -FF Some tests failed
vs
MM:SS +PP All tests passed!
With even an extra line-ending in the latter case.
Bug:
Change-Id: Ic0b5591b4867508edd3f23a61a7764aa88762be6
Reviewed-on: https://dart-review.googlesource.com/24701
Commit-Queue: Mike Fairhurst <mfairhurst@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
I was working on adding support for treating RuntimeError as Pass if
runtime == none and I noticed a lot of dead or redundant code, so
cleaned it up some.
Change-Id: I0be451a7075d99bc07d1407883861fb06d777c02
Reviewed-on: https://dart-review.googlesource.com/22800
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This CL modifies the Dart source used from test.py such that it takes
`syntax error` into account as an expected outcome in test files (so
that we can have `//# 01: syntax error` with a similar meaning as
`//# 01: compile-time error`).
For all tools except the spec_parser, `syntax error` is the same
outcome as `compile-time error`; that is, nobody else will see the
difference.
For the spec_parser, `syntax error` is the outcome where parsing has
failed; `compile-time error` is taken to mean some other compile-time
error, i.e., the spec_parser is expected to _succeed_ when the
expected outcome is `compile-time error`.
Test files in language and language_2 have been adjusted to use the
outcome `syntax error` where appropriate.
The status files in language and language_2 for the spec_parser have
been adjusted such that they fit all the new `syntax error` outcomes
in test files.
Other status files have been adjusted in a few cases where tests were
corrected (because a compile-time error which was clearly not intended
to be a syntax error turned out to be caused by a typo, which means
that the actual compile-time error has never been tested).
The spec grammar Dart.g was adjusted in a few cases, when some bugs
were discovered. In particular, the treatment of Function has been
changed: It is now known by the parser that Function does not take
any type arguments. This makes no difference for developers, because
they cannot declare a type named Function anyway, but it means that
a number of tricky parsing issues were resolved.
Dart.g was also adjusted to allow `qualified` to contain three
identifiers, which is an old bug (preventing things like metadata on
the form `@p.C.myConst`).
Change-Id: Ie420887d45c882ef97c84143365219f8aa0d2933
Reviewed-on: https://dart-review.googlesource.com/18262
Commit-Queue: Erik Ernst <eernst@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
This will make sure that the result.log file will always exist, even if no tests
are run. This also means that the delete result.log step for recipes can be
removed.
Bug:
Change-Id: Idf097acb990b047146a874715f992657deacf65a
Reviewed-on: https://dart-review.googlesource.com/16483
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Morten Krogh-jespersen <mkroghj@google.com>
- Clean up browser test failure output. Instead of barfing a giant blob
of JSON, show just the runtime error and its stack trace. In verbose
and on the buildbots, it still shows all the events, but even there
they are formatted nicely.
- Don't redundantly show the failure summary when most of the output is
already failure text.
- Color the failure summary end message.
Change-Id: I49724b204be55a297fc5e723b80b32e0e00748f0
Reviewed-on: https://dart-review.googlesource.com/15767
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Of course, if you're not using the color formatter, it stays uncolored.
But this gets rid of some unnecessary empty lines and section headers
that have no content.
Change-Id: I5b83345c583b79468af84f4e67a12ee757a3105f
Reviewed-on: https://dart-review.googlesource.com/14720
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
This will help other tools to use the same status files at test.py
uses.
Bug:
Change-Id: Ib01d0c8368089a35682d0d79e257b8df01e54de6
Reviewed-on: https://dart-review.googlesource.com/3941
Reviewed-by: William Hesse <whesse@google.com>
This required munging a bit of test.dart code too to tease out some
dependencies, but the changes are minor. I considered moving all of
test.dart out into a package and making the status file library a
public one that other packages in the repo could import but this seemed
like the less intrusive change.
R=bkonyi@google.com
Review-Url: https://codereview.chromium.org/2984203002 .
- Get rid of separate CommandBuilder class and singleton pattern. It
was being passed around explicitly even though half of the places
that received a CommandBuilder as a parameter still directly called
CommandBuilder.instance instead of using it.
- Get rid of Command caching. As far as I can tell, it makes no
measurable difference in runtime performance or memory usage. Even
with a large invocation of a lot of configurations and tests, the
Command classes don't seem to be a significant use of memory.
- Shorten the factory names. "get" adds no value, and we know it
returns a "Command" since it's on Command.
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2933973002 .
This is a pretty massive change but my hope is it will make
test.dart easier to maintain going forward. Instead of
passing around a stringly-typed Map<String, dynamic> all
throughout the program, we parse the options and then create
a Configuration object that has typed getters for all of the
various bits of configuration data.
This is a little tedious because it means declaring a new
option requires also declaring a corresponding field in the
Configuration class and passing it through the constructor.
I think it's worth it.
Also, enum-like configuration properties like architecture
and runtime now have their own classes as well. Moved a
bunch of stuff from TestUtil into those classes now that
there is an object to hang those methods off of.
In the process, I found a few typos in string literals where
the code wasn't correctly looking up a configuration
property.
Added more sanity checking and validation to status file
parsing. You will get an error at parse time if you try to
refer to a variable that isn't in the whitelist of known
variables. Also, you'll get an error if you try to compare
a variable to a value that it isn't expected to have.
Many other small-scale cleanups.
Aside from the status file validation, this should behave
mostly the same as current test.dart except that tests
may be enqueued in a slightly different order. The
rewritten code for expanding configurations iterates
through the architecture, runtime, etc. options in a
slightly different order.
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2901923003 .
This removes all implicit casts and many implicit uses of dynamic. It
adds a gratuitous number of explicit "as" casts and arguably makes the
code worse.
This is an interim step towards replacing the big configuration map
with an actual typed object. These "as" casts should help catch places
where the configuration object is being used and where the code will
need to be changed to use a new object.
R=sigmund@google.com
Review-Url: https://codereview.chromium.org/2903703002 .
- Separate out parsing a status file from applying the environment to
determine which sections are active. This makes it possible to, for
example, generate expectation sets for multiple environments without
having to reparse each time.
- Simplify expression parsing. Remove set expressions since they weren't
used for anything useful. A test's expectations are a simple
comma-separated list and don't need anything beyond that. Merge
Scanner and Tokenizer since the latter was a glorified function.
- Make more names private so that it's clearer what's used outside of
various libraries.
- Generally modernize the style.
- Add *lots* of documentation.
Again, there should be no behavioral changes. I ran:
./tools/test.py -m release,debug -c none,dart2js,dart2analyzer -r none,vm,d8 corelib
Before and after the change and verified that the output was the same
(aside from timing).
R=sigmund@google.com
Review-Url: https://codereview.chromium.org/2891753003 .
- Make the parser less error tolerant. The expression parser used to
ignore any unrecognized tokens, which means a status like
"RuntimeError CompileError" (not the missing comma) was parsed as
simply "RuntimeError", which seems bad. Now it reports an error.
Fixed a couple of status files that thought they were setting statuses
that they weren't (!).
- Separate out parsing a status file from applying the environment to
determine which sections are active. This makes it possible to, for
example, generate expectation sets for multiple environments without
having to reparse each time.
- Simplify expression parsing. Remove set expressions since they weren't
used for anything useful. A test's expectations are a simple
comma-separated list and don't need anything beyond that. Merge
Scanner and Tokenizer since the latter was a glorified function.
- Make more names private so that it's clearer what's used outside of
various libraries.
- Generally modernize the style.
- Add *lots* of documentation.
Again, there should be no behavioral changes. I ran:
./tools/test.py -m release,debug -c none,dart2js,dart2analyzer -r none,vm,d8 corelib
Before and after the change and verified that the output was the same
(aside from timing).
BUG=
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2875203005 .
This doesn't get rid of all implicit dynamic and casts, but it covers
many of them.
Also did some minor modernization when I noticed it:
- Using "var" for local variables where inference does the right thing.
- camelCase for variable names.
- More collection literals.
There are (or should be!) zero behavioral changes.
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2863253002 .
Stuff like:
- Use core lib methods like padLeft() now that they exist.
- Use collection literals (!).
- Use getters where appropriate.
- Make Formatter enum-like since it has no interesting instance state.
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2855073002 .
This is mainly so that all of the code relating to test.dart is in one
directory tree so things like "Find All Usages" work a little better.
It felt weird to me to have a .dart file two directories up importing a
bunch of stuff within "testing/dart/".
Also cleaned up the affected code since it could use a little love. I'm
working on getting test.dart running DDC tests, but from poking around,
it seems like it could use some housekeeping as well.
R=vsm@google.com, whesse@google.com
Review-Url: https://codereview.chromium.org/2848103002 .
The former doesn't seem to tell us anything useful, and is broken. The
latter is no longer needed as a flag without the former. The pkgbuild
suite implies it and other suites don't support it.
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2673133002 .
Change UnexpectedCrashDumpArchiver to generate coredumps file in the current
directory. Each line of this file specifies a core and a binary to be
uploaded in the following format:
test-name,core-file,binary-file
CoreDumpArchiver reads this file and uploads the first 10 crash dumps.
R=kustermann@google.com
BUG=
Review-Url: https://codereview.chromium.org/2645963004 .
- Fix dartanalyzer warnings about unused variables and missing returns
- Remove trailing whitespace
- Add a dummy pubspec.yaml so editors recognize it as a Dart project
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2633213003 .
William Hesse