Commit Graph

94 Commits

Author SHA1 Message Date
Bob Nystrom 881e35ee82 Migrate library_negative_test.
In the process, I discovered multitests can't handle imports to
non-existent libraries. So I fixed that and cleaned up the code.

Change-Id: I9a8557e84f91ba7858bdf98f8732cd0ded55aa1a
Reviewed-on: https://dart-review.googlesource.com/52869
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
2018-04-27 22:28:27 +00:00
Bob Nystrom 3d35875082 Clean up the multitest code.
I had a few minutes while waiting on a build. :)

Change-Id: I8683b7076040c2a4757665ecd6efd9e415d49660
Reviewed-on: https://dart-review.googlesource.com/28000
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2017-12-12 01:30:59 +00:00
Erik Ernst 40b6741778 Adds support for syntax error in tests to test.py.
This CL modifies the Dart source used from test.py such that it takes
`syntax error` into account as an expected outcome in test files (so
that we can have `//# 01: syntax error` with a similar meaning as
`//# 01: compile-time error`).

For all tools except the spec_parser, `syntax error` is the same
outcome as `compile-time error`; that is, nobody else will see the
difference.

For the spec_parser, `syntax error` is the outcome where parsing has
failed; `compile-time error` is taken to mean some other compile-time
error, i.e., the spec_parser is expected to _succeed_ when the
expected outcome is `compile-time error`.

Test files in language and language_2 have been adjusted to use the
outcome `syntax error` where appropriate.

The status files in language and language_2 for the spec_parser have
been adjusted such that they fit all the new `syntax error` outcomes
in test files.

Other status files have been adjusted in a few cases where tests were
corrected (because a compile-time error which was clearly not intended
to be a syntax error turned out to be caused by a typo, which means
that the actual compile-time error has never been tested).

The spec grammar Dart.g was adjusted in a few cases, when some bugs
were discovered. In particular, the treatment of Function has been
changed: It is now known by the parser that Function does not take
any type arguments. This makes no difference for developers, because
they cannot declare a type named Function anyway, but it means that
a number of tricky parsing issues were resolved.

Dart.g was also adjusted to allow `qualified` to contain three
identifiers, which is an old bug (preventing things like metadata on
the form `@p.C.myConst`).

Change-Id: Ie420887d45c882ef97c84143365219f8aa0d2933
Reviewed-on: https://dart-review.googlesource.com/18262
Commit-Queue: Erik Ernst <eernst@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
2017-11-06 08:56:09 +00:00
Bob Nystrom ba2e4aff24 Some minor clean-up in test.dart.
- Fix fuzzy arrow errors.
- Default the Dart repo URI so it doesn't always need to be explicitly
  set.
- Run dartfmt on everything.
- Move some members out of the garbage bin "TestUtils" class:
  - Move the stuff around the Dart repo directory to a new Repository
    class.
  - Move absolute() into Path where it belongs.
  - Move workingDirectory in Path since it is a Path.
  - Delete the random number stuff since it was apparently unused.

Change-Id: I3dab3a4f1713b7a749e64b6776149d05a0ce1b69
Reviewed-on: https://dart-review.googlesource.com/14502
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2017-10-19 19:05:58 +00:00
Bob Nystrom e0180925f6 Tighten up a bunch of types in test.dart.
This doesn't get rid of all implicit dynamic and casts, but it covers
many of them.

Also did some minor modernization when I noticed it:

- Using "var" for local variables where inference does the right thing.
- camelCase for variable names.
- More collection literals.

There are (or should be!) zero behavioral changes.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2863253002 .
2017-05-15 15:26:40 -07:00
Jacob Richman 88eb557b65 Fix warnings_checker.dart handling of multitests
Update all tests

Support //# multitests for better dartfmt compatibility and fewer multitest false positives

All files under tests were manually updated with

find . -iregex '.*\.dart$' -print0 | xargs -0 perl -pi -e 's/(\S\s+)\/\/\/ /$1\/\/# /'

For now both old and new styles are allowed to accommodate CO19 tests.

R=efortuna@google.com
BUG=

Review-Url: https://codereview.chromium.org/2765693002 .
Review-Url: https://codereview.chromium.org/2765893003 .
2017-03-21 17:29:56 -07:00
Jacob Richman bf2b545150 Revert "Update all tests"
This reverts commit ce76e9c30beaf9a193d3677b88c20a1ebb3fae8c.
2017-03-21 16:33:18 -07:00
Jacob Richman 3c7353d987 Update all tests
Support //# multitests for better dartfmt compatibility and fewer multitest false positives

All files under tests were manually updated with

find . -iregex '.*\.dart$' -print0 | xargs -0 perl -pi -e 's/(\S\s+)\/\/\/ /$1\/\/# /'

For now both old and new styles are allowed to accommodate CO19 tests.

R=efortuna@google.com

Review-Url: https://codereview.chromium.org/2765693002 .
2017-03-21 12:39:28 -07:00
William Hesse fda21e2b9a Add relative path to test when creating generated multitest directory
BUG=https://github.com/dart-lang/sdk/issues/27139
R=kustermann@google.com

Review URL: https://codereview.chromium.org/2270413002 .
2016-08-24 14:16:22 +02:00
John McCutchan ef71bc1834 Improve hot reload test mode
- [x] Start reloading even earlier.
- [x] Keep the value of FLAG_reload_every per-isolate (so spawned isolates reload eagerly too).
- [x] Adjust the reload every back off curve to be linear in the beginning.
- [x] Cap the reload every threshold at 1,000,000
- [x] Test status file updates.
- [x] Stop disabling the background compiler / OSR when running tests.
- [x] Skip multi tests with compilation errors when reloading.

BUG=
R=rmacnak@google.com

Review URL: https://codereview.chromium.org/2133823002 .
2016-07-11 12:29:13 -07:00
William Hesse c63b10b611 Format the test scripts in /tools/testing/dart using dartfmt 1.16.0-dev.3.0.
R=ahe@google.com

Review URL: https://codereview.chromium.org/1859973002 .
2016-04-06 14:49:18 +02:00
rmacnak@google.com fbc6b005a4 Support percent encoded data URIs in the standalone embedder.
BUG=
R=iposva@google.com

Review URL: https://codereview.chromium.org//1028453002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@44609 260f80e4-7a28-3924-810f-c04153c831b5
2015-03-20 20:11:53 +00:00
lrn@google.com 2b908ad153 Change Multitest files to preserve line numbers.
Instead of omitting multi-test parts that is not part of the current file,
an empty line is inserted instead.

Also move the "generated file, don't edit" warning line to the end of the file.

R=whesse@google.com

Review URL: https://codereview.chromium.org//1026693002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@44604 260f80e4-7a28-3924-810f-c04153c831b5
2015-03-20 13:34:43 +00:00
kevmoo@google.com 47de553025 tools/testing: move code into individual libraries
R=ricow@google.com

Review URL: https://codereview.chromium.org//748773004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42108 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-04 15:32:57 +00:00
kevmoo@google.com 61b056964a tools: removed unused members, tiny cleanup
R=ricow@google.com

Review URL: https://codereview.chromium.org//770633003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42106 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-04 13:56:14 +00:00
whesse@google.com 6ac0684ab8 Working SimpleHTML test front end
BUG=
R=ricow@google.com

Review URL: https://codereview.chromium.org//695893002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41517 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-05 12:54:34 +00:00
ricow@google.com d4c3fb063b Revert revisions 36163 and 36162
Review URL: https://codereview.chromium.org//288703003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@36165 260f80e4-7a28-3924-810f-c04153c831b5
2014-05-14 13:41:58 +00:00
ricow@google.com 4872c41303 Cache output of dart2js compilations that went wrong on disk.
This change adds a out.js.cached_output (or equivalent) for compilations that fails.
There are two cases where this helps:
  Commits where the sdk does not change and but we would normally recompile since we don't have the dart2js output
  Runs on the buildbot where we do several runs on different runtimes in serial

Additionally, this introduces a check on multitest files so we don't copy the file if the content is already correct. The reason for this is that a bunch of tests depend on the copied import which will then trigger a recompilation.
This will help in the same two cases as above.

This change also changes dart2js to always output the deps file, otherwise we have no way of knowing if we can reuse the output.

R=floitsch@google.com, kustermann@google.com, whesse@google.com

Review URL: https://codereview.chromium.org//256743009

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@36162 260f80e4-7a28-3924-810f-c04153c831b5
2014-05-14 12:54:01 +00:00
whesse@google.com 48b4b4550f Reduce test.dart memory usage.
BUG=18084
R=kustermann@google.com

Review URL: https://codereview.chromium.org//239003002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@35058 260f80e4-7a28-3924-810f-c04153c831b5
2014-04-15 12:50:46 +00:00
johnniwinther@google.com 1692a15f4e Add 'static type error' tags to multitests.
BUG=http://dartbug.com/16411
R=ricow@google.com

Review URL: https://codereview.chromium.org//150223002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@34412 260f80e4-7a28-3924-810f-c04153c831b5
2014-03-26 09:03:47 +00:00
sigurdm@google.com 61a5ff28fe Fixes for the test framework multitests.
Allow imports with a metadata annotation.
Allow test keys to be alphanumeric instead of just two digits.

R=floitsch@google.com, whesse@google.com

Committed: https://code.google.com/p/dart/source/detail?r=32892

Was reverted in Issue 175113002, commit r32894.

Review URL: https://codereview.chromium.org//164333013

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@32956 260f80e4-7a28-3924-810f-c04153c831b5
2014-02-24 10:13:42 +00:00
sigurdm@google.com 8e65452aa9 Revert "Fixes for the test framework multitests."
This reverts commit r32892.

Review URL: https://codereview.chromium.org//175113002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@32894 260f80e4-7a28-3924-810f-c04153c831b5
2014-02-21 12:54:11 +00:00
sigurdm@google.com 48b2d4f50b Fixes for the test framework multitests.
Allow imports with a metadata annotation.
Allow test keys to be alphanumeric instead of just two digits.

R=whesse@google.com

Review URL: https://codereview.chromium.org//164333013

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@32892 260f80e4-7a28-3924-810f-c04153c831b5
2014-02-21 12:41:49 +00:00
kustermann@google.com 19e74c6272 Added tools/status_clean.dart script
This script supports two commands:
  deflake:
    This command will go through all status file entries and check if the actual
    testoutcomes we've observed on our buildbots match the status file entries.

    If we have tests marked as flaky which are in fact not flaky, it will write
    a *.status.deflaked file which contains the fixes to the status file.

  fix:
    This command will go through all status files and ensures that all the
    status file lines correspond to existing tests.

    If we have lines that refer to non-existent tests, it will write a
    *.status.fixed file which contains the fixes to the status file.

R=whesse@google.com

Review URL: https://codereview.chromium.org//143453012

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@32694 260f80e4-7a28-3924-810f-c04153c831b5
2014-02-14 12:21:37 +00:00
kustermann@google.com bb54b776ba Checked-in binary update to v1.0.0.3
R=ricow@google.com

Review URL: https://codereview.chromium.org//91473002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30717 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-27 14:17:44 +00:00
kustermann@google.com aac5479809 Only generate multitests if content changed, enables us to dart2js compilations for multitests
BUG=
R=ricow@google.com

Review URL: https://codereview.chromium.org//52113003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@29528 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-30 10:41:17 +00:00
kustermann@google.com 94ebff2225 Bugfix in test.dart, adding of specialized test outcomes for analyzer
This CL will:
 - fix a bug in the legacy AnalyzerCommandOutputImpl class (in certain cases
   we did not report that the analyzer was missing a warning).
 - make the outcome of running the analyzer on a test be one of:
   * CompileTimeError
   * MissingCompileTimeError
   * StaticWarning
   * MissingStaticWarning
 - Make status file updates using the new Expectation markers
 - Remove support for '@static-clean':
     Previously running the analyer on a test could either result in
     'Pass' or in 'Fail'. The '@static-clean' annotation has been used
     as a (poor) mechanism to distinguish between errors and warnings.
     Having the 4 markers mentioned above means we can remove '@static-clean'
 - Remove 100 lines of rotted legacy code in our testing scripts.
 - Make a status file entry for every test that has analyzer errors or warnings

R=ricow@google.com

Review URL: https://codereview.chromium.org//46163002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@29350 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-28 10:39:38 +00:00
kustermann@google.com f6ab02857e test.py: Fixed incorrect handling of multitest in status files, status file cleanups.
R=ricow@google.com

Review URL: https://codereview.chromium.org//23721006

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@27013 260f80e4-7a28-3924-810f-c04153c831b5
2013-09-02 14:04:59 +00:00
kustermann@google.com 45c42e08f2 Update checked-in binary to 0.6.21.3_r26639 (M6)
R=ricow@google.com

Review URL: https://codereview.chromium.org//23568002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@26704 260f80e4-7a28-3924-810f-c04153c831b5
2013-08-27 11:16:37 +00:00
regis@google.com f9df206e2e Report compile-time errors for conflicting overrides as specified by latest
language spec (fix issue 12342).
Add override conflict tests.
Update status files.
Add support for 'ok' status in multi-tests.

R=hausner@google.com

Review URL: https://codereview.chromium.org//22999022

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@26302 260f80e4-7a28-3924-810f-c04153c831b5
2013-08-16 23:39:41 +00:00
kustermann@google.com a5cd38ed41 Bugfix in multitest.dart: Make sure we copy imported files to the generated_tests directory
The regular expression was no longer valid after import statements have changed.
Furthermore the guard against cyclic imports was not working because of missing
hashCode/operator== on the Path class.

R=ricow@google.com

Review URL: https://codereview.chromium.org//19774003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@25155 260f80e4-7a28-3924-810f-c04153c831b5
2013-07-18 17:11:37 +00:00
ricow@google.com af21d74d2d Fix safari browser to use Future.value
Also correct type errors found by the analyzer.

R=kustermann@google.com

Review URL: https://codereview.chromium.org//18089005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24532 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-27 17:07:17 +00:00
ricow@google.com 545f6e1341 Update checked in binary to version 0.5.19.0
This is a reland of r24086 with a fix catching errors in the websocket server in tools/testing/dart/http_server.dart.

R=kustermann@google.com

Review URL: https://codereview.chromium.org//18065003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24520 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-27 16:10:59 +00:00
ricow@google.com 4c5b2ac2ab Revert 24086 Update checked in binary to version 0.5.19.0
This is causing crashes in http server.

R=kustermann@google.com

Review URL: https://codereview.chromium.org//17274002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24096 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-17 13:22:15 +00:00
ricow@google.com 1970fc7afb Update checked in binary to version 0.5.19.0
R=kustermann@google.com

Review URL: https://codereview.chromium.org//17078007

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24086 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-17 10:47:02 +00:00
whesse@google.com 266bada744 Remove use of Expect from test scripts and idlparser_test.
BUG=

Review URL: https://codereview.chromium.org//14103003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@21287 260f80e4-7a28-3924-810f-c04153c831b5
2013-04-11 15:53:44 +00:00
sgjesse@google.com d6955d176c Reapply "Update the test runner to use the new dart:io API"
This reverts r20031.

R=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12751005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20070 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-15 11:13:39 +00:00
sgjesse@google.com f5270bf202 Revert "Update the test runner to use the new dart:io API" again
This reverts r20027 and r20029

TBR=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12475010

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20031 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 15:38:18 +00:00
sgjesse@google.com fdaca2fe83 Revert "Update the test runner to use the new dart:io API" again
This reapplies r20004 and r20008 and fixes a HTTP server issue.

TBR=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12431020

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20027 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 15:24:36 +00:00
sgjesse@google.com 3e2ffb4394 Revert "Update the test runner to use the new dart:io API" again
This reverts r20004 and r20008.

TBR=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12559013

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20009 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 12:13:37 +00:00
sgjesse@google.com db7821cdf3 Reapply "Update the test runner to use the new dart:io API"
The Linux binary will work on the bots. Fixed a number of issues.

The new binaries are from r19850.

R=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12608007

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20004 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 11:39:20 +00:00
sgjesse@google.com 78403ce605 Revert "Update the test runner to use the new dart:io API"
This reverts commit r19987.

The included binaries did not work on the bots due to a glibc version issue.

TBR=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12864003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@19988 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 08:08:56 +00:00
sgjesse@google.com 7ff44f77ac Update the test runner to use the new dart:io API
The new binaries are from r19850.

R=whesse@google.com, ager@google.com, ricow@google.com, kustermann@google.com

BUG=

Review URL: https://codereview.chromium.org//12417004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@19987 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 07:54:07 +00:00
floitsch@google.com ecce8a6293 Remove deprecated Strings class.
Review URL: https://codereview.chromium.org//12295014

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@18686 260f80e4-7a28-3924-810f-c04153c831b5
2013-02-19 13:57:03 +00:00
kustermann@google.com ecf1cdeeb8 Fixed utf8 encoding/decoding issues in the testing scripts
Review URL: https://codereview.chromium.org//11883033

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17057 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-15 10:59:24 +00:00
whesse@google.com 747f79ce1f Upload new Dart binaries and re-enable unit tests for test scripts.
BUG=

Review URL: https://codereview.chromium.org//11884029

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17031 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-14 20:45:42 +00:00
lrn@google.com bfd9b37484 Undo changes to tools-scripts.
TBR=ricow@google.com

Review URL: https://codereview.chromium.org//11856009

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16956 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-11 09:56:36 +00:00
lrn@google.com 4868ee2110 Remove Futures class, move methods to Future.
Optimize Future.forEach, Future.delay.

Review URL: https://codereview.chromium.org//11865005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16954 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-11 09:46:45 +00:00
kustermann@google.com 59a12b72c9 Migration of testing scripts in tools/ to libv2
Review URL: https://codereview.chromium.org//11817012

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16867 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-09 18:31:28 +00:00
kustermann@google.com 62c83a4443 Moved the testing scripts to the new import/library/part syntax
Review URL: https://codereview.chromium.org//11575028

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16163 260f80e4-7a28-3924-810f-c04153c831b5
2012-12-14 14:19:01 +00:00