Commit Graph

115 Commits

Author SHA1 Message Date
kevmoo@google.com 9df5e56e9f tools: fix break in test_progress
TBR

Review URL: https://codereview.chromium.org//829803005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42832 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-13 19:46:24 +00:00
kevmoo@google.com 3fdea4f366 Outputing comma-separated report with percentages
Merged:

http://crrev.com/832583003
http://crrev.com/843553004

R=ricow@google.com

Review URL: https://codereview.chromium.org//847873004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42831 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-13 19:27:47 +00:00
kevmoo@google.com 8cd05a8b90 tools: Refactored SummaryReport and move it to its own library
R=ricow@google.com

Review URL: https://codereview.chromium.org//832583003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42826 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-13 17:46:52 +00:00
kevmoo@google.com 47de553025 tools/testing: move code into individual libraries
R=ricow@google.com

Review URL: https://codereview.chromium.org//748773004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42108 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-04 15:32:57 +00:00
kevmoo@google.com 61b056964a tools: removed unused members, tiny cleanup
R=ricow@google.com

Review URL: https://codereview.chromium.org//770633003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42106 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-04 13:56:14 +00:00
whesse@google.com afbf758380 Reduce test.dart memory usage
BUG=dartbug.com/18084
R=ricow@google.com

Review URL: https://codereview.chromium.org//247223009

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@35365 260f80e4-7a28-3924-810f-c04153c831b5
2014-04-24 11:09:49 +00:00
johnniwinther@google.com 1692a15f4e Add 'static type error' tags to multitests.
BUG=http://dartbug.com/16411
R=ricow@google.com

Review URL: https://codereview.chromium.org//150223002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@34412 260f80e4-7a28-3924-810f-c04153c831b5
2014-03-26 09:03:47 +00:00
ahe@google.com 425792636f Fix a null error on empty suites.
R=whesse@google.com

Review URL: https://codereview.chromium.org//211103005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@34365 260f80e4-7a28-3924-810f-c04153c831b5
2014-03-25 15:02:43 +00:00
kustermann@google.com 19e74c6272 Added tools/status_clean.dart script
This script supports two commands:
  deflake:
    This command will go through all status file entries and check if the actual
    testoutcomes we've observed on our buildbots match the status file entries.

    If we have tests marked as flaky which are in fact not flaky, it will write
    a *.status.deflaked file which contains the fixes to the status file.

  fix:
    This command will go through all status files and ensures that all the
    status file lines correspond to existing tests.

    If we have lines that refer to non-existent tests, it will write a
    *.status.fixed file which contains the fixes to the status file.

R=whesse@google.com

Review URL: https://codereview.chromium.org//143453012

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@32694 260f80e4-7a28-3924-810f-c04153c831b5
2014-02-14 12:21:37 +00:00
ricow@google.com c2f12c4da6 Add support for copying coredumps to /tmp
This will be used on the bots to be able to archive dumps from unexpectec crashes.

The tools/archive_crash.py tool will be used to actually archive the dumps to gcs and delete the local copies. The reason for the split is to enable developers to actually utilize this locally as well (run tools/test.py in a loop but don't archive the expected crashes). Additionally, this puts the bot specific code in the small python script.

R=whesse@google.com

Review URL: https://codereview.chromium.org//145273024

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@32111 260f80e4-7a28-3924-810f-c04153c831b5
2014-01-29 09:41:47 +00:00
kustermann@google.com 54b71bdbe2 Reaply "Use a real package-root for all of our samples (and tests inside a package) instead of buildDir/packages/"
R=whesse@google.com

Review URL: https://codereview.chromium.org//133653004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@31694 260f80e4-7a28-3924-810f-c04153c831b5
2014-01-10 15:15:37 +00:00
kustermann@google.com e363ffb931 Enable pkgbuild testsuite on pub builders
R=ricow@google.com

Review URL: https://codereview.chromium.org//125833002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@31537 260f80e4-7a28-3924-810f-c04153c831b5
2014-01-07 10:12:10 +00:00
kustermann@google.com 20b8ddbfee Remove the command-specific outcome temporarily when writing out the .test-outcome.log file
TBR=ricow@google.com

Review URL: https://codereview.chromium.org//96603002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30773 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-29 09:28:15 +00:00
kustermann@google.com 26f065bdb2 Add .test-outcome.log support to test.dart
R=ricow@google.com

Review URL: https://codereview.chromium.org//95603002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30768 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-29 08:29:20 +00:00
kustermann@google.com bb54b776ba Checked-in binary update to v1.0.0.3
R=ricow@google.com

Review URL: https://codereview.chromium.org//91473002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30717 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-27 14:17:44 +00:00
kustermann@google.com e6077d093a Remove pixel/layout tests in tests/utils + special casing in the testing scripts for pixel/layout tests
R=ricow@google.com

Review URL: https://codereview.chromium.org//26998002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@28502 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-11 10:29:46 +00:00
kustermann@google.com ed3fe0161c test.py: Support for CompileTimeError,RuntimeError,MissingRuntimeError,MissingCompiletimeError markers in status files
R=ricow@google.com

Review URL: https://codereview.chromium.org//23702055

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@27804 260f80e4-7a28-3924-810f-c04153c831b5
2013-09-24 10:21:15 +00:00
kustermann@google.com b0889233cd test.py: Print the warning about left-over dirs only for directories created using Directory.createTemp()
R=ricow@google.com

Review URL: https://codereview.chromium.org//23714002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@27003 260f80e4-7a28-3924-810f-c04153c831b5
2013-09-02 09:29:12 +00:00
kustermann@google.com 45c42e08f2 Update checked-in binary to 0.6.21.3_r26639 (M6)
R=ricow@google.com

Review URL: https://codereview.chromium.org//23568002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@26704 260f80e4-7a28-3924-810f-c04153c831b5
2013-08-27 11:16:37 +00:00
kustermann@google.com 92303dffda test.py: First step towards support of caching dart2js compilations across runtimes
This CL changes the way we run tests completely: When enqueueing TestCases we
build up a dependency graph of Commands. This has the advantage that if more
TestCases share common commands, we only execute it once.

R=ricow@google.com

Review URL: https://codereview.chromium.org//21001003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@25760 260f80e4-7a28-3924-810f-c04153c831b5
2013-08-05 07:54:17 +00:00
kustermann@google.com 03f7b209b2 test.py: Report timings by adding up the time of all commands
BUT=http://dartbug.com/11847
R=ahe@google.com

Review URL: https://codereview.chromium.org//19388002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@25096 260f80e4-7a28-3924-810f-c04153c831b5
2013-07-17 13:15:22 +00:00
ricow@google.com af21d74d2d Fix safari browser to use Future.value
Also correct type errors found by the analyzer.

R=kustermann@google.com

Review URL: https://codereview.chromium.org//18089005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24532 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-27 17:07:17 +00:00
ricow@google.com 545f6e1341 Update checked in binary to version 0.5.19.0
This is a reland of r24086 with a fix catching errors in the websocket server in tools/testing/dart/http_server.dart.

R=kustermann@google.com

Review URL: https://codereview.chromium.org//18065003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24520 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-27 16:10:59 +00:00
kustermann@google.com 8984694c97 Print out experimental test.py reproduction command
R=ricow@google.com

Review URL: https://codereview.chromium.org//17361003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24231 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-20 14:19:11 +00:00
ricow@google.com 12682d3bc7 Add timing information for the individual steps of a failing test.
R=kustermann@google.com

Review URL: https://codereview.chromium.org//17302007

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24213 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-20 09:03:23 +00:00
ricow@google.com 4c5b2ac2ab Revert 24086 Update checked in binary to version 0.5.19.0
This is causing crashes in http server.

R=kustermann@google.com

Review URL: https://codereview.chromium.org//17274002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24096 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-17 13:22:15 +00:00
ricow@google.com 1970fc7afb Update checked in binary to version 0.5.19.0
R=kustermann@google.com

Review URL: https://codereview.chromium.org//17078007

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@24086 260f80e4-7a28-3924-810f-c04153c831b5
2013-06-17 10:47:02 +00:00
kustermann@google.com 3707cd889b Print output of all commands if a testcase failed
R=ricow@google.com

Review URL: https://codereview.chromium.org//15740034

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@23408 260f80e4-7a28-3924-810f-c04153c831b5
2013-05-30 13:15:26 +00:00
ricow@google.com 72cb8b7577 Catch errors on directory listings.
I saw issues on the new linux chrome bot when there was a broken symlink in /tmp

R=kustermann@google.com

Review URL: https://codereview.chromium.org//16085002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@23235 260f80e4-7a28-3924-810f-c04153c831b5
2013-05-28 06:37:18 +00:00
kustermann@google.com 9c08adf54b Print the number of failed tests
R=ricow@google.com

Review URL: https://codereview.chromium.org//14718005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@22323 260f80e4-7a28-3924-810f-c04153c831b5
2013-05-02 15:17:28 +00:00
kustermann@google.com c1df0a5418 Only print the failure summary header if there were failures
Review URL: https://codereview.chromium.org//14251009

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@21453 260f80e4-7a28-3924-810f-c04153c831b5
2013-04-15 13:57:31 +00:00
whesse@google.com d47b6701ea Fix type error in test scripts
BUG=

Review URL: https://codereview.chromium.org//14047011

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@21292 260f80e4-7a28-3924-810f-c04153c831b5
2013-04-11 16:31:02 +00:00
kustermann@google.com 1c1590da22 Remove \r in command stdout/stderr when building failure a failure output.
BUG=http://dartbug.com/8704

Review URL: https://codereview.chromium.org//13521002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20860 260f80e4-7a28-3924-810f-c04153c831b5
2013-04-03 13:20:51 +00:00
kustermann@google.com 78ac77e8c9 Fixed test_progress to print the failure summary even if no stepname was given
Review URL: https://codereview.chromium.org//13432003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20846 260f80e4-7a28-3924-810f-c04153c831b5
2013-04-03 11:32:44 +00:00
kustermann@google.com ca08052687 Do not close stdout/stderr in CompactProgressIndicator
BUG=http://dartbug.com/9419

Review URL: https://codereview.chromium.org//13052004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20497 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-25 22:29:25 +00:00
kustermann@google.com 4c45723e98 Added --failure-summary option to test.dart
Review URL: https://codereview.chromium.org//12902035

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20379 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-22 13:31:06 +00:00
sgjesse@google.com d6955d176c Reapply "Update the test runner to use the new dart:io API"
This reverts r20031.

R=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12751005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20070 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-15 11:13:39 +00:00
sgjesse@google.com f5270bf202 Revert "Update the test runner to use the new dart:io API" again
This reverts r20027 and r20029

TBR=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12475010

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20031 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 15:38:18 +00:00
sgjesse@google.com fdaca2fe83 Revert "Update the test runner to use the new dart:io API" again
This reapplies r20004 and r20008 and fixes a HTTP server issue.

TBR=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12431020

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20027 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 15:24:36 +00:00
sgjesse@google.com 3e2ffb4394 Revert "Update the test runner to use the new dart:io API" again
This reverts r20004 and r20008.

TBR=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12559013

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20009 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 12:13:37 +00:00
sgjesse@google.com db7821cdf3 Reapply "Update the test runner to use the new dart:io API"
The Linux binary will work on the bots. Fixed a number of issues.

The new binaries are from r19850.

R=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12608007

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@20004 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 11:39:20 +00:00
sgjesse@google.com 78403ce605 Revert "Update the test runner to use the new dart:io API"
This reverts commit r19987.

The included binaries did not work on the bots due to a glibc version issue.

TBR=ager@google.com

BUG=

Review URL: https://codereview.chromium.org//12864003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@19988 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 08:08:56 +00:00
sgjesse@google.com 7ff44f77ac Update the test runner to use the new dart:io API
The new binaries are from r19850.

R=whesse@google.com, ager@google.com, ricow@google.com, kustermann@google.com

BUG=

Review URL: https://codereview.chromium.org//12417004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@19987 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-14 07:54:07 +00:00
kustermann@google.com 4924f73340 Bugfix in test_progress.dart after refactoring
TBR=ricow

Review URL: https://codereview.chromium.org//12728005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@19881 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-12 17:37:28 +00:00
kustermann@google.com 53497f0c2c Refactoring of ProgressIndicator
Currently ProcessQueue creates and depends on exactly ONE ProgressIndicator.
This is not optimal for the following reasons:

 a) The used ProgressIndicator has several responsibilities (printing timing information,
    setting 'io.exitCode', printing test failures, ...)
    Splitting these responsibilities into different classes makes the code cleaner and more testable.

 b) If we make ProcessQueue depended on an abstract EventListener object (or a list of them) and make
    it send events to those listeners would make it easier to test ProcessQueue itself.

Issue: 8095

Review URL: https://codereview.chromium.org//12302016

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@19867 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-12 14:09:05 +00:00
kustermann@google.com c7857e7c0b Quote all arguments when printing a Command
Review URL: https://codereview.chromium.org//12379041

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@19291 260f80e4-7a28-3924-810f-c04153c831b5
2013-03-01 10:33:59 +00:00
kustermann@google.com 90e2760642 Start two http servers for every configuration
This should fix: http://dartbug.com/8366

Review URL: https://codereview.chromium.org//12340021

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@18964 260f80e4-7a28-3924-810f-c04153c831b5
2013-02-25 11:54:20 +00:00
kustermann@google.com 0a5a3306f7 Added '--package-root' to http_server.dart.
Review URL: https://codereview.chromium.org//12087130

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17968 260f80e4-7a28-3924-810f-c04153c831b5
2013-02-01 10:22:55 +00:00
kustermann@google.com 5d083443f8 Don't exit() in test_progress but rather set io.exitCode
Review URL: https://codereview.chromium.org//12026013

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17264 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-18 12:28:08 +00:00
kustermann@google.com ecf1cdeeb8 Fixed utf8 encoding/decoding issues in the testing scripts
Review URL: https://codereview.chromium.org//11883033

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17057 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-15 10:59:24 +00:00