Commit Graph

68 Commits

Author SHA1 Message Date
kustermann@google.com 0a5a3306f7 Added '--package-root' to http_server.dart.
Review URL: https://codereview.chromium.org//12087130

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17968 260f80e4-7a28-3924-810f-c04153c831b5
2013-02-01 10:22:55 +00:00
kustermann@google.com 5d083443f8 Don't exit() in test_progress but rather set io.exitCode
Review URL: https://codereview.chromium.org//12026013

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17264 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-18 12:28:08 +00:00
kustermann@google.com ecf1cdeeb8 Fixed utf8 encoding/decoding issues in the testing scripts
Review URL: https://codereview.chromium.org//11883033

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17057 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-15 10:59:24 +00:00
efortuna@google.com 4622dc3e76 Make browser tests all run from a server instead of the local filesystem.
BUG=

Review URL: https://codereview.chromium.org//11810004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@17007 260f80e4-7a28-3924-810f-c04153c831b5
2013-01-13 01:11:07 +00:00
kustermann@google.com a18994679b Call DumpRenderTree directly from test.dart instead of using the drt-trampoline.py indirection
Currently drt-trampoline.py is used to call DRT. Unfortunately this causes
problems on Windows: If drt-trampoline.py times out, it is killed by test.dart.
But drt-trampoline.py has no way of installing a signal handler on Windows which
would kill the actual DumpRenderTree process. So sometimes we're left with one
hanging DumpRenderTree process on the Windows buildbots.

This change removes the drt-trampoline.py indirection and calls DRT directly
from test.dart.

Committed: https://code.google.com/p/dart/source/detail?r=16214

Review URL: https://codereview.chromium.org//11586012

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16247 260f80e4-7a28-3924-810f-c04153c831b5
2012-12-18 09:52:54 +00:00
kustermann@google.com 0be2cb3fa5 Revert "Call DumpRenderTree directly from test.dart instead of using the drt-trampoline.py indirection"
TBR=ricow@google.com

Review URL: https://codereview.chromium.org//11591017

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16215 260f80e4-7a28-3924-810f-c04153c831b5
2012-12-17 17:21:48 +00:00
kustermann@google.com 8071e88618 Call DumpRenderTree directly from test.dart instead of using the drt-trampoline.py indirection
Currently drt-trampoline.py is used to call DRT. Unfortunately this causes
problems on Windows: If drt-trampoline.py times out, it is killed by test.dart.
But drt-trampoline.py has no way of installing a signal handler on Windows which
would kill the actual DumpRenderTree process. So sometimes we're left with one
hanging DumpRenderTree process on the Windows buildbots.

This change removes the drt-trampoline.py indirection and calls DRT directly
from test.dart.

Review URL: https://codereview.chromium.org//11586012

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16214 260f80e4-7a28-3924-810f-c04153c831b5
2012-12-17 16:27:11 +00:00
kustermann@google.com 62c83a4443 Moved the testing scripts to the new import/library/part syntax
Review URL: https://codereview.chromium.org//11575028

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16163 260f80e4-7a28-3924-810f-c04153c831b5
2012-12-14 14:19:01 +00:00
kustermann@google.com c01e393aa5 Added support for skipping redundant dart2js compilations.
Currently, the buildbots run tests on every commit. But often neither
the tests nor the dart2js compiler did change (e.g. on status file updates).

The added code will check if the dependencies (i.e. files listed in
test.js.deps) or the dart2js snapshot did change. If not, the output
of the previous dart2js compilation is up to date and the compilation
step is skipped.
The skipping of redundant dart2js compilations works only if the '--use-sdk'
option was specified (because in this case the dart2js snapshot is used).

Committed: https://code.google.com/p/dart/source/detail?r=15080

Review URL: https://codereview.chromium.org//11369216

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@16158 260f80e4-7a28-3924-810f-c04153c831b5
2012-12-14 11:56:00 +00:00
whesse@google.com 6220720429 Add --build-directory option to test.py.
Fix problem when no --step-name option is present.

BUG=dart:6885

Review URL: https://codereview.chromium.org//11308179

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@15280 260f80e4-7a28-3924-810f-c04153c831b5
2012-11-23 12:54:04 +00:00
kustermann@google.com 52077af7fa Revert "Added support for skipping redundant dart2js compilations."
We must disable this feature until we've made sure that the boolean
dart2js flags (e.g. --checked) are included in the directory names.

TBR=ahe

Review URL: https://codereview.chromium.org//11280061

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@15087 260f80e4-7a28-3924-810f-c04153c831b5
2012-11-19 15:47:59 +00:00
kustermann@google.com f4bc6a5f3f Added support for skipping redundant dart2js compilations.
Currently, the buildbots run tests on every commit. But often neither
the tests nor the dart2js compiler did change (e.g. on status file updates).

The added code will check if the dependencies (i.e. files listed in
test.js.deps) or the dart2js snapshot did change. If not, the output
of the previous dart2js compilation is up to date and the compilation
step is skipped.
The skipping of redundant dart2js compilations works only if the '--use-sdk'
option was specified (because in this case the dart2js snapshot is used).

Review URL: https://codereview.chromium.org//11369216

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@15080 260f80e4-7a28-3924-810f-c04153c831b5
2012-11-19 13:56:11 +00:00
kustermann@google.com 42b89c9f77 Changed: TestCase.output -> TestCase.commandOutputs[]
Currently the TestCase has exactly one TestOutput. This worked
until now, since we were only interested in the output of the
last command.

This change introduces one CommandOutput for each command which
has been run. Doing so enables us to fetch output (stdin,stderr)
information as well as exitCode ... for all commands.

See also 11293278.

Review URL: https://codereview.chromium.org//11348088

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@15006 260f80e4-7a28-3924-810f-c04153c831b5
2012-11-16 11:17:47 +00:00
kustermann@google.com 3dc5c7056a Revert of commit 14947
Review URL: https://codereview.chromium.org//11348060

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@14949 260f80e4-7a28-3924-810f-c04153c831b5
2012-11-15 09:15:43 +00:00
kustermann@google.com 5ec78286df Changed: TestCase.output -> TestCase.commandOutputs[]
Currently the TestCase has exactly one TestOutput. This worked
until now, since we were only interested in the output of the
last command.

This change introduces one CommandOutput for each command which
has been run. Doing so enables us to fetch output (stdin,stderr)
information as well as exitCode ... for all commands.

Review URL: https://codereview.chromium.org//11293278

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@14947 260f80e4-7a28-3924-810f-c04153c831b5
2012-11-15 09:05:41 +00:00
kustermann@google.com e78ad924f6 Added support for logging the output of failed flaky tests.
Failed flaky tests will be logged to 'failed_flaky_tests.log'.
The default behavior is that the old log file gets deleted before
any tests are run. To prevent this, the '--append_flaky_log' option
can be set.

Review URL: https://codereview.chromium.org//11364129

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@14735 260f80e4-7a28-3924-810f-c04153c831b5
2012-11-09 12:08:00 +00:00
ager@google.com 7f04c40b2d Land update to tools directory with new binaries.
BUG=

Review URL: https://codereview.chromium.org//11343008

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@14176 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-29 09:21:14 +00:00
ager@google.com 61e7010693 Fix issue that cases the usesWebDriver getter in TestCase to always return false.
Remove newly added code for timeout handling.

Make sure that timeout handling actually works so that dart2js
timing out will actually cause a timeout even when retrying
browser tests.

R=ahe@google.com,efortuna@google.com,ricow@google.com
BUG=

Review URL: https://codereview.chromium.org//11267008

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@14014 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-24 12:58:40 +00:00
ahe@google.com 8fafabe175 Update co19 to r397.
Review URL: https://codereview.chromium.org//11233058

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13935 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-23 09:28:31 +00:00
ricow@google.com fe7ace35e5 Fix DiffProgressIndicator to use the correct parameter name
Review URL: https://codereview.chromium.org//11187049

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13770 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-18 11:17:12 +00:00
ahe@google.com 6cac76e938 Group co19 results by runtime.
Review URL: https://codereview.chromium.org//11181004

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13680 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-16 14:23:35 +00:00
ahe@google.com 3177727403 Group co19 failures and test dart2dart.
Review URL: https://codereview.chromium.org//11099008

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13397 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-09 07:20:39 +00:00
ager@google.com 66e541d4df Update test.dart to do the same on all platforms for tests run
through selenium.

Remove the explicit exit(0) which can hide other issues. Cancel
the kill timer when the process terminates gracefully (which was
one of those issues hidden by the explicit exit(0)).

R=efortuna@google.com,gram@google.com,ahe@google.com,ricow@google.com
BUG=

Review URL: https://codereview.chromium.org//11086019

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13396 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-09 07:08:47 +00:00
efortuna@google.com 9c25f5fd61 Add back in full exit code information.
Review URL: https://codereview.chromium.org//11066065

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13380 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-08 23:09:45 +00:00
efortuna@google.com 507bc5cfc9 Add back in "graceful shutdown code" for Windows, because the fix for
dartbug.com/1756 doesn't fix Windows.

Also revert my change in r13368 to find the source of this problem.
Review URL: https://codereview.chromium.org//11017022

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13379 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-08 22:30:52 +00:00
efortuna@google.com 6ff38b9b1c Always print an exit code explicitly (no return of any exit code was causing the
Windows bots to hang).
Review URL: https://codereview.chromium.org//11090008

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13368 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-08 18:51:50 +00:00
ahe@google.com 1c51c6829d Highlight test headings with -pcolor.
Review URL: https://codereview.chromium.org//11038064

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13345 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-08 08:27:03 +00:00
efortuna@google.com 6e42838196 Fix test.dart return code -- was causing fake greenness on dartc, dart2dart
(potentially the VM) as a result of previous CL:
https://codereview.chromium.org/11032047/
Review URL: https://codereview.chromium.org//11071003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13336 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-06 00:17:57 +00:00
ager@google.com 33b4a0ac44 Fix issues with test.dart that appeared on Mac.
The issue was that we did not properly wait for subprocesses to
die. This was hidden by the fact that we always used exit to exit from
test.dart. I have removed all exit(0) calls from test.dart to keep us
honest. If nothing went wrong test.dart should terminate with exit
code 0 automatically.

R=ricow@google.com,ahe@google.com
BUG=dartbug.com/5644

Review URL: https://codereview.chromium.org//11032047

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13286 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-05 11:41:47 +00:00
ager@google.com c4c4b7f08c Add in-process http server to the dart test scripts.
Removed support for extra commands. Change pkg/intl test to use
builtin http server.

Fixed regression in test runner so the test script terminates when
passing in an unknown selector.

BUG=

Review URL: https://codereview.chromium.org//11035027

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13217 260f80e4-7a28-3924-810f-c04153c831b5
2012-10-04 10:43:48 +00:00
ahe@google.com 48a59514f3 Add progress indicator for co19 updating.
Review URL: https://codereview.chromium.org//11000049

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@13008 260f80e4-7a28-3924-810f-c04153c831b5
2012-09-28 12:13:50 +00:00
ahe@google.com 1c518f6169 Separate compile-time and runtime errors.
Review URL: https://codereview.chromium.org//10977054

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@12964 260f80e4-7a28-3924-810f-c04153c831b5
2012-09-27 14:31:05 +00:00
ahe@google.com 722e7ce28d Add --step_name option to test.dart.
Review URL: https://chromiumcodereview.appspot.com//10905208

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@12296 260f80e4-7a28-3924-810f-c04153c831b5
2012-09-13 04:19:11 +00:00
whesse@google.com 61d20e42f9 Use new syntax for getters in test.dart and its libraries.
BUG=

Review URL: https://chromiumcodereview.appspot.com//10920003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@11616 260f80e4-7a28-3924-810f-c04153c831b5
2012-08-30 14:16:05 +00:00
whesse@google.com dff5549f93 Eliminate standalone timeout_test, pass_test, and fail_test.
These are now built into standalone/io/test_runner_test.

BUG=dart:3216
TEST=tools/test.py standalone/io/test_runner_test

Review URL: https://chromiumcodereview.appspot.com//10521003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@8315 260f80e4-7a28-3924-810f-c04153c831b5
2012-06-06 09:18:37 +00:00
ahe@google.com 725984cf50 Create a unique name for each failure step.
Review URL: https://chromiumcodereview.appspot.com//10456026

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@8091 260f80e4-7a28-3924-810f-c04153c831b5
2012-05-30 09:12:51 +00:00
ahe@google.com f6d5547147 Make sure the failing build step is red.
Review URL: https://chromiumcodereview.appspot.com//10448050

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@8056 260f80e4-7a28-3924-810f-c04153c831b5
2012-05-29 13:22:13 +00:00
ahe@google.com 857a7108d9 Improve annotated steps output:
* Make sure a step is started before anything is printed (this removes the "preamble" section).

* Create a "failures" step before printing the failures again. This makes it easier to find the failures (especially, from a mobile browser).

Review URL: https://chromiumcodereview.appspot.com//10443057

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@8054 260f80e4-7a28-3924-810f-c04153c831b5
2012-05-29 13:01:22 +00:00
whesse@google.com a4e88c8b3e Remove string concatenation with + from all Dart files in tools directory.
Fix whitespace errors and rename shadowed variable in ddbg.dart.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com//10387232

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@7897 260f80e4-7a28-3924-810f-c04153c831b5
2012-05-23 08:51:38 +00:00
whesse@google.com ec58b8b233 Fix some type warnings in the test scripts, that show up when using the editor.
BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com//10001006

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@6569 260f80e4-7a28-3924-810f-c04153c831b5
2012-04-16 12:41:45 +00:00
whesse@google.com bb385c8b50 Fix some type and logical errors in test.dart, found by the Dart editor.
BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com//9662063

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@5334 260f80e4-7a28-3924-810f-c04153c831b5
2012-03-12 16:46:45 +00:00
whesse@google.com 744ca3cf46 Fix whitespace, long lines, and "boolean".
BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com//9632019

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@5225 260f80e4-7a28-3924-810f-c04153c831b5
2012-03-09 09:46:01 +00:00
zundel@google.com ab603367c0 Update test.dart for detection output of machine formatted errors
This change causes the 'dartc' tests to be more sensitive to the difference
between a static type error and a compilation error.   Instead of
relying on the return value from the test being 0 or non zero, the
code now turns on '--error_format machine' and checks type
error level 'ERROR' or 'WARNING' and the type of error for
'STATIC_TYPE' if there is a static type error.  A new annotation
format adapted from multitests is introduced for standard test cases:

  int foo = "hello";   /// static type error

The above annotation means that a static type error is expected.
Currently enforcement is very lenient.  There must be at least as
many STATIC_TYPE error messages as there are '/// static type error'
annotations (and no ERROR level messages.)

Review URL: https://chromiumcodereview.appspot.com//9559007

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@4895 260f80e4-7a28-3924-810f-c04153c831b5
2012-03-02 18:50:09 +00:00
sigmund@google.com 6cefc269d6 test.dart: add support for compiling multiple scripts for a single test.
Review URL: https://chromiumcodereview.appspot.com//9475038

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@4708 260f80e4-7a28-3924-810f-c04153c831b5
2012-02-28 17:47:43 +00:00
efortuna@google.com 36b88ced7a Don't retry browser tests if the bot is pretty red. Also, retry properly all
times that DRT crashes.
Review URL: https://chromiumcodereview.appspot.com//9426005

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@4370 260f80e4-7a28-3924-810f-c04153c831b5
2012-02-17 21:44:34 +00:00
sigmund@google.com c656cc1b62 test.dart progress bar: a small tweak when --report is not used.
Review URL: https://chromiumcodereview.appspot.com//9378028

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@4190 260f80e4-7a28-3924-810f-c04153c831b5
2012-02-13 16:25:26 +00:00
whesse@google.com e7860a4f33 Change output of test.dart --list option, so it can be parsed automatically.
The framework that lets shared tests be run in Eclipse uses the output of this command, and this framework is tested when running test.dart --component=dartc dartc/junit_tests.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com//9315036

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@3891 260f80e4-7a28-3924-810f-c04153c831b5
2012-02-03 12:19:20 +00:00
ager@google.com 4efd1f98fa Split dart:builtin into dart:builtin and dart:io.
- dart:builtin is implicitly imported.
- dart:io has to be explicitly imported.

I will update the macos and windows stable test binaries when we
are happy with the solution.

R=iposva@google.com
BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com//9254026

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@3537 260f80e4-7a28-3924-810f-c04153c831b5
2012-01-24 07:36:59 +00:00
whesse@google.com ca63b5de74 Add configuration information to test.dart output.
BUG=dart:1168
TEST=

Review URL: https://chromiumcodereview.appspot.com//9150027

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@3431 260f80e4-7a28-3924-810f-c04153c831b5
2012-01-19 12:37:41 +00:00
whesse@google.com c220130314 Add summary of failures to end of test.dart output.
BUG=
TEST=

Review URL: http://codereview.chromium.org//9234002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@3336 260f80e4-7a28-3924-810f-c04153c831b5
2012-01-16 12:31:50 +00:00