247ff772d6
Update status files, replace "dartbug.com" with "Issue ".
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//845793004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42900 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-15 10:27:11 +00:00
5bba992965
Enable co19 dart2js browser tests on content shell (drt).
...
Update co19 status files for Firefox update to 35, and other flakes.
TBR=ricow@google.com
Review URL: https://codereview.chromium.org//847393002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42873 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-14 15:56:28 +00:00
9df5e56e9f
tools: fix break in test_progress
...
TBR
Review URL: https://codereview.chromium.org//829803005
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42832 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-13 19:46:24 +00:00
3fdea4f366
Outputing comma-separated report with percentages
...
Merged:
http://crrev.com/832583003
http://crrev.com/843553004
R=ricow@google.com
Review URL: https://codereview.chromium.org//847873004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42831 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-13 19:27:47 +00:00
8cd05a8b90
tools: Refactored SummaryReport and move it to its own library
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//832583003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42826 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-13 17:46:52 +00:00
4598291567
Add info about which command line was used to start a browser.
...
This will help us if we see hanging browsers.
R=kustermann@google.com
Review URL: https://codereview.chromium.org//790793009
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42728 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-09 10:46:40 +00:00
b9a61a835a
Allow for reporting with the --list option
...
Also, be better at catagorizing tests and have a category for tests that we don't know.
R=whesse@google.com
Review URL: https://codereview.chromium.org//846433002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42695 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-08 15:23:55 +00:00
261200c587
Build Observatory as part of runtime
...
Relanding https://codereview.chromium.org/810623005/
Review URL: https://codereview.chromium.org//837723004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42645 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-06 21:59:42 +00:00
aa5a986d5d
Revert "Build Observatory with runtime"
...
This reverts commit 76df7b3c1bf83c08d3994d61df4c9c530fadb4e5.
This commit breaks dartium compilation.
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//839543002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42619 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-06 10:07:58 +00:00
9d6ac72b46
1) Remove prebuilt Observatory sources
...
2) Move observatory from runtime/bin/vmservice/observatory to runtime/observatory
3) Add two tools scripts:
- tools/run_pub.py (runs pub before SDK is built)
- tools/obs_tool.py (helper for get, build, and deploy stages)
4) Build Observatory with runtime:
- pub get --offline
- pub build
- deploy
5) Build artifacts are now in standard output directory, for example, out/DebugIA32/observatory.
6) Add a new 'dart_boostrap' host target (no snapshot, no observatory) that can be used as the Dart executable to run pub when building Observatory. This is behind a build.py flag --use-bootstrap-for-observatory because:
- It is only necessary on older Linux distributions that are incompatible with the prebuilt Dart testing executable.
- running pub build with the boostrap Debug build is significantly slower.
7) Detect if the prebuilt executable doesn't work and automatically switch to the 'dart_bootstrap' executable. Also, warn the user and provide a Wiki link with more information.
R=turnidge@google.com
Review URL: https://codereview.chromium.org//810623005
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42614 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-05 21:26:52 +00:00
84669b632d
Report the correct number of tests when tests are skipped due to compile time errors.
...
The current setup will count tests that are skipped when runtime is a browser due to compile time errors twice. Example:
tools/test.py -mrelease -rdrt -cdart2js --report language/bad_override
<snip>
Total: 10 tests
* 0 tests will be skipped (0 skipped by design)
* 0 tests are expected to be flaky but not crash
* 5 tests are expected to pass
* 0 tests are expected to fail that we won't fix
* 2 tests are expected to fail that we should fix
* 0 tests are expected to crash that we should fix
* 0 tests are allowed to timeout
* 3 tests are skipped on browsers due to compile-time error
This is a multitest that has 7 tests (including none). 3 of these are giving compile-time errors, 2 of these are failing (as a compile-time error when they should not, but that is not the point), 2 are passing.
This is preparation of a bigger refactoring of the summary reporting, so that I can have sane numbers for validation.
R=kustermann@google.com
Review URL: https://codereview.chromium.org//816703003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42460 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-18 15:15:12 +00:00
4723aee28d
Retry up to 4 times on safari until we have a resolution for issue 21434
...
I worry more that people will actually ignore these then retrying it for now.
R=floitsch@google.com
Review URL: https://codereview.chromium.org//807813004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42424 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-17 09:57:11 +00:00
5f8aa66fde
Rename legacy path library in tools/testing to avoid name collision.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//756423003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42129 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-05 07:21:47 +00:00
47de553025
tools/testing: move code into individual libraries
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//748773004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42108 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-04 15:32:57 +00:00
61b056964a
tools: removed unused members, tiny cleanup
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//770633003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42106 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-04 13:56:14 +00:00
0059ade0dd
Add android device id to test output for failing tests.
...
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//755263002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41960 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-25 16:11:42 +00:00
12845acbf3
Revert "Add android device id to test output in test scripts."
...
This reverts r41933
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//758563004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41934 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-24 17:53:30 +00:00
d91010512a
Add android device id to test output in test scripts.
...
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//752173002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41933 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-24 17:40:13 +00:00
6601198a60
Do not retry intentionally failing tests on browsers.
...
BUG=dartbug.com/21434
R=ricow@google.com
Review URL: https://codereview.chromium.org//726563002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41714 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-13 14:20:50 +00:00
f82e0e339a
Put test shared options before filename in compilation commands.
...
BUG=dartbug.com/21582
R=ricow@google.com
Review URL: https://codereview.chromium.org//718073002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41708 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-13 09:17:00 +00:00
3b4613ca3a
Enable dart2js compilation for new HTML tests.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//700033003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41668 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-11 14:59:14 +00:00
beaa60020c
Refactor test scripts browser multitest loop.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//688323006
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41635 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-10 14:25:59 +00:00
ea27275aa6
Refactor browser test enqueuing in test scripts.
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//700753004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41633 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-10 13:37:40 +00:00
55dac6a763
Allow testing script to run without the package directories being present
...
R=sigmund@google.com , whesse@google.com
Review URL: https://codereview.chromium.org//705093002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41570 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-06 17:14:49 +00:00
562fb318a8
Support dart and js scripts in HTML tests, on dartium.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//705733003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41569 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-06 17:03:24 +00:00
84b0209a77
Add test/xml content type to test server.
...
BUG=dartbug.com/21525
R=ricow@google.com
Review URL: https://codereview.chromium.org//709483002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41566 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-06 16:37:16 +00:00
1c840e6f01
Re-land r41544 "Update dart binaries used for running tests to version 1.7.2"
...
This also re-lands r41546, and some status file updates.
BUG=dartbug.com/21525 21526 21527
R=ricow@google.com
Review URL: https://codereview.chromium.org//701413002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41562 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-06 15:28:56 +00:00
802c618118
Move test config files under tools/testing/dart
...
R=whesse@google.com
Review URL: https://codereview.chromium.org//708783002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41561 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-06 14:21:19 +00:00
d71ecd85d9
Fix hanging test script batch runners.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//707733003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41553 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-06 13:03:00 +00:00
0624b4f7b7
Revert r41544 "Update dart binaries used for running tests to version 1.7.2"
...
Also revert fixes for test scripts for this update, r41546
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//708733003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41547 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-06 11:41:49 +00:00
207f9eacb4
Fix test scripts and xhr test for new default HTTP headers
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//704133003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41546 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-06 09:59:51 +00:00
151dfd105e
Fix multitest display name bug in test scripts.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//702873002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41520 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-05 13:42:25 +00:00
cecc8d7de5
Fix HTML tests on non-browser runtimes.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//704733004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41519 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-05 13:09:50 +00:00
6ac0684ab8
Working SimpleHTML test front end
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//695893002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41517 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-05 12:54:34 +00:00
49acd886b9
Improve handling of intentionally broken links in test scripts.
...
BUG=
Review URL: https://codereview.chromium.org//702673002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41492 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-04 15:00:00 +00:00
0928369fc9
Improve browser testing scripts to better handle tests that reload themselves.
...
BUG=dartbug.com/18558
R=ricow@google.com
Review URL: https://codereview.chromium.org//702543006
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41490 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-04 14:46:45 +00:00
c6102a3b5b
Retry failing tests on Safari, because it is flaky.
...
BUG=dartbug.com/21434
Review URL: https://codereview.chromium.org//691583002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41397 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-29 17:53:14 +00:00
67e8b965bb
_Temporarily_ retry failing browser tests on Safari, because versions 6.2 and 7.1 are flaky on dart2js output.
...
Add support for macos 10.9 to buildbot scripts.
BUG=dartbug.com/21434
R=ricow@google.com
Review URL: https://codereview.chromium.org//687213002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41394 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-29 13:07:11 +00:00
aeba67e44e
Add builder tag for Safari 6.2 slave. Update status files for Safari 6.2.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//679153002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41323 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-27 16:09:00 +00:00
f7db779401
Clean up scripts to run Java-based analyzer.
...
sdk/bin/dartanalyzer_developer is renamed sdk/bin/dartanalyzer_java
(replacing the old sdk/bin/dartanalyzer_java, which was unused), and
the test infrastructure is modified to use sdk/bin/dartanalyzer_java
to start the Java-based analyzer.
This frees up sdk/bin/dartanallyzer_developer to be the --host-checked
version of sdk/bin/dartanalyzer (paralleling what we do for
/sdk/bin/dart2js_developer).
R=ricow@google.com
Review URL: https://codereview.chromium.org//668743003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41227 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-21 13:02:37 +00:00
d83ee35044
Terminate browsers in sequence.
...
Also make iframe larger.
R=whesse@google.com
Review URL: https://codereview.chromium.org//660823002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41189 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-20 12:32:50 +00:00
b9116653bd
Modify dartanalyzer scripts to look more like dart2js scripts.
...
The dartanalyzer script (and its Windows companion, dartanalyzer.bat)
now support:
- Executing from snapshot if present, otherwise from source (needed
for buildbot tests).
- Passing in the appropriate package root when executind from source.
- Passing in VM tuning parameters to speed up analysis.
- Passing through extra DART_VM_OPTIONS to the VM.
Also, the old editor/tools/analyzer script has been removed and
the test infrastructure now uses sdk/bin/dartanalyzer, in the
same way that it works for dart2js.
R=ahe@google.com , johnniwinther@google.com , ricow@google.com
Review URL: https://codereview.chromium.org//628363002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41150 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-16 14:21:37 +00:00
6650aeb3d6
Clean up test_runner Command subclass hash/equality.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//645533002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41037 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-10 12:05:52 +00:00
b21c902607
Add support in test scripts backend for simple HTML tests.
...
BUG=dartbug.com/20698
R=ricow@google.com
Review URL: https://codereview.chromium.org//632283002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@40956 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-07 15:29:39 +00:00
33e51a5295
Update co19 test suite to r786.
...
R=kustermann@google.com , ricow@google.com
Review URL: https://codereview.chromium.org//621753002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@40832 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-01 09:36:32 +00:00
7efd38d195
Start implementing checked mode compile time errors in analyzer.
...
This enables the Java-based command-line analyzer to accept the
"--enable_type_checks" flag so that it can be told whether to report
type checking errors during constant evaluation. Also it fixes the
tests language/compile_time_constant_checked2_test and
language/compile_time_constant_checked3_test in unchecked mode, by
causing CONST_FIELD_INITIALIZER_NOT_ASSIGNABLE to be escalated to an
error only in checked mode.
Note: in order to test this properly, I had to remove a hack from
test_runner.dart which caused it to effectively ignore the "checked
mode compile-time error" test annotation when testing analyzer. With
this hack removed, several new lines needed to be added to the .status
files.
BUG=dartbug.com/16391
R=jwren@google.com , kustermann@google.com
Review URL: https://codereview.chromium.org//610863004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@40780 260f80e4-7a28-3924-810f-c04153c831b5
2014-09-29 18:59:14 +00:00
94e1160661
Add PKGTestSuite that will be used for package waterfall testers.
...
This is based on the standard test suite but we specialcase html files.
There is another html file aproach currently being done by whesse@ that we may be able to utilize when it is done instead.
R=sigmund@google.com
Review URL: https://codereview.chromium.org//567803004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@40255 260f80e4-7a28-3924-810f-c04153c831b5
2014-09-15 17:16:35 +00:00
4acbe3d75a
Find pub status file in right directory.
...
R=nweiz@google.com
Review URL: https://codereview.chromium.org//561893002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@40123 260f80e4-7a28-3924-810f-c04153c831b5
2014-09-10 22:38:24 +00:00
13e1c5d54b
Allow trailing / in directory paths
...
I run into errors as soon as I use tab completion and include a trailing slash
in the suite_dir, this help a bit.
R=ricow@google.com
Review URL: https://codereview.chromium.org//560643002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@40093 260f80e4-7a28-3924-810f-c04153c831b5
2014-09-10 15:37:49 +00:00
17c1642748
Store the async-await compiled pub code directly in the repo.
...
R=nweiz@google.com
Review URL: https://codereview.chromium.org//557563002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@40041 260f80e4-7a28-3924-810f-c04153c831b5
2014-09-09 16:55:16 +00:00
whesse@google.com