8257b1874e
Restart ie11 browser while running tests. Update status files.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//888013002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@43329 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-30 10:33:07 +00:00
5406811398
Restart IE 10 browser as we do with Android browsers on buildbot, to improve stability
...
BUG=22203
Review URL: https://codereview.chromium.org//867083003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@43294 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-29 16:24:00 +00:00
014a89563a
Add debug logging for android testing.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//863663002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@43005 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-20 11:39:09 +00:00
4598291567
Add info about which command line was used to start a browser.
...
This will help us if we see hanging browsers.
R=kustermann@google.com
Review URL: https://codereview.chromium.org//790793009
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42728 260f80e4-7a28-3924-810f-c04153c831b5
2015-01-09 10:46:40 +00:00
47de553025
tools/testing: move code into individual libraries
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//748773004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@42108 260f80e4-7a28-3924-810f-c04153c831b5
2014-12-04 15:32:57 +00:00
0059ade0dd
Add android device id to test output for failing tests.
...
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//755263002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41960 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-25 16:11:42 +00:00
12845acbf3
Revert "Add android device id to test output in test scripts."
...
This reverts r41933
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//758563004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41934 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-24 17:53:30 +00:00
d91010512a
Add android device id to test output in test scripts.
...
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//752173002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41933 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-24 17:40:13 +00:00
6ac0684ab8
Working SimpleHTML test front end
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//695893002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41517 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-05 12:54:34 +00:00
0928369fc9
Improve browser testing scripts to better handle tests that reload themselves.
...
BUG=dartbug.com/18558
R=ricow@google.com
Review URL: https://codereview.chromium.org//702543006
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41490 260f80e4-7a28-3924-810f-c04153c831b5
2014-11-04 14:46:45 +00:00
d83ee35044
Terminate browsers in sequence.
...
Also make iframe larger.
R=whesse@google.com
Review URL: https://codereview.chromium.org//660823002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41189 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-20 12:32:50 +00:00
b21c902607
Add support in test scripts backend for simple HTML tests.
...
BUG=dartbug.com/20698
R=ricow@google.com
Review URL: https://codereview.chromium.org//632283002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@40956 260f80e4-7a28-3924-810f-c04153c831b5
2014-10-07 15:29:39 +00:00
f9b0a27ee9
test.dart: Serve the browser test driver page from the test file HTTP server.
...
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//540933002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@39907 260f80e4-7a28-3924-810f-c04153c831b5
2014-09-05 11:01:26 +00:00
48065a1f91
Update test scripts to run under the Dart 1.6 vm.
...
BUG=
R=sgjesse@google.com
Review URL: https://codereview.chromium.org//483353003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@39443 260f80e4-7a28-3924-810f-c04153c831b5
2014-08-21 09:30:10 +00:00
e89bd4ab1d
Try running ie10 in a new window instead of an iframe.
...
We are seeing issues where it seems that we are getting unittest-suite-success messages out of the blue and this may be related to:
https://code.google.com/p/dart/issues/detail?id=17825
This could be the old javascript still running in the iframe posting the event.
R=whesse@google.com
Review URL: https://codereview.chromium.org//477613002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@39245 260f80e4-7a28-3924-810f-c04153c831b5
2014-08-14 12:36:54 +00:00
4431b50125
Browser testing: Run android content shell tests on hosts with multiple devices connected.
...
The adb_reverse_forwarder.py script in [dartium]/build/android/ needs the --all-devices option added. We will also try to upstream this flag to chromium.
This change makes the test host reinstall the ContentShell apk on the device each time the device is restarted due to timeout or periodic reset, which takes about 10 seconds, but this is simpler and more like the existing behavior with android chrome.
R=ricow@google.com
Review URL: https://codereview.chromium.org//467643002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@39142 260f80e4-7a28-3924-810f-c04153c831b5
2014-08-12 14:40:26 +00:00
75e14b393a
Test infrastructure: Avoid restarting browser app on Android twice simultaneously.
...
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//442673003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@38898 260f80e4-7a28-3924-810f-c04153c831b5
2014-08-05 14:01:23 +00:00
60a9ad6c0b
Reland r37983, "Add periodic browser restart to ContentShellOnAndroid test runtime"
...
This was reverted in r38002, because it didn't seem needed. It now looks like it will make the builder more stable.
Original CL: https://codereview.chromium.org/363323002/
TBR=ricow@google.com
Review URL: https://codereview.chromium.org//439713006
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@38866 260f80e4-7a28-3924-810f-c04153c831b5
2014-08-04 16:25:47 +00:00
9f85c90367
Make ie11 use a fresh window again
...
Review URL: https://codereview.chromium.org//379923002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@38088 260f80e4-7a28-3924-810f-c04153c831b5
2014-07-09 16:36:54 +00:00
33ed771f0a
Don't use new windows for ie11
...
R=whesse@google.com
Review URL: https://codereview.chromium.org//377493002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@38023 260f80e4-7a28-3924-810f-c04153c831b5
2014-07-07 11:10:35 +00:00
bb59b9fa85
Revert r37983, "Add periodic browser restart to ContentShellOnAndroid test runtime. Update test status for ContentShellOnAndroid."
...
This reverts CL https://codereview.chromium.org/363323002/
We found that the flaky timeouts are real renderer crashes, and this change does not seem to help. We will instead report more data on the renderer crashes.
BUG=
R=ricow@google.com
Review URL: https://codereview.chromium.org//373453002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@38002 260f80e4-7a28-3924-810f-c04153c831b5
2014-07-04 09:19:08 +00:00
419bf8df29
Add periodic browser restart to ContentShellOnAndroid test runtime. Update test status for ContentShellOnAndroid.
...
BUG=dartbug.com/19127
R=kustermann@google.com
Review URL: https://codereview.chromium.org//363323002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@37983 260f80e4-7a28-3924-810f-c04153c831b5
2014-07-03 12:59:59 +00:00
edcdb5e02d
Update co19 test suite to r737
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//330633002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@37305 260f80e4-7a28-3924-810f-c04153c831b5
2014-06-13 08:15:06 +00:00
6be37386e0
Add mobile safari simulator support in the testing scripts
...
R=whesse@google.com
Review URL: https://codereview.chromium.org//262763005
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@35626 260f80e4-7a28-3924-810f-c04153c831b5
2014-05-01 11:32:08 +00:00
9a9ca47034
Add clear_browser_cache flag to test.dart
...
This can be used by any browser implementation to flush browser specific caches if set
clear_safari_cache is left in as a synonym for now, I will remove if after I update the bots
Review URL: https://chromiumcodereview.appspot.com//252653007
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@35477 260f80e4-7a28-3924-810f-c04153c831b5
2014-04-28 09:38:31 +00:00
d8a23ea25a
"Reverting 35157"
...
BUG=
R=blois@google.com
Review URL: https://codereview.chromium.org//243253002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@35211 260f80e4-7a28-3924-810f-c04153c831b5
2014-04-21 17:51:42 +00:00
b5ffc1968e
Fix Browser Controller to not reuse iFrames, which can sometimes cause bad state to be left between tests.
...
BUG=
R=jmesserly@google.com
Review URL: https://codereview.chromium.org//241783003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@35157 260f80e4-7a28-3924-810f-c04153c831b5
2014-04-17 22:56:05 +00:00
8fb7ab6c74
test.dart: Add timeout for browsers that are started but do not fetch tests.
...
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//218683009
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@34701 260f80e4-7a28-3924-810f-c04153c831b5
2014-04-03 12:35:22 +00:00
618140cd95
Get the correct version of ie from the reg database
...
version is 9.0.FOO.BAR on ie9
9.10.XFOO.XBAR on ie10
9.11.ZFOO.ZBAR on ie11
svcVersion is a more sane number, i.e., on ie10 it is 10.0.9200.16843 (atm) and on ie11 it is 11.0.9600.16521 (atm)
R=whesse@google.com
Review URL: https://codereview.chromium.org//210783004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@34355 260f80e4-7a28-3924-810f-c04153c831b5
2014-03-25 12:47:59 +00:00
6e6bc22e12
Add support for ie11 in our testing scripts
...
We explicitly need this since we need to be able to distinguesh between the
different versions in the status files. We could generalize this by having browser groups for everything but the actual status file parsing.
Review URL: https://codereview.chromium.org//206623010
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@34349 260f80e4-7a28-3924-810f-c04153c831b5
2014-03-25 08:51:53 +00:00
72b0b89102
Allow empty messages coming from the iframe to the driving page
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//188543003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@33377 260f80e4-7a28-3924-810f-c04153c831b5
2014-03-06 14:29:55 +00:00
7e29f42833
Change tests to use test.py's test_controller.js, cleanup in browser_controller
...
This change makes all remaining tests use test.py's test_controller.js:
We have our own way of testing using test.py and that might change any time.
It's therefore not wise to rely on functionality in the unittest framework. If
we kept test_controller.js in the unittest package, we'd need to make a breaking
change whenever we change the behavior of test_controller.js. So this file was
forked into tools/testing/dart/test_controller.js.
After this, all our tests will use tools/testing/dart/test_controller.js. It is
therefore no longer necessary for the browser_controller to support the old
mechanism of posting back results. By removing this legacy code, we can ensure,
that the driving page of browser_controller will only react on valid messages
which were ment for it.
Context:
Some tests (e.g. dromaeo) might use postMessage internally, maybe even for
testing that postMessage is working. But the driving window should only react
on valid messages ment for it, ignoring any other messages.
R=blois@google.com , efortuna@google.com , ricow@google.com , whesse@google.com
Review URL: https://codereview.chromium.org//177333004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@33365 260f80e4-7a28-3924-810f-c04153c831b5
2014-03-06 10:37:53 +00:00
9d53b1d682
Replace run_selenium.py with Dart script that uses the browser controller for performance tests.
...
BUG=
R=kustermann@google.com , ricow@google.com , vsm@google.com
Review URL: https://codereview.chromium.org//178223008
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@33171 260f80e4-7a28-3924-810f-c04153c831b5
2014-02-28 21:06:28 +00:00
338fabdb6c
Cancel stdout/stderr subscriptions once subprocesses are dead, add more debugging information in case the debug timer expires
...
R=whesse@google.com
Review URL: https://codereview.chromium.org//128703002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@31649 260f80e4-7a28-3924-810f-c04153c831b5
2014-01-09 09:04:46 +00:00
c8bfa98017
Test infrastructure: add retry to browser controller sending "done" message.
...
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//107513004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30950 260f80e4-7a28-3924-810f-c04153c831b5
2013-12-06 16:38:15 +00:00
e634fb20a4
Make status updates after 10 seconds + fix handling of status updates
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//94213002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30748 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-28 08:50:58 +00:00
bb54b776ba
Checked-in binary update to v1.0.0.3
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//91473002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30717 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-27 14:17:44 +00:00
8649be6036
Implement a periodic timer which sends the current state of a browser test back to the browser controller
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//77763004
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30468 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-20 13:32:16 +00:00
e3d8608ca2
test.py: Sending JSON between test_controller.js <-> browser_controller
...
Currently test.py will look for FAIL/PASS messages in the DOM to determine if a
test has failed or not. test_controller.js will add these PASS/FAIL messages to
the DOM (in addition to other messages).
This approach has several issues:
- The test itself could create nodes in the DOM containing FAIL/PASS. This means
that the outcome of a test cannot be reliably determined.
- The test ifself can remove nodes from the DOM (e.g. setting
document.body.innerHTML). This could result in a situation where the
stacktraces/... test_controller.js prints to the DOM are lost.
- It is unclear which parts of the DOM we see in test failure messages
(reported by test.py) come from the test itself and which parts come from
test_controller.js.
This CL tries to fix this by recording all events that happend with global
variables in test_controller.js and send back a JSON string with information
about
- debug messages
- posted messages
- print() calls of the test
- DOM
- errors (from window.onerror, script.onerror ...)
Furthermore this CL fixes unittest to report
unittest-suite-success/unittest-suite-fail instead of unittest-suite-done (which
is broken).
R=ricow@google.com
Review URL: https://codereview.chromium.org//36913002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@30407 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-19 12:54:41 +00:00
0f2f482388
Add browser path options to testing scripts.
...
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org//60503002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@29981 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-06 16:24:33 +00:00
a69e4a5e8d
Adds checked mode to Dartium on Android browser test controller.
...
R=kustermann@google.com
Review URL: https://codereview.chromium.org//52803007
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@29832 260f80e4-7a28-3924-810f-c04153c831b5
2013-11-04 18:26:41 +00:00
b5373a419a
Run all tests inside an iframe + restrict number of chrome browser processes on mac
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//49043003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@29367 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-28 16:31:18 +00:00
bdc3343090
Adds test target for Dartium on Android.
...
R=kustermann@google.com
Review URL: https://codereview.chromium.org//43773005
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@29293 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-25 21:34:25 +00:00
5572278a23
Changes to run tests on Dartium content shell on Android.
...
- In browser_controller.dart, adds a drtOnAndroid Browser.
- Adds flags for starting test servers at fixed ports.
- This is needed so that android reverse port forwarding
can be set up in advance.
- Adds setProp to android.dart to be used in lieu of
environment variables when testing on Android.
R=kustermann@google.com
Review URL: https://codereview.chromium.org//29773006
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@29259 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-25 16:39:48 +00:00
b3ba5c5c95
test.py: Support for running tests in checked mode on none-dartium with the browser controller
...
R=ricow@google.com
Review URL: https://codereview.chromium.org//38753008
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@29175 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-24 15:37:33 +00:00
006645db9e
Capture and report stdout/stderr of the browser while running a test
...
Currently we have no way of getting stdout/stderr information when using the
browser controller.
This CL will capture stdout/stderr of the browser and tries to align it with the
currently executed test (which might be slightly off due to the fact that
Process.{stderr,stdout} is reported asynchronously).
R=ricow@google.com
Review URL: https://codereview.chromium.org//28533003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@28858 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-18 15:28:08 +00:00
c9ba1fcbe4
test.py: Propagate the global --dartium option to the browser controller
...
R=kasperl@google.com
Review URL: https://codereview.chromium.org//25876002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@28222 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-03 14:55:09 +00:00
22d20313b4
Implement testing support for "--compiler=none --runtime=dartium"
...
This is implemented without the broken unittest-based wrapping of tests.
R=kasperl@google.com
Review URL: https://codereview.chromium.org//25514002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@28141 260f80e4-7a28-3924-810f-c04153c831b5
2013-10-02 07:37:50 +00:00
7ff576944f
Don't try to get chrome version on windows.
...
We still make sure that the binary is valid.
Enable the browser controller on windows again
R=kustermann@google.com
Review URL: https://codereview.chromium.org//23815007
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@27333 260f80e4-7a28-3924-810f-c04153c831b5
2013-09-10 12:29:43 +00:00
1e32f03118
Disable new browser controller on win-chrome
...
test.dart is not able to determine the version of chrome when running
'chrome.exe --version'. This causes an open browser window to hang and
makes the bot timeout.
TBR=ricow@google.com
Review URL: https://codereview.chromium.org//23561005
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@27245 260f80e4-7a28-3924-810f-c04153c831b5
2013-09-06 12:22:32 +00:00
whesse@google.com