Commit Graph

64 Commits

Author SHA1 Message Date
Siva Chandra 6ff0951bdf Remove the use of redundant flags --dfe and --kernel-binaries.
This change helps in confining flag manipulation to just the embedder.
When we are ready to turn all dart 2 flags to true by default, the
change can be confined to just the VM.

Change-Id: I4ac550c331b1f236a2bd9d0453b7c567eb0c05d3
Reviewed-on: https://dart-review.googlesource.com/38881
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-02-08 20:56:22 +00:00
Erik Corry 5ed2f25f40 There is no longer a safarimobilesim test runtime.
And so there is no need to maintain status files for this platform.

R=whesse@google.com

Change-Id: Ie70e4b7308cfd9b1b179b2d1eeeb8f36dd1a62dc
Reviewed-on: https://dart-review.googlesource.com/34824
Commit-Queue: Erik Corry <erikcorry@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2018-01-17 13:13:17 +00:00
Martin Kustermann 0b8d270434 Do not use pkg/vm wrapper script when running fully-linked kernel files
Issue https://github.com/dart-lang/sdk/issues/31585

Change-Id: I5211c7d8fb16ea60509916bf525cc36ac05dd3a9
Reviewed-on: https://dart-review.googlesource.com/29591
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2017-12-14 16:03:15 +00:00
Martin Kustermann a11d7a1643 Add support for running simdbc64 in batch mode
This makes isolate tests fail, since we no longer run from "source" (or
rather use the kernel-isolate to to "source -> dill" for us).

The special vm/cc suite will continue to be run via the kerne-isolate, so we
have the coverage for these (which probably include reload tests).

Issue https://github.com/dart-lang/sdk/issues/31585

Change-Id: I51bd2f9345d650b4ff2a98aa1c8365c765e0d013
Reviewed-on: https://dart-review.googlesource.com/28722
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2017-12-12 15:26:41 +00:00
Bob Nystrom ba2e4aff24 Some minor clean-up in test.dart.
- Fix fuzzy arrow errors.
- Default the Dart repo URI so it doesn't always need to be explicitly
  set.
- Run dartfmt on everything.
- Move some members out of the garbage bin "TestUtils" class:
  - Move the stuff around the Dart repo directory to a new Repository
    class.
  - Move absolute() into Path where it belongs.
  - Move workingDirectory in Path since it is a Path.
  - Delete the random number stuff since it was apparently unused.

Change-Id: I3dab3a4f1713b7a749e64b6776149d05a0ce1b69
Reviewed-on: https://dart-review.googlesource.com/14502
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2017-10-19 19:05:58 +00:00
Bob Nystrom 012774c78e Remove Dartium support from test.dart.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2981223002 .
2017-07-20 12:46:50 -07:00
Zachary Anderson ac16656161 Remove MIPS support
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2858623002 .
2017-06-22 08:49:22 -07:00
Bob Nystrom a6ca718e98 Simplify Command classes.
- Get rid of separate CommandBuilder class and singleton pattern. It
  was being passed around explicitly even though half of the places
  that received a CommandBuilder as a parameter still directly called
  CommandBuilder.instance instead of using it.
- Get rid of Command caching. As far as I can tell, it makes no
  measurable difference in runtime performance or memory usage. Even
  with a large invocation of a lot of configurations and tests, the
  Command classes don't seem to be a significant use of memory.
- Shorten the factory names. "get" adds no value, and we know it
  returns a "Command" since it's on Command.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2933973002 .
2017-06-15 14:19:03 -07:00
Bob Nystrom 19cb0cafb7 Simplify test_suite.dart.
- Get rid of baseCommand, which is only ever an empty list.
- Move some utility classes out of the giant test_suite.dart.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2915843003 .
2017-06-09 16:46:49 -07:00
Bob Nystrom 7e2d9ac85d Revert "Revert "Replace the configuration map with a typed object.""
This reverts commit 8bada4873a.

R=nweiz@google.com

Review-Url: https://codereview.chromium.org/2919573003 .
2017-05-31 15:39:23 -07:00
Bob Nystrom 8bada4873a Revert "Replace the configuration map with a typed object."
This reverts commit 06f75fe5cd.

Review-Url: https://codereview.chromium.org/2914893003 .
2017-05-31 15:08:21 -07:00
Bob Nystrom 06f75fe5cd Replace the configuration map with a typed object.
This is a pretty massive change but my hope is it will make
test.dart easier to maintain going forward. Instead of
passing around a stringly-typed Map<String, dynamic> all
throughout the program, we parse the options and then create
a Configuration object that has typed getters for all of the
various bits of configuration data.

This is a little tedious because it means declaring a new
option requires also declaring a corresponding field in the
Configuration class and passing it through the constructor.
I think it's worth it.

Also, enum-like configuration properties like architecture
and runtime now have their own classes as well. Moved a
bunch of stuff from TestUtil into those classes now that
there is an object to hang those methods off of.

In the process, I found a few typos in string literals where
the code wasn't correctly looking up a configuration
property.

Added more sanity checking and validation to status file
parsing. You will get an error at parse time if you try to
refer to a variable that isn't in the whitelist of known
variables. Also, you'll get an error if you try to compare
a variable to a value that it isn't expected to have.

Many other small-scale cleanups.

Aside from the status file validation, this should behave
mostly the same as current test.dart except that tests
may be enqueued in a slightly different order. The
rewritten code for expanding configurations iterates
through the architecture, runtime, etc. options in a
slightly different order.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2901923003 .
2017-05-31 14:47:31 -07:00
Bob Nystrom 7ce978bf91 Tighten types in test.dart even more.
This removes all implicit casts and many implicit uses of dynamic. It
adds a gratuitous number of explicit "as" casts and arguably makes the
code worse.

This is an interim step towards replacing the big configuration map
with an actual typed object. These "as" casts should help catch places
where the configuration object is being used and where the code will
need to be changed to use a new object.

R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2903703002 .
2017-05-30 14:02:24 -07:00
Bob Nystrom 92267afc66 Refactor and clean up the status file parsing code. - Make the parser less error tolerant. The expression parser used to ignore any unrecognized tokens, which means a status like "RuntimeError CompileError" (not the missing comma) was parsed as simply "RuntimeError", which seems bad. Now it reports an error. Fixed a couple of status files that thought they were setting statuses that they weren't (!).
- Separate out parsing a status file from applying the environment to
  determine which sections are active. This makes it possible to, for
  example, generate expectation sets for multiple environments without
  having to reparse each time.

- Simplify expression parsing. Remove set expressions since they weren't
  used for anything useful. A test's expectations are a simple
  comma-separated list and don't need anything beyond that. Merge
  Scanner and Tokenizer since the latter was a glorified function.

- Make more names private so that it's clearer what's used outside of
  various libraries.

- Generally modernize the style.

- Add *lots* of documentation.

Again, there should be no behavioral changes. I ran:

  ./tools/test.py -m release,debug -c none,dart2js,dart2analyzer -r none,vm,d8 corelib

Before and after the change and verified that the output was the same
(aside from timing).

R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2891753003 .
2017-05-18 12:42:52 -07:00
Bob Nystrom c269ef53a5 Revert "Re-apply status file parser changes from 0b7728da1bef08c1c1e092005d9fd8c8bff5fa6c."
This reverts commit a881aed354.

TBR.

R=paulberry@google.com

Review-Url: https://codereview.chromium.org/2888213002 .
2017-05-17 13:45:40 -07:00
Bob Nystrom a881aed354 Re-apply status file parser changes from 0b7728da1b.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2880203004 .
2017-05-17 13:04:35 -07:00
Bob Nystrom 01e5b1b40c Revert "Refactor and clean up the status file parsing code."
This reverts commit 0b7728da1b.

R=nbosch@google.com

Review-Url: https://codereview.chromium.org/2885623002 .
2017-05-15 15:58:05 -07:00
Bob Nystrom 0b7728da1b Refactor and clean up the status file parsing code.
- Make the parser less error tolerant. The expression parser used to
  ignore any unrecognized tokens, which means a status like
  "RuntimeError CompileError" (not the missing comma) was parsed as
  simply "RuntimeError", which seems bad. Now it reports an error.
  Fixed a couple of status files that thought they were setting statuses
  that they weren't (!).

- Separate out parsing a status file from applying the environment to
  determine which sections are active. This makes it possible to, for
  example, generate expectation sets for multiple environments without
  having to reparse each time.

- Simplify expression parsing. Remove set expressions since they weren't
  used for anything useful. A test's expectations are a simple
  comma-separated list and don't need anything beyond that. Merge
  Scanner and Tokenizer since the latter was a glorified function.

- Make more names private so that it's clearer what's used outside of
  various libraries.

- Generally modernize the style.

- Add *lots* of documentation.

Again, there should be no behavioral changes. I ran:

  ./tools/test.py -m release,debug -c none,dart2js,dart2analyzer -r none,vm,d8 corelib

Before and after the change and verified that the output was the same
(aside from timing).

BUG=
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2875203005 .
2017-05-15 15:42:15 -07:00
Erik Corry aa6353b6da Dart SDK Spelling b, c, and d.
R=kmillikin@google.com
BUG=

Review-Url: https://codereview.chromium.org/2850783002 .
2017-05-01 08:28:10 +02:00
Jacob Richman 119b2d58ce Format all files under tools and utils directory.
BUG=
R=terry@google.com

Review-Url: https://codereview.chromium.org/2827793002 .
2017-04-20 09:08:31 -07:00
Vyacheslav Egorov b026aed3cf Remove dartk and DFE workers support from testing scripts and kernel-service.
We only support running -c dark and -c dartkp configuration through Kernel isolate and fasta in non-batch mode.

R=kustermann@google.com
BUG=

Review-Url: https://codereview.chromium.org/2721543003 .
2017-02-27 13:31:03 +01:00
Vyacheslav Egorov de56824429 Switch -c dartk/dartkp configuration to use fasta.
R=kustermann@google.com
BUG=

Review-Url: https://codereview.chromium.org/2721443002 .
2017-02-26 20:39:02 +01:00
Vyacheslav Egorov d5a11f3d34 [Kernel] Enable Kernel Isolate to use Fasta instead of DartK.
We are not switching yet, but now everybody can try fasta by doing:

$ dart -DDFE_USE_FASTA=true --dfe=utils/kernel-service.dart hello.dart

Note that when using fasta we expect that patched_sdk contains platform.dill
Kernel binary containing compiled patched sdk. This file can be obtained by
doing:

$ export DART_AOT_SDK=<path-to-patched_sdk>
$ dart pkg/front_end/lib/src/fasta/bin/compile_platform.dart \
  ${DART_AOT_SDK}/platform.dill

We are also adding --use-fasta to testing script to allow end-to-end testing
of fasta (though platform.dill file needs to be generated manually prior to
running tests):

$ tools/test.py -c dartk -m release -a x64 --nobatch --use-fasta

Current test status:

$ tools/test.py -c dartk -m release -a x64 --nobatch --use-fasta
[27:34 | 100% | +12566 | - 1941]

BUG=
R=ahe@google.com, kustermann@google.com

Review-Url: https://codereview.chromium.org/2684943003 .
2017-02-08 16:48:35 +01:00
Ryan Macnak 6b1315a06f Revert "Delete app jit or aot snapshots as each test is completed to reduce the amount of disk space required to run the test suite."
This reverts commit 302e54a55d.

Review-Url: https://codereview.chromium.org/2671503002 .
2017-02-01 10:51:28 -08:00
Ryan Macnak 302e54a55d Delete app jit or aot snapshots as each test is completed to reduce the amount of disk space required to run the test suite.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2666833004 .
2017-02-01 09:20:48 -08:00
Vyacheslav Egorov a7ffe24177 VM: [Kernel] Switch to DFE for testing -c dartk configuration.
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2668503003 .
2017-01-31 18:25:02 +01:00
Martin Kustermann 5d7e89cb4e Fix android precompilation builders after testing script refactoring
Review-Url: https://codereview.chromium.org/2640063002 .
2017-01-18 19:07:22 +01:00
Martin Kustermann ef019665e7 Remove some hacks in the testing scripts & prepare for checked mode handling with kernel
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2638033002 .
2017-01-18 12:11:11 +01:00
Florian Schneider bef4967ea1 Remove dart_noopt and related parts from the VM.
dart_noopt is not needed anymore because we have testing
and builbot integration of the real precompilation pipeline in place now.

Fixes #24569
Fixes #25726
Fixes #25845

R=rmacnak@google.com

Review-Url: https://codereview.chromium.org/2624393002 .
2017-01-11 16:55:55 -08:00
Siva Annamalai b583d74608 Add flutter_engine runtime to the test script so we can run standard dart tests from the test suite against the flutter engine on linux/x64
The test suite can be invoked as :
tools/test.py -rflutter --packages=<path_to_packages_dir>/.packages --flutter=<path_to_flutter_engine_workspace>/sky-shell --exclude-suite=pkg

Currently the sky_shell executable does not yet support the proper return
codes:
Success - 0
API error - 253
Compilation error - 254
Unhandled error - 255
Once this is fixed we can start running the tests against our
dart/flutter engine bleeding edge build bot.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2602783002 .
2017-01-05 14:41:27 -08:00
Asger Feldthaus 2e718613aa Add 'self_check' runtime for running self-checking unit tests
Some of the kernel unit tests can now be run using:

  tools/test.py -cdartk -rself_check language co19

This will search the pkg/ folder for files matching *_self_check.dart
and run each program with the compiled output as argument. If there is
no compiler, the test case itself is given as argument. These testers
are always run in batch-mode.

This type of test has no expected output, but is intended to check
itself by testing that certain invariants are not violated while
processing the given data set.

The 'self_check' runtime is not specifically tied to kernel,
although only kernel is using it at the moment.

There is also a new option --skip-compilation which skips the
compiler step.  It doesn't interact nicely with the status files,
but can still be useful for a quick offline test.

Current limitations:
- All self-check tests are treated as the same test case. If one fails,
  the remaining self-check testers don't run for that input.
- There is no way to run a subset of the self-check tests, or filter
  them based on what compiler was used.
- Tests that are expected to fail in the compiler show up as
  errors when skipping compilation.

BUG=
R=kustermann@google.com, whesse@google.com

Review-Url: https://codereview.chromium.org/2549793002 .
2016-12-15 13:07:03 +01:00
William Hesse 191184ba3d Remove dead code with analyzer warning in test scripts
BUG=

Review URL: https://codereview.chromium.org/2548773002 .
2016-12-02 11:29:45 +01:00
Ryan Macnak 1ebeb79942 Remove runtime 'dart_app' as it became identical to 'vm'.
Merge compiler 'dart2app' and 'dart2appjit' and rename to 'app_jit' since they came to mean the same thing.

Update status files to reflect these changes.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2541533004 .
2016-12-01 16:47:33 -08:00
Ryan Macnak 30517b2d1e Also push process_test when testing on Android hardware.
R=fschneider@google.com

Review URL: https://codereview.chromium.org/2505483002 .
2016-11-15 09:26:36 -08:00
Ryan Macnak 3d00ecdc8f Reapply "Use a single file for app snapshots."
- Don't read script to check for an app snapshot if it is a pipe. Fixes loading source from a pipe.
 - Fix windows and fushsia builds.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2416973003 .
2016-10-14 09:57:54 -07:00
Ryan Macnak 6839a23240 Revert "Use a single file for app snapshots."
This reverts commit 4f8b16df3b.

Review URL: https://codereview.chromium.org/2410303008 .
2016-10-13 15:29:23 -07:00
Ryan Macnak 4f8b16df3b Use a single file for app snapshots.
For blobs, put all four pieces into a single file with a header describing their offsets.
For dynamic libraries, move the vm isolate and isolate pieces into the dynamic library as additional read-only sections.

Automatically detect if the script argument is an app snapshot and initialize the VM appropriately, similar to automatic detection of script snapshots.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2405393002 .
2016-10-13 14:33:56 -07:00
Siva Annamalai 9eb719b105 Fix build break.
Review URL: https://codereview.chromium.org/2194413002 .
2016-08-01 15:03:55 -07:00
Siva Annamalai a794b54273 Increase test runtime timeout configuration for hot_reload tests in debug mode as we are about to add additional verification which involves iterating over the heap.
R=rmacnak@google.com

Review URL: https://codereview.chromium.org/2197653005 .
2016-08-01 14:37:10 -07:00
Ryan Macnak ef8baa8760 Add test harness option for app snapshots with unoptimized code.
./tools/test.py -m all -c dart2appjit -r dart_app --use-blobs

Rename 'dart_product' runtime to 'dart_app'.

Rename Dart_PrecompiledJITSnapshotBlob to Dart_CreateAppJITSnapshot and add documentation.

Remove out-of-date scripts in tools/precompilation.

R=fschneider@google.com

Review URL: https://codereview.chromium.org/1992703005 .
2016-05-26 10:37:47 -07:00
Florian Schneider 6c778cab08 Pass command line options of tests to precompiled runtime correctly.
This fixes the precompilation tests on Android that have command line
options specified in their test files.

BUG=
R=kustermann@google.com

Review URL: https://codereview.chromium.org/1988133003 .
2016-05-18 21:00:40 +02:00
Ryan Macnak 7546ef1709 Simplify command line options for creating snapshots.
Also makes codeless app snapshots work in debug and release modes.

dart --snapshot=file.snap [--snapshot-kind=script] script.dart
dart file.snap

dart_bootstrap --snapshot=dir --snapshot-kind=app-after-run script.dart
dart --run-app-snapshot=dir script.dart

dart_bootstrap --snapshot=dir --snapshot-kind=app-jit-after-run script.dart
dart --run-app-snapshot=dir script.dart

dart_bootstrap --snapshot=dir --snapshot-kind=app-aot script.dart
dart_precompiled_runtime --run-app-snapshot=dir script.dart

R=asiva@google.com

Review URL: https://codereview.chromium.org/1947783003 .
2016-05-06 12:40:04 -07:00
Martin Kustermann 620b896e2f Initial support to test.dart for running precompiler tests on android devices
Steps to make this work:

a) Make sure you have an android sdk checkout:

  => See https://github.com/dart-lang/sdk/wiki/Building-Dart-SDK-for-Android

b) Building using android-toolchain crosscompiler:

  $ ./tools/build.py -mrelease -aarm --os=android dart_precompiled_runtime

  => Notice --os=android

c) Testing using attached android phones:

  $ export PATH=$PATH:$PWD/third_party/android_tools/sdk/platform-tools
  $ ./tools/test.py -mrelease -aarm --system=android -cprecompiler -rdart_precompiled --use-blobs

  => Notice --system=android

Failing tests on android can be marked in status files via:

[ $compiler == precompiler && $runtime == dart_precompiled && $system == android ]

R=whesse@google.com

Review URL: https://codereview.chromium.org/1922163002 .
2016-05-02 12:41:22 +02:00
Ryan Macnak f262fad639 Option to output precompiled instructions as a blob for use with mmap instead of assembly for use in a shared library.
BUG=http://dartbug.com/26199
R=fschneider@google.com

Review URL: https://codereview.chromium.org/1915853004 .
2016-04-25 16:19:34 -07:00
Zachary Anderson 3d7202bad3 DBC: Adds simdbc64 target, adds arm64 arithmetic overflow logic
There were differences between linux gcc, android gcc,
MacOS clang, and 32 vs. 64-bit w.r.t the __builtin
functions for detecting arithmetic overflow. I couldn't get
them all working at the same time. Instead, I removed them,
and changed to always use the inline assembly. This works
in all the configurations above.

This change also adds a simdbc64 target for building simdbc
for 64-bit, and sets up the android targets. simdbc targets
arm, and simdbc64 targets arm64. You can build them with:

$ ./tools/build.py -m release -a simdbc{64} --os=android runtime

R=iposva@google.com

Review URL: https://codereview.chromium.org/1904153003 .
2016-04-25 12:48:34 -07:00
Florian Schneider e905d64058 VM: Remove redundant build target dart_product.
It has been replaced by out/Product<arch>/dart

Building and testing product mode works via the -mproduct flag
with build.py/test.py.

BUG=
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/1902073003 .
2016-04-20 12:29:21 +02:00
Vyacheslav Egorov ee0f608ce4 Dart Byte Code interpreter.
This version is Clang/GCC only and does not support Windows because it uses computed goto's.

Only unoptimized mode is supported.

Architecture is described in constants_dbc.h and stack_frame_dbc.h.

R=fschneider@google.com, zra@google.com

Review URL: https://codereview.chromium.org/1858283002 .
2016-04-18 23:02:01 +02:00
William Hesse c63b10b611 Format the test scripts in /tools/testing/dart using dartfmt 1.16.0-dev.3.0.
R=ahe@google.com

Review URL: https://codereview.chromium.org/1859973002 .
2016-04-06 14:49:18 +02:00
Ryan Macnak a0e3bfdd68 Qualify the precompiled shared library name with the snapshot directory instead of using LD_LIBRARY_PATH.
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/1771423002 .
2016-03-08 12:01:53 -08:00
John McCutchan 7b7f2b05c6 Add dart_product binary
- Add dart_product binary that doesn't include the service isolate, Observatory, or the snapshot. It can only be used for running full snapshots.
- Use this binary when running tests.

R=asiva@google.com

Review URL: https://codereview.chromium.org/1690303002 .
2016-02-12 11:04:56 -08:00