Commit Graph

164 Commits

Author SHA1 Message Date
Aart Bik 19e844ed5b [dart/fuzzing] Use current directory as SDK top (by default)
Rationale:
DART_TOP can now be provided through switch, as
an environment variable or, by default, using the
current directory from which script is started.
This simplifies set up in cluster runs.
Change-Id: Icb4f720b7a7bb7b349ce158d39574c271132c224
Reviewed-on: https://dart-review.googlesource.com/76201
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-09-24 19:02:01 +00:00
Alexander Markov 19e2040f35 [vm] Cleanup the ability to specify entry points via JSON and text files
Entry points files (both JSON and text) are replaced with
@pragma("vm:entry-point") annotations.

This change removes obsolete:
* --embedder_entry_points_manifest option in gen_snapshot;
* --entry-points option in gen_kernel and frontend_server;
* --print_precompiler_entry_points VM option;
* 'entry_points' argument in Dart_Precompile() API.

Change-Id: I503f50f51df27e8e9635388c013058686b3b6ff1
Reviewed-on: https://dart-review.googlesource.com/75793
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-09-21 16:51:48 +00:00
Aart Bik cbf60679eb [vm/fuzzer] Migrated driver from Python to Dart.
Rationale:
Remove last part of Python in favor of Dart. Note that
this is still under heavy development, with many new
features planned. But it is a runnable system now which
we be deployed from the comfort of your own desktop or
on a cluster if needed.
Change-Id: I96fc8973bcc00c69849434087c57ff897d01b79a
Reviewed-on: https://dart-review.googlesource.com/75621
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-09-20 17:41:59 +00:00
Aart Bik dd2d4af442 [vm/fuzzer] Migrated from Python to Dart.
Rationale:
I suppose that "Drinking the Kool-Aid" requires me to migrate
the Dart fuzzer test tools from Python to Dart. This CL is a first
step, migrating dartfuzz.py to dartfuzz.dart. In addition, more
constructs were added to the generation, although it is still
far from finished of course!

Todo:
Migrate the driver (run_dartfuzz_test.py) to Dart as well.
Keep on adding more constructs to generate.
Change-Id: I9a8cb4fa9482d0d41c9af3e9bd3ac2e59286c949
Reviewed-on: https://dart-review.googlesource.com/72361
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-09-11 17:52:32 +00:00
Alexander Aprelev d00a5783c9 [vm] Fix bin_to_coff.py so it produces 8-byte wide kKernelServiceDillSize on 64-bit build.
Change-Id: Ic0d69fcccb42388129b39019d945fa9dfd2b1732
Reviewed-on: https://dart-review.googlesource.com/66522
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2018-07-25 04:32:43 +00:00
Zach Anderson 54c7754beb [build] Use the right section type in bin_to_coff.py
Change-Id: Ia4a761e757293aa117b909e0ee919031dce58bd9
Reviewed-on: https://dart-review.googlesource.com/64765
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-07-12 19:47:48 +00:00
Ben Konyi 370f2e8ff8 [ VM / Build ] Added bin_to_coff.py to generate object files from binaries. Speeds up Windows builds significantly since .dill files are no longer converted to assembly and run through asm.exe.
Change-Id: Iaaaf054c0202e8547227b2bfd19c4d9df1bcdf6b
Reviewed-on: https://dart-review.googlesource.com/63986
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2018-07-11 18:42:13 +00:00
Aart Bik 85350b0c53 [vm/fuzzer] Added configuration support in DartFuzz.
Rationale:
This change allows running Dart in various
different configurations (e.g. jit-ia32 and
aot-arm64).
Change-Id: I16700cd462873766f47a33bd866c7fa7cba603d1
Reviewed-on: https://dart-review.googlesource.com/63985
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-07-11 05:15:26 +00:00
Aart Bik bde2a8a981 [vm/fuzzer] Start of Dart fuzz tester.
Rationale:
Generation-based fuzz testing for Dart that
finds divergences for different execution paths
(e.g. AOT vs JIT) on randomly generated Dart programs.
This project is still under heavy development, this
first CL just introduces the concepts.

Note:
The JS mode may report quite a few divergences due to
dart2js - VM semantic differences on integer ops.

Change-Id: I2854341d9c2744556f19e44a825cddb7437405ee
Reviewed-on: https://dart-review.googlesource.com/63323
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-07-02 17:59:19 +00:00
asiva 6dfa10d33d Generate the kernel buffers using bin_to_assembly rule
instead of dill_to_cc.
This should fix the clang tool errors we are seeing.

Change-Id: I9a17a8028bc4f3e2352793ffe41a7444ae06956b
Reviewed-on: https://dart-review.googlesource.com/61107
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-06-20 02:31:12 +00:00
Zach Anderson 736ddd9b0e [Observatory] Refactor observatory build to prepare for Fuchsia prebuilt
This splits create_archive.py into two steps. One that creates the .tar
and one that creates the .cc for the .tar. This way on Fuchsia, the
second step will take the prebuilt instead of building it.

Change-Id: I7f407f9e70db3135884eeeb61aa5aec3091f1fcc
Reviewed-on: https://dart-review.googlesource.com/56291
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-05-29 16:35:34 +00:00
Peter von der Ahé 861550a28c Read patch files in strong mode
Closes https://github.com/dart-lang/sdk/issues/28822

Change-Id: I05e1d9885a39efca131d78d06cfd8a0e84276bb9
Reviewed-on: https://dart-review.googlesource.com/36641
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-02-05 14:35:36 +00:00
Siva Chandra 4b73d12082 Link kernel service dill with the command line dart executable.
This linked in kernel service dill file will be used to load the kernel
isolate if the attempt to lookup the kernel service snapshot fails. The
kernel service snapshot is looked up in the following order.

1. If the "--dfe" option is specified, the file specified is used.
2. If the kernel service snapshot is found next to the executable,
then it is used.
3. If the kernel service snapshot is found in the "snapshots" directory
next to the executable, then it is used.

Change-Id: I5a0e757eb27b26a274b22b4bc36350fee59a100f
Reviewed-on: https://dart-review.googlesource.com/32446
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Chandra <sivachandra@google.com>
2018-01-12 20:40:41 +00:00
Ryan Macnak d366f9619a [vm] Toward deterministic builds.
- Remove random build-id.
 - Replace build time in embedded version string with commit time.
 - Remove timestamps from Observatory tarball.
 - Zero-initialize skipped bytes in snapshot streams.
 - Fix uninitialized fields in PatchClass, Script and Library.
 - Disable (under flag) random identity hashes and concurrent GC.

Bug: https://github.com/dart-lang/sdk/issues/31427
Change-Id: I3e95de679c8372841cd27ca60df78d9b00ffbfe1
Reviewed-on: https://dart-review.googlesource.com/22901
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2017-11-23 00:07:56 +00:00
Alexander Aprelev d822bdb797 Prefix scripts from core builtin libraries so the URIs are unique.
After https://dart-review.googlesource.com/c/sdk/+/7588 we ended up with two crypto.dart files in the core prebuilt libraries(io and _http), which caused problem with coverage tool that relied on the fact that script uris can uniquely identify files. https://github.com/dart-lang/coverage/issues/194 was filed to handle non-unique uris.
This CL ensures that core builtin libraries URIs stays unique by prefixing them with 'dart:' library prefix. This makes core builtin libraries scripts have names similarly structured to script names for other core libraries.

Bug:
Change-Id: I79960f4f24e6e958836df866365355584c28df27
Reviewed-on: https://dart-review.googlesource.com/11140
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2017-10-05 15:47:43 +00:00
Zach Anderson 783f1e8c04 [infra] Remove dependency on gyp
R=rmacnak@google.com, whesse@google.com

Review-Url: https://codereview.chromium.org/3003433002 .
2017-08-18 08:51:40 -07:00
Zach Anderson 0394d3a0cf [infra] Begin removing gyp
This CL begins removing gyp from Dart's tree.

Removing gyp will go in three stages:
1. Remove unused .gyp and .gypi files.
2. Translate *_sources.gypi files to *_sources.gni files
3. Remove //third_party/gyp from DEPS

This CL is stage 1.

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2992593002 .
2017-08-04 22:19:35 -07:00
Zachary Anderson ac16656161 Remove MIPS support
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2858623002 .
2017-06-22 08:49:22 -07:00
Jonas Termansen 742822af73 Fix runtime/tools/bin_to_assembly.py not setting symbol type and size.
The absence of symbol sizes broke build comparison that looks for and
ignores snapshots. Add symbol types for good measure to match the compiler
behavior.

Consider unknown operating systems to use as(1)-style assembly, which is
generally true. If an unknown operating system uses another syntax, the
generated files will fail to assemble and the porter should soon find
this script and add support for their operating system.

R=whesse@google.com

Review-Url: https://codereview.chromium.org/2939013002 .
2017-06-15 13:40:46 +02:00
Ryan Macnak 812c3423bc core-jit:
Adjust compilation trace format to be plain text friendly.
  Avoid writing functions we can't lookup in the trace.
  Fix tear off fuzzy matching.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2924013004 .
2017-06-08 16:04:29 -07:00
Siva Annamalai f436b4986c Related to issue 27860
- Removed code which stored the source code paths for core and
  builtin libraries.

BUG=27860
R=rmacnak@google.com

Review-Url: https://codereview.chromium.org/2921513002 .
2017-06-01 11:11:09 -07:00
Ryan Macnak 76df24b076 CoreJIT snapshots without training.
Only include OSR and field guards in the features descriptor for JIT code to avoid gen_snapshot and dart having different default values.

Disabled since core snapshots with code break tests with non-default flags for type checks, assertions, strict errors, OSR, or field guards.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2902313004 .
2017-05-31 13:49:12 -07:00
Ryan Macnak 14ec8fa979 Use assembly instead of C array literals to link the core snapshot into the VM.
Allows for including instructions.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2901163002 .
2017-05-24 14:38:15 -07:00
Erik Corry fc2fcf9bc8 Add more safe points in compiler.
The Scavenge (young-gen) GCs on the main thread have to wait for other
threads to check in at a safe point. We were seeing big waits here, often
20ms, occasionally up to 180ms where the main thread is idling, waiting
for the optimizing compiler.  By adding more safe points the wait is
reduced and is now rarely over 10ms, often under 1ms.

This also changes the --verbose-gc output to be better aligned with the
column headings, and to add the time needed to get to
the safe point to the output, eg:

[ GC(784211551): Scavenge(new space), 18, 2.209, 76.009, 32768, 0, 32768, 32768, 0, 0, 144912, 154425, 152064, 154880, 0, 0, 46.984, 2.752, 7.407, 18.657, 0.033, 5421, 0, 0, 0, ]
                                                 ^^^^^^ Scavenge time                                                        ^^^^^^  safe point time.

R=vegorov@google.com
BUG=

Review-Url: https://codereview.chromium.org/2771013002 .
2017-03-31 11:19:46 +02:00
Jacob Richman 4360e99d18 Revert "Fix observatory tests broken by running dartfmt."
This reverts commit b1f7e6d94673b098c93c187fe3efc45c15f27edc.

BUG=

Committed: https://github.com/dart-lang/sdk/commit/62be0eacfbb36bdc92a05c7c35bb4506621b6a3a
Review-Url: https://codereview.chromium.org/2767533002 .
2017-03-21 10:07:26 -07:00
Jacob Richman 62be0eacfb Revert "Fix observatory tests broken by running dartfmt."
This reverts commit b1f7e6d94673b098c93c187fe3efc45c15f27edc.

BUG=

Review-Url: https://codereview.chromium.org/2767533002 .
2017-03-20 22:19:22 -07:00
Jacob Richman 88f8aacd5f Re-apply rolled back CL to apply dartfmt to runtime.
Fix observatory tests broken by running dartfmt due to line and column changes.
Temporarily reverted formatting for evaluate_activation_test.dart as dartfmt doesn't yet handle multitests.

BUG=
R=johnmccutchan@google.com

Review-Url: https://codereview.chromium.org/2759973004 .
2017-03-20 21:44:59 -07:00
Ryan Macnak ea4744a06f Revert "Run dartfmt on all files under runtime."
This reverts commit 15f393f189.

The change caused many service test failures due to changed line numbers.

TBR=jacobr@google.com

Review-Url: https://codereview.chromium.org/2762583003 .
2017-03-20 13:42:55 -07:00
Jacob Richman 15f393f189 Run dartfmt on all files under runtime.
I've omitted files where the formatter output is significantly uglier
than the original code and I'll send those files in a separate CL
with options for how to make the code look reasonable while still
taking advantage of the formatter.

BUG=
R=johnmccutchan@google.com

Review-Url: https://codereview.chromium.org/2751423005 .
2017-03-20 13:13:50 -07:00
Chris Bracken c9442bb8a2 Unset GREP_OPTIONS when extracting processor count
If GREP_OPTIONS includes -n or --line-number, matches will be emitted
with a line number following the filename. Unset GREP_OPTIONS to ensure
we're getting expected output.

BUG=
R=johnmccutchan@google.com

Review-Url: https://codereview.chromium.org/2721953004 .
2017-02-28 16:00:32 -08:00
Peter von der Ahé 00bed94e0c Fix various nits in VM patch files.
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2705593002 .
2017-02-17 16:28:57 +01:00
Ryan Macnak b6a9be6e39 Teach gen_snapshot how to create script snapshots if given a core snapshot to create them against.
Require an explicit --snapshot_kind option instead of inferring the desired snapshot kind from the combination of output file options.

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2692223002 .
2017-02-15 14:28:35 -08:00
Florian Schneider 15a4d5d6d1 Windows: Link library sources into gen_snapshot/dart_bootstrap
This allows gen_snapshot and dart_bootstrap to run without having
a full SDK snapshot around.

The generated cc file looks like:

static const char source_array_1[] = {
 '\x2f', '\x2f', '\x20', '\x43', '\x6f', ...
};
static const char source_array_2[] = {
 '\x2f', '\x2f', '\x20', '\x43', '\x6f', ...
};
...

const char* dart::Bootstrap::core_source_paths_[] = {
 "dart:core",
 "/path/to/sdk/lib/core/core.dart",
 source_array_1,

 "annotations.dart",
 "/path/to/sdk/lib/core/annotations.dart",
 source_array_2,
...

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2668353004 .
2017-02-01 15:55:59 -08:00
Ryan Macnak 9eb216a495 Reapply "Create an app snapshot of the Dart front end."
Skip training on Windows, issue #28532.

R=hausner@google.com

Review-Url: https://codereview.chromium.org/2665753002 .
2017-01-31 09:47:51 -08:00
Vyacheslav Egorov a7ffe24177 VM: [Kernel] Switch to DFE for testing -c dartk configuration.
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2668503003 .
2017-01-31 18:25:02 +01:00
Vyacheslav Egorov 43a0500e37 VM: [Kernel] Fix bootstraping when Kernel isolate is used.
We must bootstrap from Kernel instead of Source when running with --dfe

BUG=
R=asiva@google.com, kustermann@google.com

Review-Url: https://codereview.chromium.org/2651633002 .
2017-01-30 20:52:59 +01:00
Ryan Macnak 728f460798 Revert "Create an app snapshot of the Dart front end."
This reverts commit 8c7111c967.
This reverts commit 27c8e59d2e.

Review-Url: https://codereview.chromium.org/2657123002 .
2017-01-26 17:42:25 -08:00
Ryan Macnak 8c7111c967 Create an app snapshot of the Dart front end.
R=asiva@google.com, hausner@google.com, zra@google.com

Review-Url: https://codereview.chromium.org/2655173002 .
2017-01-26 16:28:59 -08:00
Ryan Macnak b46af1e75f Refactor snapshots pieces to include a section for loading instructions into the heap of a regular isolate.
Progress toward allowing each isolate to load a different snapshot.

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2622053002 .
2017-01-23 10:25:02 -08:00
Zachary Anderson 375322d3ab GN: Add option to build with TSAN
fixes #28216

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2614493002 .
2017-01-03 14:37:27 -08:00
Kasper Lund 53dd819a4c Cleanup the kernel service implementation and avoid printing on the console when no tracing flags are provided.
R=kmillikin@google.com, hausner@google.com
BUG=

Review-Url: https://codereview.chromium.org/2570483003 .
2016-12-13 09:08:10 +01:00
Matthias Hausner f5ab173c77 Add Kernel Isolate, second attempt
To run a script with the Dart->Kernel parser, use the -dfe option:

dart --dfe=runtime/tools/kernel-service.dart  --packages=/src/d2/sdk/.packages  foo.dart

The front end expects a directory named patched_sdk in the build directory (patched_sdk should be a sibling of the directory that contains the dart executable.)

Warning: using a debug build dart executable is very slow. It takes 2 minutes to run Hello World in a debug build, and about 7 seconds in a release build.

BUG=
R=asiva@google.com

Review URL: https://codereview.chromium.org/2558673002 .
2016-12-07 09:11:16 -08:00
Matthias Hausner 2ea399285a Revert "Add Kernel Isolate"
This reverts commit c9399fc557.

The change broke Dartium and Fuchsia.

BUG=

Review URL: https://codereview.chromium.org/2554953004 .
2016-12-06 14:17:09 -08:00
Matthias Hausner c9399fc557 Add Kernel Isolate
To run a script with the Dart->Kernel parser, use the -dfe option:

dart --dfe=runtime/tools/kernel-service.dart  --packages=/src/d2/sdk/.packages  foo.dart

The front end expects a directory named patched_sdk in the build directory (patched_sdk should be a sibling of the directory that contains the dart executable.)

Warning: using a debug build dart executable is very slow. It takes 2 minutes to run Hello World in a debug build, and about 7 seconds in a release build.

BUG=
R=asiva@google.com

Review URL: https://codereview.chromium.org/2483373002 .
2016-12-06 12:23:09 -08:00
Zachary Anderson 63cb4565d7 Fuchsia: Adds a script to download the prebuild SDK
The Fuchsia world doesn't run gclient runhooks or use depot_tools so
the advice is to check in a custom script to pull the prebuilt SDK
from google storage.

R=asiva@google.com, phosek@chromium.org, rmacnak@google.com

Review URL: https://codereview.chromium.org/2541123006 .
2016-12-02 13:18:54 -08:00
Jonas Termansen 3676af3168 Ensure consistent file ordering in observatory tarball
R=whesse@google.com
BUG=

Review URL: https://codereview.chromium.org/2545783002 .
2016-12-01 16:01:09 +01:00
Adam Barth 0ea1c155ae Fix the Fuchsia build
Python scripts need to be marked executable to run in the Fuchsia build.

TBR=zra@google.com
2016-10-27 09:59:58 -07:00
Vyacheslav Egorov eb8c24e133 Reland "Merge more Kernel infrastructure from kernel_sdk SDK fork."
R=zra@google.com
BUG=

Review URL: https://codereview.chromium.org/2451623006 .
2016-10-27 10:09:22 +02:00
Vyacheslav Egorov 12968edb9c Revert "Reland "Merge more Kernel infrastructure from kernel_sdk SDK fork.""
This reverts commit 4e7a31262e.

Windows bots are broken now.

R=zra@google.com
BUG=

Review URL: https://codereview.chromium.org/2451893004 .
2016-10-26 17:07:18 +02:00
Vyacheslav Egorov 4e7a31262e Reland "Merge more Kernel infrastructure from kernel_sdk SDK fork."
This relands commit 2fed1c3905 with fixes to ensure that analyzer summaries are correctly created and XARM build does not try to use target binaries on the host.

R=zra@google.com
BUG=

Review URL: https://codereview.chromium.org/2453773002 .
2016-10-26 16:48:28 +02:00