Commit Graph

126 Commits

Author SHA1 Message Date
Alexander Thomas 54a3e11d5b [infra] Upload core dumps if BUILDBOT_BUILDERNAME is set
Recipes pass output_dir to local shards as well but it points to a temp
directory that isn't accessible to users.

Change-Id: I0618abf78cf03957dd5cb8c9a10150bc4964684e
Reviewed-on: https://dart-review.googlesource.com/c/82985
Commit-Queue: Alexander Thomas <athom@google.com>
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-11-06 11:56:11 +00:00
Vyacheslav Egorov c413b61969 [testing] Improvements to crash dump archiving
* On Windows dump stacks from all collected dumps into the output so
that one would not need to download the dump to analyse it.
* Document crash dump archiving so that engineers know where to look
for dumps and how look at them.

Change-Id: Ibf53bb597c07702b3e051cf24bea8f6bfb1d5e8f
Reviewed-on: https://dart-review.googlesource.com/c/82066
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
2018-10-30 20:41:03 +00:00
Vyacheslav Egorov 64635263ff [testing] Fix code in utils.py to use Python not Dart
In Dart we do `x.containsKey(k)` in Python we do `k in x`.

Change-Id: I60d0e95d0c4402c42a914270dbb1828f714f9d08
Reviewed-on: https://dart-review.googlesource.com/c/81760
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2018-10-29 10:34:24 +00:00
Vyacheslav Egorov 040dd40ef3 [testing] Making core dump archiving more robust.
* Catch and suppress errors so that core dump archiving can succeed even
if it can't delete some files.
* On Windows also archive PDB files that contains debugging information.

Change-Id: If1d7caa7c793a71bf8c0852c38098933e88fadba
Reviewed-on: https://dart-review.googlesource.com/c/81680
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-10-26 18:17:18 +00:00
Vyacheslav Egorov 245ec25943 [testing] Make core dump archiving machinery more verbose
Change-Id: Ifadbc3e5d0740697aaea7336a738daa20223ad5e
Reviewed-on: https://dart-review.googlesource.com/c/81412
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-10-25 15:31:27 +00:00
Alexander Thomas bf26f760b1 [infra] Ensure that core dump upload triggers on main builders
https://crrev.com/c/1296514 will add --output_directory to non-shards as
well. BUILDBOT_BUILDERNAME is not set on shards and a more reliable
indication that this is a main bot that should gsutil upload core dumps.

Change-Id: Ic6cddc4937b8e83d2e15da4acd90c193e412773b
Reviewed-on: https://dart-review.googlesource.com/c/81362
Reviewed-by: Jonas Termansen <sortie@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-10-25 07:36:06 +00:00
Vyacheslav Egorov 2c83eb550e [vm] Integrate dart binary with Crashpad on Windows.
dart binary would instantiate CrashpadClient if DART_CRASHPAD_HANDLER
and DART_CRASHPAD_CRASHES_DIR environment variables are set.

- DART_CRASHPAD_HANDLER should contain the path to the crashpad_handler
binary that would handle the crash and write minidump;
- DART_CRASHPAD_CRASHES_DIR should contain the path to the crashpad database
which would be used to store minidumps.

Rewrite --copy-crash-dumps support on windows to use Crashpad integration
instead of editing Windows registry.

Embedding crashpad required to roll a new zlib version because Crashpad
depends on the zlib. This version of zlib is buildable with its own
BUILD.gn so our custom BUILD.gn is removed.

Change-Id: I048aad16b234e1d750f0a24782b04e3b6e19703d
Reviewed-on: https://dart-review.googlesource.com/c/81007
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-10-24 13:30:16 +00:00
Martin Kustermann ed73cc379d Increase file descriptor limit when running test.py on MacOS
This is an attempt to solve io test flakiness on MacOS. The symptoms
include "connection refused", "broken pipe" error messages.

Change-Id: If6759f4ef9cd2c1b9bd083db9a469db43f12c4e0
Reviewed-on: https://dart-review.googlesource.com/c/80120
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-10-16 09:25:10 +00:00
Alexander Thomas 5ce132a912 [infra] Copy core dumps to ISOLATED_OUTDIR on shards (#34746)
* Upload to Cloud Storage requires credentials not available to shards.
* Remove "SiteConfigBotoFile" buildbot hack.
* Remove obsolete Java utility method.

Change-Id: If03412a09644845d0f9467fa5fffc1cbec1cc216
Reviewed-on: https://dart-review.googlesource.com/c/79900
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-10-16 09:11:45 +00:00
Vyacheslav Egorov 051108c004 [tools] Re-enable core-dump archiving on Linux.
Make core_pattern checks non-fatal for now because underlying issues with
bots are not yet fixed.

We need core-dumps from debug kernel bots to look into some pthread flakes.

Bug: https://github.com/dart-lang/sdk/issues/31979

Change-Id: I38c79013cb2ed5a82ce10c909ff729f73cc34536
Reviewed-on: https://dart-review.googlesource.com/c/37320
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-10-10 13:47:17 +00:00
Zach Anderson b4004f6476 [vm] Reland: Move the interpreter under a flag.
After this CL, the interpreter is included by default in the
JIT VM under the flag --enable-interpreter.

Reland with fix to NativeArgument setup in simulator_arm.cc

Change-Id: Ib9b4df6eb4d997dfbe361188b8a127828c1d9c6f
Reviewed-on: https://dart-review.googlesource.com/74003
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-09-07 19:25:31 +00:00
Zach Anderson 21406f576a Revert "[vm] Move the interpreter under a flag."
This reverts commit 715ab7b30b.

Reason for revert: simarm and kernel precomp builds failing

Original change's description:
> [vm] Move the interpreter under a flag.
> 
> After this CL, the interpreter is included by default in the
> JIT VM under the flag --enable-interpreter.
> 
> Change-Id: I06458f20288d67981822788ee6e867c2127ebdd3
> Reviewed-on: https://dart-review.googlesource.com/71800
> Commit-Queue: Zach Anderson <zra@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com,zra@google.com,asiva@google.com,regis@google.com

Change-Id: I3622f4dbe53288d07fc21b34ee5ff68f2682b36d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/74001
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-09-07 17:14:21 +00:00
Zach Anderson 715ab7b30b [vm] Move the interpreter under a flag.
After this CL, the interpreter is included by default in the
JIT VM under the flag --enable-interpreter.

Change-Id: I06458f20288d67981822788ee6e867c2127ebdd3
Reviewed-on: https://dart-review.googlesource.com/71800
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-09-07 16:46:07 +00:00
Zach Anderson d0510501d2 [build] Add -a kbc for interpreter
This CL adds support to build.py and test.py for building/testing
a VM with the interpreter on x64, e.g.:

$ ./tools/gn.py -m release -a x64 --bytecode
$ ./tools/build.py -m release -a x64 --bytecode runtime
$ ./tools/test.py -m release -a x64 -r vm -c dartkb language_2

Change-Id: I956d23790636609d4a2e71129481fcbd7afef9a0
Reviewed-on: https://dart-review.googlesource.com/65206
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2018-07-17 21:51:41 +00:00
Alexander Thomas 72b77ac6ab [infra] Remove last remnants of old checked in SDK
* Update README.
* Create 32-bit Windows CIPD package in update.sh.
* Remove architecture specific executable names.

Change-Id: I553b282188bc76e82df0a6e40200e3311fdc6f1d
Reviewed-on: https://dart-review.googlesource.com/62842
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-06-28 18:15:57 +00:00
Alexander Thomas 66c9109caf [infra] Download checked in SDKs from CIPD
#33659

Change-Id: I94fccf4bbce2f0140bf0d6efa953d44b290df856
Reviewed-on: https://dart-review.googlesource.com/62840
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-06-28 16:08:14 +00:00
Zach Anderson 0b4f01f759 [infra] Fixes to custom version string
related https://github.com/flutter/flutter/issues/14751

Change-Id: Id11048777e9e27fc63b94bc49c45e575b4333d24
Reviewed-on: https://dart-review.googlesource.com/42460
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-02-20 17:53:06 +00:00
Zach Anderson 0d5cf900b0 [infra,flutter] Adds semantically meaningful custom version strings
See https://github.com/flutter/flutter/issues/14751

Change-Id: I3b35c2a32c0dd1301e92abe23b1c6fe2c2e04a6c
Reviewed-on: https://dart-review.googlesource.com/42026
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-02-16 20:44:48 +00:00
Ryan Macnak d366f9619a [vm] Toward deterministic builds.
- Remove random build-id.
 - Replace build time in embedded version string with commit time.
 - Remove timestamps from Observatory tarball.
 - Zero-initialize skipped bytes in snapshot streams.
 - Fix uninitialized fields in PatchClass, Script and Library.
 - Disable (under flag) random identity hashes and concurrent GC.

Bug: https://github.com/dart-lang/sdk/issues/31427
Change-Id: I3e95de679c8372841cd27ca60df78d9b00ffbfe1
Reviewed-on: https://dart-review.googlesource.com/22901
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2017-11-23 00:07:56 +00:00
Zachary Anderson 336478d7d9 Really disable core dump archiving on Linux
Trying again.

Review-Url: https://codereview.chromium.org/2984653002 .
2017-07-19 11:49:21 -07:00
Zachary Anderson 3e354bcc58 Disable core dump archive check on Linux
related #30205

Review-Url: https://codereview.chromium.org/2983093002 .
2017-07-19 11:34:15 -07:00
Zachary Anderson ac16656161 Remove MIPS support
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2858623002 .
2017-06-22 08:49:22 -07:00
Terry Lucas f27144d7b5 Roll 50: Updated for push to origin/master.
TBR=jacobr@google.com

Review-Url: https://codereview.chromium.org/2875773003 .
2017-05-10 18:16:51 -07:00
Zachary Anderson 713b89fbe7 [infra] Stop console spam from utils.py on a cross-build
R=rmacnak@google.com

Review-Url: https://codereview.chromium.org/2873613004 .
2017-05-10 09:41:45 -07:00
Martin Kustermann a7e6da3a93 Add support for core dump archiving on macos
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2815553003 .
2017-04-19 10:36:23 +02:00
Martin Kustermann b40a9e8608 Add link to cit bug in the exception for coredumps
Review-Url: https://codereview.chromium.org/2769683002 .
2017-03-22 14:36:20 +01:00
Martin Kustermann eb903b955e Add more debugging support to coredump archiving code
It seems sometimes we cannot find the coredumps but we are unable to
reproduce it (even on the bots). So let's add some more debugging
information.

R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2713193002 .
2017-02-24 17:06:28 +01:00
Martin Kustermann cc45d27027 Disable usage of site_config boto file
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2710333004 .
2017-02-24 17:04:40 +01:00
Martin Kustermann f3bef9983b Convert arguments to ''.join() to strings in tools/utils.py
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2710333003 .
2017-02-24 12:23:26 +01:00
Martin Kustermann affb786582 Enable support for coredump archiving on windows
BUG=https://github.com/dart-lang/sdk/issues/28648
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2692883002 .
2017-02-14 16:34:31 +01:00
Vyacheslav Egorov e8f456edd5 Buildbot: Only archive core dumps from unexpected crashes.
Change UnexpectedCrashDumpArchiver to generate coredumps file in the current
directory. Each line of this file specifies a core and a binary to be
uploaded in the following format:

              test-name,core-file,binary-file

CoreDumpArchiver reads this file and uploads the first 10 crash dumps.

R=kustermann@google.com
BUG=

Review-Url: https://codereview.chromium.org/2645963004 .
2017-01-20 17:44:51 +01:00
Vyacheslav Egorov 6c9ca0cd2b Buildbot: Change --copy-crashdumps to do the actual archiving to CloudStorage.
This flag now both enables core dump collection by setting appropriate rlimits
and takes care of archiving cores and crashed binaries.

R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2640093005 .
2017-01-19 17:25:39 +01:00
Vyacheslav Egorov e9483132ed Revert "Buildbot: Change --copy-crashdumps to do the actual archiving to CloudStorage."
This reverts commit dec352c1ac.

Bots are completely broken

BUG=
TBR=kustermann@google.com

Review-Url: https://codereview.chromium.org/2640823005 .
2017-01-19 16:17:23 +01:00
Vyacheslav Egorov dec352c1ac Buildbot: Change --copy-crashdumps to do the actual archiving to CloudStorage.
This flag now both enables core dump collection by setting appropriate rlimits
and takes care of archiving cores and crashed binaries.

BUG=
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2644593002 .
2017-01-19 16:13:28 +01:00
Zachary Anderson 566ed7ff55 GN: Fix armsimdbc target. Add armsimdbc64
R=johnmccutchan@google.com

Review-Url: https://codereview.chromium.org/2596333002 .
2016-12-22 10:24:24 -08:00
Jonas Termansen c0172aba48 Don't store git commit in observatory version string
The observatory assets tarball would be different every time due to the
version string, which makes it compress to a different size. The size
changes would then move the program data around, making it much more
difficult to determine if dart builds of different revisions are
actually identical.

Google Analytics doesn't need to know the exact version numbers, it will
be fine to say they are bleeding edge builds of a particular version.

BUG=
R=whesse@google.com

Review URL: https://codereview.chromium.org/2540383003 .
2016-12-01 14:16:23 +01:00
William Hesse e82a20b180 Change default Windows compiler to MSVS 2015 (VS version 14.0)
BUG=
R=zra@google.com

Review URL: https://codereview.chromium.org/2217393002 .
2016-08-06 00:12:56 +02:00
Zachary Anderson a274332ed7 Remove Fuchsia build from the gyp build
This isn't used anymore. The build happens in the Fuchsia tree using GN.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2194223002 .
2016-08-01 12:38:11 -07:00
Zachary Anderson a503570e3a Fuchsia: Initial check-in.
Instructions to build and run are in README.fuchsia.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2117593002 .
2016-07-01 12:42:05 -07:00
Zach Anderson 44796ed02d Move Windows build output from //build to //out
This is needed in order to begin migration to GN because
third-party dependencies and GN boilerplate expect GN
boilerplate to be in //build.

R=asiva@google.com, whesse@google.com

Review URL: https://codereview.chromium.org/2027063003 .
2016-06-03 08:28:48 -07:00
Vyacheslav Egorov 43dc5c223d Introduce a new ARMSIMDBC configuration.
This allows us to build dart binary with SIMDBC for non-Android ARM devices.

We are going to use it to measure performance.

BUG=http://dartbug.com/26421
R=zra@google.com

Review URL: https://codereview.chromium.org/1967613002 .
2016-05-10 20:06:54 +02:00
Zachary Anderson 3d7202bad3 DBC: Adds simdbc64 target, adds arm64 arithmetic overflow logic
There were differences between linux gcc, android gcc,
MacOS clang, and 32 vs. 64-bit w.r.t the __builtin
functions for detecting arithmetic overflow. I couldn't get
them all working at the same time. Instead, I removed them,
and changed to always use the inline assembly. This works
in all the configurations above.

This change also adds a simdbc64 target for building simdbc
for 64-bit, and sets up the android targets. simdbc targets
arm, and simdbc64 targets arm64. You can build them with:

$ ./tools/build.py -m release -a simdbc{64} --os=android runtime

R=iposva@google.com

Review URL: https://codereview.chromium.org/1904153003 .
2016-04-25 12:48:34 -07:00
Vyacheslav Egorov ee0f608ce4 Dart Byte Code interpreter.
This version is Clang/GCC only and does not support Windows because it uses computed goto's.

Only unoptimized mode is supported.

Architecture is described in constants_dbc.h and stack_frame_dbc.h.

R=fschneider@google.com, zra@google.com

Review URL: https://codereview.chromium.org/1858283002 .
2016-04-18 23:02:01 +02:00
Rico Wind 2b8c5d2450 Print git number when running tools/utils.py
R=whesse@google.com
BUG=

Review URL: https://codereview.chromium.org/1708993002 .
2016-02-18 11:15:18 +01:00
William Hesse 2573093c50 Switch to downloaded SDK executable, drop executables in testing/bin.
BUG=https://github.com/dart-lang/sdk/issues/24311
R=ricow@google.com

Review URL: https://codereview.chromium.org/1697203002 .
2016-02-15 17:10:59 +01:00
William Hesse 9ed0e9c97f Update downloaded SDKs to version 1.14.2. Add arm64 dart binary.
BUG=
R=ricow@google.com

Review URL: https://codereview.chromium.org/1700823002 .
2016-02-15 16:36:01 +01:00
Ivan Posva 9c0c6cb5c2 Add product mode:
- Add PRODUCT define and build mode to gyp configurations.
- Add product mode to test harness.
- Start to unify list of flags.
- Allow flags to be constant for particular build configurations.

R=fschneider@google.com

Review URL: https://codereview.chromium.org/1663863002 .
2016-02-03 21:10:30 -08:00
Zachary Anderson 80f358dede Adds targets for simarmv6 and armv6
related #25562

R=rmacnak@google.com

Review URL: https://codereview.chromium.org/1624593002 .
2016-01-22 13:07:48 -08:00
Ryan Macnak 642f754219 Note that R19 is a C preserved register, fixing crash returning from the first Dart entry on Linux ARM64.
Bug was introduced by 91bc8005fe.

 - Move the Dart stack pointer from R19 (C preserved) to R15 (C volatile) so it isn't among the registers saved by the InvokeDartCode stub.
 - Set a default cross compiler for ARM64.
 - Use ARM32 instead of IA32 binary to build Observatory on ARM64 hosts.

R=regis@google.com

Review URL: https://codereview.chromium.org/1559223003 .
2016-01-07 09:30:17 -08:00
Ivan Posva 6d0453594a - Prevent running of the dart_bootstrap script by default.
- Fix argument handling in Python: argparse does not properly handle booleans.

BUG=
R=rmacnak@google.com

Review URL: https://codereview.chromium.org/1519063004 .
2015-12-11 13:23:12 -08:00