Commit Graph

136 Commits

Author SHA1 Message Date
Keerti Parthasarathy d7f4763e79 Add optional version_file param
Change-Id: I7ff1d6dc7a492bfce3e1458b51b92dd47f6de674
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101681
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Keerti Parthasarathy <keertip@google.com>
2019-05-21 16:24:41 +00:00
Vyacheslav Egorov 125c5cbf4a [vm/compiler] Instroduce SIMARM_X64 build architecture.
This is variant of SIMARM build (ARM target and non-ARM host) but with a
word size mismatch (32-bit target and 64-bit host). We only expect to
build gen_snapshot binary in this mode.

This is the first step towards enabling AOT compilation targeting ARM
using 64-bit gen_snapshot binary.

This change also introduces --gen-snapshot flag for test.py which allows
to specify which gen_snapshot binary should be used for running tests.

Expected workflow with SIMARM_X64 build:

$ tools/build.py -a simarm_x64 -m release gen_snapshot
$ tools/build.py -a simarm -m release dart_precompiled_runtime
vm_platform
$ tools/test.py -a simarm -m release -c dartkp --gen-snapshot
out/ReleaseSIMARM_X64/gen_snapshot

Note that our ARM simulator can't be built as a 64-binary so we are
going to be using SIMARM runtime to test AOT compiled code produced by
SIMARM_X64 binary.

Issue https://github.com/dart-lang/sdk/issues/36839
Issue https://github.com/flutter/flutter/issues/22598

Change-Id: Id003900e8b46fc9d57975ab82b0c21852a176079
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100968
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-05-06 10:48:08 +00:00
Keerti Parthasarathy b378eef54a Add a option to pass in version file to tools
Change-Id: I3b7224b108cdb8693d17da90d499c0648f140bc3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100721
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Keerti Parthasarathy <keertip@google.com>
2019-04-29 16:40:50 +00:00
Joseph Richey 86fe7ca75c Fix build when python=python3
Right now most of the dart SDK's python is compatible with python2
or python3. This change fixes a few of the build scripts to make
that completely true (at least when building the standard build on
Linux). There are only four types of changes:
  - Bare `print` statements now use the `print ()` function
  - `commands.getoutput` becomes `subprocess.check_output` with `shell=True`
  - `xrange` becomes `range`
  - `print >> sys.stderr` becomes `sys.stderr.write`

Starts work on addressing (but does not completely fix):
https://github.com/dart-lang/sdk/issues/28793

See related issue:
https://fuchsia-review.googlesource.com/c/fuchsia/+/272925

This change applys to both the `dev` and `master` branches.

Change-Id: Ibd3eb9b1f57520d2d745f05c2ac430b1d20943da

Closes #36662
https://github.com/dart-lang/sdk/pull/36662

GitOrigin-RevId: beab165294982a7e369daf6d61aea63efcab1b9b
Change-Id: I6d240749a9ba0889b5a45a08f3c4c2c20291f484
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99707
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2019-04-23 07:48:15 +00:00
Jason Simmons bc5cc91fef Fix the command line for getting the Git head short hash
Fixes https://github.com/flutter/flutter/issues/30433

Change-Id: I77d85963412117f6592326d9ca45625a9f58731e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99266
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-04-12 17:52:58 +00:00
Martin Kustermann a0e82ba23e [gardening] Disable exception caused by missing core dumps on windows
Issue https://github.com/dart-lang/sdk/issues/36469

Change-Id: I57d53d1ef02841e1f5c9f9993f896a8d20ee6f18
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98670
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-04-04 18:07:33 +00:00
Alexander Thomas b89d08355b [infra] Only move coredump files on shards
On shards, the files don't need to be tar.gz'ed because the isolate
server will zip them and there aren't many files so tar'ing them doesn't
add much.

Change-Id: I696fe4c64fcee879bb9240e9d1990d2053aa8346
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96655
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2019-03-12 17:43:16 +00:00
Alexander Thomas 567d552de8 [infra] Add versionchecker and debianpackage to test matrix
* Run Debian package creation in a Docker container.
* Upload archives from Docker host (the container lacks credentials).
* Remove BUILDBOT_BUILDERNAME environment variables from test matrix:
  They are overwritten in the recipe.

Change-Id: I90d7747e64231bdb745c85030a9cecf256adbaf0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/94542
Reviewed-by: William Hesse <whesse@google.com>
2019-03-11 12:52:03 +00:00
Liam Appelbe 2f158b9997 Add ABI_VERSION to the version file and a script to upload dills to CPID
Bug: https://github.com/dart-lang/sdk/issues/36047
Change-Id: I19d825eb2addaa32680d18a21d237ca172cd4b73
Reviewed-on: https://dart-review.googlesource.com/c/94442
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2019-03-01 23:00:41 +00:00
Martin Kustermann cf63db2594 [VM/infra] Archive kernel-service.dart.snapshot together with coredumps
Change-Id: I59f9356de0f63b3da68ea794480b1a0a6e58827e
Reviewed-on: https://dart-review.googlesource.com/c/93220
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2019-02-15 18:00:51 +00:00
Alexander Thomas 54a3e11d5b [infra] Upload core dumps if BUILDBOT_BUILDERNAME is set
Recipes pass output_dir to local shards as well but it points to a temp
directory that isn't accessible to users.

Change-Id: I0618abf78cf03957dd5cb8c9a10150bc4964684e
Reviewed-on: https://dart-review.googlesource.com/c/82985
Commit-Queue: Alexander Thomas <athom@google.com>
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-11-06 11:56:11 +00:00
Vyacheslav Egorov c413b61969 [testing] Improvements to crash dump archiving
* On Windows dump stacks from all collected dumps into the output so
that one would not need to download the dump to analyse it.
* Document crash dump archiving so that engineers know where to look
for dumps and how look at them.

Change-Id: Ibf53bb597c07702b3e051cf24bea8f6bfb1d5e8f
Reviewed-on: https://dart-review.googlesource.com/c/82066
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
2018-10-30 20:41:03 +00:00
Vyacheslav Egorov 64635263ff [testing] Fix code in utils.py to use Python not Dart
In Dart we do `x.containsKey(k)` in Python we do `k in x`.

Change-Id: I60d0e95d0c4402c42a914270dbb1828f714f9d08
Reviewed-on: https://dart-review.googlesource.com/c/81760
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2018-10-29 10:34:24 +00:00
Vyacheslav Egorov 040dd40ef3 [testing] Making core dump archiving more robust.
* Catch and suppress errors so that core dump archiving can succeed even
if it can't delete some files.
* On Windows also archive PDB files that contains debugging information.

Change-Id: If1d7caa7c793a71bf8c0852c38098933e88fadba
Reviewed-on: https://dart-review.googlesource.com/c/81680
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-10-26 18:17:18 +00:00
Vyacheslav Egorov 245ec25943 [testing] Make core dump archiving machinery more verbose
Change-Id: Ifadbc3e5d0740697aaea7336a738daa20223ad5e
Reviewed-on: https://dart-review.googlesource.com/c/81412
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-10-25 15:31:27 +00:00
Alexander Thomas bf26f760b1 [infra] Ensure that core dump upload triggers on main builders
https://crrev.com/c/1296514 will add --output_directory to non-shards as
well. BUILDBOT_BUILDERNAME is not set on shards and a more reliable
indication that this is a main bot that should gsutil upload core dumps.

Change-Id: Ic6cddc4937b8e83d2e15da4acd90c193e412773b
Reviewed-on: https://dart-review.googlesource.com/c/81362
Reviewed-by: Jonas Termansen <sortie@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-10-25 07:36:06 +00:00
Vyacheslav Egorov 2c83eb550e [vm] Integrate dart binary with Crashpad on Windows.
dart binary would instantiate CrashpadClient if DART_CRASHPAD_HANDLER
and DART_CRASHPAD_CRASHES_DIR environment variables are set.

- DART_CRASHPAD_HANDLER should contain the path to the crashpad_handler
binary that would handle the crash and write minidump;
- DART_CRASHPAD_CRASHES_DIR should contain the path to the crashpad database
which would be used to store minidumps.

Rewrite --copy-crash-dumps support on windows to use Crashpad integration
instead of editing Windows registry.

Embedding crashpad required to roll a new zlib version because Crashpad
depends on the zlib. This version of zlib is buildable with its own
BUILD.gn so our custom BUILD.gn is removed.

Change-Id: I048aad16b234e1d750f0a24782b04e3b6e19703d
Reviewed-on: https://dart-review.googlesource.com/c/81007
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-10-24 13:30:16 +00:00
Martin Kustermann ed73cc379d Increase file descriptor limit when running test.py on MacOS
This is an attempt to solve io test flakiness on MacOS. The symptoms
include "connection refused", "broken pipe" error messages.

Change-Id: If6759f4ef9cd2c1b9bd083db9a469db43f12c4e0
Reviewed-on: https://dart-review.googlesource.com/c/80120
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-10-16 09:25:10 +00:00
Alexander Thomas 5ce132a912 [infra] Copy core dumps to ISOLATED_OUTDIR on shards (#34746)
* Upload to Cloud Storage requires credentials not available to shards.
* Remove "SiteConfigBotoFile" buildbot hack.
* Remove obsolete Java utility method.

Change-Id: If03412a09644845d0f9467fa5fffc1cbec1cc216
Reviewed-on: https://dart-review.googlesource.com/c/79900
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-10-16 09:11:45 +00:00
Vyacheslav Egorov 051108c004 [tools] Re-enable core-dump archiving on Linux.
Make core_pattern checks non-fatal for now because underlying issues with
bots are not yet fixed.

We need core-dumps from debug kernel bots to look into some pthread flakes.

Bug: https://github.com/dart-lang/sdk/issues/31979

Change-Id: I38c79013cb2ed5a82ce10c909ff729f73cc34536
Reviewed-on: https://dart-review.googlesource.com/c/37320
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-10-10 13:47:17 +00:00
Zach Anderson b4004f6476 [vm] Reland: Move the interpreter under a flag.
After this CL, the interpreter is included by default in the
JIT VM under the flag --enable-interpreter.

Reland with fix to NativeArgument setup in simulator_arm.cc

Change-Id: Ib9b4df6eb4d997dfbe361188b8a127828c1d9c6f
Reviewed-on: https://dart-review.googlesource.com/74003
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-09-07 19:25:31 +00:00
Zach Anderson 21406f576a Revert "[vm] Move the interpreter under a flag."
This reverts commit 715ab7b30b.

Reason for revert: simarm and kernel precomp builds failing

Original change's description:
> [vm] Move the interpreter under a flag.
> 
> After this CL, the interpreter is included by default in the
> JIT VM under the flag --enable-interpreter.
> 
> Change-Id: I06458f20288d67981822788ee6e867c2127ebdd3
> Reviewed-on: https://dart-review.googlesource.com/71800
> Commit-Queue: Zach Anderson <zra@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com,zra@google.com,asiva@google.com,regis@google.com

Change-Id: I3622f4dbe53288d07fc21b34ee5ff68f2682b36d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/74001
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-09-07 17:14:21 +00:00
Zach Anderson 715ab7b30b [vm] Move the interpreter under a flag.
After this CL, the interpreter is included by default in the
JIT VM under the flag --enable-interpreter.

Change-Id: I06458f20288d67981822788ee6e867c2127ebdd3
Reviewed-on: https://dart-review.googlesource.com/71800
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-09-07 16:46:07 +00:00
Zach Anderson d0510501d2 [build] Add -a kbc for interpreter
This CL adds support to build.py and test.py for building/testing
a VM with the interpreter on x64, e.g.:

$ ./tools/gn.py -m release -a x64 --bytecode
$ ./tools/build.py -m release -a x64 --bytecode runtime
$ ./tools/test.py -m release -a x64 -r vm -c dartkb language_2

Change-Id: I956d23790636609d4a2e71129481fcbd7afef9a0
Reviewed-on: https://dart-review.googlesource.com/65206
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2018-07-17 21:51:41 +00:00
Alexander Thomas 72b77ac6ab [infra] Remove last remnants of old checked in SDK
* Update README.
* Create 32-bit Windows CIPD package in update.sh.
* Remove architecture specific executable names.

Change-Id: I553b282188bc76e82df0a6e40200e3311fdc6f1d
Reviewed-on: https://dart-review.googlesource.com/62842
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-06-28 18:15:57 +00:00
Alexander Thomas 66c9109caf [infra] Download checked in SDKs from CIPD
#33659

Change-Id: I94fccf4bbce2f0140bf0d6efa953d44b290df856
Reviewed-on: https://dart-review.googlesource.com/62840
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-06-28 16:08:14 +00:00
Zach Anderson 0b4f01f759 [infra] Fixes to custom version string
related https://github.com/flutter/flutter/issues/14751

Change-Id: Id11048777e9e27fc63b94bc49c45e575b4333d24
Reviewed-on: https://dart-review.googlesource.com/42460
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-02-20 17:53:06 +00:00
Zach Anderson 0d5cf900b0 [infra,flutter] Adds semantically meaningful custom version strings
See https://github.com/flutter/flutter/issues/14751

Change-Id: I3b35c2a32c0dd1301e92abe23b1c6fe2c2e04a6c
Reviewed-on: https://dart-review.googlesource.com/42026
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-02-16 20:44:48 +00:00
Ryan Macnak d366f9619a [vm] Toward deterministic builds.
- Remove random build-id.
 - Replace build time in embedded version string with commit time.
 - Remove timestamps from Observatory tarball.
 - Zero-initialize skipped bytes in snapshot streams.
 - Fix uninitialized fields in PatchClass, Script and Library.
 - Disable (under flag) random identity hashes and concurrent GC.

Bug: https://github.com/dart-lang/sdk/issues/31427
Change-Id: I3e95de679c8372841cd27ca60df78d9b00ffbfe1
Reviewed-on: https://dart-review.googlesource.com/22901
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2017-11-23 00:07:56 +00:00
Zachary Anderson 336478d7d9 Really disable core dump archiving on Linux
Trying again.

Review-Url: https://codereview.chromium.org/2984653002 .
2017-07-19 11:49:21 -07:00
Zachary Anderson 3e354bcc58 Disable core dump archive check on Linux
related #30205

Review-Url: https://codereview.chromium.org/2983093002 .
2017-07-19 11:34:15 -07:00
Zachary Anderson ac16656161 Remove MIPS support
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2858623002 .
2017-06-22 08:49:22 -07:00
Terry Lucas f27144d7b5 Roll 50: Updated for push to origin/master.
TBR=jacobr@google.com

Review-Url: https://codereview.chromium.org/2875773003 .
2017-05-10 18:16:51 -07:00
Zachary Anderson 713b89fbe7 [infra] Stop console spam from utils.py on a cross-build
R=rmacnak@google.com

Review-Url: https://codereview.chromium.org/2873613004 .
2017-05-10 09:41:45 -07:00
Martin Kustermann a7e6da3a93 Add support for core dump archiving on macos
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2815553003 .
2017-04-19 10:36:23 +02:00
Martin Kustermann b40a9e8608 Add link to cit bug in the exception for coredumps
Review-Url: https://codereview.chromium.org/2769683002 .
2017-03-22 14:36:20 +01:00
Martin Kustermann eb903b955e Add more debugging support to coredump archiving code
It seems sometimes we cannot find the coredumps but we are unable to
reproduce it (even on the bots). So let's add some more debugging
information.

R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2713193002 .
2017-02-24 17:06:28 +01:00
Martin Kustermann cc45d27027 Disable usage of site_config boto file
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2710333004 .
2017-02-24 17:04:40 +01:00
Martin Kustermann f3bef9983b Convert arguments to ''.join() to strings in tools/utils.py
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2710333003 .
2017-02-24 12:23:26 +01:00
Martin Kustermann affb786582 Enable support for coredump archiving on windows
BUG=https://github.com/dart-lang/sdk/issues/28648
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2692883002 .
2017-02-14 16:34:31 +01:00
Vyacheslav Egorov e8f456edd5 Buildbot: Only archive core dumps from unexpected crashes.
Change UnexpectedCrashDumpArchiver to generate coredumps file in the current
directory. Each line of this file specifies a core and a binary to be
uploaded in the following format:

              test-name,core-file,binary-file

CoreDumpArchiver reads this file and uploads the first 10 crash dumps.

R=kustermann@google.com
BUG=

Review-Url: https://codereview.chromium.org/2645963004 .
2017-01-20 17:44:51 +01:00
Vyacheslav Egorov 6c9ca0cd2b Buildbot: Change --copy-crashdumps to do the actual archiving to CloudStorage.
This flag now both enables core dump collection by setting appropriate rlimits
and takes care of archiving cores and crashed binaries.

R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2640093005 .
2017-01-19 17:25:39 +01:00
Vyacheslav Egorov e9483132ed Revert "Buildbot: Change --copy-crashdumps to do the actual archiving to CloudStorage."
This reverts commit dec352c1ac.

Bots are completely broken

BUG=
TBR=kustermann@google.com

Review-Url: https://codereview.chromium.org/2640823005 .
2017-01-19 16:17:23 +01:00
Vyacheslav Egorov dec352c1ac Buildbot: Change --copy-crashdumps to do the actual archiving to CloudStorage.
This flag now both enables core dump collection by setting appropriate rlimits
and takes care of archiving cores and crashed binaries.

BUG=
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2644593002 .
2017-01-19 16:13:28 +01:00
Zachary Anderson 566ed7ff55 GN: Fix armsimdbc target. Add armsimdbc64
R=johnmccutchan@google.com

Review-Url: https://codereview.chromium.org/2596333002 .
2016-12-22 10:24:24 -08:00
Jonas Termansen c0172aba48 Don't store git commit in observatory version string
The observatory assets tarball would be different every time due to the
version string, which makes it compress to a different size. The size
changes would then move the program data around, making it much more
difficult to determine if dart builds of different revisions are
actually identical.

Google Analytics doesn't need to know the exact version numbers, it will
be fine to say they are bleeding edge builds of a particular version.

BUG=
R=whesse@google.com

Review URL: https://codereview.chromium.org/2540383003 .
2016-12-01 14:16:23 +01:00
William Hesse e82a20b180 Change default Windows compiler to MSVS 2015 (VS version 14.0)
BUG=
R=zra@google.com

Review URL: https://codereview.chromium.org/2217393002 .
2016-08-06 00:12:56 +02:00
Zachary Anderson a274332ed7 Remove Fuchsia build from the gyp build
This isn't used anymore. The build happens in the Fuchsia tree using GN.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2194223002 .
2016-08-01 12:38:11 -07:00
Zachary Anderson a503570e3a Fuchsia: Initial check-in.
Instructions to build and run are in README.fuchsia.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2117593002 .
2016-07-01 12:42:05 -07:00
Zach Anderson 44796ed02d Move Windows build output from //build to //out
This is needed in order to begin migration to GN because
third-party dependencies and GN boilerplate expect GN
boilerplate to be in //build.

R=asiva@google.com, whesse@google.com

Review URL: https://codereview.chromium.org/2027063003 .
2016-06-03 08:28:48 -07:00