Commit Graph

48 Commits

Author SHA1 Message Date
Alexander Thomas b5c63ce757 [infra] Migrate scripts to python3
* Migrate to python3; drop python support.
* Update Windows toolchain support.
* Remove some unused methods.
* Python 2.7 is still needed on Windows.
* Update gsutil to a version that supports python3.

Fixes: https://github.com/dart-lang/sdk/issues/28793

TEST=Manually tested common user journeys.

Change-Id: I663a22b237a548bb82dc2e601e399e3bc3649211
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192182
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-04-15 10:10:20 +00:00
Alexander Thomas bbe16d5dfe [infra] Remove dead code in tools
* Remove bot.py and all it's uses.
* Remove unused buildbot pre-recipe framework recipes.
* Remove abiversions folder.
* Remove old browsers folder.

Change-Id: Iebd0a374c5bd1f17fed9d1fa9dbf750639242312
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170432
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2020-11-06 07:29:12 +00:00
Janice Collins 77be14d718 Update dartdoc to 0.36.0 and remove unnecessary extra parameter to
invocation on the bots.

Release notes: https://github.com/dart-lang/dartdoc/releases/tag/v0.36.0

Change-Id: Ie15dfd538ee54b17b04c7d556fc61deddab463f6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168600
Commit-Queue: Janice Collins <jcollins@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2020-10-21 19:11:14 +00:00
Sam Rawlins 4585d4af0a Bump dartdoc to ab98003fc368f484fcc4c055770adea47bc83fbd
Change-Id: I1e1713428bbd5ab713a62aff70d83cc502dde9f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162250
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Samuel Rawlins <srawlins@google.com>
2020-09-10 02:20:30 +00:00
Martin Kustermann 168539d61d Reland "[vm] Build dart2native dependencies with the normal "create_sdk" target"
The flutter engine DEPS got new icu/boringssl so this CL can safely
land and roll into engine. A small change is made in Patchset 1..3.

This removes special logic for creating the `dart-sdk` we distribute
which used to build release and product mode and copied some binaries
from the latter into the former, before the SDK was actuallly ready to
test and distribute.

This changes the GN build rules to build the necessary
gen_snapshot/dart_precompiled_runtime product binaries during the normal
release build.

Normally during --mode=product builds the global build config in
//build/config/BUILDCONFIG.gn will set `-fvisibility=false`.

=> Doing so results in much smaller binaries - because only explicitly
   exported symbols are visible, the rest can be tree shaken by the linker.

Since we are building --mode=release, the `-fvisibility=false` will not
be set. In order to set the flag for the 2 special product-mode binaries
we need to add -fvisibility=hidden manually, in:

  * dart_product_config: Which is used for compiling VM sources.
  * 3rd party double-conversion library
  * 3rd party boringssl library
  * 3rd party icu library

The upstream CLs are:

  * BoringSSL: https://dart-review.googlesource.com/c/boringssl_gen/+/150482
  * ICU: https://chromium-review.googlesource.com/c/chromium/deps/icu/+/2236407

Issue https://github.com/dart-lang/sdk/issues/42230

Change-Id: I8d9b37acbb5eb94e44d97652f838948a946fd372
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151505
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2020-06-17 16:12:26 +00:00
Martin Kustermann 21222b64c4 Revert "[vm] Build dart2native dependencies with the normal "create_sdk" target"
This reverts commit a88a328b7f.

Reason for revert: Auto roller hasn't rolled previous changes from
Dart yet. Those changes updated DEPS for icu/boringssl. We have to
wait with this CL until the DEPS changes have been done in the engine.

Original change's description:
> [vm] Build dart2native dependencies with the normal "create_sdk" target
> 
> This removes special logic for creating the `dart-sdk` we distribute
> which used to build release and product mode and copied some binaries
> from the latter into the former, before the SDK was actuallly ready to
> test and distribute.
> 
> This changes the GN build rules to build the necessary
> gen_snapshot/dart_precompiled_runtime product binaries during the normal
> release build.
> 
> Normally during --mode=product builds the global build config in
> //build/config/BUILDCONFIG.gn will set `-fvisibility=false`.
> 
> => Doing so results in much smaller binaries - because only explicitly
>    exported symbols are visible, the rest can be tree shaken by the linker.
> 
> Since we are building --mode=release, the `-fvisibility=false` will not
> be set. In order to set the flag for the 2 special product-mode binaries
> we need to add -fvisibility=hidden manually, in:
> 
>   * dart_product_config: Which is used for compiling VM sources.
>   * 3rd party double-conversion library
>   * 3rd party boringssl library
>   * 3rd party icu library
> 
> The upstream CLs are:
> 
>   * BoringSSL: https://dart-review.googlesource.com/c/boringssl_gen/+/150482
>   * ICU: https://chromium-review.googlesource.com/c/chromium/deps/icu/+/2236407
> 
> Issue https://github.com/dart-lang/sdk/issues/42230
> 
> Change-Id: I3e47664d9fadb9ed1ad033bb17d46e769442f741
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150524
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Alexander Thomas <athom@google.com>
> Reviewed-by: Zach Anderson <zra@google.com>

TBR=kustermann@google.com,zra@google.com,athom@google.com,asiva@google.com

Change-Id: Ied0f2f5b58dceb2b59bc2ac6e11e5fd717c8885e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151382
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-06-16 10:41:27 +00:00
Martin Kustermann a88a328b7f [vm] Build dart2native dependencies with the normal "create_sdk" target
This removes special logic for creating the `dart-sdk` we distribute
which used to build release and product mode and copied some binaries
from the latter into the former, before the SDK was actuallly ready to
test and distribute.

This changes the GN build rules to build the necessary
gen_snapshot/dart_precompiled_runtime product binaries during the normal
release build.

Normally during --mode=product builds the global build config in
//build/config/BUILDCONFIG.gn will set `-fvisibility=false`.

=> Doing so results in much smaller binaries - because only explicitly
   exported symbols are visible, the rest can be tree shaken by the linker.

Since we are building --mode=release, the `-fvisibility=false` will not
be set. In order to set the flag for the 2 special product-mode binaries
we need to add -fvisibility=hidden manually, in:

  * dart_product_config: Which is used for compiling VM sources.
  * 3rd party double-conversion library
  * 3rd party boringssl library
  * 3rd party icu library

The upstream CLs are:

  * BoringSSL: https://dart-review.googlesource.com/c/boringssl_gen/+/150482
  * ICU: https://chromium-review.googlesource.com/c/chromium/deps/icu/+/2236407

Issue https://github.com/dart-lang/sdk/issues/42230

Change-Id: I3e47664d9fadb9ed1ad033bb17d46e769442f741
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150524
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2020-06-15 15:17:35 +00:00
Janice Collins 83277a4f37 Enable the non-nullable experiment for dartdoc.
For Dartdoc 0.32.1 this will only impact generated docs for the 'dev'
branch -- 'edge', 'be' and other builds will still generate legacy
documentation.  This may change with future versions as the SDK
allow-list feature gets pipelined through the analyzer to dartdoc.

Depends on https://dart-review.googlesource.com/c/sdk/+/149363
(will result in undefined behavior if applied to 0.32.0 or earlier).

Change-Id: Idf7d37f5b06deec1c4e02708d24029f47a869df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149344
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2020-05-29 12:55:05 +00:00
Alexander Thomas a1b895a900 [infra] Disable builds of macOS 32bit SDKs
These are currently broken because our xcode version no longer supports
i386 builds.

See https://github.com/dart-lang/sdk/issues/39810 for the corresponding
breaking change request.

Change-Id: Iaf9c20d090e7d9914a4c395e4aa17c13a1d336fe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128583
Reviewed-by: William Hesse <whesse@google.com>
2019-12-17 13:32:48 +00:00
Martin Kustermann 635aaa8adf [vm/dart2native] Bundle product-mode vm_platform.dill with the dart-sdk and let dart2native use it
Currently we bundle only one platform file, which dart2native uses as
well as other tools. Since the platform file has annotations already
evaluated, our AOT compiler ends up including service related
functionality in the AOT snapshot.

This CL bundles the product-mode vm_platform.dill as well, which will
reduce dart2native-created apps slightly.

Change-Id: I223811b30704cde1739739d97a8ea2d34a681292
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125665
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2019-11-20 16:50:09 +00:00
Nate Bosch 55f81f2210 Mass format python with yapf
- Add `.style.yapf` with configuration to use Google style.
- Run `yapf` on all `.py` files in this repo.
- Manually fix one trailing space in a doc string.
- Run `git cl format runtime` to satisfy presubmit.

Change-Id: I7e6bd11e91f07926b9188362599af398551eed79
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/111600
Commit-Queue: Nate Bosch <nbosch@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2019-08-05 20:34:31 +00:00
Clement Skau f1fcd89c2d [SDK] Fix missing argument in tools/bots/dart_sdk.py
This should make the build cycle green again.

Change-Id: I7a355020247d9edb18ad93cb6d890baef312a206
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101295
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2019-05-07 07:34:33 +00:00
Clement Skau 19c228115a [SDK] Adds smoke test for dartaot.
Change-Id: I0e840f9e6489268d64b458872f19d0a2964f1271
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100244
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2019-05-06 11:11:38 +00:00
Clement Skau 26874860e3 [SDK] Adds .exe extension for Windows binaries in dart_sdk.py.
This fixes the current build breakage:
https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8917013092125339168/+/steps/upload_sdk/0/stdout

Bug: None
Change-Id: Ic47812a5f3ce0ab7f090a3f19429aa5d5e3cb279
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98846
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2019-04-08 13:22:53 +00:00
Clement Skau 907c514c89 [SDK] Adds build targets, wrappers and SDK builds for Dart AOT.
Adds:
- dart2aot, a script similar to dart2js which compiles my.dart to my.dart.aot.
- dartaotruntime, a minimal Dart runtime that only runs AOT blobs.
- some extra tooling like gen_kernel and gen_snapshot used by the above.
- build rules for all of the above, including adding it to the full SDK builds.

Bug:https://github.com/dart-lang/sdk/issues/27596
Change-Id: Ic35f832b2b86be959212b8d21cfc5a082da5ced4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97627
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2019-04-05 13:23:44 +00:00
William Hesse 9a2b222cba [infra] dart_sdk builder script won't upload from tryjobs
Change-Id: I8b00936784ffd9ca9ff97aeefeaf07f16f16c11e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98481
Reviewed-by: Alexander Thomas <athom@google.com>
2019-04-03 09:22:19 +00:00
Alexander Thomas 770ab5275a [infra] Upgrade gsutil to version 4.34
Change-Id: Id353fa11a8f86415569a505ef5ecd7bf03fb51e1
Reviewed-on: https://dart-review.googlesource.com/c/90487
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2019-01-22 16:58:58 +00:00
Jonas Termansen 0a0eac5038 [infra] Build Linux ARM architectures in tools/bots/dart_sdk.py.
Change-Id: I0ab9a2c798bf95866f3139b2d3bf25fea5475a17
Reviewed-on: https://dart-review.googlesource.com/c/88327
Reviewed-by: William Hesse <whesse@google.com>
2019-01-03 14:37:35 +00:00
Alexander Thomas a787e53198 Add Windows SDK builder to CQ
Add experimental Linux SDK builder to CQ
Support building API docs on try builders

Bug:
Change-Id: I163d2a4a9e5f8a062ea005e9b50d9351d867ba02
Reviewed-on: https://dart-review.googlesource.com/4000
Reviewed-by: William Hesse <whesse@google.com>
2017-09-08 11:36:23 +00:00
William Hesse 002917b79c Fix error in commit bae6ab5a7a
"Split the dart_sdk bot script into two runs ..." used sys.argv[2]
instead of sys.argv[1]

BUG=
TBR=terry@google.com

Review-Url: https://codereview.chromium.org/3012713002 .
2017-09-01 17:21:03 +02:00
William Hesse bae6ab5a7a Split the dart_sdk bot script into two runs, one with only apidocs.
BUG=
R=terry@google.com, vegorov@google.com

Review-Url: https://codereview.chromium.org/3004233002 .
2017-09-01 16:50:00 +02:00
William Hesse 616789b6d4 Use wheezy sysroot when building SDK
This avoids a dependency of the linux SDK on glibc version
2.16 or greater.

BUG=https://github.com/dart-lang/sdk/issues/28476
R=zra@google.com

Committed: https://github.com/dart-lang/sdk/commit/a99211821cfdc0836576325d1d9d8c8d00ce72de
Review-Url: https://codereview.chromium.org/2650763003 .
2017-01-23 19:23:01 +01:00
William Hesse a99211821c Use wheezy sysroot when building SDK
This avoids a dependency of the linux SDK on glibc version
2.16 or greater.

BUG=https://github.com/dart-lang/sdk/issues/28476
R=zra@google.com

Review-Url: https://codereview.chromium.org/2650763003 .
2017-01-23 19:15:43 +01:00
keertip bb74bc6a49 fix arguments for dartdoc run
BUG=
R=whesse@google.com

Review URL: https://codereview.chromium.org/2534173003 .
2016-11-29 08:43:14 -08:00
William Hesse 61eee9ee52 Use checked-in .package file for building and testing
Stop creating [build dir]/packages directory with symbolic links.

BUG=https://github.com/dart-lang/sdk/issues/23565
R=rnystrom@google.com

Committed: https://github.com/dart-lang/sdk/commit/5d3e356ca75be8aeff3efde5df946d746b9f8352

Review URL: https://codereview.chromium.org/1746743002 .

Reverted: https://github.com/dart-lang/sdk/commit/6d238ba197c8bda71afe790f16f6e9adaf7abd13
2016-09-16 15:03:18 +02:00
William Hesse 9d2ab4d016 Change location of sysroot download so bot doesn't erase it.
BUG=https://github.com/dart-lang/sdk/issues/27048
TBR=sgjesse@google.com

Review URL: https://codereview.chromium.org/2230333002 .
2016-08-10 18:06:05 +02:00
William Hesse ab74e91c51 Fix error in SDK build script
BUG=
R=sgjesse@google.com

Review URL: https://codereview.chromium.org/2233923002 .
2016-08-10 17:34:19 +02:00
William Hesse 81f2226225 Change SDK builder bot to use sysroots when compiling
BUG=https://github.com/dart-lang/sdk/issues/27048
R=sgjesse@google.com

Review URL: https://codereview.chromium.org/2231073002 .
2016-08-10 17:28:21 +02:00
William Hesse 8473409b9c Archive SDK by git hash in addition to version number
This is needed for buildbot testers to download the SDK instead of building it.

BUG=
R=kustermann@google.com

Review URL: https://codereview.chromium.org/2218723003 .
2016-08-05 13:38:55 +02:00
Rico Wind d8fa5d5b31 Archive unstripped dart binaries on the sdk builders
R=whesse@google.com
BUG=

Review URL: https://codereview.chromium.org/1704323002 .
2016-02-18 08:48:13 +01:00
keertip c88841c041 use bot_utils.run to run dartdoc so that the bot will fail if there are errors when dartdoc is run. This will catch failures to generate docs.
BUG=https://github.com/dart-lang/sdk/issues/25015
R=whesse@google.com

Review URL: https://codereview.chromium.org/1514193002 .
2015-12-11 08:03:48 -08:00
keertip 895abbf8dd rev up dartdoc, markdown
BUG=
R=whesse@google.com

Review URL: https://codereview.chromium.org//1360823002 .
2015-09-22 09:44:56 -07:00
keertip d6cff42ea5 fix build (TBR)
BUG=

Review URL: https://codereview.chromium.org//1353213002 .
2015-09-18 13:35:38 -07:00
keertip 3d37a55dce remove docgen from the sdk
BUG=
R=whesse@google.com

Review URL: https://codereview.chromium.org//1356943002 .
2015-09-18 13:18:10 -07:00
keertip a6d128832c update to dartdoc 0.6.1
BUG=
R=pquitslund@google.com

Review URL: https://codereview.chromium.org//1321513002 .
2015-08-26 09:35:40 -07:00
keertip d2a067e22b add footer for dartdocs generated for the sdk for analytics and survey
BUG=
R=sethladd@google.com, whesse@google.com

Review URL: https://codereview.chromium.org//1239543002 .
2015-07-13 13:14:09 -07:00
keertip 72ac998b8d add generation of docs by dartdoc on the bots
BUG=
R=pquitslund@google.com

Review URL: https://codereview.chromium.org//1180773005.
2015-06-18 10:30:28 -07:00
keertip 0079275f18 revert cl to run dartdoc on bots
BUG=
R=danrubel@google.com

Review URL: https://codereview.chromium.org//1184153002.
2015-06-15 10:12:56 -07:00
keertip 91b2281e08 add dartdoc to third_party/pkg, run dartdoc to generate docs for sdk and upload to GCS
BUG=
R=ricow@google.com

Review URL: https://codereview.chromium.org//1164423008.
2015-06-15 09:25:44 -07:00
Rico Wind b35b14e94f Fix annotated steps for dart_sdk
Stray : at the end of the force line (used to be an if)

R=whesse@google.com
BUG=

Review URL: https://codereview.chromium.org//1167523006
2015-06-08 15:27:12 +02:00
Rico Wind 74759d5771 Remove GetSVNRevision from tools/utils.py
Also change all callsites to use git.

I have a pending request to Vijay for info on what tools/dartium/deploy_aar.py is for.

BUG=
R=whesse@google.com

Review URL: https://codereview.chromium.org//1166743005
2015-06-08 14:56:58 +02:00
Rico Wind f273093a4b Actually clobber on be sdk bots if the flag is set
Still clobber unconditionally on the dev/stable bots

R=whesse@google.com
BUG=

Review URL: https://codereview.chromium.org//1160793008
2015-06-08 14:54:32 +02:00
Rico Wind 77321a7586 Always clobber sdk builds on dev/stable.
We should build from scratch to eliminate any consistency issues and guarantee that versions we ship can always build.

R=whesse@google.com
BUG=

Review URL: https://codereview.chromium.org//1161413002
2015-06-02 11:36:19 +02:00
Rico Wind af3b6b2b61 Add python based VERSION file generation for sdk bots
We used to have this file generated by the editor build

BUG=23516
R=whesse@google.com

Review URL: https://codereview.chromium.org//1153033007
2015-05-28 11:20:36 +02:00
ricow@google.com ad6bf85994 Fix bug in dart_sdk annotated.
We don't explicitly import join

R=kustermann@google.com
BUG=

Review URL: https://codereview.chromium.org//1133123002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@45695 260f80e4-7a28-3924-810f-c04153c831b5
2015-05-11 16:22:17 +00:00
ricow@google.com 4b4a3202df Fix annotated steps, now really only do api docs on linux (but still do zips on mac and windows
BUG=
R=kustermann@google.com

Review URL: https://codereview.chromium.org//1135593003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@45691 260f80e4-7a28-3924-810f-c04153c831b5
2015-05-11 15:30:27 +00:00
ricow@google.com 77c28dc835 Only build and archive docs on linux
TBR=kustermann@google.com
BUG=

Review URL: https://codereview.chromium.org//1133113002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@45688 260f80e4-7a28-3924-810f-c04153c831b5
2015-05-11 15:08:29 +00:00
ricow@google.com 581c169dd7 Add an annotated steps for driving our sdk and api archiving.
This is based on old code from editor/build/build.py, but I have cleaned it up a bit:
  Simplified zip and sdk bundle creation
  Use bot_utils.gsutil for almost everything (except checking for the existense of the api_docs)

BUG=
R=kustermann@google.com

Review URL: https://codereview.chromium.org//1138703003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@45684 260f80e4-7a28-3924-810f-c04153c831b5
2015-05-11 13:41:00 +00:00