Jens Johansen
e2b6208871
This CL shards the CFE try bot tests better. * co19_2 runs was previously made faster, so using 10 shards on that is way too much. * sdk_test runs was previously made faster, so using 5 shard on that is way too much. * unit_tests was taking ~10 minutes and wasn't sharded. This CL shards it. Note that this sharding is semi-complicated because a few tests use git which is not usable on shard-runs. To overcome this a previous CL renamed the tests that use git and here the sharded runs only run the tests that doesn't require git. Then the "main bot" runs the tests that require git. * unit tests suites was taking ~12 minutes and wasn't sharded. This CL shards it. Note that this sharding is semi-complicated because a few of the suites use git which is not usable on shard-runs. Also a single suite uses many more files that it is reasonable to include in the fileset for the shards. In both cases these are run on the "main bot" instead. The suit runnier was already "threaded", but simply launched all threads (say around 20 or something like that) at the same time. That's not ideal if having, say, 8 cores to work with (which is what the bots seem to have) - or 12 (which is what my computer has). Now only 'cores - 1' "threads" are run at any one time, and most "sub-suites" are sharded, so that if one finishes early that core can start another "sub-suite" - hopefully utilizing more of the resources avaiable on the computer running it. In total - in my 'benchmarks' (i.e. try bot runs) - the front-end-linux-release-x64-try bot previously rook around 35 minutes to complete, and with these changes it takes around 15 minutes to complete. The biggest entry in the timeline is now building dart which takes almost half that time. It does so with fewer shards than before, but each might be utilized for longer. Change-Id: Ie034058d8f33aafd21bc49f2bc878484563ba01c Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181383 Commit-Queue: Jens Johansen <jensj@google.com> Reviewed-by: Johnni Winther <johnniwinther@google.com>
Package validation
The packages in pkg/ are automatically validated on the LUCI CI bots. The
validation is largely done by the tools/package_deps package; it can be tested
locally via:
dart tools/package_deps/bin/package_deps.dart
Packages which are published
There are several packages developed in pkg/ which are published to pub.
Validation of these packages is particularly important because the pub tools are
not used for these packages during development; we get our dependency versions
from the DEPS file. Its very easy for the dependencies specified in a package's
pubspec file to get out of date wrt the packages and versions actually used.
In order to better ensure we're publishing correct packages, we validate some properties of the pubspec files on our CI system. These validations include:
- that the dependencies listed in the pubspec are used in the package
- that all the packages used by the source are listed in the pubspec
- that we don't use relative path deps to pkg/ or third_party/ packages
Packages which are not published
For packages in pkg/ which we do not intend to be published, we put the following comment in the pubspec.yaml file:
# This package is not intended for consumption on pub.dev. DO NOT publish.
publish_to: none
These pubspecs are still validated by the package validation tool. The contents are more informational as the pubspecs for these packages are not consumed by the pub tool or ecosystem.
We validate:
- that the dependencies listed in the pubspec are used in the package
- that all the packages used by the source are listed in the pubspec
- that a reference to a pkg/ package is done via a relative path dependency