[CFE] Speedup dill serialization after slowdown in 548bff1

548bff1 was meant to do less.
It actually turns out slower though:

Before 548bff1:
[ 07:52 | 100.0% | + 2252 | -    0 ]: weak/variance/unconstrained_inference

real    7m58.017s
user    11m15.944s
sys     2m42.890s

=> 209.5 ms per test.

After 548bff1:
[ 08:47 | 100.0% | + 2252 | -    0 ]: weak/variance/unconstrained_inference

real    8m53.362s
user    10m27.086s
sys     1m10.925s

=> 234.0 ms per test.

What it did was, that it introduced - I'm guessing this is the reason -
some non-monomorphism in the serialization code by having sinks of
different types.

Running
`time out/ReleaseX64/dart --enable-asserts pkg/front_end/test/fasta/weak_suite.dart --traceStepTiming -DupdateExpectations=false -DsemiFuzz=false`
(with code changed so it's three shards, running only the first one)

I get numbers like this for writing the dill file though:

Before 548bff1: 0:01:28.369107
 After 548bff1: 0:01:44.803383

(again this is for 1 out of 3 shards, but illustrates the point ---
serialization is slower).

This CL changes this to always serialize into the same sink (in memory),
then - if needed (as in the original CL) - write it to disk.
This gets (again with 1 out of 3 shards) writing the dill down to
0:01:21.872812, and the total run looks like this:

[ 07:18 | 100.0% | + 2252 | -    0 ]: weak/variance/unconstrained_inference

real    7m23.361s
user    10m48.069s
sys     1m30.336s

=> 194.4 ms per test.

(rebased run:

[ 07:13 | 100.0% | + 2254 | -    0 ]: weak/variance/unconstrained_inference

real    7m18.828s
user    10m44.163s
sys     1m31.777s

=> 192.1 ms per test.)

Change-Id: I2906807bbab407fb499aa910afd20f41347bc3c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/245374
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This commit is contained in:
Jens Johansen
2022-05-31 08:17:33 +00:00
committed by Commit Bot
parent a801368f80
commit b99901c433
+19 -60
View File
@@ -7,8 +7,6 @@ library fasta.testing.kernel_chain;
import 'dart:async';
import 'dart:io' show Directory, File, IOSink, Platform;
import 'dart:typed_data' show Uint8List;
import 'package:_fe_analyzer_shared/src/util/relativize.dart'
show isWindows, relativizeUri;
@@ -454,51 +452,38 @@ class WriteDill extends Step<ComponentResult, ComponentResult, ChainContext> {
writeToFile = false;
}
}
Sink<List<int>> sink;
String writeMessage;
if (writeToFile && !skipVm) {
Directory tmp = await Directory.systemTemp.createTemp();
Uri uri = tmp.uri.resolve("generated.dill");
File generated = new File.fromUri(uri);
sink = generated.openWrite();
result = new ComponentResult(
result.description,
result.component,
result.userLibraries,
result.compilationSetup,
result.sourceTarget,
uri);
writeMessage = "Wrote component to `${generated.path}`";
} else {
sink = new DevNullSink();
writeMessage = "Wrote component to /dev/null";
}
ByteSink sink = new ByteSink();
bool good = false;
try {
// TODO(johnniwinther,jensj): Avoid serializing the sdk.
new BinaryPrinter(sink).writeComponentFile(component);
good = true;
} catch (e, s) {
return fail(result, e, s);
} finally {
print(writeMessage);
if (sink is IOSink) {
await sink.close();
if (good && writeToFile && !skipVm) {
Directory tmp = await Directory.systemTemp.createTemp();
Uri uri = tmp.uri.resolve("generated.dill");
File generated = new File.fromUri(uri);
IOSink ioSink = generated.openWrite();
ioSink.add(sink.builder.takeBytes());
await ioSink.close();
result = new ComponentResult(
result.description,
result.component,
result.userLibraries,
result.compilationSetup,
result.sourceTarget,
uri);
print("Wrote component to `${generated.path}`.");
} else {
sink.close();
print("Wrote component to memory.");
}
}
return pass(result);
}
}
class DevNullSink<T> extends Sink<T> {
@override
void add(T data) {}
@override
void close() {}
}
class ReadDill extends Step<Uri, Uri, ChainContext> {
const ReadDill();
@@ -516,32 +501,6 @@ class ReadDill extends Step<Uri, Uri, ChainContext> {
}
}
class BytesCollector implements Sink<List<int>> {
final List<List<int>> lists = <List<int>>[];
int length = 0;
@override
void add(List<int> data) {
lists.add(data);
length += data.length;
}
Uint8List collect() {
Uint8List result = new Uint8List(length);
int offset = 0;
for (List<int> list in lists) {
result.setRange(offset, offset += list.length, list);
}
lists.clear();
length = 0;
return result;
}
@override
void close() {}
}
Future<String> runDiff(Uri expected, String actual) async {
if (Platform.isWindows) {
// TODO(johnniwinther): Work-around for Windows. For some reason piping