f169abcee0
This CL: * Makes the standard filesystem more synchronous by default. This makes reading files for normal compiles faster. * Adds a more asynchronous version of `exists` and `readAsBytes`. This potentially allows for reading files faster when in a context where the files can actually be read in parallel. The client have to make the choise though. * Skips `Uri.base.resolveUri` when the uri has a scheme. This makes it slightly faster in my tests, but does not remove any `.` and `..`. Having those when having a scheme seems sort of weird though. If this turns out to be a problem we can add it back. * Re-orders what the standard filesystem checks in `exists` to assume files which seems more likely to happen. This - for files - means less checks and should make checking for existance faster. (This was noticed by Siggi). Using the same measurement procedure as in https://dart-review.googlesource.com/c/sdk/+/85442 timing how long fasta spends on *reading* files when compiling dart2js (when compiling via the VM) I get a good speedup. Note that the measurements are in microseconds: Before this CL (run like `out/ReleaseX64/dart pkg/compiler/bin/dart2js.dart 2>&1 | grep "Read file (total)" | tail -n 1` 10 times): Read file (total): 94678 Read file (total): 94232 Read file (total): 91726 Read file (total): 77472 Read file (total): 95582 Read file (total): 88113 Read file (total): 89245 Read file (total): 91575 Read file (total): 96291 Read file (total): 95730 With this CL (run like `out/ReleaseX64/dart pkg/compiler/bin/dart2js.dart 2>&1 | grep "Read file (total)" | tail -n 1` 10 times): Read file (total): 68379 Read file (total): 69320 Read file (total): 72930 Read file (total): 69692 Read file (total): 68685 Read file (total): 73548 Read file (total): 64649 Read file (total): 71951 Read file (total): 73486 Read file (total): 70621 Difference at 95.0% confidence -21138.3 +/- 4193 -23.111% +/- 4.5843% (Student's t, pooled s = 4462.56) Furthermore, using an internal benchmark (discussed in an email thread) I get these numbers (note that the measurements are in milliseconds): before: 4453 4249 4187 4190 4216 after: 2310 2379 2449 2467 2407 Difference at 95.0% confidence -1856.6 +/- 131.443 -43.5924% +/- 3.08625% (Student's t, pooled s = 90.1257) Using the asynchronous versions of exists and readAsBytes and checking for existance and reading in parallel on the same internal benchmark it gets to 1137 1185 1067 1089 1189 For completion, comparing to using "raw" `File` (again using the internal benchmark): sequential standard 2377 2419 2485 2373 2431 sequential io_sync 2279 2182 2237 2312 2324 io_sync faster: -150.2 +/- 76.3122 -6.21432% +/- 3.15731% (Student's t, pooled s = 52.3245) sequential standardasync 4865 4915 4753 5185 4760 sequential io_async 4746 4834 4989 4840 4891 No difference proven at 95.0% confidence parallel standard 2526 2563 2652 2616 2685 parallel io_sync 2335 2184 2244 2341 2401 io_async faster Difference at 95.0% confidence -307.4 +/- 111.037 -11.785% +/- 4.25691% (Student's t, pooled s = 76.1341) parallel standardasync 1137 1185 1067 1089 1189 parallel io_async 1105 1088 1145 1048 1113 No difference proven at 95.0% confidence TEST=Existing tests. Change-Id: I8ba56ab0768df8672bcdb693782d3f1eec86b683 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185101 Commit-Queue: Jens Johansen <jensj@google.com> Reviewed-by: Johnni Winther <johnniwinther@google.com> Reviewed-by: Gary Roumanis <grouma@google.com>
120 lines
3.8 KiB
Dart
120 lines
3.8 KiB
Dart
// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
|
|
// for details. All rights reserved. Use of this source code is governed by a
|
|
// BSD-style license that can be found in the LICENSE file.
|
|
|
|
import 'dart:async';
|
|
|
|
import 'package:front_end/src/api_unstable/build_integration.dart';
|
|
|
|
/// Wraps a file system to create an overlay of files from multiple roots.
|
|
///
|
|
/// Regular `file:` URIs are resolved directly in the underlying file system,
|
|
/// but URIs that use a special [markerScheme] are resolved by searching
|
|
/// under a set of given roots in order.
|
|
///
|
|
/// For example, consider the following inputs:
|
|
///
|
|
/// - markerScheme is `multi-root`
|
|
/// - the set of roots are `file:///a/` and `file:///b/`
|
|
/// - the underlying file system contains files:
|
|
/// a/c/a1.dart
|
|
/// a/c/a2.dart
|
|
/// b/c/b1.dart
|
|
/// b/c/a2.dart
|
|
/// c/c1.dart
|
|
///
|
|
/// Then:
|
|
///
|
|
/// - file:///c/c1.dart is resolved as file:///c/c1.dart
|
|
/// - multi-root:///c/a1.dart is resolved as file:///a/c/a1.dart
|
|
/// - multi-root:///c/b1.dart is resolved as file:///b/c/b1.dart
|
|
/// - multi-root:///c/a2.dart is resolved as file:///a/c/b2.dart
|
|
class MultiRootFileSystem implements FileSystem {
|
|
final String markerScheme;
|
|
final List<Uri> roots;
|
|
final FileSystem original;
|
|
|
|
MultiRootFileSystem(this.markerScheme, List roots, this.original)
|
|
: roots = roots.map(_normalize).toList();
|
|
|
|
@override
|
|
FileSystemEntity entityForUri(Uri uri) =>
|
|
new MultiRootFileSystemEntity(this, uri);
|
|
}
|
|
|
|
/// Entity that searches the multiple roots and resolve a, possibly multi-root,
|
|
/// entity to a plain entity under `multiRootFileSystem.original`.
|
|
class MultiRootFileSystemEntity implements FileSystemEntity {
|
|
final MultiRootFileSystem multiRootFileSystem;
|
|
final Uri uri;
|
|
FileSystemEntity _delegate;
|
|
Future<FileSystemEntity> get delegate async =>
|
|
_delegate ??= await _resolveEntity();
|
|
|
|
Future<FileSystemEntity> _resolveEntity() async {
|
|
if (uri.scheme == multiRootFileSystem.markerScheme) {
|
|
if (!uri.isAbsolute) {
|
|
throw new FileSystemException(
|
|
uri, "This MultiRootFileSystem only handles absolutes URIs: $uri");
|
|
}
|
|
var original = multiRootFileSystem.original;
|
|
assert(uri.path.startsWith('/'));
|
|
var path = uri.path.substring(1);
|
|
for (var root in multiRootFileSystem.roots) {
|
|
var candidate = original.entityForUri(root.resolve(path));
|
|
if (await candidate.exists()) return candidate;
|
|
}
|
|
return MissingFileSystemEntity(uri);
|
|
}
|
|
return multiRootFileSystem.original.entityForUri(uri);
|
|
}
|
|
|
|
MultiRootFileSystemEntity(this.multiRootFileSystem, this.uri);
|
|
|
|
@override
|
|
Future<bool> exists() async => (await delegate).exists();
|
|
|
|
@override
|
|
Future<bool> existsAsyncIfPossible() async =>
|
|
(await delegate).existsAsyncIfPossible();
|
|
|
|
@override
|
|
Future<List<int>> readAsBytes() async => (await delegate).readAsBytes();
|
|
|
|
@override
|
|
Future<List<int>> readAsBytesAsyncIfPossible() async =>
|
|
(await delegate).readAsBytes();
|
|
|
|
@override
|
|
Future<String> readAsString() async => (await delegate).readAsString();
|
|
}
|
|
|
|
class MissingFileSystemEntity implements FileSystemEntity {
|
|
@override
|
|
final Uri uri;
|
|
|
|
MissingFileSystemEntity(this.uri);
|
|
|
|
@override
|
|
Future<bool> exists() => Future.value(false);
|
|
|
|
@override
|
|
Future<bool> existsAsyncIfPossible() => exists();
|
|
|
|
@override
|
|
Future<List<int>> readAsBytes() =>
|
|
Future.error(FileSystemException(uri, 'File not found'));
|
|
|
|
@override
|
|
Future<List<int>> readAsBytesAsyncIfPossible() => readAsBytes();
|
|
|
|
@override
|
|
Future<String> readAsString() =>
|
|
Future.error(FileSystemException(uri, 'File not found'));
|
|
}
|
|
|
|
Uri _normalize(root) {
|
|
Uri uri = root;
|
|
return uri.path.endsWith('/') ? uri : uri.replace(path: '${uri.path}/');
|
|
}
|