Files
sdk/runtime/vm/kernel_loader.h
T
Jens Johansen 3e599229b9 [kernel] Don't scan strings up front when serializing.
Currently serializing the ast for kernel is done in two passe:

1) Scan the program to find and index all strings. These are then
   sorted based on frequency and assigned an id. All string-
   references are refering to that id. As small numbers use less
   space in the binary than big numbers, sorting the numbers by
   frequency saves a certain amount of space.
   In addition the string indexing is "hijacked" for the
   "LimitedBinaryPrinter" to also perform some CanonicalName
   re-indexing.

2) We then serialize the entire thing.

This CL gets rid of a pass by not indexing the strings up-front.
Whenever it is asked to serialize a string it adds it to the index
(if not already there). The serialization is otherwise the same.
This means that:

1) Strings are not sorted by frequency, i.e. the binary output size
   can by bigger (numbers below).

2) The stringindex and canonical names are moved to the end of the
   binary instead of the front. As we still need it up front for
   deserialization some additional data is added to the
   ProgramIndex.

3) The "hijacking" done in "LimitedBinaryPrinter" is replaced by
   an alternative.

4) We don't spend time on walking the tree twice.

The cost is the binary size. Compiling helloworld with fasta,
as well as looking at outline.dill, platform.dill and
vmservice_io.dill reveals these numbers:

* helloworld.dill is 0.657248732% bigger (26573 bytes)
* outline.dill is 1.686911399% bigger (9395 bytes)
* platform.dill is 0.657062238% bigger (26565 bytes)
* vmservice_io.dill is 0.44991899% bigger (19147 bytes)

The cost does thus not appear to be very big.

The gain is the serialization time.

From 20 runs of an instrumented VM/serialization, running numbers
through calculations stolens from ministat
(https://www.freebsd.org/cgi/man.cgi?query=ministat) reveals the
following:

* Serialization time: -21.69% +/- 1.44%

* Total time spend in relevant parts of bootstrap_nocore.cc,
dart_api_impl.cc (Dart_LoadKernel), bootstrap_nocore.cc,
dart_api_impl.cc (LoadKernelProgram) as well as serialization:
-14.01% +/- 1.58%

From 5 runs of
"time python tools/test.py -m release -cdartk language -j6"
(again run through ministat calculations):

* real: -4.18% +/- 0.5%
* user: -4.2% +/- 0.29%
* sys: No difference at 95%
* user+sys: -3.3% +/- 0.36%

Change-Id: I1c220eac083496994f0a9f1e2a2445b3707c9a93
Reviewed-on: https://dart-review.googlesource.com/2880
Reviewed-by: Samir Jindel <sjindel@google.com>
2017-09-06 11:55:16 +00:00

161 lines
5.0 KiB
C++

// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
#ifndef RUNTIME_VM_KERNEL_LOADER_H_
#define RUNTIME_VM_KERNEL_LOADER_H_
#if !defined(DART_PRECOMPILED_RUNTIME)
#include <map>
#include "vm/compiler/frontend/kernel_binary_flowgraph.h"
#include "vm/compiler/frontend/kernel_to_il.h"
#include "vm/kernel.h"
#include "vm/object.h"
namespace dart {
namespace kernel {
class KernelLoader;
class BuildingTranslationHelper : public TranslationHelper {
public:
BuildingTranslationHelper(KernelLoader* loader, Thread* thread)
: TranslationHelper(thread), loader_(loader) {}
virtual ~BuildingTranslationHelper() {}
virtual RawLibrary* LookupLibraryByKernelLibrary(NameIndex library);
virtual RawClass* LookupClassByKernelClass(NameIndex klass);
private:
KernelLoader* loader_;
};
template <typename VmType>
class Mapping {
public:
bool Lookup(intptr_t canonical_name, VmType** handle) {
typename MapType::Pair* pair = map_.LookupPair(canonical_name);
if (pair != NULL) {
*handle = pair->value;
return true;
}
return false;
}
void Insert(intptr_t canonical_name, VmType* object) {
map_.Insert(canonical_name, object);
}
private:
typedef IntMap<VmType*> MapType;
MapType map_;
};
class KernelLoader {
public:
explicit KernelLoader(Program* program);
// Returns the library containing the main procedure, null if there
// was no main procedure, or a failure object if there was an error.
Object& LoadProgram();
// Finds all libraries that have been modified in this incremental
// version of the kernel program file.
void FindModifiedLibraries(Isolate* isolate,
BitVector* modified_libs,
bool force_reload);
void LoadLibrary(intptr_t kernel_offset);
const String& DartSymbol(StringIndex index) {
return translation_helper_.DartSymbol(index);
}
const String& LibraryUri(intptr_t library_index) {
return translation_helper_.DartSymbol(
translation_helper_.CanonicalNameString(
library_canonical_name(library_index)));
}
intptr_t library_offset(intptr_t index) {
kernel::Reader reader(program_->kernel_data(),
program_->kernel_data_size());
reader.set_offset(reader.size() - (4 * LibraryCountFieldCountFromEnd) -
(4 * (program_->library_count() - index)));
return reader.ReadUInt32();
}
NameIndex library_canonical_name(intptr_t index) {
kernel::Reader reader(program_->kernel_data(),
program_->kernel_data_size());
reader.set_offset(reader.size() - (4 * LibraryCountFieldCountFromEnd) -
(4 * (program_->library_count() - index)));
reader.set_offset(reader.ReadUInt32());
// Start reading library.
reader.ReadFlags();
return reader.ReadCanonicalNameReference();
}
uint8_t CharacterAt(StringIndex string_index, intptr_t index);
private:
friend class BuildingTranslationHelper;
void LoadPreliminaryClass(Class* klass,
ClassHelper* class_helper,
intptr_t type_parameter_count);
Class& LoadClass(const Library& library, const Class& toplevel_class);
void LoadProcedure(const Library& library, const Class& owner, bool in_class);
void LoadAndSetupTypeParameters(const Object& set_on,
intptr_t type_parameter_count,
const Class& parameterized_class,
const Function& parameterized_function);
RawArray* MakeFunctionsArray();
// If klass's script is not the script at the uri index, return a PatchClass
// for klass whose script corresponds to the uri index.
// Otherwise return klass.
const Object& ClassForScriptAt(const Class& klass, intptr_t source_uri_index);
Script& ScriptAt(intptr_t source_uri_index,
StringIndex import_uri = StringIndex());
void GenerateFieldAccessors(const Class& klass,
const Field& field,
FieldHelper* field_helper,
intptr_t field_offset);
void SetupFieldAccessorFunction(const Class& klass, const Function& function);
Library& LookupLibrary(NameIndex library);
Class& LookupClass(NameIndex klass);
RawFunction::Kind GetFunctionType(ProcedureHelper::Kind procedure_kind);
Program* program_;
Thread* thread_;
Zone* zone_;
Isolate* isolate_;
Array& scripts_;
Array& patch_classes_;
ActiveClass active_class_;
BuildingTranslationHelper translation_helper_;
StreamingFlowGraphBuilder builder_;
Mapping<Library> libraries_;
Mapping<Class> classes_;
GrowableArray<const Function*> functions_;
GrowableArray<const Field*> fields_;
};
} // namespace kernel
} // namespace dart
#endif // !defined(DART_PRECOMPILED_RUNTIME)
#endif // RUNTIME_VM_KERNEL_LOADER_H_