This is a reland of 5909fd111d
Does a large refactoring on the V8 snapshot profile writer
to clean things up, add more debugging support, and to fix
the problems that surfaced during the original landing.
Other changes:
Changes Serializer::CreateArtificialNodeIfNeeded() to create
artificial nodes for Code objects and immutable arrays.
Fixes CodeSerializationCluster::Trace() to only push needed parts of
discarded code objects, instead of tracing them like full code objects.
Adds test cases to v8_snapshot_profile_writer_test that exercise
the following situations (both separately and together):
* Non-symbolic stack traces are enabled and code and function objects
are dropped when not needed at runtime.
* Creation of the dispatch table is disabled.
TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test
Original change's description:
> [vm] Fix V8 snapshot profile handling of the dispatch table.
>
> Fixes https://github.com/dart-lang/sdk/issues/45702.
>
> TEST=Tests listed in the issue above.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
> Change-Id: Ibf5e3ccf3828c01f9dda47de360314dabe8cb8a9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195272
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I8e7030267fe190079a8f68d00fe20bf7170e5719
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195513
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
In the codebase, we have several different interfaces for instance
methods that return a hash:
- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;
This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.
TEST=Existing test suites, as this is an internal refactoring.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Instead of special-casing the current fields that may have
WeakSerializationReferences, handle WSRs appearing as elements
or properties of objects more generally. This removes existing
special casing and avoids the need for it in case of new
future uses of WSRs.
For artificial nodes being added for dropped
WeakSerializationReference targets, add them as kArtificial nodes
(not kSnapshot) that has the original offset (element) or
name (property). The replacement is added as a kSnapshot node
that has a negative offset with the same magnitude as the
artificial node (element) or ":real_<property name>" (property).
This simplifies the work done in pkg/vm_snapshot_analysis to
use the artificial nodes instead of replacement ones for
reassembling hierarchies and the like.
This CL also cleans up the old SerializerWritingObjectScope
class, both moving it to Serializer::WritingObjectScope and
allowing nesting of WritingObjectScopes with the correct
semantics.
Thanks to this, not only can we recur when under a WritingObjectScope
instead of lifting recursion outside of those scopes, but we can
also create artificial nodes for non-empty per-code object pools and
static call target tables instead of attributing their contents
as supposed elements of the Code object being written.
TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-mac-release-try,pkg-win-release-try
Change-Id: Ib945c5afcd89b1458b8be3559b6eae24048aba2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194243
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This partially undoes changes from acbd51c08
because they are now known to increate RSS of AOT
compiled applications.
This CL also removes length adjustment code from *String::InstanceSize
which tried to ensure that external string fits into a non-external
string. This code is no longer necessary because APIs for
string externalization were removed.
TEST=covered by the existing tests
Cq-Include-Trybots: luci.dart.try:app-kernel-linux-release-x64-try,app-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: Ife4f16d6939d98bb7966121b5899cef9f0121220
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170080
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Instead of having two similar concrete implementations of WriteText
that differ mostly in the output format, abstract out the parts that
differ and create a single, non-virtual WriteText that calls the
abstracted-out methods as appropriate.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ic728cd4d3376af32780d428efba19cd36e73c3ff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164562
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Before, picking a symbol name for a Code object when generating
assembly required adding a placeholder entry in the Dwarf object,
because the Dwarf object separately used a SnapshotTextObjectNamer
to determine the appropriate symbol name to use in relocations.
Instead, now the ImageWriter subclasses are entirely responsible for
creating the symbol name for a given Code object, and that name is
passed when adding the Code object to the Dwarf object.
For Elf output, we eagerly create symbols for sections and Code objects
again in the appropriate symbol table, and so looking up the start of a
given section or Code object can be done simply by asking for the
corresponding symbol's address, removing special case methods and other
workarounds.
We also clean up the generated static and dynamic symbol tables by
only generating local symbols for Code objects and BSS sections (these
are local in assembly output also), and only adding global symbols to
the dynamic symbol table.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie40c0c836681e98f345483136bb6860e5588ef30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166002
Reviewed-by: Martin Kustermann <kustermann@google.com>
Before, we only wrapped the Instructions payloads in an
InstructionsSection in precompiled snapshots with bare instructions mode
enabled. This CL changes that to always add an InstructionsSection
wrapper in precompiled snapshots, even when not in bare instructions
mode, and moves all the information from the old ImageHeader object into
the InstructionsSection object instead. Note that in non-bare mode, the
InstructionsSection has no payload, and the Instructions objects follow
the InstructionsSection object instead of being contained by it.
This CL also wraps all implementations of AssemblyImageWriter and
SnapshotTextObjectNamer methods in a single DART_PRECOMPILER ifdef
block, since we only generate assembly in precompiled mode. This
removes the need for the smaller DART_PRECOMPILER ifdef blocks scattered
across the AssemblyImageWriter method implementations.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: I3676fb5c6675845463707acbbb4759a465cfa342
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165563
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This CL adds a new PAYLOAD_SIZEOF specification to
runtime_offsets_list.h which defines InstanceSize methods given an
method name to invoke to get the header size (i.e., the size of
the object portion before the payload).
It uses this new specification to create appropriate InstanceSize()
methods for objects written to read-only sections of snapshots,
instead of needing separate size calculations for SIMARM_X64.
It adds more methods to Instructions to avoid special casing for bare
instructions mode. It also removes the special casing for SIMARM_X64,
serializing all read-only objects in the same manner even when not
in a crossword situation.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie3e4009f4bc03688998c32281e42fa22a255731d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165501
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Fixes errors introduced by adding PRINTF_ATTRIBUTE to
BaseWriteStream::Printf due to attempting to print a
compiler::target::uword using the format for uword.
Original description:
For snapshot writing, we generally only have aligned writes so the old
methods worked well, but in some formats (like DWARF) we're not
guaranteed to have aligned writes and so had to use WriteBytes.
Instead, have WriteFixed call WriteBytes and replace any other uses of
WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.
Also makes the following additional changes:
* Expose WriteByte(value), instead of forcing use of
WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.
* Rename Print/VPrint to Printf/VPrintf, so methods taking format
strings are named consistently with others in the codebase.
* Add format string checking to Printf when supported.
* Add AddString() for non-formatted string writing.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Ib70ccaa752dbf9ead5ea57e386a67725723bac84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164324
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 4e2556215a.
Reason for revert: Adding PRINTF_ATTRIBUTE broke more than expected on Mac bots.
Original change's description:
> [vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions.
>
> For snapshot writing, we generally only have aligned writes so the old
> methods worked well, but in some formats (like DWARF) we're not
> guaranteed to have aligned writes and so had to use WriteBytes.
> Instead, have WriteFixed call WriteBytes and replace any other uses of
> WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.
>
> Also makes the following additional changes:
>
> * Expose WriteByte(value), instead of forcing use of
> WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.
>
> * Rename Print/VPrint to Printf/VPrintf, so methods taking format
> strings are named consistently with others in the codebase.
>
> * Add format string checking to Printf when supported.
>
> * Add AddString() for non-formatted string writing.
>
> Change-Id: I5a2e91380bc41b0502e1a576acec786dfe66abbf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164161
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com,sstrickl@google.com
Change-Id: Ie90ac4477f4c581692b37429536ce8dd66788540
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164323
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
For snapshot writing, we generally only have aligned writes so the old
methods worked well, but in some formats (like DWARF) we're not
guaranteed to have aligned writes and so had to use WriteBytes.
Instead, have WriteFixed call WriteBytes and replace any other uses of
WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.
Also makes the following additional changes:
* Expose WriteByte(value), instead of forcing use of
WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.
* Rename Print/VPrint to Printf/VPrintf, so methods taking format
strings are named consistently with others in the codebase.
* Add format string checking to Printf when supported.
* Add AddString() for non-formatted string writing.
Change-Id: I5a2e91380bc41b0502e1a576acec786dfe66abbf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164161
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
All *WriteStream classes are now part of the same hierarchy:
class BaseWriteStream : ValueObject;
Base class for all *WriteStreams. Provides all the methods from
the old WriteStream except for buffer() and SetPosition()
(relegated to NonStreamingWriteStreams). Has one pure virtual
method Realloc that must be overridden by concrete subclasses.
class NonStreamingWriteStream : BaseWriteStream;
Base class for all *WriteStreams where the entire stream is available
at all times (i.e., no flushing to an external sink). Extends the
public BaseWriteStream API with buffer() (for accessing the stream
contents) and SetPosition() (for changing the current stream pointer
to the given absolute position in the stream).
class MallocWriteStream : NonStreamingWriteStream;
Uses realloc to reallocate the internal buffer. Almost the same as
the old WriteStream, except that it only takes an initial size. Adds
one public method Steal() for taking ownership of the current buffer
contents (after which the buffer is reset to an empty state). Instead
of passing a pointer to a buffer, the internal buffer must be accessed
via either Steal() or buffer(), which allows access to the current
stream contents without changing ownership or resetting the stream.
The internal buffer is freed on stream destruction.
class ZoneWriteStream: NonStreamingWriteStream;
Takes a zone and reallocates the internal buffer in that zone. No
additional public methods beyond those available from
NonStreamingWriteStream.
class StreamingWriteStream : BaseWriteStream;
Uses realloc to reallocate the internal buffer. Generally same as
before, where the contents of the stream are periodically flushed
using Dart_StreamingWriteCallback. Since it extends BaseWriteStream,
there are now more methods available for writing data to the stream
than just Print/VPrint/WriteBytes. Since portions of the stream may be
flushed and thus no longer in the internal buffer, does not provide
access to the contents of the stream or a way to reposition the
current stream pointer. Flushes any unflushed data and frees the
internal buffer on stream destruction.
Also refactor things so that write streams are passed to appropriate
recipients, instead of the recipients taking arguments they only used to
create a WriteStream internally. Thus, recipients now can just specify
the appropriate base class for the public API used:
* BaseWriteStream for just writing to the stream, or
* NonStreamingWriteStream if re-positioning or the stream contents are
needed.
Change-Id: I419096ecd9331483d168b079fca55b69ef397f15
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164080
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Since we've run out of room for more fields in the Image object header
on 64-bit architectures, the serializer instead creates an ImageHeader
object for precompiled snapshots that is placed at the start of text
segments. The new ImageHeader object contains the following information:
* The offset of the BSS segment from the text segment, previously
stored in the Image object header.
* The relocated address of the text segment in the dynamic shared
object. Due to restrictions when generating assembly snapshots, this
field is only set for ELF snapshots, and so it can also be used to
detect whether a snapshot was compiled to assembly or ELF.
* The offset of the build ID description field from the text segment.
* The length of the build ID description field.
We replace the BSS offset in the Image object header with the offset of
the ImageHeader object within the text segment, so that we can detect
when a given Image has an ImageHeader object available.
There are no methods available on ImageHeader objects, but instead the
Image itself controls access to the information. In particular, the
relocated address method either returns the relocated address
information from the ImageHeader object or from the initialized BSS
depending on the type of snapshot, so the caller need not do this work.
Also, instead of returning the raw offset to the BSS section and having
the caller turn that into an appropriate pointer, the method for
accessing the BSS segment now returns a pointer to the segment.
Bug: https://github.com/dart-lang/sdk/issues/43274
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try
Change-Id: I15eae4ad0a088260b127f3d07da79374215b7f56
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/163207
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Instead, we just always add a BSS section and we add the ROData
sections for ELF snapshots in BlobImageWriter::WriteText() (similar
to how AssemblyImageWriter::WriteText() handles it for assembly).
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I87ce64037821ce10f2344964e7f379376c4538df
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150937
Reviewed-by: Martin Kustermann <kustermann@google.com>
Previously, we created a separate DWARF object for each possible output.
For the most part, the information stored in each DWARF object was the
same, but the DWARF object was responsible for keeping track of abstract
origin addresses (in the non-assembly case) and for actually generating
the desired output (assembly or binary).
Instead, abstract out the output formatting into a DwarfWriteStream,
whose subclasses handle both the final output format and certain details
that differ between assembly and binary outputs. With this, the DWARF
class no longer depends on having direct access to the ELF file if being
used to generate ELF sections, and as such is output agnostic.
Now, if we have one or more outputs that contain DWARF information, we
only generate a single DWARF object that is shared across these outputs.
The only additional change required is in Dwarf::AddCode(), which now
handles being called multiple times for the same Code object.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I71de887a07a673164b3ccfa7fbbab2ccef80c576
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150302
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF."
Storing the relocated address as an extra field in the Image header,
which requires increasing the Image header size on 64-bit platforms,
means Image pages cannot be used reliably as HeapPages as objects no
longer start after kMaxObjectAlignment bytes.
Instead, we return to an older design that just uses the lowest bit
in the BSS offset to store whether the instructions in an Image were
compiled directly to ELF.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Change-Id: I3819b0dc2719d69f5e8764ca8be8c6ae7171a7bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146560
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This reverts commit 250b84987d.
Reason for revert: Failures on FFI CI tests.
Original change's description:
> [vm] Mark snapshots that were compiled directly to ELF.
>
> Since the virtual addresses in ELF snapshots are the same as in
> separately saved debugging information, print the virtual address in
> non-symbolic frames again when running from a snapshot compiled directly
> to ELF.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
> Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
TBR=vegorov@google.com,rmacnak@google.com,dacoharkes@google.com,sstrickl@google.com
Change-Id: I1c5f58e9d9e11b994c1016970839f95b92885d43
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146220
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
These changes were originally submitted separately on different days,
and a major performance regression was seen after the first change
when creating snapshots that led to both being reverted. However,
that performance regression should be addressed by the followup.
First change:
"[vm] Treat the dispatch table as a root in the snapshot.
Additional changes:
* Only serialize a dispatch table in precompiled snapshots.
* Add information in v8 snapshot profiles for the dispatch table.
* Fix a typo in a field name.
* Print the number of Instructions objects (or payloads, for
precompiled bare instructions mode) in the fake cluster for
the data section.
* Fix v8 snapshots profiles so objects in memory mapped segments
and only those are prefixed with "(RO) ".
* Add names for Instructions objects in v8 snapshot profiles
when we can use the assembly namer.
* Add command line flag for old #define'd false flag."
Second change:
"[vm/aot] Keep GC-visible references to dispatch table Code entries.
This change splits dispatch table handling into four distinct
parts:
* The dispatch table generator does not make a dispatch table
directly, but rather creates an Array that contains the Code
objects for dispatch table entries.
* The precompiler takes this Array and puts it in the object
store, which makes it a new GC root.
* The serializer takes this information and serializes the
dispatch table information in the same form as before.
* The deserializer creates a DispatchTable object and populates
it using the serialized information.
The change in the precompiler ensures that the Code objects
used in the dispatch table have GC-visible references. Thus,
even if all other references to them from the other GC roots
were removed, they would be accessible in the serializer in
the case of a GC pass between the precompiler and serializer.
This change also means that the serializer can retrieve and
trace the Code objects directly rather than first looking up
the Code objects by their entry point."
Bug: https://github.com/dart-lang/sdk/issues/41022
Change-Id: I52c83b0536fc588da0bef9aed1f0c72e8ee4663f
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139285
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Additional changes:
* Only serialize a dispatch table in precompiled snapshots.
* Add information in v8 snapshot profiles for the dispatch table.
* Fix a typo in a field name.
* Print the number of Instructions objects (or payloads, for
precompiled bare instructions mode) in the fake cluster for
the data section.
* Fix v8 snapshots profiles so objects in memory mapped segments
and only those are prefixed with "(RO) ".
* Add names for Instructions objects in v8 snapshot profiles
when we can use the assembly namer.
* Add command line flag for old #define'd false flag.
Change-Id: I1e3be59c7a6312efd76afca560ba465cc17cb22b
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138089
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Now, when writing an AOT snapshot in bare instructions mode, only
the actual instructions in the RawInstructions payload are serialized
instead of the entire RawInstructions object.
Since there are no longer RawInstructions objects in these AOT
snapshots, we also change how Code objects are serialized. Instead
of just containing a reference to the RawInstructions object, we
serialize two pieces of information: where the instructions
payload for this Code object begins and whether there was a single
entry for the instructions payload. (To save space, the single
entry bit is serialized as the low order bit of the unchecked
offset, which was already being serialized).
While we also need the length of the instructions payload, we
approximate it for all but the last Code object by subtracting
the next Code object's payload start from this Code object's
payload start. For the last Code object, we assume it extends
to the end of the instructions image.
Changes on flutter gallery in release mode:
armv7: instructions size -2.70%, total size -1.73%
armv8: instructions size -6.04%, total size -3.63%
Fixes https://github.com/dart-lang/sdk/issues/38451.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ia0a5c4e5e47c956776dc62503da38ec55a143c04
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134325
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 7475c637c3.
Reason for revert: Breakage on simarm_x64 (and thus arm_x64)
Original change's description:
> [vm/aot] Remove object wrapping of bare instructions for AOT snapshots.
>
> Now, when writing an AOT snapshot in bare instructions mode, only
> the actual instructions in the RawInstructions payload are serialized
> instead of the entire RawInstructions object.
>
> Since there are no longer RawInstructions objects in these AOT
> snapshots, we also change how Code objects are serialized. Instead
> of just containing a reference to the RawInstructions object, we
> serialize two pieces of information: where the instructions
> payload for this Code object begins and whether there was a single
> entry for the instructions payload. (To save space, the single
> entry bit is serialized as the low order bit of the unchecked
> offset, which was already being serialized).
>
> While we also need the length of the instructions payload, we
> approximate it for all but the last Code object by subtracting
> the next Code object's payload start from this Code object's
> payload start. For the last Code object, we assume it extends
> to the end of the instructions image.
>
> Changes on flutter gallery in release mode:
> armv7: instructions size -2.66%, total size -1.68%
> armv8: instructions size -5.81%, total size -3.49%
>
> Fixes https://github.com/dart-lang/sdk/issues/38451.
>
> Change-Id: Ia458e8d99bae18f5c3b6e849df2519027f06f574
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131067
> Commit-Queue: Teagan Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com,sstrickl@google.com
Change-Id: If2c8ca6b0993211b2509b275af7ff11a8fa2baa3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-simarm_x64-try, vm-kernel-precomp-android-release-arm64-try, vm-kernel-precomp-android-release-arm_x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134322
Reviewed-by: Teagan Strickland <sstrickl@google.com>
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Now, when writing an AOT snapshot in bare instructions mode, only
the actual instructions in the RawInstructions payload are serialized
instead of the entire RawInstructions object.
Since there are no longer RawInstructions objects in these AOT
snapshots, we also change how Code objects are serialized. Instead
of just containing a reference to the RawInstructions object, we
serialize two pieces of information: where the instructions
payload for this Code object begins and whether there was a single
entry for the instructions payload. (To save space, the single
entry bit is serialized as the low order bit of the unchecked
offset, which was already being serialized).
While we also need the length of the instructions payload, we
approximate it for all but the last Code object by subtracting
the next Code object's payload start from this Code object's
payload start. For the last Code object, we assume it extends
to the end of the instructions image.
Changes on flutter gallery in release mode:
armv7: instructions size -2.66%, total size -1.68%
armv8: instructions size -5.81%, total size -3.49%
Fixes https://github.com/dart-lang/sdk/issues/38451.
Change-Id: Ia458e8d99bae18f5c3b6e849df2519027f06f574
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131067
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Instead of making the ELF object handle stripping and keep track of two
possible output stream, just revert ELF objects to only output to a
single stream. Now, layers that use ELF, like the ImageWriters, handle
the stripping by deciding whether or not to add certain sections or
static symbols to the generated ELF, and we create separate DWARF
objects for each unstripped ELF object.
We also avoid duplication of writing segment and section table entries
by just creating special Sections that correspond to reserved entries,
the program table header entries for itself, etc, and then move the
section and segment entry writing into Section itself. We also lift
a lot of the same calculations used in subclass construction to the
Section level, and replace the old fields with four different kind
of object members:
* Fields that are known at construction time and thus now const
(e.g., section_type)
* Fields that are not known at construction time that have reasonable
defaults for most instances and so need not be changed from their
default value (e.g., section_link)
* Fields that are not known at construction time which must be set once
(and only once) before use. These are now accessed via getters/setters
that check whether or not the field has been set (e.g., section_name)
* Fields that are calculated from the contents of the particular
subclass, which are now just methods (e.g., FileSize())
We also change the snapshot profile writer test to test the size of
internally stripped ELF output on all platforms without running it
through an external stripping utility (previously untested), and test
assembled or externally stripped output on more platforms where we
support those. Instead of depending on pkg/vm/tool/precompiler2 for
assembling (which always uses gcc), I've added assembleSnapshot and
stripSnapshot to use_flag_test_helper.dart, which mimics the way the
test_runner determines the executable and flags to use.
Change-Id: I4ade45c5caffb443abb3f5c567343504c8085f89
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/129083
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Previously when --save-debugging-info=<...> was used when not
compiling to ELF, we generated the requested snapshot format
and the separate debugging information as two separate snapshot
generation passes. However, we want to avoid this, to remove the
assumption that instruction snapshot serialization must be
deterministic.
Fixes https://github.com/dart-lang/sdk/issues/39803.
Also allows --strip to be used with --snapshot-kind=app-aot-assembly,
so that DWARF sections are not output in the generated assembly code.
Fixes https://github.com/dart-lang/sdk/issues/37560.
Change-Id: Ibdf1f5c771bae21870af252b38e64250fde53603
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128735
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Fixes the changes to assembly snapshot writer in SIMARM_X64 mode. This unfortunately can't be tested until the ELF loader is available.
Take 2 is in patchset 1.
Change-Id: Ib8b067dd4f09dcba3b142705e9cb4dfbf4e3eb53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117726
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The original change is in Patchset 1.
The dependency contains the fix for the non-bare-instructions bot.
Change-Id: Iab02ab0736a2ee9144b667d2777ca526341b2a36
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-android-release-arm-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-bare-linux-release-x64-try,vm-kernel-precomp-mac-debug-simarm_x64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/115241
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This reverts commit ecbea5a58e.
Reason for revert: broken with bare instructions and ABI bot
Original change's description:
> [vm/ffi] Implement FFI callbacks on AOT for ELF and Asm snapshots (excl. blobs).
>
> To do this, we add writable data sections (currently uninitialzed) to ELF and Asm snapshots
> and allow Instructions to have patchable relocations against (the start of) these sections.
>
> Issue https://github.com/dart-lang/sdk/issues/37295 (see also for design & discussion).
>
> Change-Id: If20bfa55776f4044aaa6bb8ea2101d2ada41842c
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-android-release-arm-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/110221
> Commit-Queue: Samir Jindel <sjindel@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com,alexmarkov@google.com,sjindel@google.com
Change-Id: I9787da6d42575ca4f5ae0a698052a19ac4275afd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-android-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/115240
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
To do this, we add writable data sections (currently uninitialzed) to ELF and Asm snapshots
and allow Instructions to have patchable relocations against (the start of) these sections.
Issue https://github.com/dart-lang/sdk/issues/37295 (see also for design & discussion).
Change-Id: If20bfa55776f4044aaa6bb8ea2101d2ada41842c
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-android-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/110221
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL adds support to the ImageWriter to write opaque blocks of
trampoline bytes
The AOT code relocator is adapted to allow limited range calls and
inserts trampolines if need be. The algorithm tries to minimize the
number of trampolines added, which for small applications will be 0.
The unconditional pc-relative calls have limited range:
* on ARM (+/-32 MB)
* on ARM64 (+/-128 MB)
To avoid verbose code for doubly-linked list, this CL adds double_list.h
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I0354cf4b2dd58ed5de25d67fc818f0603a2ec501
Reviewed-on: https://dart-review.googlesource.com/c/89283
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Tess Strickland