Fixes the case where the Image name and type was not written
to the v8 snapshot profile in non-precompiled mode.
Adds core-jit snapshot tests to the v8 snapshot profile writer test.
In core-jit and app-aot-elf modes, verify that all bytes in the
snapshot sections are accounted for in the profile.
Bug: https://github.com/dart-lang/sdk/issues/45787
TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try
Change-Id: I88be309cdea6293d768e2fa458492387d70fde22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196323
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This is a reland of 5909fd111d
Does a large refactoring on the V8 snapshot profile writer
to clean things up, add more debugging support, and to fix
the problems that surfaced during the original landing.
Other changes:
Changes Serializer::CreateArtificialNodeIfNeeded() to create
artificial nodes for Code objects and immutable arrays.
Fixes CodeSerializationCluster::Trace() to only push needed parts of
discarded code objects, instead of tracing them like full code objects.
Adds test cases to v8_snapshot_profile_writer_test that exercise
the following situations (both separately and together):
* Non-symbolic stack traces are enabled and code and function objects
are dropped when not needed at runtime.
* Creation of the dispatch table is disabled.
TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test
Original change's description:
> [vm] Fix V8 snapshot profile handling of the dispatch table.
>
> Fixes https://github.com/dart-lang/sdk/issues/45702.
>
> TEST=Tests listed in the issue above.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
> Change-Id: Ibf5e3ccf3828c01f9dda47de360314dabe8cb8a9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195272
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I8e7030267fe190079a8f68d00fe20bf7170e5719
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195513
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
In the codebase, we have several different interfaces for instance
methods that return a hash:
- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;
This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.
TEST=Existing test suites, as this is an internal refactoring.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Instead of special-casing the current fields that may have
WeakSerializationReferences, handle WSRs appearing as elements
or properties of objects more generally. This removes existing
special casing and avoids the need for it in case of new
future uses of WSRs.
For artificial nodes being added for dropped
WeakSerializationReference targets, add them as kArtificial nodes
(not kSnapshot) that has the original offset (element) or
name (property). The replacement is added as a kSnapshot node
that has a negative offset with the same magnitude as the
artificial node (element) or ":real_<property name>" (property).
This simplifies the work done in pkg/vm_snapshot_analysis to
use the artificial nodes instead of replacement ones for
reassembling hierarchies and the like.
This CL also cleans up the old SerializerWritingObjectScope
class, both moving it to Serializer::WritingObjectScope and
allowing nesting of WritingObjectScopes with the correct
semantics.
Thanks to this, not only can we recur when under a WritingObjectScope
instead of lifting recursion outside of those scopes, but we can
also create artificial nodes for non-empty per-code object pools and
static call target tables instead of attributing their contents
as supposed elements of the Code object being written.
TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-mac-release-try,pkg-win-release-try
Change-Id: Ib945c5afcd89b1458b8be3559b6eae24048aba2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194243
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Also includes some example changes using C++14/C++17 features.
TEST=vm/cc/BitField_Assert
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-linux-release-x64-try,dart-sdk-win-try,vm-kernel-win-release-x64-try,vm-kernel-win-release-ia32-try,dart-sdk-mac-try,vm-kernel-mac-release-x64-try
Change-Id: Icf5c5267431daf9cea2e61b67131021557675f3c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192183
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Assembler now always knows whether LR contains return address
or can be safely cloberred.
TEST=tested through normal test suite
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-ffi-android-release-arm64-try,vm-ffi-android-release-arm-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-android-release-arm64-try
Change-Id: I0b436509389cfd29ee413fd28930fb2d6b07616e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174468
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
An address containing an object header is sometimes interpreted as a object slot due to array truncation and concurrent marking/sweeping. Always accessing it as a word assuages TSAN that it won't be subject to tearing, and allows for the removal of TSAN suppressions from header access. In turn, this will allow TSAN to verify some acquire/release header accesses.
TEST=tsan
Bug: https://github.com/dart-lang/sdk/issues/44091
Change-Id: I7bf6449bf7dfeabd9db76e4bb37456e3be2d5124
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172861
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This partially undoes changes from acbd51c08
because they are now known to increate RSS of AOT
compiled applications.
This CL also removes length adjustment code from *String::InstanceSize
which tried to ensure that external string fits into a non-external
string. This code is no longer necessary because APIs for
string externalization were removed.
TEST=covered by the existing tests
Cq-Include-Trybots: luci.dart.try:app-kernel-linux-release-x64-try,app-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: Ife4f16d6939d98bb7966121b5899cef9f0121220
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170080
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Instead of having two similar concrete implementations of WriteText
that differ mostly in the output format, abstract out the parts that
differ and create a single, non-virtual WriteText that calls the
abstracted-out methods as appropriate.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ic728cd4d3376af32780d428efba19cd36e73c3ff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164562
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Before, picking a symbol name for a Code object when generating
assembly required adding a placeholder entry in the Dwarf object,
because the Dwarf object separately used a SnapshotTextObjectNamer
to determine the appropriate symbol name to use in relocations.
Instead, now the ImageWriter subclasses are entirely responsible for
creating the symbol name for a given Code object, and that name is
passed when adding the Code object to the Dwarf object.
For Elf output, we eagerly create symbols for sections and Code objects
again in the appropriate symbol table, and so looking up the start of a
given section or Code object can be done simply by asking for the
corresponding symbol's address, removing special case methods and other
workarounds.
We also clean up the generated static and dynamic symbol tables by
only generating local symbols for Code objects and BSS sections (these
are local in assembly output also), and only adding global symbols to
the dynamic symbol table.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie40c0c836681e98f345483136bb6860e5588ef30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166002
Reviewed-by: Martin Kustermann <kustermann@google.com>
Before, we only wrapped the Instructions payloads in an
InstructionsSection in precompiled snapshots with bare instructions mode
enabled. This CL changes that to always add an InstructionsSection
wrapper in precompiled snapshots, even when not in bare instructions
mode, and moves all the information from the old ImageHeader object into
the InstructionsSection object instead. Note that in non-bare mode, the
InstructionsSection has no payload, and the Instructions objects follow
the InstructionsSection object instead of being contained by it.
This CL also wraps all implementations of AssemblyImageWriter and
SnapshotTextObjectNamer methods in a single DART_PRECOMPILER ifdef
block, since we only generate assembly in precompiled mode. This
removes the need for the smaller DART_PRECOMPILER ifdef blocks scattered
across the AssemblyImageWriter method implementations.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: I3676fb5c6675845463707acbbb4759a465cfa342
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165563
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This CL adds a new PAYLOAD_SIZEOF specification to
runtime_offsets_list.h which defines InstanceSize methods given an
method name to invoke to get the header size (i.e., the size of
the object portion before the payload).
It uses this new specification to create appropriate InstanceSize()
methods for objects written to read-only sections of snapshots,
instead of needing separate size calculations for SIMARM_X64.
It adds more methods to Instructions to avoid special casing for bare
instructions mode. It also removes the special casing for SIMARM_X64,
serializing all read-only objects in the same manner even when not
in a crossword situation.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie3e4009f4bc03688998c32281e42fa22a255731d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165501
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Fixes errors introduced by adding PRINTF_ATTRIBUTE to
BaseWriteStream::Printf due to attempting to print a
compiler::target::uword using the format for uword.
Original description:
For snapshot writing, we generally only have aligned writes so the old
methods worked well, but in some formats (like DWARF) we're not
guaranteed to have aligned writes and so had to use WriteBytes.
Instead, have WriteFixed call WriteBytes and replace any other uses of
WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.
Also makes the following additional changes:
* Expose WriteByte(value), instead of forcing use of
WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.
* Rename Print/VPrint to Printf/VPrintf, so methods taking format
strings are named consistently with others in the codebase.
* Add format string checking to Printf when supported.
* Add AddString() for non-formatted string writing.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Ib70ccaa752dbf9ead5ea57e386a67725723bac84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164324
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 4e2556215a.
Reason for revert: Adding PRINTF_ATTRIBUTE broke more than expected on Mac bots.
Original change's description:
> [vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions.
>
> For snapshot writing, we generally only have aligned writes so the old
> methods worked well, but in some formats (like DWARF) we're not
> guaranteed to have aligned writes and so had to use WriteBytes.
> Instead, have WriteFixed call WriteBytes and replace any other uses of
> WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.
>
> Also makes the following additional changes:
>
> * Expose WriteByte(value), instead of forcing use of
> WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.
>
> * Rename Print/VPrint to Printf/VPrintf, so methods taking format
> strings are named consistently with others in the codebase.
>
> * Add format string checking to Printf when supported.
>
> * Add AddString() for non-formatted string writing.
>
> Change-Id: I5a2e91380bc41b0502e1a576acec786dfe66abbf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164161
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com,sstrickl@google.com
Change-Id: Ie90ac4477f4c581692b37429536ce8dd66788540
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164323
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
For snapshot writing, we generally only have aligned writes so the old
methods worked well, but in some formats (like DWARF) we're not
guaranteed to have aligned writes and so had to use WriteBytes.
Instead, have WriteFixed call WriteBytes and replace any other uses of
WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.
Also makes the following additional changes:
* Expose WriteByte(value), instead of forcing use of
WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.
* Rename Print/VPrint to Printf/VPrintf, so methods taking format
strings are named consistently with others in the codebase.
* Add format string checking to Printf when supported.
* Add AddString() for non-formatted string writing.
Change-Id: I5a2e91380bc41b0502e1a576acec786dfe66abbf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164161
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
All *WriteStream classes are now part of the same hierarchy:
class BaseWriteStream : ValueObject;
Base class for all *WriteStreams. Provides all the methods from
the old WriteStream except for buffer() and SetPosition()
(relegated to NonStreamingWriteStreams). Has one pure virtual
method Realloc that must be overridden by concrete subclasses.
class NonStreamingWriteStream : BaseWriteStream;
Base class for all *WriteStreams where the entire stream is available
at all times (i.e., no flushing to an external sink). Extends the
public BaseWriteStream API with buffer() (for accessing the stream
contents) and SetPosition() (for changing the current stream pointer
to the given absolute position in the stream).
class MallocWriteStream : NonStreamingWriteStream;
Uses realloc to reallocate the internal buffer. Almost the same as
the old WriteStream, except that it only takes an initial size. Adds
one public method Steal() for taking ownership of the current buffer
contents (after which the buffer is reset to an empty state). Instead
of passing a pointer to a buffer, the internal buffer must be accessed
via either Steal() or buffer(), which allows access to the current
stream contents without changing ownership or resetting the stream.
The internal buffer is freed on stream destruction.
class ZoneWriteStream: NonStreamingWriteStream;
Takes a zone and reallocates the internal buffer in that zone. No
additional public methods beyond those available from
NonStreamingWriteStream.
class StreamingWriteStream : BaseWriteStream;
Uses realloc to reallocate the internal buffer. Generally same as
before, where the contents of the stream are periodically flushed
using Dart_StreamingWriteCallback. Since it extends BaseWriteStream,
there are now more methods available for writing data to the stream
than just Print/VPrint/WriteBytes. Since portions of the stream may be
flushed and thus no longer in the internal buffer, does not provide
access to the contents of the stream or a way to reposition the
current stream pointer. Flushes any unflushed data and frees the
internal buffer on stream destruction.
Also refactor things so that write streams are passed to appropriate
recipients, instead of the recipients taking arguments they only used to
create a WriteStream internally. Thus, recipients now can just specify
the appropriate base class for the public API used:
* BaseWriteStream for just writing to the stream, or
* NonStreamingWriteStream if re-positioning or the stream contents are
needed.
Change-Id: I419096ecd9331483d168b079fca55b69ef397f15
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164080
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Since we've run out of room for more fields in the Image object header
on 64-bit architectures, the serializer instead creates an ImageHeader
object for precompiled snapshots that is placed at the start of text
segments. The new ImageHeader object contains the following information:
* The offset of the BSS segment from the text segment, previously
stored in the Image object header.
* The relocated address of the text segment in the dynamic shared
object. Due to restrictions when generating assembly snapshots, this
field is only set for ELF snapshots, and so it can also be used to
detect whether a snapshot was compiled to assembly or ELF.
* The offset of the build ID description field from the text segment.
* The length of the build ID description field.
We replace the BSS offset in the Image object header with the offset of
the ImageHeader object within the text segment, so that we can detect
when a given Image has an ImageHeader object available.
There are no methods available on ImageHeader objects, but instead the
Image itself controls access to the information. In particular, the
relocated address method either returns the relocated address
information from the ImageHeader object or from the initialized BSS
depending on the type of snapshot, so the caller need not do this work.
Also, instead of returning the raw offset to the BSS section and having
the caller turn that into an appropriate pointer, the method for
accessing the BSS segment now returns a pointer to the segment.
Bug: https://github.com/dart-lang/sdk/issues/43274
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try
Change-Id: I15eae4ad0a088260b127f3d07da79374215b7f56
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/163207
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
For proper crashpad integration, we need to generate a build ID, as the
build ID generated by crashpad if there is not one will be a simple XOR
of the first text page, which rarely changes for Dart snapshots.
Assembly snapshots already have a build ID included by the assembler, so
we currently only do this for ELF snapshots.
Currently the build ID is a 128-bit hash value that is four separate
32-bit hash values concatenated together. Those hash values come from
the contents of the VM and isolate .text and .rodata sections.
This change also contains work to separate out the concepts of sections
and segments in the ELF builder. Now, consecutive allocated sections
with the same write and execute flags are combined into a single PT_LOAD
segment when possible, which reduces the padding needed to ensure that
segments start on page boundaries in ELF snapshots.
Bug: https://github.com/dart-lang/sdk/issues/42020
Change-Id: I42a837dae665a3902d881b8d151b49ede87d6c67
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150625
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Instead, we just always add a BSS section and we add the ROData
sections for ELF snapshots in BlobImageWriter::WriteText() (similar
to how AssemblyImageWriter::WriteText() handles it for assembly).
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I87ce64037821ce10f2344964e7f379376c4538df
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150937
Reviewed-by: Martin Kustermann <kustermann@google.com>
Since we no longer add the symbol for a given code object to the Elf
object when writing the contents of the text segment, we don't need to
calculate the relocated address that early. Instead, just store which
text segment the instructions payload belongs to as well as its offset
into the segment contents and let Elf::Finalize() calculate the
relocated address when creating the symbol.
This removes one need for knowing the segment base address when writing
the segment contents. The remaining needs are related to BSS usage:
* Calculating the BSS offset from the text segment for the Image header.
* Fixing up BSS relocations in instructions payloads.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I8be860a7a7111721e6f546154a0814f4d16ac634
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150929
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Previously, we created a separate DWARF object for each possible output.
For the most part, the information stored in each DWARF object was the
same, but the DWARF object was responsible for keeping track of abstract
origin addresses (in the non-assembly case) and for actually generating
the desired output (assembly or binary).
Instead, abstract out the output formatting into a DwarfWriteStream,
whose subclasses handle both the final output format and certain details
that differ between assembly and binary outputs. With this, the DWARF
class no longer depends on having direct access to the ELF file if being
used to generate ELF sections, and as such is output agnostic.
Now, if we have one or more outputs that contain DWARF information, we
only generate a single DWARF object that is shared across these outputs.
The only additional change required is in Dwarf::AddCode(), which now
handles being called multiple times for the same Code object.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I71de887a07a673164b3ccfa7fbbab2ccef80c576
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150302
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This avoids attributing the header and padding bytes in Image objects
to the artificial root, instead making it clear why these bytes exist.
It also ensures Instructions objects (in the non-bare-payload case) are
never considered roots of the profile.
Also unify semantics of text_offset in AssemblyImageWriter::WriteText()
and BlobImageWriter::WriteText(). Originally the former was the offset
from the start of the text section and the latter was the offset from
the start of the instructions objects/payloads. Now they're both the
offset in the text section.
Bug: https://github.com/dart-lang/sdk/issues/41249
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I617e76c9c390a93abb6bc37efc1355368c64691e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149286
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
The relocated addresses are populated during BSS::Initialize() for
natively loaded ELF snapshots, and during the non-native loader when it
is used. Putting this information in the BSS segment avoids having to
change the embedder interface, since we only need this information for
AOT snapshots. This also avoids depending on our ELF snapshot layout
to reverse-engineer the DSO base for ELF-compiled snapshots.
We now always print the DSO base for both the VM and isolate in
non-symbolic stack traces, not just for ELF-compiled snapshots. However,
we still only print the relocated addresses in individual stack frames
if we're guaranteed they match those in separately saved debugging
information.
Bug: https://github.com/dart-lang/sdk/issues/41880
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-mac-release-simarm64-try
Change-Id: I4837262f78e6e73a32eb7e24ef7a68ccb8ec2669
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148441
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
We used Align on the assembly_stream_ directly, which
calls StreamingWriteStream::Align and introduces null
bytes into the stream. When running with -Werror, this
will stop builds.
```
.../ios__app_ios_aot.S:2510:1: error: null character ignored [-Werror,-Wnull-character]
<U+0000><U+0000><U+0000><U+0000><U+0000>_kDartVmSnapshotBss:
```
Instead, we should have used AssemblyImageWriter::Align,
which introduces a .balign instruction instead.
Fixes b/156346855.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-release-simarm64-try
Change-Id: If99f66ccd7eee67f86d654e17a639fad7096dc53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147720
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF."
Storing the relocated address as an extra field in the Image header,
which requires increasing the Image header size on 64-bit platforms,
means Image pages cannot be used reliably as HeapPages as objects no
longer start after kMaxObjectAlignment bytes.
Instead, we return to an older design that just uses the lowest bit
in the BSS offset to store whether the instructions in an Image were
compiled directly to ELF.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Change-Id: I3819b0dc2719d69f5e8764ca8be8c6ae7171a7bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146560
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Previously the output used scrubbed qualified names which hides some
of the information and causes names of various synthetic functions
collide. We change it to use internal qualified names instead and add
additional information for disambiguation:
- For functions this means adding a prefix for various synthetic
methods;
- For anonymous closures we add their token position;
- For types we prefix type parameters with their owner's name.
We also add a unit test covering parsing of
--print-instructions-sizes-to output.
As part of adding a test dart2native was changed to support passing
extra arguments to gen_snapshot binary.
Fixes https://github.com/dart-lang/sdk/issues/41137
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: I299ab0fb5993c8af9b51246483608d1e65970d67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146581
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This reverts commit 250b84987d.
Reason for revert: Failures on FFI CI tests.
Original change's description:
> [vm] Mark snapshots that were compiled directly to ELF.
>
> Since the virtual addresses in ELF snapshots are the same as in
> separately saved debugging information, print the virtual address in
> non-symbolic frames again when running from a snapshot compiled directly
> to ELF.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
> Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
TBR=vegorov@google.com,rmacnak@google.com,dacoharkes@google.com,sstrickl@google.com
Change-Id: I1c5f58e9d9e11b994c1016970839f95b92885d43
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146220
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Symbolic and non-symbolic stack traces had separate ToCString
implementations with the same general structure, but those
implementations diverged enough to cause issues in certain async stack
modes. They are now merged, with the --dwarf-stack-traces-mode flag
checked where appropriate within the merged method.
Also, now non-symbolic stack traces do not include frames for Code with
invisible Function owners unless the flag --show-invisible-frames is
enabled. With this change, pkg/native_stack_traces no longer needs to
guess at which frames corresponding to Dart code should be treated as
internal.
Tested by adding --dwarf-stack-traces version of the tests
in runtime/tests/vm/dart/causal_stacks.
Fixes https://github.com/dart-lang/sdk/issues/41578
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Change-Id: I41a887129616c88acd7729492addf7364d95df33
Bug: 41578
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143816
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Make them form its own source set (libdart_compiler) and completely exclude
them from AOT runtime targets.
Previously we had some inconsistencies, with some files were using
DART_PRECOMPILED_RUNTIME to fully or partially exclude either contents
from their headers or from implementation, while other files did nothing
and relied on linker to throw their contents away.
This change tries to address this inconsistency.
A follow up change would include a check in most compiler headers which
would prohibit to use them while building AOT runtime.
Change-Id: Ief11b11cbc518b301d3e93fce80580a31bbad151
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142993
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
In particular, don't serialize a function if its code is used by the
dispatch table, but there's no need for the function object itself
at runtime like dynamic lookup or dynamic invocation forwarders.
However, just because the owning function is not needed at runtime,
it may be used during serialization for things like assembly label
creation. Thus, instead of just clearing out the Code's owner field,
we add a new WeakSerializationReference (WSR) object whose target is
the original contents of the owner field. The WSR allows the
serializer to access the original owner, but also signals to the
serializer that the target should not be serialized in AOT snapshots
unless there are additional strong (non-WSR) references.
If no strong references are found, then the references to the WSR
in the snapshot are replaced with a reference to a WSR that only
contains the class ID of the original target. The serializer creates
only one WSR per class ID.
If strong references are found, then the target is still serialized.
In this case, the WSR is dropped entirely and any reference to it is
replaced with a direct reference to the serialized target. Thus, WSRs
only exist in the precompiled runtime for targets with no strong
references.
Changes on the Flutter gallery in release mode:
arm7: total size -1.10%, isolate size -6.08%
arm8: total size -1.16%, isolate size -6.09%
Bug: https://github.com/dart-lang/sdk/issues/41052
Change-Id: I5b435ca71ef85f50fe3484789087471a91aa4fe2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137104
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
These changes were originally submitted separately on different days,
and a major performance regression was seen after the first change
when creating snapshots that led to both being reverted. However,
that performance regression should be addressed by the followup.
First change:
"[vm] Treat the dispatch table as a root in the snapshot.
Additional changes:
* Only serialize a dispatch table in precompiled snapshots.
* Add information in v8 snapshot profiles for the dispatch table.
* Fix a typo in a field name.
* Print the number of Instructions objects (or payloads, for
precompiled bare instructions mode) in the fake cluster for
the data section.
* Fix v8 snapshots profiles so objects in memory mapped segments
and only those are prefixed with "(RO) ".
* Add names for Instructions objects in v8 snapshot profiles
when we can use the assembly namer.
* Add command line flag for old #define'd false flag."
Second change:
"[vm/aot] Keep GC-visible references to dispatch table Code entries.
This change splits dispatch table handling into four distinct
parts:
* The dispatch table generator does not make a dispatch table
directly, but rather creates an Array that contains the Code
objects for dispatch table entries.
* The precompiler takes this Array and puts it in the object
store, which makes it a new GC root.
* The serializer takes this information and serializes the
dispatch table information in the same form as before.
* The deserializer creates a DispatchTable object and populates
it using the serialized information.
The change in the precompiler ensures that the Code objects
used in the dispatch table have GC-visible references. Thus,
even if all other references to them from the other GC roots
were removed, they would be accessible in the serializer in
the case of a GC pass between the precompiler and serializer.
This change also means that the serializer can retrieve and
trace the Code objects directly rather than first looking up
the Code objects by their entry point."
Bug: https://github.com/dart-lang/sdk/issues/41022
Change-Id: I52c83b0536fc588da0bef9aed1f0c72e8ee4663f
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139285
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This was already done when generating ELF snapshots, but not when
generating assembly ones. This unifies the approaches so that
the static symbol is created while adding the Code payload instead
of as a separate step.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: I191120728e03fdcd4d8df8a8fd3db00ce9175723
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139202
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Tess Strickland