Commit Graph

58 Commits

Author SHA1 Message Date
Tess Strickland 84a1b114cd [vm] Add base class for shared object writers.
This CL pulls out the refactorings used to support the new MachOWriter
in a followup CL to allow them to be reviewed separately.

Rename Elf -> ElfWriter. Also rename model classes used by ElfWriter
for concepts that exist both in ELF and Mach-O to ElfX. For example,
the old ELF-specific SymbolTable is renamed to ElfSymbolTable.

Adds SharedObjectWriter to serve as a base class for both ElfWriter
and the upcoming MachOWriter.

Adds a new AbstractWriteStream that serves as a common superclass
of both BaseWriteStream and SharedObjectWriter::WriteStream and
allows the creation of fully delegating WriteStreams that do not
maintain a local buffer.

Abstract the old Elf::SymbolData class into
SharedObjectWriter::SymbolData, which stores an enum value as the type
of the symbol instead of storing the ELF encoding of the type.

Rename the DwarfElfStream (which actually wasn't ELF specific, as
all the ELF-specific DWARF information is handled by ElfWriter) to
DwarfSharedObjectStream and put it in a separate header file.

Rename Image::compiled_to_elf() to Image::compiled_to_shared_object()
and add a separate Image::compiled_to_elf() that checks for the ELF
magic value at the DSO base. Also add Image::shared_object_start()
and Image::build_id_start() to return pointers to the DSO base and
the build ID note, respectively.

Refactor Image::build_id() and Image::build_id_length() to check
compiled_for_elf() prior to decoding the data pointed to by
build_id_start() as an ELF note section.

Create an AOTSnapshotType enum to specific the snapshot writer to use
in CreateAppAOTSnapshot instead of using an as_elf boolean.

TEST=refactorings, so existing tests on ci

Change-Id: Ia3ab37a4dff93b6e00390b123753be5a51fbdaaa
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/421301
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2025-05-21 01:36:22 -07:00
Ryan Macnak 2bfecc160b [vm] Update to constexpr in runtime/vm.
TEST=build
Change-Id: I2dd8ae69764af27f480a19995b491e98f52476ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293902
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-04-12 22:18:54 +00:00
Alexander Thomas b9b6511ca6 Spelling sdk
Closes https://github.com/dart-lang/sdk/pull/50918

Co-authored-by: Josh Soref <jsoref@gmail.com>
GitOrigin-RevId: 1fd275051c561b63d374fb47e76a22424c4a12a9
Change-Id: I97790d9c79ff659f2c1fa2d2d46d041fe67957cb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278530
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2023-01-20 12:37:49 +00:00
Josh Soref 0633041729 Spelling runtime vm
Closes https://github.com/dart-lang/sdk/pull/50786

TEST=ci

GitOrigin-RevId: 781a38e2925ba505f0223ecd578faa9be8e1485c
Change-Id: Ia3c615012d1bac3f0cf64dc01c2ceb40d50d1193
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276684
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2022-12-29 10:18:02 +00:00
Ryan Macnak 28c4d87b34 [vm, gc] Extend alignment helpers to work with alignment offsets.
TEST=ci
Change-Id: I02bad1feba3a16360aa52414c94cdef81b158ed6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262000
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2022-10-06 00:50:45 +00:00
Ryan Macnak 72c153b930 [vm] Keep snapshot cursor etc in registers during ReadFill.
Startup.ReadProgramSnapshot(StartupTime): on a Moto x4
  arm32  37510 -> 35212 us (-6.13%)
  arm64c 45080 -> 42634 us (-5.42%)

TEST=ci
Change-Id: Ie97cf8cc1bd48a7ace16baded48d656224c5252b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242921
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-04-28 22:21:40 +00:00
Ryan Macnak 5748add35c [vm] Use a big-endian encoding for snapshot ref ids.
When unconditionally adding in each byte, this causes the last-byte marker to appear in a constant position. The correction factor for each exit branch is the same, allowing it to be folded into subsequent load offsets.

Startup.ReadProgramSnapshot(StartupTime): on a Moto x4
  arm32  40456 -> 37414 us (-7.52%)
  arm64c 46012 -> 44457 us (-3.38%)

TEST=ci
Change-Id: Ie7fd69504f9d9edf0c17458281a3750931d7d97b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242392
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-04-27 16:08:19 +00:00
Ryan Macnak d9bb68f93a [vm] Fix serialization of SIMD vectors.
Broken by bbefc05748.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/46793
Change-Id: I8e5869caee2d705cecaf974cca3ee3f0c542ad3c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208840
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-08-03 19:48:50 +00:00
Ivan Inozemtsev a5ff656da1 Revert "[vm] Switch datastream Write/WriteUnsigned to (S)LEB128."
This reverts commit 80102c981b.

Reason for revert: b/186359854

Original change's description:
> [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
>
> This reduces the number of variable-length integer encodings in our
> database from 2 to 1, and chooses the more standard one.
>
> TEST=Existing tests, in particular any that involve snapshots.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
> Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Idf32bdd879cf8bb7407f6dae764312140ad6eeb2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196920
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
2021-04-26 07:52:47 +00:00
Tess Strickland 80102c981b [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
This reduces the number of variable-length integer encodings in our
database from 2 to 1, and chooses the more standard one.

TEST=Existing tests, in particular any that involve snapshots.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-04-23 07:35:41 +00:00
Tess Strickland 45a46ca2b8 [vm] Add (S)LEB128 encoding/decoding to BaseWriteStream.
Unlike cfc8e6de, this does _not_ replace the default variable length
encoding for {Read,Write}Streams, but insteads adds separate
{Read,Write}{S,}LEB128 methods to the appropriate classes. If we later
find the cause of the issues that led to the revert of cfc8e6de, it'll
be easy to switch over then.

Note that WriteLEB128 asserts that the value is non-negative if used
with a signed type (since negative values suggests that SLEB128 should
be used instead for minimal encoding).

Also removes the various other encoding and decoding methods for
(S)LEB128 across the codebase and changes those clients to use
{Read,Write}Streams instead.

Other cleanups:

* Various constant-related cleanups in datastream.h.

* Adds DART_FORCE_INLINE to ReadStream::ReadByte and uses it in the
  default variable length decoding methods for retrieving bytes
  from the stream instead of managing current_ by hand.

* Creates a canonical empty CompressedStackMaps instance and uses
  that instead of the null CompressedStackMaps instance in most cases.
  The only remaining (expected) use of the null CompressedStackMaps
  instance is for the global table in the object store when no global
  table exists (e.g., in JIT mode before any snapshotting).

* Moves CompressedStackMapsIterator from code_descriptors.h to an
  Iterator class within CompressedStackMaps in object.h (similar to
  PcDescriptors::Iterator), to limit friend declarations and because it
  conceptually makes more sense as part of CompressedStackMaps.

* Removed CompressedStackMaps::PayloadByte, since existing clients
  (CompressedStackMaps::Iterator, StackMapEntry in program_visitor.cc)
  are better served by just operating on the payload buffer directly
  (with appropriate NoSafepointScopes).

* WriteStreams no longer allocate their initial space on construction,
  but rather on the first write, so no allocation is performed by
  constructing a never-used WriteStream.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-ubsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: Ice63321abaa79157fbe9f230a864c8bba0e6dea9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166421
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-10-09 10:08:16 +00:00
Tess Strickland df79bd7c53 [vm] Unify the concrete implementations of ImageWriter::WriteText.
Instead of having two similar concrete implementations of WriteText
that differ mostly in the output format, abstract out the parts that
differ and create a single, non-virtual WriteText that calls the
abstracted-out methods as appropriate.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ic728cd4d3376af32780d428efba19cd36e73c3ff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164562
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-10-06 17:10:05 +00:00
Tess Strickland 72cfc5c638 [vm] Clean up handling of payload-containing objects.
This CL adds a new PAYLOAD_SIZEOF specification to
runtime_offsets_list.h which defines InstanceSize methods given an
method name to invoke to get the header size (i.e., the size of
the object portion before the payload).

It uses this new specification to create appropriate InstanceSize()
methods for objects written to read-only sections of snapshots,
instead of needing separate size calculations for SIMARM_X64.

It adds more methods to Instructions to avoid special casing for bare
instructions mode. It also removes the special casing for SIMARM_X64,
serializing all read-only objects in the same manner even when not
in a crossword situation.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie3e4009f4bc03688998c32281e42fa22a255731d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165501
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-10-06 07:59:33 +00:00
Tess Strickland a36d17730a Reland "[vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions."
Fixes errors introduced by adding PRINTF_ATTRIBUTE to
BaseWriteStream::Printf due to attempting to print a
compiler::target::uword using the format for uword.

Original description:

For snapshot writing, we generally only have aligned writes so the old
methods worked well, but in some formats (like DWARF) we're not
guaranteed to have aligned writes and so had to use WriteBytes.
Instead, have WriteFixed call WriteBytes and replace any other uses of
WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.

Also makes the following additional changes:

* Expose WriteByte(value), instead of forcing use of
  WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.

* Rename Print/VPrint to Printf/VPrintf, so methods taking format
  strings are named consistently with others in the codebase.

* Add format string checking to Printf when supported.

* Add AddString() for non-formatted string writing.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Ib70ccaa752dbf9ead5ea57e386a67725723bac84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164324
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-09-24 16:58:30 +00:00
Tess Strickland 074d6e1309 Revert "[vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions."
This reverts commit 4e2556215a.

Reason for revert: Adding PRINTF_ATTRIBUTE broke more than expected on Mac bots.

Original change's description:
> [vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions.
>
> For snapshot writing, we generally only have aligned writes so the old
> methods worked well, but in some formats (like DWARF) we're not
> guaranteed to have aligned writes and so had to use WriteBytes.
> Instead, have WriteFixed call WriteBytes and replace any other uses of
> WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.
>
> Also makes the following additional changes:
>
> * Expose WriteByte(value), instead of forcing use of
>   WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.
>
> * Rename Print/VPrint to Printf/VPrintf, so methods taking format
>   strings are named consistently with others in the codebase.
>
> * Add format string checking to Printf when supported.
>
> * Add AddString() for non-formatted string writing.
>
> Change-Id: I5a2e91380bc41b0502e1a576acec786dfe66abbf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164161
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,rmacnak@google.com,sstrickl@google.com

Change-Id: Ie90ac4477f4c581692b37429536ce8dd66788540
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164323
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-09-24 13:57:52 +00:00
Tess Strickland 4e2556215a [vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions.
For snapshot writing, we generally only have aligned writes so the old
methods worked well, but in some formats (like DWARF) we're not
guaranteed to have aligned writes and so had to use WriteBytes.
Instead, have WriteFixed call WriteBytes and replace any other uses of
WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.

Also makes the following additional changes:

* Expose WriteByte(value), instead of forcing use of
  WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.

* Rename Print/VPrint to Printf/VPrintf, so methods taking format
  strings are named consistently with others in the codebase.

* Add format string checking to Printf when supported.

* Add AddString() for non-formatted string writing.

Change-Id: I5a2e91380bc41b0502e1a576acec786dfe66abbf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164161
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-09-24 12:45:00 +00:00
Tess Strickland a4a714ebf9 [vm] Consolidate the *WriteStream hierarchy.
All *WriteStream classes are now part of the same hierarchy:

class BaseWriteStream : ValueObject;

  Base class for all *WriteStreams. Provides all the methods from
  the old WriteStream except for buffer() and SetPosition()
  (relegated to NonStreamingWriteStreams). Has one pure virtual
  method Realloc that must be overridden by concrete subclasses.

class NonStreamingWriteStream : BaseWriteStream;

  Base class for all *WriteStreams where the entire stream is available
  at all times (i.e., no flushing to an external sink). Extends the
  public BaseWriteStream API with buffer() (for accessing the stream
  contents) and SetPosition() (for changing the current stream pointer
  to the given absolute position in the stream).

class MallocWriteStream : NonStreamingWriteStream;

  Uses realloc to reallocate the internal buffer.  Almost the same as
  the old WriteStream, except that it only takes an initial size.  Adds
  one public method Steal() for taking ownership of the current buffer
  contents (after which the buffer is reset to an empty state). Instead
  of passing a pointer to a buffer, the internal buffer must be accessed
  via either Steal() or buffer(), which allows access to the current
  stream contents without changing ownership or resetting the stream.
  The internal buffer is freed on stream destruction.

class ZoneWriteStream: NonStreamingWriteStream;

  Takes a zone and reallocates the internal buffer in that zone. No
  additional public methods beyond those available from
  NonStreamingWriteStream.

class StreamingWriteStream : BaseWriteStream;

  Uses realloc to reallocate the internal buffer.  Generally same as
  before, where the contents of the stream are periodically flushed
  using Dart_StreamingWriteCallback. Since it extends BaseWriteStream,
  there are now more methods available for writing data to the stream
  than just Print/VPrint/WriteBytes. Since portions of the stream may be
  flushed and thus no longer in the internal buffer, does not provide
  access to the contents of the stream or a way to reposition the
  current stream pointer. Flushes any unflushed data and frees the
  internal buffer on stream destruction.

Also refactor things so that write streams are passed to appropriate
recipients, instead of the recipients taking arguments they only used to
create a WriteStream internally. Thus, recipients now can just specify
the appropriate base class for the public API used:

* BaseWriteStream for just writing to the stream, or
* NonStreamingWriteStream if re-positioning or the stream contents are
  needed.

Change-Id: I419096ecd9331483d168b079fca55b69ef397f15
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164080
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-09-23 21:05:59 +00:00
Tess Strickland 33e4a6bc2a Revert "[vm] Have Read/WriteStream use (S)LEB128 for variable-length encoding."
This reverts commit cfc8e6de3f.

Reason for revert: b/167520550

Original change's description:
> [vm] Have Read/WriteStream use (S)LEB128 for variable-length encoding.
>
> Previously, the non-fixed read and write methods of ReadStream and
> WriteStream used a similar but slightly different encoding to LEB128 for
> unsigned values and SLEB128 for signed values. The only difference was
> the high continuation marker bit, which was unset in non-final parts and
> set in final parts in the previous encoding, but is set for non-final
> parts and unset in final parts in (S)LEB128.
>
> This CL changes it so that they instead use the standard LEB128 and
> SLEB128 encodings. Among other things, this means that one-byte
> encodings of values in snapshots will be the value itself, not the value
> with the high bit set. This also means that when outputting specific
> formats that use (S)LEB128, like DWARF, the format writers can just use
> WriteStream::Write(Unsigned) directly instead of needing to write a
> separate encoder.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-ubsan-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
> Change-Id: I409a22e7da9e4672011dfaccd3790c122ec3e522
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150526
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=kustermann@google.com,rmacnak@google.com,sstrickl@google.com

Change-Id: Ie5b9e595e4dbe9bbed5f0dd5bdc6f78bd1233f5d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161540
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2020-09-02 14:17:53 +00:00
Tess Strickland cfc8e6de3f [vm] Have Read/WriteStream use (S)LEB128 for variable-length encoding.
Previously, the non-fixed read and write methods of ReadStream and
WriteStream used a similar but slightly different encoding to LEB128 for
unsigned values and SLEB128 for signed values. The only difference was
the high continuation marker bit, which was unset in non-final parts and
set in final parts in the previous encoding, but is set for non-final
parts and unset in final parts in (S)LEB128.

This CL changes it so that they instead use the standard LEB128 and
SLEB128 encodings. Among other things, this means that one-byte
encodings of values in snapshots will be the value itself, not the value
with the high bit set. This also means that when outputting specific
formats that use (S)LEB128, like DWARF, the format writers can just use
WriteStream::Write(Unsigned) directly instead of needing to write a
separate encoder.

Cq-Include-Trybots: luci.dart.try:vm-kernel-ubsan-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I409a22e7da9e4672011dfaccd3790c122ec3e522
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150526
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-09-02 01:16:21 +00:00
Ryan Macnak 40d876382a [vm] Fix various UBSan failures: AOT.
Bug: https://github.com/dart-lang/sdk/issues/39427
Change-Id: I2594e28a277f1d9f838aa93d546714d4f300b5b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147682
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-05-12 20:28:11 +00:00
Ryan Macnak 2707880f1b [vm] Fix various UBSan failures.
Bug: https://github.com/dart-lang/sdk/issues/39427
Change-Id: I74e0eee623d88005fb2893d03e284a87daa09260
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146696
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-05-07 19:40:18 +00:00
Ryan Macnak 60493d58fb [vm] Remove undefined shifts from ReadStream.
Change-Id: I3570739b77d431e412af94c87fa544805f6dbbcb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146028
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-05-04 22:53:47 +00:00
Victor Lima b68d95ec9e [vm] Reland support for real unboxed floating point fields in AOT
Issue https://github.com/dart-lang/sdk/issues/40404

Change-Id: Icfa801ff0640a6b27bb3c13d0b737c40452cbf7d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/133983
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-01-31 12:56:31 +00:00
Zichang Guo 162d6c5634 Revert "[vm] Add support for real unboxed floating point fields in AOT"
This reverts commit 9eb531bde4.

Reason for revert: Bots are red. Some tests are failing.

https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-obfuscate-linux-release-x64/6039

https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-android-release-arm_x64/957

Original change's description:
> [vm] Add support for real unboxed floating point fields in AOT
> 
> Non-nullable floating point fields (double, Float32x4, Float64x2)
> are fully unboxed in their classes.
> 
> A bitmap for each class was added to the shared class table in order to keep
> track of the pointers of the classes. Since all classes in Flutter Gallery
> have less than 64 fields, the bitmap is represented by a 64 bit integer and
> fields whose offset is more than 64 words are not unboxed.
> 
> The instance sizes and field offsets might change between target and host
> in cross-compilation, since the number of words used to store unboxed fields
> may differ.
> 
> dart-aot Xeon
> 
>   SplayLatency               -4.62%
>   SplayHarderLatency         -4.17%
>   NavierStokes               -2.20%
>   Tracer                      8.72%
>   ParticleSystemPaint         2.90%
>   NBodySIMD                   8.35%
>   NBody                      25.59%
> 
> With hack TFA to make doubles in Rect/Offset/Size classes in flutter non-nullable:
> 
> flutter arm-v8:
> 
>   gallery total size: -1%
> 
>   matrix_utils_transform_rect_perspective   -16.70% (less is better)
>   matrix_utils_transform_rect_affine        -31.82% (less is better)
>   matrix_utils_transform_point_perspective  -24.90% (less is better)
>   matrix_utils_transform_point_affine)      -27.26% (less is better)
>   rrect_contains_bench                      -4.719% (less is better)
> 
> Change-Id: I9ae09c9c3167d99f9efd071a92937aa51093fd1d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131824
> Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Samir Jindel <sjindel@google.com>

TBR=kustermann@google.com,rmacnak@google.com,sjindel@google.com,victoragnez@google.com

Change-Id: Ic73858f6adb7f55c4129d4f46ff4731b378cb634
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134020
Reviewed-by: Zichang Guo <zichangguo@google.com>
Commit-Queue: Zichang Guo <zichangguo@google.com>
2020-01-30 18:13:28 +00:00
Victor Lima 9eb531bde4 [vm] Add support for real unboxed floating point fields in AOT
Non-nullable floating point fields (double, Float32x4, Float64x2)
are fully unboxed in their classes.

A bitmap for each class was added to the shared class table in order to keep
track of the pointers of the classes. Since all classes in Flutter Gallery
have less than 64 fields, the bitmap is represented by a 64 bit integer and
fields whose offset is more than 64 words are not unboxed.

The instance sizes and field offsets might change between target and host
in cross-compilation, since the number of words used to store unboxed fields
may differ.

dart-aot Xeon

  SplayLatency               -4.62%
  SplayHarderLatency         -4.17%
  NavierStokes               -2.20%
  Tracer                      8.72%
  ParticleSystemPaint         2.90%
  NBodySIMD                   8.35%
  NBody                      25.59%

With hack TFA to make doubles in Rect/Offset/Size classes in flutter non-nullable:

flutter arm-v8:

  gallery total size: -1%

  matrix_utils_transform_rect_perspective   -16.70% (less is better)
  matrix_utils_transform_rect_affine        -31.82% (less is better)
  matrix_utils_transform_point_perspective  -24.90% (less is better)
  matrix_utils_transform_point_affine)      -27.26% (less is better)
  rrect_contains_bench                      -4.719% (less is better)

Change-Id: I9ae09c9c3167d99f9efd071a92937aa51093fd1d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131824
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2020-01-30 13:45:08 +00:00
Ryan Macnak 36985859e4 [vm] Don't copy when reading typed data in messages sent to native ports.
Bug: https://github.com/dart-lang/sdk/issues/37833
Change-Id: Iac906e58320e003365c4c0624b4587579f352db1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114970
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2019-08-30 17:49:28 +00:00
Liam Appelbe a6eb2cd309 [vm] Support for simarm_x64
Third sub-CL of https://dart-review.googlesource.com/c/sdk/+/100644
The first was here https://dart-review.googlesource.com/c/sdk/+/103487
The second was here https://dart-review.googlesource.com/c/sdk/+/104286

This is the last major CL to support simarm_x64. Now that the other two
CLs have put the required infrastructure in place, this CL actually
adds the new mode..

Bug: https://github.com/dart-lang/sdk/issues/36839
Change-Id: Ie2f850bf33544f2462f37854e762191c78b1bde6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105500
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-06-21 04:00:35 +00:00
Ryan Macnak 8da53e3120 [vm] Add Dart_SaveTypeFeedback and Dart_LoadTypeFeedback.
Change-Id: I8c20271b20638972b28299f0488e7f3dd37f8797
Reviewed-on: https://dart-review.googlesource.com/c/88604
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2019-01-10 01:04:11 +00:00
Alexander Markov a4dc8cd22e [vm/kernel/bytecode] Make sure bytecode instructions are aligned in kernel
Change-Id: I109123e88b8352b198d5ef9aad9eedc1df7141c8
Reviewed-on: https://dart-review.googlesource.com/c/83543
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-11-09 17:08:46 +00:00
Ryan Macnak 22ccad1aa6 [vm] Stream writing snapshots as assembly to reduce peak memory usage.
Bug: US-449
Change-Id: Ic1ce6881b54c8c0eb7c93cbbda8057d7268bb3fd
Reviewed-on: https://dart-review.googlesource.com/45580
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2018-03-08 19:26:17 +00:00
Ryan Macnak 558a931409 [vm] Move some string utils from runtime/vm to runtime/platform.
Change-Id: I552086445a7e07792f9da85afa5edf23b6c450d5
Reviewed-on: https://dart-review.googlesource.com/44402
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-03-01 02:07:46 +00:00
Ryan Macnak d366f9619a [vm] Toward deterministic builds.
- Remove random build-id.
 - Replace build time in embedded version string with commit time.
 - Remove timestamps from Observatory tarball.
 - Zero-initialize skipped bytes in snapshot streams.
 - Fix uninitialized fields in PatchClass, Script and Library.
 - Disable (under flag) random identity hashes and concurrent GC.

Bug: https://github.com/dart-lang/sdk/issues/31427
Change-Id: I3e95de679c8372841cd27ca60df78d9b00ffbfe1
Reviewed-on: https://dart-review.googlesource.com/22901
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2017-11-23 00:07:56 +00:00
Zachary Anderson 6cd8a79078 VM: Re-format to use at most one newline between functions
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2974233002 .
2017-07-13 08:08:37 -07:00
Ryan Macnak b09552cd62 Reapply "DWARF and unwind support for AOT assembly output."
This reverts commit 8fcdbb4a7f.

R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2761593002 .
2017-03-20 13:02:41 -07:00
Erik Ernst 8fcdbb4a7f Revert "DWARF and unwind support for AOT assembly output."
Gardening.

This reverts commit ca20aab384.

That commit causes generation of incorrect register names on
vm-noopt-simarm64-mac-be, and it seems to cause a disk full
error later on (making the bot purple).

Review-Url: https://codereview.chromium.org/2754233002 .
2017-03-17 12:57:45 +01:00
Ryan Macnak ca20aab384 DWARF and unwind support for AOT assembly output.
Emit CFI information to teach gdb how to unwind Dart frames. Fixes unwinding for Mac x64 and Linux x64/ARM/ARM64. Unwinding already worked for iOS ARM64. Fixes unwinding on Android 6+ ARM.

Emit DWARF information about function inlining and line number mappings. Works with gdb, addr2line and llvm-symbolizer.

With --dwarf-stack-traces, also remove CodeSourceMaps from snapshots and change stack traces to a non-compliant form resembling Android's debuggerd output, which can be passed to ndk-stack.

Issue #28785

R=fschneider@google.com, johnmccutchan@google.com, vegorov@google.com

Review-Url: https://codereview.chromium.org/2723213002 .
2017-03-16 13:08:54 -07:00
Ryan Macnak 909ce02b12 Encode inlining information in CodeSourceMap and remove inlining interval arrays.
CodeSourceMap now contains a small bytecode that describes changes to the token position and inlining stack as one moves through a function. This information is used by the disassembler to display inlining comments and by the profiler to properly blame ticks against source code.

Slightly shrinks the total size of code metadata; on dart2js compiling hello world, the final heap is

 before 59.6MB total, 12.8MB List, 1.1MB CodeSourceMap
  after 59.4MB total, 12.1MB List, 1.5MB CodeSourceMap

To be filtered and included in AOT code to expand inline frames in stack traces.

R=johnmccutchan@google.com

Review-Url: https://codereview.chromium.org/2670843006 .
2017-02-07 12:52:21 -08:00
Martin Kustermann 89de0c9267 Reland "VM: Fix memory leaks during isolate spawning"
R=vegorov@google.com

Review-Url: https://codereview.chromium.org/2649283003 .
2017-01-24 12:04:41 +01:00
Martin Kustermann 50150deaf6 Revert "VM: Fix memory leaks during isolate spawning"
This reverts commit d7e884b05f.

Review-Url: https://codereview.chromium.org/2654443003 .
2017-01-24 10:32:59 +01:00
Martin Kustermann d7e884b05f VM: Fix memory leaks during isolate spawning
Fixes #28351

R=rmacnak@google.com

Review-Url: https://codereview.chromium.org/2650693004 .
2017-01-24 09:59:13 +01:00
Ryan Macnak b46af1e75f Refactor snapshots pieces to include a section for loading instructions into the heap of a regular isolate.
Progress toward allowing each isolate to load a different snapshot.

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2622053002 .
2017-01-23 10:25:02 -08:00
Ryan Macnak 2ec03cba70 Fix leak of message snapshot buffer when attempting to send an illegal object.
Fix leak with FLAG_enable_debug_break.

R=asiva@google.com, fschneider@google.com

Review-Url: https://codereview.chromium.org/2629533002 .
2017-01-11 16:50:04 -08:00
Zachary Anderson a1bcf051d8 clang-format runtime/vm
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2481873005 .
2016-11-08 13:54:47 -08:00
Zachary Anderson 103881d01c Make header include guards great again
i.e. #ifndef VM_WHATEVER -> #ifndef RUNTIME_VM_WHATEVER

This lets us remove a hack from the PRESUBMIT.py script that existed
for reasons that are no longer valid, and sets us up to add some
presubmit checks for the GN build.

R=asiva@google.com, rmacnak@google.com

Review URL: https://codereview.chromium.org/2450713004 .
2016-10-26 00:26:03 -07:00
Ryan Macnak f262fad639 Option to output precompiled instructions as a blob for use with mmap instead of assembly for use in a shared library.
BUG=http://dartbug.com/26199
R=fschneider@google.com

Review URL: https://codereview.chromium.org/1915853004 .
2016-04-25 16:19:34 -07:00
Ryan Macnak 83585cfbf1 Emit assembly instead of a blob for the instructions snapshot, and provide labels for the entry point of each Instructions.
R=asiva@google.com

Review URL: https://codereview.chromium.org//1309563006 .
2015-09-04 15:15:50 -07:00
asiva@google.com 6a41d797cb Change snapshot versioning code to use plain strings instead of Dart strings
- This ensures that the versioning itself is not dependent on the Dart object
  layout
- It should also fix the strange performance regression we noticed in tracer
  because of the allocation of an initial version string in the old space.

R=srdjan@google.com

Review URL: https://codereview.chromium.org//522893002

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@39708 260f80e4-7a28-3924-810f-c04153c831b5
2014-08-29 20:39:21 +00:00
zra@google.com 0ef376e2bd Changes to improve the performance of reading a snapshot.
There are two changes:
1. The decoding loop in ReadStream is unrolled.
2. Small reorganizaiton of PageSpace::TryAllocate to avoid
   copying SpaceUsuage structs around.

These changes improve performance of CorelibIsolateStartup
~10-12% on ia32.

R=iposva@google.com

Review URL: https://codereview.chromium.org//313393003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@37141 260f80e4-7a28-3924-810f-c04153c831b5
2014-06-09 22:16:06 +00:00
iposva@google.com 781f737c6a First round of http://dartbug.com/15922:
- Address warnings about 64-bit to 32-bit conversions.
- Remove heap profiler.

R=asiva@google.com

Review URL: https://codereview.chromium.org//139043003

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@31867 260f80e4-7a28-3924-810f-c04153c831b5
2014-01-16 05:05:35 +00:00
asiva@google.com fbc931e6a6 Report OOM errors instead of asserting on allocation failures when sending
messages to other isolates.

Review URL: https://codereview.chromium.org//14273021

git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@21980 260f80e4-7a28-3924-810f-c04153c831b5
2013-04-24 20:52:22 +00:00