Commit Graph

91 Commits

Author SHA1 Message Date
Alexander Markov f7790bb65b [gardening] Fix gcc and msvc builds
Fix the following compile-time errors:

C:\b\s\w\ir\cache\builder\sdk\runtime\vm\elf.cc(601) : error C2220: the following warning is treated as an error
C:\b\s\w\ir\cache\builder\sdk\runtime\vm\elf.cc(601) : warning C4715: 'dart::ElfSymbolTable::ElfSymbolType': not all control paths return a value

../../runtime/vm/elf.cc: In static member function ‘static intptr_t dart::ElfSymbolTable::ElfSymbolType(dart::SharedObjectWriter::SymbolData::Type)’:
../../runtime/vm/elf.cc:601:3: error: control reaches end of non-void function [-Werror=return-type]
  601 |   }
      |   ^

TEST=ci

Change-Id: Ib06339405b406ea4a6f6d395a04a16573d92c704
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430321
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2025-05-22 07:50:49 -07:00
Tess Strickland 84a1b114cd [vm] Add base class for shared object writers.
This CL pulls out the refactorings used to support the new MachOWriter
in a followup CL to allow them to be reviewed separately.

Rename Elf -> ElfWriter. Also rename model classes used by ElfWriter
for concepts that exist both in ELF and Mach-O to ElfX. For example,
the old ELF-specific SymbolTable is renamed to ElfSymbolTable.

Adds SharedObjectWriter to serve as a base class for both ElfWriter
and the upcoming MachOWriter.

Adds a new AbstractWriteStream that serves as a common superclass
of both BaseWriteStream and SharedObjectWriter::WriteStream and
allows the creation of fully delegating WriteStreams that do not
maintain a local buffer.

Abstract the old Elf::SymbolData class into
SharedObjectWriter::SymbolData, which stores an enum value as the type
of the symbol instead of storing the ELF encoding of the type.

Rename the DwarfElfStream (which actually wasn't ELF specific, as
all the ELF-specific DWARF information is handled by ElfWriter) to
DwarfSharedObjectStream and put it in a separate header file.

Rename Image::compiled_to_elf() to Image::compiled_to_shared_object()
and add a separate Image::compiled_to_elf() that checks for the ELF
magic value at the DSO base. Also add Image::shared_object_start()
and Image::build_id_start() to return pointers to the DSO base and
the build ID note, respectively.

Refactor Image::build_id() and Image::build_id_length() to check
compiled_for_elf() prior to decoding the data pointed to by
build_id_start() as an ELF note section.

Create an AOTSnapshotType enum to specific the snapshot writer to use
in CreateAppAOTSnapshot instead of using an as_elf boolean.

TEST=refactorings, so existing tests on ci

Change-Id: Ia3ab37a4dff93b6e00390b123753be5a51fbdaaa
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/421301
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2025-05-21 01:36:22 -07:00
Ryan Macnak 57ee563bc9 Roll Clang from 0454dd8c48cd to 8c7a2ce01a77
TEST=ci
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try;luci.dart.try:dart-sdk-mac-try;luci.dart.try:dart-sdk-mac-arm64-try;luci.dart.try:dart-sdk-win-try
Change-Id: I9d4e7abe9541f74a67e469c4ba4ba3cecfaec431
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/427280
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2025-05-07 09:22:20 -07:00
Ivan Inozemtsev a5077aa352 Reland "Add gen_snapshot binaries, producing Linux ARM64/x64 snapshots"
This is a reland of commit e6d55c6c2f

Reworks guards for 64-bit Windows unwinding support code so that
it is not included in any 64-bit Windows gen_snapshot executable that
targets a non-64-bit Windows platform.

TEST=ci, build only

Original change's description:
> Add gen_snapshot binaries, producing Linux ARM64/x64 snapshots
>
> Inspired by `//runtime/bin:gen_snapshot_fuchsia`, required for go/dart-cross-compile.
>
> TEST=ci, build only
>
> Change-Id: Ie521c984a8f44d25f3f78d946af9416582a572dc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/410402
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>

Change-Id: I4f1323ac8f7ab215c2dffeef188f70c466fb62ea
Cq-Include-Trybots: luci.dart.try:vm-win-release-arm64-try,vm-win-release-ia32-try,vm-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/413402
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
2025-03-13 07:23:39 -07:00
Ryan Macnak 4924cf9dab [vm] Fill in ELF attributes for RISC-V.
TEST=readelf
Change-Id: Ida1069f12998c8d12750d23843a7f964bf618077
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/396100
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2024-11-19 17:49:09 +00:00
Vyacheslav Egorov dfccc4a268 [vm/aot] Fix debug-info generation
The algorithm for ordering sections in `CreateProgramTable` permitted
skew between the order of sections in snapshot and corresponding
debug info. This skew can lead to inconsistencies in `.text` section
memory offset between two generated ELF files making debug info
invalid.

This skew was left unnoticed for a long time because executable
segment has a huge alignment requirement (64K) while the effect
of the skew was usually rather small and thus hidden by realignment.

This CL adds a strict verification pass (always enabled) which
guards against such skew in the future.

A better architecture would be to ditch parallel generation of
snapshot and corresponding ELF files altogether and other implement
splitting as a post processing path over generated ELF or
rely on external tooling. But this is a much bigger change
than I am willing to invest time into right now.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try
Change-Id: I0881783237faa6534eb3c076b6cae472bbdc7243
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384742
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2024-09-11 12:51:23 +00:00
Tess Strickland b1b3e34b88 [vm] Add non-symbolic stack trace support for deferred loading units.
Dart VM changes:

Note that the following changes are backwards compatible in the
case that a Dart program has no deferred loading units (i.e., the
Dart program is contained in a single shared object snapshot).

When there are non-root loading units, the non-symbol stack trace
header now includes information about loading units as follows:

loading_unit: N, build_id: S, dso_base: A, instructions: A

where N is an integer, S is a string of hex digits (0-9a-f), and A
is a word-sized address printed as a hex string (without prefix).

In addition, all non-symbolic stack frames for isolate instructions
include a unit field, including those for the root loading unit, e.g.,

   #NN abs <address> unit <id> virt <address> <symbol>+<offset>

If there are no non-root loading units, then the non-symbolic stack
trace is unchanged from its previous format.

Adds a build ID to split deferred loading unit snapshots.
Fixes: https://github.com/dart-lang/sdk/issues/43516

If separate debugging information is requested, the loading unit
manifest includes a 'debugPath' field for each loading unit,
which contains the path to its separate debugging information.

Removes the attempt to store the relocated address of the instructions
section when running from an assembled snapshot in the initialized BSS.

Adds OS::GetAppDSOBase, which takes a pointer to the instructions
section and returns a pointer to its loaded shared object in memory.
For compiled-to-ELF snapshots, it does this using the relocated address
of the instructions in the Image, and for assembled snapshots, it
delegates to NativeSymbolResolver::LookupSharedObject.

-----

Changes to package:native_stack_traces:

PCOffset now has two new fields:

* int? unitId: the unit ID of the loading unit, when available.
* String? buildId: the build ID of the loading unit, when available.

For PCOffsets in the VM section, the unitId and buildId are those of
the root loading unit.

The constructor for the DwarfStackTraceDecoder now takes two
additional optional named arguments:

* Map<int, Dwarf>? dwarfByUnitId: A map associating loading unit IDs
  with the appropriate Dwarf object. May or may not contain an entry
  for the root loading unit.
* Iterable<Dwarf>? unitDwarfs: An iterable container holding Dwarf
  objects. May or may not contain an entry for the root loading unit.

The Dwarf object that is passed to the DwarfStackTraceDecoder as a
positional argument is used for all lookups within the root loading
unit. If the dwarfByUnitId or unitDwarfs arguments contain an entry
for the root loading unit, it should be the same as the positional
argument.

When decoding a non-symbolic stack frame with a non-root loading unit
id, the decoder first looks in the map for the appropriate Dwarf object.
If one is not found, the decoder uses the build ID for the loading unit
to find the appropriate Dwarf object in the iterable container. If an
appropriate Dwarf object cannot be found in either manner, the
non-symbolic stack frame is emitted without change.

The native_stack_traces:decode executable now takes two additional
multi-options for the translate command:

* -u, --unit_debug: Takes a path to the associated DWARF information.
* --unit_id_debug: Takes N=FILE, where N is the loading unit ID and
  FILE is a path to the associated DWARF information.

The arguments to -u are collected into an iterable container to be
passed as the unitDwarfs argument to the DwarfStackTraceDecoder, and
the arguments to --unit-id-debug are collected into a map to be passed
as the dwarfByUnitId argument.

TEST=vm/dart/use_dwarf_stack_traces_flag_deferred

Issue: https://github.com/dart-lang/sdk/issues/53902
Change-Id: I210d4f69e4ae9fd37275a96beb1aac55c5e9d080
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-product-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2024-04-19 08:10:18 +00:00
Daco Harkes f98a2138b7 [vm] Run clang-format on code base
When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.

However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.

This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
  whole file, or (2) the diff lines overlap.

`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.

This CL formats the runtime to be in line with the current pinned
`clang-format`:

```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```

`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)

TEST=A variety of bots including GCC, MacOS and Windows.

Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2024-04-17 19:14:41 +00:00
Ryan Macnak fc85488000 [vm] Unwind records for Windows ARM64.
TEST=ffi_induce_a_crash_test
Bug: https://github.com/dart-lang/sdk/issues/51726
Change-Id: I7df3b56a150434d4c7b0cfbadda4fd9d57606eef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336203
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-11-21 20:39:29 +00:00
Liam Appelbe 07f587504b Reland "[vm] Migrate FFI callbacks to the new metadata system."
This reverts https://dart-review.googlesource.com/c/sdk/+/306674

Patchset 1 is a pure rollback of the rollback.
Patchset 2 is https://dart-review.googlesource.com/c/sdk/+/306316
Patchset 4+ is the forward fix for the Fuchsia issues.

The Fuchsia bug that we're fixing (or working around), is that
VirtualMemory::DuplicateRX doesn't work on Fuchsia. A proper fix will
require special casing it, like on MacOS. In the mean time we can avoid
using this function by only allowing one page of trampolines on Fuchsia.
Unfortunately, when I removed the BSS stuff from the original CL, it
was necessary to duplicate even the first page, so I've had to add that
stuff back just for Fuchsia.

Change-Id: Id42de78ee5de126bcc83bfa4148f6efb4045f976
Bug: https://github.com/dart-lang/sdk/issues/52579
Bug: https://buganizer.corp.google.com/issues/284959841
Fixes: https://github.com/dart-lang/sdk/issues/52581
TEST=CI, especially vm-fuchsia-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306676
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-06-06 02:07:58 +00:00
Liam Appelbe 8ea3e9fd0d Revert "[vm] Migrate FFI callbacks to the new metadata system."
Roll back https://dart-review.googlesource.com/c/sdk/+/302903
and https://dart-review.googlesource.com/c/sdk/+/306124

Since the bug is in VirtualMemory::DuplicateRX, also roll back
https://dart-review.googlesource.com/c/sdk/+/303960
https://dart-review.googlesource.com/c/sdk/+/304400
https://dart-review.googlesource.com/c/sdk/+/304620

Bug: https://buganizer.corp.google.com/issues/284959841
Change-Id: Id212de209fb57b2c3395bb61064dc8f569832884
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306674
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-06-01 01:00:58 +00:00
Liam Appelbe f0f732b1a6 [vm] Migrate FFI callbacks to the new metadata system.
This is the same metadata system that will support async callbacks. The
main thing this does is take the trampolines that used to be JIT only,
and use them in AOT too. This is partly a safety thing (there's some
extra checks that used to be skipped on AOT), but mostly just so that
the metadata system is unified between the sync and async callbacks.

More details about the design:
https://docs.google.com/document/d/1QDjyY_6wOTOgURwpeYMKU9qEz0gKxx2MUrdruC6Kp6c/edit?usp=sharing

I split this off the async CL. Some of the comments refer to async
stuff that doesn't exist yet, but it's coming immediately after this so
I didn't update them.

Change-Id: Icd5e86934ee9ae34c2c0e2ed2bbd1b928a7184ac
TEST=ffi_callback_metadata_test.cc and CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302903
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-05-24 22:22:04 +00:00
Alexander Aprelev 3a668dcf03 [vm/win/aot] Provide unwinding information for Windows AOT snapshots.
Fixes https://github.com/dart-lang/sdk/issues/52045
TEST=ffi_induce_a_crash_test

Change-Id: Ic047df10732d6ff8cf695ce098d80ec3a098dbf9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2023-05-11 03:14:27 +00:00
Ryan Macnak 2bfecc160b [vm] Update to constexpr in runtime/vm.
TEST=build
Change-Id: I2dd8ae69764af27f480a19995b491e98f52476ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293902
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-04-12 22:18:54 +00:00
Ryan Macnak f9a6a5bdd2 [vm] Update NULL to nullptr in runtime/vm.
TEST=build
Change-Id: I2834ef7cf7cb7c8770f8167a2438cbedcee5c623
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292063
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-04-10 18:15:12 +00:00
Josh Soref f7a2ea5e06 Spelling
Closes https://github.com/dart-lang/sdk/pull/50922

GitOrigin-RevId: 58fd7cfd5ef470a65a52ea28e0407244d853c917
Change-Id: I2e5a5ed991cb05270170a18b8f0169daa9eabdb7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278537
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-01-24 18:00:25 +00:00
Josh Soref 0633041729 Spelling runtime vm
Closes https://github.com/dart-lang/sdk/pull/50786

TEST=ci

GitOrigin-RevId: 781a38e2925ba505f0223ecd578faa9be8e1485c
Change-Id: Ia3c615012d1bac3f0cf64dc01c2ceb40d50d1193
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276684
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2022-12-29 10:18:02 +00:00
Tess Strickland 3b2c41d789 [vm] Make symbol names more user-readable.
Instead of trying to make names assembler-friendly, instead just
quote the symbols names in the assembly output and use user visible
names for function and object symbols.

More specifically:

* This CL changes the symbol name from an assembler-safe version of the
  scrubbed name to an assembler-unsafe version of the user visible name
  that is then quoted when used in assembly output.

* For code symbols, regular stubs are prefixed with `stub`, allocation
  stubs are prefixed with `new` and type testing stubs are prefixed with
  `assert type is` (e.g., `stub Await`, `new RegExp`,
  `assert type is List<Int>`). Function symbols have no prefix.

* Readonly data symbols begin with the type of the readonly data. If
  there's any additional information (e.g., the function name for
  objects like `PcDescriptors` when `SNAPSHOT_BACKTRACE` is defined),
  that follows the type in parentheses.

* For direct-to-ELF snapshots, we allow symbols to have duplicate names.
  Thus, internally we create unique ids for all created symbols and
  refactor the ELF and DWARF subsystems to pass and store these ids
  instead of the symbol names.

* For assembly output, symbols must have unique names. The namer now
  has a `CStringIntMap` that keeps counts of the number of times a
  given symbol name has been generated for assembly output. If the
  symbol name is in the `CStringIntMap`, then the new count in the map
  is added as a suffix to ensure uniqueness.

  For example, the first symbol created using the name `_StringBase.[]`
  has no suffix. If a second symbol is created using the same name,
  the second symbol is instead named `_StringBase.[] (#2)`.

  This also happens for read-only objects with additional information if
  that additional information is not sufficient to make it unique, and
  is added separately from that additional information.

* A further restriction for assembly output is that symbols that begin
  with 'L' have special meaning to the assembler. Thus, we keep the
  old system of prepending such symbol names with an underscore to
  avoid this behavior.

TEST=vm/dart{,_2}/use_add_readonly_data_symbol_flag

Bug: b/248012965
Change-Id: I1b6e876f4fb4f42f4a3b90e03110a34a87a03a7c
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262242
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-10-06 10:33:09 +00:00
Ryan Macnak 32715d1d5b [vm] Mark ELF libraries produced directly by the VM as not requiring an executable stack.
Even if the main program disables executable stacks, dlopen'ing a library that doesn't itself disable executable stack will switch the stack to executable. (Presumably in the name of compatability with GNU nested functions.)

Dart does not need executable stacks, and executable stacks are an additional attack surface.

TEST=readelf
Bug: b/229648756
Change-Id: Ia8c234ebc6178a26528d37b1359a10dd42039a9b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241540
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-04-19 18:51:47 +00:00
Vyacheslav Egorov 3e697cd1dd [vm/elf] Use correct kWordSize in cfa_offset.
We were generating unwinding information using host word size
instead of target word size.

TEST=manually

Bug: b/220804295
Change-Id: I5adacf2b7ecdf8af90ed6e7b25ae76e12b76124f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238085
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2022-03-22 23:16:10 +00:00
Ryan Macnak 51b22b8197 [vm] Add missing .eh_frame for X64 and RISC-V.
TEST=gdb bt
Bug: b/220804295
Change-Id: Id4dccdf65b4ccbec74ab61a3881726fa4f883ad5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/235768
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-03-15 16:33:23 +00:00
Vyacheslav Egorov 9a0be16a9d [vm/compiler] Fix unwinding issues
We used to emit an unwinding rule for caller's SP, however
this does not really work as we expected it to work. Instead there is
an unwritten rule that caller's SP is equal to the CFA which overwrites
our unwinding rule (at least in libunwindstack library used by
Android).

This leads to unwinding issues on the boundary between C++ and Dart:
when we unwind through the InvokeDartCode stub frame we arrive to
C++ code with SP equal to callee's CFA, which is set to FP by our
unwinding rules. In this case if C++ code does not use FP based frames
we will fail to unwind C++ frame property - because SP will be
incorrect. This manifest in simpleperf profiles as stacks which
abruptly truncate at DartEntry::InvokeCode.

This CL rewrites unwinding rules to fix this: instead of using
FP as CFA we define CFA to be caller's SP (FP+2*kWordSize). This ensures
that unwinding on the boundary between C++ and Dart works correctly:
as CFA will always be consistently set to caller's SP.

Additionally this CL fixes InvokeDartCode stub to remove a value
that this stub pushes above saved return address in PRODUCT mode.

This value acts as a profiler marker used by builtin profiler
which is only included in non-PRODUCT builds - but the presence of this
value interfers with unwinding (caller's SP will be off by 1 due the
presence of this value).

We could produce custom unwinding information for this stub instead, but
just removing this value in PRODUCT builds should solve the problem
as we usually only profile Flutter release builds with simpleperf.

Fixes b/220804295

TEST=manually profiled a Flutter app and confirmed correct flamegraph.

Change-Id: I2094afaab6e54e89625c0b5a89aebc6b9823d67c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/235226
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2022-03-06 15:29:22 +00:00
Ryan Macnak 04ba20aa98 [vm] Support RISC-V.
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.

The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.

Quirks:
  - RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
  - There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
  - The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
  - For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
  - Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
  - There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
  - The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
  - We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.

Unimplemented:
 - non-trivial FFI cases
 - Compressed pointers - No intention to implement.
 - Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
 - BigInt intrinsics

TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-01-20 00:57:57 +00:00
Tess Strickland d16ad3d64b [vm/elf] Reorder non-NOBITS sections before NOBITS sections.
This ensures that the relocated addresses of sections with file contents
are contained in the loaded segment.

Fixes https://github.com/dart-lang/sdk/issues/47289

TEST=vm/dart{,_2}/use_save_debugging_info_flag

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: I6f7f94900ab1f9f0cb5ead4b0dd63bd2402e5a19
Fixed: 47289
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/214800
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-09-28 17:23:32 +00:00
Ryan Macnak a56c6ccab9 [vm] Fix gcc build.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/46748
Change-Id: I21807884d974a918adf2aec91f2e2a8d06504776
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207841
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-07-28 19:57:54 +00:00
Tess Strickland d9fa9accac [vm/elf] Remove outdated comment.
As of 1d54dab2 this is no longer true.

TEST=No code changed.

Change-Id: I0e3a8bcf565b495ddb33bb1c555834e79dea68ad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207081
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-07-15 18:04:15 +00:00
Vyacheslav Egorov 1d54dab284 [vm/elf] Reorder sections into RO, RX, RW sequence
On Android native libraries can be mapped directly from an APK
they are stored uncompressed in it. In such situations the name
of the mapping no longer provides enough information for libunwindstack
to find the original ELF file and instead it has to rely on heuristics
to locate program header table. These heuristics currently assume that
program header table will be located in the RO mapping which precedes
RX mapping. This implies the following order of segments: RO (program
header, build id section if present, .rodata), RX (.text), RW (.dynamic
and .bss).

TEST=ci, manual

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: I3fa4de75c5d8841a90fd4cae1ab230d02c554f94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206620
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-07-15 11:13:29 +00:00
Tess Strickland 0ee638e78a [vm] Change encoding of snapshot and position-relative relocations.
Previously in Elf::Relocation, symbol names had this mapping:

* nullptr: relative to the start of the snapshot
* ".": relative to the position of the relocation
* otherwise, relative to the given static symbol value

Change this as follows:

* nullptr: relative to the position of the relocation
* "": relative to the start of the snapshot
* otherwise, relative to the given static symbol value

This uses the fact that in ELF, symbol tables always have an initial
symbol with a name of "" that has a value of 0, so now we only have to
handle the nullptr case specially there, instead of having to handle
three different cases.

TEST=Refactoring, so existing tests of snapshots.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: Icb428551091e5fcf04facd34a5bf57ea0d664fa4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206544
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-07-13 10:45:26 +00:00
Tess Strickland 298777c07d [vm] Merge text and data sections in ELF snapshots.
To do this, BitsContainer is changed to be a more rope-like
representation of section portions. In addition to containing
most of the same information stored per-section previously, each
portion also has a section-relative offset that is calculated when
it is added. Thus, merging two compatible BitsContainer sections
is just adding the portions from the second to the first, tweaking
the section-relative offset for each.

Other changes in this CL:

* Create PseudoSections subclasses for the elf header, program
  header table, and section header table, so we can treat them
  more uniformly with the other parts of the ELF snapshot.

* We now only allocate as much BSS space in the snapshot as is needed
  for any text sections in the snapshot, instead of always allocating
  a big enough BSS space for both VM and isolate, even for deferred
  snapshots where there is no VM isolate.

* We already separated segment and section alignment in previous CLs,
  so the fact that our own ELF loader needs load segments to be
  page-aligned no longer means that the sections within those segments
  also needs to be. Thus, we align individual instructions sections to
  kMaxObjectAlignment, like readonly data sections, since both hold a
  single Image object. This removes unnecessary intra-section padding.

TEST=Tests that check DWARF information and trybots that use ELF
     snapshots.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: If0315c8b7b0f31481b676a8901f49cd3a44b5561
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206365
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-07-13 07:40:15 +00:00
Tess Strickland 20ba6afe4c [vm] Delay symbol table creation further.
Create all symbol table-related sections after all other sections, right
before section re-ordering. Also remove the need to keep fields in the
Elf object for most of these sections, only keeping fields for the
static symbol table and the dynamic table, which handles updates
and finalization for the dynamic symbol table.

Reworks symbols so that they are stored in a growable array in the
symbol table instead of as separately allocated objects. Since this
disallows constant fields in Symbol objects due to the need for a copy
constructor, initialize the offset of symbols with the section-relative
offset. This can be adjusted to a snapshot-relative offset after
section memory offsets are computed.

Instead of updating indices and offsets by individually looking up
symbols via their name, just create a map from section indices to new
section indices (for index updates) or to memory offsets (for offset
adjustment) and iterate over all the symbols, applying the map
appropriately.

Move non-PT_LOAD, non-PT_PHDR segment creation into
OrderSectionsAndCreateSegments.

TEST=Refactoring, so existing ELF-based tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: Ic22f1bf3ab0b00ff3e73c431f92209526e787927
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206220
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-07-13 07:39:25 +00:00
Tess Strickland 2f38c2094b [vm] Move Symbol from elf.cc into SymbolTable.
It's not used outside of elf.cc and rarely used as a type outside
of SymbolTable, so nest it to avoid defining dart::Symbol.

TEST=Refactoring, so existing tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: Idc8895b95b021edf831bc310aff0cf3106de8eb1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206367
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-07-09 16:38:20 +00:00
Vyacheslav Egorov 892279cca8 [vm/elf] Fix .eh_frame encoding
Ensure starting address in FDEs is correctly encoded
as PC-relative reference.

TEST=manually

Change-Id: I8297ef38ca50182c0fd87ad434a78389f476243d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206368
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2021-07-09 14:22:25 +00:00
Vyacheslav Egorov 6d503a7c13 [vm/elf] Fix .eh_frame encoding
When FDE refers to CIE it needs to use the offset to CIE start
not CIE's payload start.

TEST=manually

Change-Id: Ic0661e882ab9284d34f5768070d05f9dad440dda
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206362
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-07-09 13:48:25 +00:00
Tess Strickland 2362cd5508 [vm] Reorder ELF sections more sensibly.
Previous CLs made it so that we don't need to calculate memory offsets
until finalization. This means that we can now reorder sections as
desired and then calculate the memory offsets at the same time as the
file offsets.

Thus, we reorder them so that writable, non-executable allocated
sections come first (due to requirements by some loaders to not sandwich
writable segments between non-writable ones), followed by non-writable,
non-executable allocated sections, followed by non-writable, executable
sections, and finally unallocated sections (to avoid differences in
memory offsets between snapshots and separate debugging information).
This ordering minimizes the number of segments that need to be created
(5 or 6, depending on whether a build ID exists).

Since we can reorder, we also delay the creation of the build ID (if
generated) and BSS sections until finalization, instead of creating them
up front in the constructor.

TEST=Further refactorings, so existing tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: Id4142a021c2bb800938e8bc711b1b8a7529bff51
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205780
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-07-02 17:25:34 +00:00
Tess Strickland 2c48cf7f3d [vm] Delay calculating offsets for ELF symbols further.
This CL splits the creation of ELF symbols into two parts: an
initialization that creates the symbol with all information but
the offset, and a finalization that sets the offset appropriately.
The initialization happens eagerly when adding sections, and the
finalization happens just prior to writing out the ELF contents,
after all memory and file offsets have been calculated.

To ensure safety, retrieving the offset from a symbol checks that
the symbol has been properly finalized.

This removes another obstacle to reordering sections and segments.

TEST=Further refactorings, so existing tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: I5dcb0d2c685fb341478e9a7d90c67fc4649a4e75
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205481
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-07-02 17:24:55 +00:00
Tess Strickland d25fdc43ba [vm] Lazily calculate ELF relocations and symbols.
This removes the need for the image snapshot writer to know the
starting memory offset of any ELF sections. Instead, relocation
and symbol information are recorded in the ELF writer to be
resolved during finalization.

This is a step towards being able to reorder sections in the
ELF memory space for better packing into segments. Delaying
relocation and symbol resolution also moves towards merging
multiple text or data sections into a single section if desired.

In addition, this CL removes redundant rows from the DWARF line
number program matrix to reduce the size of debugging information.
Originally, rows were written for every PC offset visited by the
CodeSourceMap. However, no row is needed if the source information
(file, line, column) is the same as the previously written row.

There are also fixes to Utils::{IsInt,IsUint,IsAbsoluteUint} to
allow N >= the bit size of the value.

TEST=Refactoring, so existing tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: I09f87cea214bca06b6fca60cd66138dae6da9e83
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203766
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-07-02 08:52:49 +00:00
Vyacheslav Egorov c3880f466f [vm/aot] Fix issues with .eh_frame
* Make sure to emit .eh_frame before .dynamic section to satisfy
a single non-writable segment requirement (see Elf::WriteProgramTable)
* Use compiler::target::kWordSize instead of kWordSize when aligning
CIE/FDE and for data alignment factor.

TBR=aam@google.com
TEST=dartkp-linux-debug-simarm-crossword

Fixes https://github.com/dart-lang/sdk/issues/46403

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try
Change-Id: Ieb73b7b90edcce91709201dc92caf0bb74ac1189
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204201
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2021-06-19 04:56:16 +00:00
Vyacheslav Egorov 2c6009efaf Reland "[vm/aot] Emit simple .eh_frame in ARM/ARM64 ELF snapshots."
TEST=tested manually

This is a reland of ddfbe05eee

Original change's description:
> [vm/aot] Emit simple .eh_frame in ARM/ARM64 ELF snapshots.
>
> This allows simpleperf to unwind through AOT compiled frames and
> consequently allows Android tooling to produce useful
> timelines and flamegraphs.
>
> Note: emitted .eh_frame is rather primitive and does not cover all possible
> asynchronous unwinding cases (e.g. tick might arrive before Dart frame was
> fully setup). We will address this in a separate CL.
>
> Additionally, this CL tweaks .eh_frame emitted in assembly snapshots to
> workaround Android-specific unwinding bug in libunwindstack (b/191113792):
> current versions of libunwindstack don't push CFA value when evaluating
> DW_CFA_expression.
>
> TEST=tested manually
>
> Bug: b/184102680,b/190248517
> Change-Id: Iea209b2354cca0753385093f165b0320857104b1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203771
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>


Bug: b/184102680,b/190248517
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try
Change-Id: Id888f906307ec3b2e73deec3cf1c2ead5672a2af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204100
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-06-18 13:18:22 +00:00
Slava Egorov 72d0d9981e Revert "[vm/aot] Emit simple .eh_frame in ARM/ARM64 ELF snapshots."
This reverts commit ddfbe05eee.

Reason for revert: broke ARM32 builds

Original change's description:
> [vm/aot] Emit simple .eh_frame in ARM/ARM64 ELF snapshots.
>
> This allows simpleperf to unwind through AOT compiled frames and
> consequently allows Android tooling to produce useful
> timelines and flamegraphs.
>
> Note: emitted .eh_frame is rather primitive and does not cover all possible
> asynchronous unwinding cases (e.g. tick might arrive before Dart frame was
> fully setup). We will address this in a separate CL.
>
> Additionally, this CL tweaks .eh_frame emitted in assembly snapshots to
> workaround Android-specific unwinding bug in libunwindstack (b/191113792):
> current versions of libunwindstack don't push CFA value when evaluating
> DW_CFA_expression.
>
> TEST=tested manually
>
> Bug: b/184102680,b/190248517
> Change-Id: Iea209b2354cca0753385093f165b0320857104b1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203771
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>

TBR=vegorov@google.com,sstrickl@google.com

Change-Id: I57ab5df1d567bb4ddea813024245ef606712a79c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: b/184102680,b/190248517
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203778
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2021-06-17 15:56:12 +00:00
Vyacheslav Egorov ddfbe05eee [vm/aot] Emit simple .eh_frame in ARM/ARM64 ELF snapshots.
This allows simpleperf to unwind through AOT compiled frames and
consequently allows Android tooling to produce useful
timelines and flamegraphs.

Note: emitted .eh_frame is rather primitive and does not cover all possible
asynchronous unwinding cases (e.g. tick might arrive before Dart frame was
fully setup). We will address this in a separate CL.

Additionally, this CL tweaks .eh_frame emitted in assembly snapshots to
workaround Android-specific unwinding bug in libunwindstack (b/191113792):
current versions of libunwindstack don't push CFA value when evaluating
DW_CFA_expression.

TEST=tested manually

Bug: b/184102680,b/190248517
Change-Id: Iea209b2354cca0753385093f165b0320857104b1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203771
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-06-17 14:13:11 +00:00
Ryan Macnak b105a2bed0 [vm] Don't include ELF support in the JIT.
Reduces VM code size by about 32k.

TEST=ci
Change-Id: Id29619379fc2fd1d0fbe85715863df0bae5bb3df
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/202866
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-06-09 18:39:12 +00:00
Ivan Inozemtsev a5ff656da1 Revert "[vm] Switch datastream Write/WriteUnsigned to (S)LEB128."
This reverts commit 80102c981b.

Reason for revert: b/186359854

Original change's description:
> [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
>
> This reduces the number of variable-length integer encodings in our
> database from 2 to 1, and chooses the more standard one.
>
> TEST=Existing tests, in particular any that involve snapshots.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
> Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Idf32bdd879cf8bb7407f6dae764312140ad6eeb2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196920
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
2021-04-26 07:52:47 +00:00
Tess Strickland 80102c981b [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
This reduces the number of variable-length integer encodings in our
database from 2 to 1, and chooses the more standard one.

TEST=Existing tests, in particular any that involve snapshots.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-04-23 07:35:41 +00:00
Tess Strickland 6c8f4aa190 Reland "[vm] Fix V8 snapshot profile handling of the dispatch table."
This is a reland of 5909fd111d

Does a large refactoring on the V8 snapshot profile writer
to clean things up, add more debugging support, and to fix
the problems that surfaced during the original landing.

Other changes:

Changes Serializer::CreateArtificialNodeIfNeeded() to create
artificial nodes for Code objects and immutable arrays.

Fixes CodeSerializationCluster::Trace() to only push needed parts of
discarded code objects, instead of tracing them like full code objects.

Adds test cases to v8_snapshot_profile_writer_test that exercise
the following situations (both separately and together):

* Non-symbolic stack traces are enabled and code and function objects
  are dropped when not needed at runtime.

* Creation of the dispatch table is disabled.

TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test

Original change's description:
> [vm] Fix V8 snapshot profile handling of the dispatch table.
>
> Fixes https://github.com/dart-lang/sdk/issues/45702.
>
> TEST=Tests listed in the issue above.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
> Change-Id: Ibf5e3ccf3828c01f9dda47de360314dabe8cb8a9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195272
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

Change-Id: I8e7030267fe190079a8f68d00fe20bf7170e5719
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195513
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-20 11:17:36 +00:00
Ryan Macnak 8267bb9072 [vm] Fix gcc build.
TEST=build
Change-Id: I9e8af9172a3f25b88cbc97d1ccd9dd053d8ab8a4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171808
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2020-11-16 23:52:40 +00:00
Tess Strickland 45a46ca2b8 [vm] Add (S)LEB128 encoding/decoding to BaseWriteStream.
Unlike cfc8e6de, this does _not_ replace the default variable length
encoding for {Read,Write}Streams, but insteads adds separate
{Read,Write}{S,}LEB128 methods to the appropriate classes. If we later
find the cause of the issues that led to the revert of cfc8e6de, it'll
be easy to switch over then.

Note that WriteLEB128 asserts that the value is non-negative if used
with a signed type (since negative values suggests that SLEB128 should
be used instead for minimal encoding).

Also removes the various other encoding and decoding methods for
(S)LEB128 across the codebase and changes those clients to use
{Read,Write}Streams instead.

Other cleanups:

* Various constant-related cleanups in datastream.h.

* Adds DART_FORCE_INLINE to ReadStream::ReadByte and uses it in the
  default variable length decoding methods for retrieving bytes
  from the stream instead of managing current_ by hand.

* Creates a canonical empty CompressedStackMaps instance and uses
  that instead of the null CompressedStackMaps instance in most cases.
  The only remaining (expected) use of the null CompressedStackMaps
  instance is for the global table in the object store when no global
  table exists (e.g., in JIT mode before any snapshotting).

* Moves CompressedStackMapsIterator from code_descriptors.h to an
  Iterator class within CompressedStackMaps in object.h (similar to
  PcDescriptors::Iterator), to limit friend declarations and because it
  conceptually makes more sense as part of CompressedStackMaps.

* Removed CompressedStackMaps::PayloadByte, since existing clients
  (CompressedStackMaps::Iterator, StackMapEntry in program_visitor.cc)
  are better served by just operating on the payload buffer directly
  (with appropriate NoSafepointScopes).

* WriteStreams no longer allocate their initial space on construction,
  but rather on the first write, so no allocation is performed by
  constructing a never-used WriteStream.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-ubsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: Ice63321abaa79157fbe9f230a864c8bba0e6dea9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166421
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-10-09 10:08:16 +00:00
Tess Strickland 1b50200abd [vm] Simplify symbol naming/address retrieval.
Before, picking a symbol name for a Code object when generating
assembly required adding a placeholder entry in the Dwarf object,
because the Dwarf object separately used a SnapshotTextObjectNamer
to determine the appropriate symbol name to use in relocations.

Instead, now the ImageWriter subclasses are entirely responsible for
creating the symbol name for a given Code object, and that name is
passed when adding the Code object to the Dwarf object.

For Elf output, we eagerly create symbols for sections and Code objects
again in the appropriate symbol table, and so looking up the start of a
given section or Code object can be done simply by asking for the
corresponding symbol's address, removing special case methods and other
workarounds.

We also clean up the generated static and dynamic symbol tables by
only generating local symbols for Code objects and BSS sections (these
are local in assembly output also), and only adding global symbols to
the dynamic symbol table.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie40c0c836681e98f345483136bb6860e5588ef30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166002
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-10-06 17:10:05 +00:00
Tess Strickland fca36cae39 [vm] Merge ImageHeader into InstructionsSection.
Before, we only wrapped the Instructions payloads in an
InstructionsSection in precompiled snapshots with bare instructions mode
enabled. This CL changes that to always add an InstructionsSection
wrapper in precompiled snapshots, even when not in bare instructions
mode, and moves all the information from the old ImageHeader object into
the InstructionsSection object instead. Note that in non-bare mode, the
InstructionsSection has no payload, and the Instructions objects follow
the InstructionsSection object instead of being contained by it.

This CL also wraps all implementations of AssemblyImageWriter and
SnapshotTextObjectNamer methods in a single DART_PRECOMPILER ifdef
block, since we only generate assembly in precompiled mode.  This
removes the need for the smaller DART_PRECOMPILER ifdef blocks scattered
across the AssemblyImageWriter method implementations.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: I3676fb5c6675845463707acbbb4759a465cfa342
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165563
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-10-06 17:10:05 +00:00
Tess Strickland a36d17730a Reland "[vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions."
Fixes errors introduced by adding PRINTF_ATTRIBUTE to
BaseWriteStream::Printf due to attempting to print a
compiler::target::uword using the format for uword.

Original description:

For snapshot writing, we generally only have aligned writes so the old
methods worked well, but in some formats (like DWARF) we're not
guaranteed to have aligned writes and so had to use WriteBytes.
Instead, have WriteFixed call WriteBytes and replace any other uses of
WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.

Also makes the following additional changes:

* Expose WriteByte(value), instead of forcing use of
  WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.

* Rename Print/VPrint to Printf/VPrintf, so methods taking format
  strings are named consistently with others in the codebase.

* Add format string checking to Printf when supported.

* Add AddString() for non-formatted string writing.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Ib70ccaa752dbf9ead5ea57e386a67725723bac84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164324
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-09-24 16:58:30 +00:00
Tess Strickland 074d6e1309 Revert "[vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions."
This reverts commit 4e2556215a.

Reason for revert: Adding PRINTF_ATTRIBUTE broke more than expected on Mac bots.

Original change's description:
> [vm] Allow BaseWriteStream::WriteFixed to be used at unaligned positions.
>
> For snapshot writing, we generally only have aligned writes so the old
> methods worked well, but in some formats (like DWARF) we're not
> guaranteed to have aligned writes and so had to use WriteBytes.
> Instead, have WriteFixed call WriteBytes and replace any other uses of
> WriteBytes(&value, sizeof(value)) with WriteFixed(value) instead.
>
> Also makes the following additional changes:
>
> * Expose WriteByte(value), instead of forcing use of
>   WriteFixed<uint8_t> or Raw<1,T>::Write to get the same behavior.
>
> * Rename Print/VPrint to Printf/VPrintf, so methods taking format
>   strings are named consistently with others in the codebase.
>
> * Add format string checking to Printf when supported.
>
> * Add AddString() for non-formatted string writing.
>
> Change-Id: I5a2e91380bc41b0502e1a576acec786dfe66abbf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164161
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,rmacnak@google.com,sstrickl@google.com

Change-Id: Ie90ac4477f4c581692b37429536ce8dd66788540
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164323
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-09-24 13:57:52 +00:00