In addition to adding a parser for the universal binary format, this
also requires major reworks to handle files that contain different
DWARF information for different architectures, and how to pass the
architecture down to where it's needed.
Also fix dSYM handling: instead of assuming the name of the MachO file
corresponds exactly to the basename of the dSYM with the extension
stripped, just look for the single file within the
Contents/Resources/DWARF directory.
Also add `unrecognized` enum entries for DW_TAG, DW_AT, and DW_FORM
values that aren't handled.
Issue: https://github.com/flutter/flutter/pull/101586
Change-Id: Ief5edc275ccd1192669252140d128136cd2bed26
Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-nnbd-mac-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252821
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Examples of the new line added to non-symbolic stack traces:
os: linux arch: x64 comp: yes sim: no
(Running on linux-x64c)
os: macos arch: arm64 comp: no sim: yes
(Running on mac-simarm64)
This CL also abstracts out the separate hardcoded strings across
the codebase for host and target OS and architecture into
definitions in platform/globals.h to ensure that they stay
in sync across different uses.
TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag
Issue: https://github.com/flutter/flutter/pull/101586
Change-Id: Ifdfea5138dd1003f561da0174e89aebc165bf9b0
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253283
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This is a reland of commit 08c13f173c
Fixes test failures on non-x64 architectures, both in the test
harness and due to DWARF5 line number program headers having a
non-backwards compatible format. (We generate DWARF2 in the
ELF snapshot writer, but the assembler used for assembly snapshots
may generate DWARF5.)
TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag
Original change's description:
> [pkg/native_stack_traces] Support Mach-O dSYM debugging information.
>
> TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag
>
> Bug: https://github.com/dart-lang/sdk/issues/43612
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
> Change-Id: Icda21bb14dcc0cf4784cea118e6ba7dd4edd35aa
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250381
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
Bug: https://github.com/dart-lang/sdk/issues/43612
Change-Id: I8a9cb70e78bc8594bcae004809c5a1be778d691d
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-mac-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251464
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This ensures that the relocated addresses of sections with file contents
are contained in the loaded segment.
Fixes https://github.com/dart-lang/sdk/issues/47289
TEST=vm/dart{,_2}/use_save_debugging_info_flag
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: I6f7f94900ab1f9f0cb5ead4b0dd63bd2402e5a19
Fixed: 47289
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/214800
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Instead of trying to lazily create artificial nodes when needed for
WeakSerializationReference (WSR) targets in all cases, create them
eagerly for targets in reachable WSRs, since those are guaranteed
to be needed.
Those for unreachable WSRs are still lazily created as needed, since the
WSR won't even be accessed by the clustered snapshot writer unless the
unreachable WSR is part of another unreachable object that has an
artificial node created.
This rework avoids some issues seen on upcoming CLs where the artificial
nodes for WSR targets weren't getting correctly generated.
-----
Also extend the v8 snapshot profile writer tests to check the sizes of
the text and data sections in ELF snapshots. That means the v8 snapshot
profile writer tests check up to three different measures, from most
precise to least precise, depending on the output mode:
* If writing an ELF snapshot directly: the sum of the sizes attributed
to the text and data section symbols are checked to be exactly the
same as the sum of the sizes of the objects in the profile.
* If writing an ELF snapshot either directly or via assembly: the sum
of the sizes of the text and data sections are checked to be the same
as the sum of the sizes of the objects in the profile. If using an
assembler that merges text and data sections, then account for the
padding between the sections using an approximate check.
* For all: Check that the size of the snapshot as a whole is
approximately equal to the sum of the sizes of the objects in the
profile, accounting both for possible padding and for the global
header information and non-data/non-text sections like the dynamic
symbol section.
TEST=vm/data{,_2}/v8_snapshot_profile_writer_test
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-mac-release-simarm64-try
Change-Id: I66e0e7fdb5bb98045621bf516f150a4723e08147
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198942
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Previously, Code objects of invisible functions were retained
in order to omit frames corresponding to such functions from stack
traces when stack trace is printed.
This change drops Code objects of invisible functions. That also
means that frames corresponding to such functions are no longer
skipped in binary DWARF stack traces.
In order to account for that, DW_AT_artificial attribute is added to
generated DWARF debug information to mark invisible functions.
Stack trace decoding now looks at this attribute and skips those
frames when symbolizing stack trace.
Flutter gallery in release-sizeopt mode:
Heap size of snapshot objects -4.2% (arm), -4.4% (arm64).
A large application in --dwarf_stack_traces mode:
Number of discarded Code objects increased from 72.4% to 83.7%
(out of all Code objects).
Heap size of Code objects -37.4%.
Heap size of all snapshot objects -5%.
TEST=tests/standalone/dwarf_stack_trace_invisible_functions_test.dart
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: Ib804852aba1e083670f1d9b9d66cbaab7dcdcff9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196583
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Fixes the case where the Image name and type was not written
to the v8 snapshot profile in non-precompiled mode.
Adds core-jit snapshot tests to the v8 snapshot profile writer test.
In core-jit and app-aot-elf modes, verify that all bytes in the
snapshot sections are accounted for in the profile.
Bug: https://github.com/dart-lang/sdk/issues/45787
TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try
Change-Id: I88be309cdea6293d768e2fa458492387d70fde22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196323
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Changes:
Doing this always in precompiled mode meant increased data segment sizes
when CodeSourceMaps are stored, since encoded line/column information is
larger in the LEB-like encoding used. Now we only store column
information when we produce non-symbolic stacks, since the increased
space needed to store the columns is instead in DWARF sections and can
be stripped or elided.
Original description:
Previously, we passed line number information to the stack trace printer
and to DWARF by changing the non-special positions in the CodeSourceMap
to line numbers in precompiled mode. However, doing this lost column
information.
We get the column information back in the majority of cases by encoding
the line number and column information when neither is too large to pack
together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
Otherwise, we just store the line information as before, though due to
using a bit to encode whether column info exists, it's reduced to 30
bits. If the line info is too big for that, we just return kNoSourcePos.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Ia8baee71468da6100a170fa305d03059ffd17f78
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151822
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 7c34a80c11.
Reason for revert: This change seems to cause >2% code size regression
in flutter-release mode.
Original change's description:
> [vm/aot] Keep column information when possible for precompiled mode.
>
> Previously, we passed line number information to the stack trace printer
> and to DWARF by changing the non-special positions in the CodeSourceMap
> to line numbers in precompiled mode. However, doing this lost column
> information.
>
> We get the column information back in the majority of cases by encoding
> the line number and column information when neither is too large to pack
> together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
> Otherwise, we just store the line information as before, though due to
> using a bit to encode whether column info exists, it's reduced to 30
> bits. If the line info is too big for that, we just return kNoSourcePos.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
> Change-Id: Id1c826f10871e2f304fa40a59d8b704404d3a2c9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151507
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TBR=rmacnak@google.com,alexmarkov@google.com,sstrickl@google.com
Change-Id: I5cf97543d1ac2731bb27bdb58ae97af6f22f2cfb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151820
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Previously, we passed line number information to the stack trace printer
and to DWARF by changing the non-special positions in the CodeSourceMap
to line numbers in precompiled mode. However, doing this lost column
information.
We get the column information back in the majority of cases by encoding
the line number and column information when neither is too large to pack
together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
Otherwise, we just store the line information as before, though due to
using a bit to encode whether column info exists, it's reduced to 30
bits. If the line info is too big for that, we just return kNoSourcePos.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Id1c826f10871e2f304fa40a59d8b704404d3a2c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151507
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
For proper crashpad integration, we need to generate a build ID, as the
build ID generated by crashpad if there is not one will be a simple XOR
of the first text page, which rarely changes for Dart snapshots.
Assembly snapshots already have a build ID included by the assembler, so
we currently only do this for ELF snapshots.
Currently the build ID is a 128-bit hash value that is four separate
32-bit hash values concatenated together. Those hash values come from
the contents of the VM and isolate .text and .rodata sections.
This change also contains work to separate out the concepts of sections
and segments in the ELF builder. Now, consecutive allocated sections
with the same write and execute flags are combined into a single PT_LOAD
segment when possible, which reduces the padding needed to ensure that
segments start on page boundaries in ELF snapshots.
Bug: https://github.com/dart-lang/sdk/issues/42020
Change-Id: I42a837dae665a3902d881b8d151b49ede87d6c67
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150625
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF."
Storing the relocated address as an extra field in the Image header,
which requires increasing the Image header size on 64-bit platforms,
means Image pages cannot be used reliably as HeapPages as objects no
longer start after kMaxObjectAlignment bytes.
Instead, we return to an older design that just uses the lowest bit
in the BSS offset to store whether the instructions in an Image were
compiled directly to ELF.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Change-Id: I3819b0dc2719d69f5e8764ca8be8c6ae7171a7bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146560
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This reverts commit 250b84987d.
Reason for revert: Failures on FFI CI tests.
Original change's description:
> [vm] Mark snapshots that were compiled directly to ELF.
>
> Since the virtual addresses in ELF snapshots are the same as in
> separately saved debugging information, print the virtual address in
> non-symbolic frames again when running from a snapshot compiled directly
> to ELF.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
> Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
TBR=vegorov@google.com,rmacnak@google.com,dacoharkes@google.com,sstrickl@google.com
Change-Id: I1c5f58e9d9e11b994c1016970839f95b92885d43
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146220
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Symbolic and non-symbolic stack traces had separate ToCString
implementations with the same general structure, but those
implementations diverged enough to cause issues in certain async stack
modes. They are now merged, with the --dwarf-stack-traces-mode flag
checked where appropriate within the merged method.
Also, now non-symbolic stack traces do not include frames for Code with
invisible Function owners unless the flag --show-invisible-frames is
enabled. With this change, pkg/native_stack_traces no longer needs to
guess at which frames corresponding to Dart code should be treated as
internal.
Tested by adding --dwarf-stack-traces version of the tests
in runtime/tests/vm/dart/causal_stacks.
Fixes https://github.com/dart-lang/sdk/issues/41578
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Change-Id: I41a887129616c88acd7729492addf7364d95df33
Bug: 41578
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143816
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Static symbol offsets from Dart stack traces can now be used directly
as locations for the find command.
Change-Id: Ie2971099cbc33b871f8e0e213865854b5ee6890b
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142362
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This allows for the translation of partial stack traces that do not
include the instructions header line.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: I5b490c99cb844d37c63943e780f3a5613faac119
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140285
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Now when -v/--verbose is used, we will also print call information
when those calls appear in stubs that do not correspond to Dart code.
The returned information contains the name of the stub (the name of
the static symbol in the ELF file generated for the stub) as well as
the offset of the frame PC within the stub payload.
This change also adds a flag --dump-debug-file-contents that
outputs the parsed information from the file passed via -d/--debug.
Change-Id: Ic52dd6825f2f3564efa3c2c12b46a8bef23d56bc
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139280
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The Dart VM uses two dynamic symbols, _kDartVmSnapshotInstructions and
_kDartIsolateSnapshotInstructions, to store the virtual addresses of the
start of these sections. By using these symbols to find the sections,
we don't bake in the assumption that the two sections appear as distinct
.text segments or an assumption about the order in which they appear.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: I02c4d61a1cf743e2455252df16a47dbb5c718195
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134165
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
Tess Strickland