Commit Graph

59 Commits

Author SHA1 Message Date
Tess Strickland bc021bbc01 [vm] Output accompaning relocatable object for Mach-O snapshots.
In order to appropriately generate .dSYMs for a snapshot and allow
the strip tool to be used on the snapshot afterwards, add a mode that
not only outputs the snapshot (sans DWARF information) but also
an associated relocatable object file that contains the program's DWARF
information. That allows dsymutil to retrieve the DWARF information
from the relocatable object when run on the snapshot prior to stripping.

To specify that the relocatable object file should be output, use
the new --macho-object command line argument to gen_snapshot
to specify where the object file should be written.

This CL also adds an additional command line argument,
--macho-reduce-padding, which reduces the alignment used for segments
and the text/const sections in Mach-O outputs from 16KB to 64 bytes.
The larger padding is needed for some uses, like non-native loading
of Mach-O objects, but can be elided for other uses like Flutter builds.

TEST=vm/dart/use_dwarf_stack_traces_flag_test
     vm/dart/use_macho_reduce_padding_flag_test

Change-Id: I2bf4bacb70c41299b8b6fdb7635c2374acf7a07d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/457420
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2025-11-19 07:45:52 -08:00
Tess Strickland 83e9bd72ea [vm] Additional Mach-O related command line options for gen_snapshot.
Adds the following command line options to gen_snapshot:

* --macho-install-name: The name to use as the install name for the
  dynamic library (used in the LC_ID_DYLIB and LC_CODE_SIGNATURE
  load commands). If not provided, the output filename is used.

* --[no-]macho-linker-signature: Whether or not gen_snapshot should
  generate an ad-hoc linker-signed signature. Defaults to true.

The following command line option is macOS/iOS-specific:

* --macho-rpath: Comma-delimited run paths that should be added at
  runtime to the current run path used for finding @rpath-prefixed
  dynamic libraries.

Adds support to parsing rpath and dylib load commands to
pkg/native_stack_traces's Mach-O parser in order to test the
command line options appropriately.

TEST=vm/dart/use_macho_options

Issue: https://github.com/dart-lang/sdk/issues/60307
Change-Id: I17618578a7bff7851a88cdaf27f299f058f98dd3
Cq-Include-Trybots: luci.dart.try:vm-aot-mac-debug-x64-try,vm-aot-mac-debug-arm64-try,vm-aot-linux-debug-arm64-try,vm-aot-linux-debug-x64-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-linux-release-arm64-try,vm-aot-win-release-arm64-try,vm-aot-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/441920
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2025-07-29 06:01:14 -07:00
Tess Strickland 1f5fd27c61 [vm] Add --macho-min-os-version flag for macOS/iOS targets.
The command line flag allows the user to specify the minimum MacOS or
iOS version used in the build version load command for the Mach-O
dynamic library snapshot. If not specified, the minimum OS version
specified in macOS Mach-O snapshots is 10.15 (Catalina) and the minimum
OS version specified in iOS Mach-O snapshots is 13.

This CL also changes the targeted OS version in the build version load
command to be the same as the minimum OS version.

Adds support to parsing build version load commands to
pkg/native_stack_traces's Mach-O parser in order to test the
existence and contents of build version load commands in
vm/dart/unobfuscated_static_symbols.

TEST=vm/dart/unobfuscated_static_symbols

Issue: https://github.com/dart-lang/sdk/issues/60307
Change-Id: I3ee3ba34297d3261be7e3a1d2fb3c1da1ef0ef05
Cq-Include-Trybots: luci.dart.try:vm-aot-mac-debug-x64-try,vm-aot-mac-debug-arm64-try,vm-aot-linux-debug-arm64-try,vm-aot-linux-debug-x64-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-linux-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/438901
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2025-07-08 05:29:19 -07:00
Tess Strickland 6b53073eae Reland "[vm, gen_snapshot] Add app-aot-macho-dylib option for AOT snapshots."
This is a reland of commit 38ef28a058

Fixes:

* Fix comparisons in ASSERT_EQUAL statements on 32-bit architectures.
* Take simulated architectures into account when deciding whether
  to use dlopen() for native shared object formats.
* Fix struct/field name collision for GCC.
* Use CPU_TYPE_ANY/CPU_SUBTYPE_ANY for architectures that do not
  have more specific cpu_type_t/cpu_subtype_t constants defined.

Original change's description:
> [vm, gen_snapshot] Add app-aot-macho-dylib option for AOT snapshots.
>
> This is the initial framework for creating snapshots as Mach-O dynamic
> libraries. Note that this framework is not 100% feature complete
> compared to generating Mach-O snapshots via assembly. In particular,
> the directly-compiled Mach-O dylib does not yet contain compact
> unwinding information.
>
> Other changes:
>
> * Adds UuidCommand to the native_stack_traces package's Mach-O reader,
>   which now appropriately returns the UUID as the build ID for Mach-O
>   shared objects.
>
> * Adds Utils::Basename(path) for portably retrieving the basename
>   from a path. (Returns nullptr for all arguments where it is not
>   currently implemented on Fuchsia or Windows.)
>
> * Adjusts vm/timeline.h to avoid pulling in <mach_o/loader.h> on MacOS,
>   as that interferes with uses of the namespaced Mach-O definitions
>   in platform/mach_o.h.
>
> * Only attempt to dlopen() a snapshot if ELF is the native format
>   for the host platform or the snapshot is not an ELF shared object.
>   If dlopen() is used, report the error message if it fails rather
>   than attempting to manually load the snapshot as an ELF shared object.
>
> * Fix the magic number stored in DylibAppSnapshot for loaded non-ELF
>   dynamic libraries.
>
> * Remove the detection of reverse-endian Mach-O magic numbers in
>   DartUtils::SniffForMagicNumber(), since all our Mach-O related code
>   assumes host-endian Mach-O files and so there's no point other than
>   to give a slightly better error message when failing.
>
> TEST=vm/dart/exported_symbols_test
>      vm/dart/unobfuscated_static_symbols_test
>      vm/dart/use_dwarf_stack_traces_flag_test
>      vm/cc/CanDetectMachOFiles
>
> Issue: https://github.com/dart-lang/sdk/issues/60307
> Change-Id: Idf5b49d6c6d035ab033509613212b95520d65965
> Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-mac-release-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-fuchsia-release-x64-try,vm-fuchsia-release-arm64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415020
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

TEST=vm/dart/exported_symbols_test
     vm/dart/unobfuscated_static_symbols_test
     vm/dart/use_dwarf_stack_traces_flag_test
     vm/cc/CanDetectMachOFiles
     ci on trybots that failed on the original CL

Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-mac-release-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-fuchsia-release-x64-try,vm-fuchsia-release-arm64-try,vm-linux-debug-ia32-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv32-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-release-simarm_x64-try,vm-gcc-linux-try,vm-ubsan-linux-release-arm64-try
Change-Id: Iaffea0ddc6173100c8b5b2a9fe46c45f4f611a2e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/431240
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2025-05-27 06:30:20 -07:00
Tess Strickland 38ea3a8f5e Revert "[vm, gen_snapshot] Add app-aot-macho-dylib option for AOT snapshots."
This reverts commit 38ef28a058.

Reason for revert: Breaks build on debug mode 32-bit arches,
                   release mode simarm_x64

Issue: https://github.com/dart-lang/sdk/issues/60307
Original change's description:
> [vm, gen_snapshot] Add app-aot-macho-dylib option for AOT snapshots.
>
> This is the initial framework for creating snapshots as Mach-O dynamic
> libraries. Note that this framework is not 100% feature complete
> compared to generating Mach-O snapshots via assembly. In particular,
> the directly-compiled Mach-O dylib does not yet contain compact
> unwinding information.
>
> Other changes:
>
> * Adds UuidCommand to the native_stack_traces package's Mach-O reader,
>   which now appropriately returns the UUID as the build ID for Mach-O
>   shared objects.
>
> * Adds Utils::Basename(path) for portably retrieving the basename
>   from a path. (Returns nullptr for all arguments where it is not
>   currently implemented on Fuchsia or Windows.)
>
> * Adjusts vm/timeline.h to avoid pulling in <mach_o/loader.h> on MacOS,
>   as that interferes with uses of the namespaced Mach-O definitions
>   in platform/mach_o.h.
>
> * Only attempt to dlopen() a snapshot if ELF is the native format
>   for the host platform or the snapshot is not an ELF shared object.
>   If dlopen() is used, report the error message if it fails rather
>   than attempting to manually load the snapshot as an ELF shared object.
>
> * Fix the magic number stored in DylibAppSnapshot for loaded non-ELF
>   dynamic libraries.
>
> * Remove the detection of reverse-endian Mach-O magic numbers in
>   DartUtils::SniffForMagicNumber(), since all our Mach-O related code
>   assumes host-endian Mach-O files and so there's no point other than
>   to give a slightly better error message when failing.
>
> TEST=vm/dart/exported_symbols_test
>      vm/dart/unobfuscated_static_symbols_test
>      vm/dart/use_dwarf_stack_traces_flag_test
>      vm/cc/CanDetectMachOFiles
>
> Issue: https://github.com/dart-lang/sdk/issues/60307
> Change-Id: Idf5b49d6c6d035ab033509613212b95520d65965
> Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-mac-release-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-fuchsia-release-x64-try,vm-fuchsia-release-arm64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415020
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

Issue: https://github.com/dart-lang/sdk/issues/60307
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-mac-release-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-fuchsia-release-x64-try,vm-fuchsia-release-arm64-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: Iff2ab4c84a513a184784129f456bd357ae8e3a67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/431220
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Tess Strickland <sstrickl@google.com>
2025-05-27 01:05:48 -07:00
Tess Strickland 38ef28a058 [vm, gen_snapshot] Add app-aot-macho-dylib option for AOT snapshots.
This is the initial framework for creating snapshots as Mach-O dynamic
libraries. Note that this framework is not 100% feature complete
compared to generating Mach-O snapshots via assembly. In particular,
the directly-compiled Mach-O dylib does not yet contain compact
unwinding information.

Other changes:

* Adds UuidCommand to the native_stack_traces package's Mach-O reader,
  which now appropriately returns the UUID as the build ID for Mach-O
  shared objects.

* Adds Utils::Basename(path) for portably retrieving the basename
  from a path. (Returns nullptr for all arguments where it is not
  currently implemented on Fuchsia or Windows.)

* Adjusts vm/timeline.h to avoid pulling in <mach_o/loader.h> on MacOS,
  as that interferes with uses of the namespaced Mach-O definitions
  in platform/mach_o.h.

* Only attempt to dlopen() a snapshot if ELF is the native format
  for the host platform or the snapshot is not an ELF shared object.
  If dlopen() is used, report the error message if it fails rather
  than attempting to manually load the snapshot as an ELF shared object.

* Fix the magic number stored in DylibAppSnapshot for loaded non-ELF
  dynamic libraries.

* Remove the detection of reverse-endian Mach-O magic numbers in
  DartUtils::SniffForMagicNumber(), since all our Mach-O related code
  assumes host-endian Mach-O files and so there's no point other than
  to give a slightly better error message when failing.

TEST=vm/dart/exported_symbols_test
     vm/dart/unobfuscated_static_symbols_test
     vm/dart/use_dwarf_stack_traces_flag_test
     vm/cc/CanDetectMachOFiles

Issue: https://github.com/dart-lang/sdk/issues/60307
Change-Id: Idf5b49d6c6d035ab033509613212b95520d65965
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-mac-release-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-fuchsia-release-x64-try,vm-fuchsia-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415020
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2025-05-26 08:52:26 -07:00
Sigurd Meldgaard 38d0ce70b2 Bump language version for pkg/native_stack_traces
Extracted from https://dart-review.googlesource.com/c/sdk/+/397164
Which migrates the sdk to resolve as a pub workspace

Pub workspaces requires a language version of 3.5.

Change-Id: I44dc73114c9c794cb60ab29b19aa4b1b00f3241d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/412661
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
2025-02-27 07:02:41 -08:00
Devon Carew 1f8aea0c81 Bump lints to a97011b4b031b1df94765dbacd7d92b0b765c8b9
Changes:
```
> git log --format="%C(auto) %h %s" af68bf0..a97011b
 https://dart.googlesource.com/lints.git/+/a97011b add unintended_html_in_doc_comment; rev to 5.1.0 (211)

```

Diff: https://dart.googlesource.com/lints.git/+/af68bf0cbad88002a2ae19ee8db37b64eb120c32..a97011b4b031b1df94765dbacd7d92b0b765c8b9/
Change-Id: I998d73f5f72e94d5f2922b0751d31620a837555e
Tested: analysis only change
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/387601
Reviewed-by: Nate Bosch <nbosch@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2024-09-30 21:48:03 +00:00
Paul Berry 7826451495 Remove unreachable default clauses from dwarf.dart.
I'm currently working on a fix to the analyzer that will cause
unreachable default clauses to be properly flagged as a warning
(https://github.com/dart-lang/sdk/issues/54575). To avoid a bot
failure when the fix lands, this unreachable default clause needs to
be removed.

Bug: https://github.com/dart-lang/sdk/issues/54575
Change-Id: I8b44341fad3146d4fdd64831e263513fd5092102
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/380075
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Auto-Submit: Paul Berry <paulberry@google.com>
2024-08-19 09:10:59 +00:00
Tess Strickland 953b1a9b0b [vm] Refactor vm/dart/use_dwarf_stack_traces_flag tests.
Both the regular and deferred versions of this test contain
up to six different test cases:

All platforms:
 * ELF snapshot, using DWARF from the snapshot
 * ELF snapshot, using DWARF from the separate debugging information
For platforms where the test can assemble snapshots:
 * assembled snapshot, using DWARF from the snapshot or
   the separate .dSYM package on MacOS
 * assembled snapshot, using DWARF from the separate debugging
   information
For MacOS only:
 * creating a single-architecture universal binary from the
   separate .dSYM package and extracting DWARF information from it
 * creating a multi-architecture universal binary from the
   separate .dSYM package and extracting DWARF information from it

Originally the tests were written using package:expect, performing
program compilation and execution separately before each test and
lazily reading DWARF information within the test itself. Since tests
using package:expect stop the program immediately on a failure,
a failing expectation keeps other independent test cases from being
checked. However, it's useful to know if the failure is limited to
only a subset of the test cases, since that helps point at which code
is to blame for the test failure(s).

Now the tests are refactored to first set up the tests by performing
all program compilation and execution first, collecting all outputs and
DWARF information as test state. Then the test cases are defined over
the collected test state using package:test instead of package:expect.
This way, as long as there is not a failure in the initial setup, all
of the applicable test cases are run even if one or more of them fail.

-----

In pkg/native_stack_traces, changes the return types of the reader
retrieval methods in the DwarfContainer class and
DwarfSnapshot.fromDwarfContainer to be nullable. If the shared object
does not contain the expected DWARF information, these methods now
return null instead of causing null check exceptions to be thrown.

Issue: https://github.com/dart-lang/sdk/issues/55612

Change-Id: I253965a95894f455e51d021e32dbf2703d8b99cf
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-product-arm64-try,vm-aot-mac-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/375240
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2024-07-18 10:00:11 +00:00
Tess Strickland b1b3e34b88 [vm] Add non-symbolic stack trace support for deferred loading units.
Dart VM changes:

Note that the following changes are backwards compatible in the
case that a Dart program has no deferred loading units (i.e., the
Dart program is contained in a single shared object snapshot).

When there are non-root loading units, the non-symbol stack trace
header now includes information about loading units as follows:

loading_unit: N, build_id: S, dso_base: A, instructions: A

where N is an integer, S is a string of hex digits (0-9a-f), and A
is a word-sized address printed as a hex string (without prefix).

In addition, all non-symbolic stack frames for isolate instructions
include a unit field, including those for the root loading unit, e.g.,

   #NN abs <address> unit <id> virt <address> <symbol>+<offset>

If there are no non-root loading units, then the non-symbolic stack
trace is unchanged from its previous format.

Adds a build ID to split deferred loading unit snapshots.
Fixes: https://github.com/dart-lang/sdk/issues/43516

If separate debugging information is requested, the loading unit
manifest includes a 'debugPath' field for each loading unit,
which contains the path to its separate debugging information.

Removes the attempt to store the relocated address of the instructions
section when running from an assembled snapshot in the initialized BSS.

Adds OS::GetAppDSOBase, which takes a pointer to the instructions
section and returns a pointer to its loaded shared object in memory.
For compiled-to-ELF snapshots, it does this using the relocated address
of the instructions in the Image, and for assembled snapshots, it
delegates to NativeSymbolResolver::LookupSharedObject.

-----

Changes to package:native_stack_traces:

PCOffset now has two new fields:

* int? unitId: the unit ID of the loading unit, when available.
* String? buildId: the build ID of the loading unit, when available.

For PCOffsets in the VM section, the unitId and buildId are those of
the root loading unit.

The constructor for the DwarfStackTraceDecoder now takes two
additional optional named arguments:

* Map<int, Dwarf>? dwarfByUnitId: A map associating loading unit IDs
  with the appropriate Dwarf object. May or may not contain an entry
  for the root loading unit.
* Iterable<Dwarf>? unitDwarfs: An iterable container holding Dwarf
  objects. May or may not contain an entry for the root loading unit.

The Dwarf object that is passed to the DwarfStackTraceDecoder as a
positional argument is used for all lookups within the root loading
unit. If the dwarfByUnitId or unitDwarfs arguments contain an entry
for the root loading unit, it should be the same as the positional
argument.

When decoding a non-symbolic stack frame with a non-root loading unit
id, the decoder first looks in the map for the appropriate Dwarf object.
If one is not found, the decoder uses the build ID for the loading unit
to find the appropriate Dwarf object in the iterable container. If an
appropriate Dwarf object cannot be found in either manner, the
non-symbolic stack frame is emitted without change.

The native_stack_traces:decode executable now takes two additional
multi-options for the translate command:

* -u, --unit_debug: Takes a path to the associated DWARF information.
* --unit_id_debug: Takes N=FILE, where N is the loading unit ID and
  FILE is a path to the associated DWARF information.

The arguments to -u are collected into an iterable container to be
passed as the unitDwarfs argument to the DwarfStackTraceDecoder, and
the arguments to --unit-id-debug are collected into a map to be passed
as the dwarfByUnitId argument.

TEST=vm/dart/use_dwarf_stack_traces_flag_deferred

Issue: https://github.com/dart-lang/sdk/issues/53902
Change-Id: I210d4f69e4ae9fd37275a96beb1aac55c5e9d080
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-product-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2024-04-19 08:10:18 +00:00
Devon Carew 3b0b1bd863 Bump package:lints to the latest; address instances of new lints.
Changes:
```
> git log --format="%C(auto) %h %s" 8d5f750..b044aca
 https://dart.googlesource.com/lints.git/+/b044aca add several rules to core and recommended (150)
 https://dart.googlesource.com/lints.git/+/81100a2 fix a dangling table link (146)
```

Diff: https://dart.googlesource.com/lints.git/+/8d5f7500024320654adb1e799e49fc10c5304ae7..b044acab9f6669b3d8e781923a8ff86877801177/
Change-Id: I031333ade99af700a7009b14a36d3aadba12fc94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/327321
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: Phil Quitslund <pquitslund@google.com>
2023-09-25 21:15:15 +00:00
Vyacheslav Egorov a52f2b9617 [vm] Rework awaiter stack unwinding.
The main contribution of this CL is unification of disparate
handling of various functions like `Future.timeout`,
`Future.wait`, `_SuspendState.createAsyncCallbacks` and
`_SuspendState._createAsyncStarCallback` into a single
`@pragma('vm:awaiter-link')` which allows Dart developers
to specify where awaiter unwinder should look for the next
awaiter.

For example this allows unwinding to succeed for the code like this:

    Future<int> outer(Future<int> inner) {
      @pragma('vm:awaiter-link')
      final completer = Completer<int>();

      inner.then((v) => completer.complete(v));

      return completer.future;
   }

This refactoring also ensures that we preserve information
(including Function & Code objects) required for awaiter
unwinding across all modes (JIT, AOT and AOT with DWARF stack
traces). This guarantees users will get the same information
no matter which mode they are running in. Previously
we have been disabling awaiter_stacks tests in some AOT
modes - which led to regressions in the quality of produced
stacks.

This CL also cleans up relationship between debugger and awaiter
stack returned by StackTrace.current - which makes stack trace
displayed by debugger (used for stepping out and determinining
whether exception is caught or not) and `StackTrace.current`
consistent.

Finally we make one user visible change to the stack trace:
awaiter stack will no always include intermediate listeners
created through `Future.then`. Previously we would sometimes
include these listeners at the tail of the stack trace,
which was inconsistent.

Ultimately this means that code like this:

    Future<int> inner() async {
      await null;  // asynchronous gap
      print(StackTrace.current); // (*)
      return 0;
    }

    Future<int> outer() async {
      int process(int v) {
        return v + 1;
      }

      return await inner().then(process);
    }

    void main() async {
      await outer();
    }

Produces stack trace like this:

    inner
    <asynchronous suspension>
    outer.process
    <asynchronous suspension>
    outer
    <asynchronous suspension>
    main
    <asynchronous suspension>

And when stepping out of `inner` execution will stop at `outer.process`
first and the next step out will bring execution to `outer` next.

Fixes https://github.com/dart-lang/sdk/issues/52797
Fixes https://github.com/dart-lang/sdk/issues/52203
Issue https://github.com/dart-lang/sdk/issues/47985

TEST=ci

Bug: b/279929839
CoreLibraryReviewExempt: CL just adds @pragma to facilitate unwinding
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-dwarf-linux-product-x64-try
Change-Id: If377d5329d6a11c86effb9369dc603a7ae616fe7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311680
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2023-06-30 14:03:03 +00:00
Tess Strickland ecf1968122 [vm] Do not obfuscate code or object symbols in the static symbol table.
Also changes pkg/native_stack_traces to return a parsed MachO file
if the MachO file doesn't contain DWARF information, so we can examine
its static symbol information for the new test.

Issue: https://github.com/flutter/flutter/issues/124715

TEST=vm/dart/unobfuscated_static_symbols

Change-Id: I07d3ced56eeba852ebe4178dfd2b66ebb899eb76
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-mac-product-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305720
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-05-31 08:06:08 +00:00
Josh Soref 77978889eb Spelling
Closes https://github.com/dart-lang/sdk/pull/51143

GitOrigin-RevId: 9e21c99a222d588e4fc95980725a2f8c9784965c
Change-Id: If0870e8936c7649935dce7e23cd783d62aa5610c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279916
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-01-30 18:29:59 +00:00
Tess Strickland 73c639a4ba Reland "[native_stack_traces] Remove initial spaces check in stack trace lines."
This is a reland of commit ae4ee87601

Our Dart tests that check for symbolic stack traces assume the frame
number comes at the very start of the line. Thus, be lenient about
how many spaces we see before non-symbolic stack traces, but only
generate a prefix for the resulting symbolic stack traces if the
original stack trace had any initial non-whitespace content.

Original change's description:
> [native_stack_traces] Remove initial spaces check in stack trace lines.
>
> The strictness of the old "check for four spaces" was causing failures
> in `flutter symbolize`, and there's no reason to check how much initial
> whitespace we got prior to the line contents anyway.
>
> TEST=pkg/native_stack_traces/test/convert/regress_262474517_test
>
> Change-Id: I6a6e31732cb2a5b5d40a088b9a04877052726be2
> Bug: b/262474517
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276905
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

Bug: b/262474517
Change-Id: I3b0753404e00d535cf438e79078736f5d9a10dbc
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/277001
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-12-22 11:52:18 +00:00
Tess Strickland ba8db1976c Revert "[native_stack_traces] Remove initial spaces check in stack trace lines."
This reverts commit ae4ee87601.

Reason for revert: Broke integration tests on precompiled bots.

Original change's description:
> [native_stack_traces] Remove initial spaces check in stack trace lines.
>
> The strictness of the old "check for four spaces" was causing failures
> in `flutter symbolize`, and there's no reason to check how much initial
> whitespace we got prior to the line contents anyway.
>
> TEST=pkg/native_stack_traces/test/convert/regress_262474517_test
>
> Change-Id: I6a6e31732cb2a5b5d40a088b9a04877052726be2
> Bug: b/262474517
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276905
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

TBR=dacoharkes@google.com,sstrickl@google.com,dart-scoped@luci-project-accounts.iam.gserviceaccount.com

Change-Id: I83bfa2d09570d8ae4d244b987bd5acf855e57306
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: b/262474517
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/277000
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2022-12-21 17:13:48 +00:00
Tess Strickland ae4ee87601 [native_stack_traces] Remove initial spaces check in stack trace lines.
The strictness of the old "check for four spaces" was causing failures
in `flutter symbolize`, and there's no reason to check how much initial
whitespace we got prior to the line contents anyway.

TEST=pkg/native_stack_traces/test/convert/regress_262474517_test

Change-Id: I6a6e31732cb2a5b5d40a088b9a04877052726be2
Bug: b/262474517
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276905
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-12-21 12:47:54 +00:00
Tess Strickland 1d081c8bdd [vm] Add symbol size and type information to the assembly output.
Add symbol size and type information to the assembly output when
compiling for Linux-based platforms, so that the symbol tables in the
assembled output include that information.

Since symbol tables in Mach-O files do not include symbol size
information, we don't output either currently for MacOS or iOS targets.

TEST=vm/dart{,_2}/use_add_readonly_data_symbols_flag

Change-Id: I4219b898249153dc84214565e85ac9d3cf802538
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-gcc-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260820
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-09-27 16:45:53 +00:00
Tess Strickland 823934f1c5 Reland "[vm/compiler] Add symbols for read-only data when requested."
This is a reland of commit 286326f834

Checking into https://github.com/flutter/flutter/issues/108378, it
looks like the stripped snapshot indeed is unchanged, but the .ipa
being benchmarked includes .dSYMs, which is why there was a size
regression in adding this extra debugging information.

If that changes, we can remove the flag, but for now, I've added the
flag back so these symbols are not added by default.

TEST=vm/dart{,_2}/readonly_data_symbols

Original change's description:
> [vm/compiler] Add symbols for read-only data when requested.
>
> Symbols for non-clustered objects in the read-only data section are
now added to the static symbol tables for unstripped snapshots and
separate debugging information.
>
> In DEBUG mode, the name for a non-String read-only data object also
includes the name of the parent object.
>
> TEST=vm/dart{,_2}/readonly_data_symbols
>
> Change-Id: I623b023138aeca0580bc76392882eac5686f8f50
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251104
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Change-Id: I41be1c494c4324f1f3fae648d9832772c45bfbaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260522
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-27 16:45:53 +00:00
Tess Strickland 0a4cb4d43e [pkg/native_stack_traces] Add support for MacOS universal binaries.
In addition to adding a parser for the universal binary format, this
also requires major reworks to handle files that contain different
DWARF information for different architectures, and how to pass the
architecture down to where it's needed.

Also fix dSYM handling: instead of assuming the name of the MachO file
corresponds exactly to the basename of the dSYM with the extension
stripped, just look for the single file within the
Contents/Resources/DWARF directory.

Also add `unrecognized` enum entries for DW_TAG, DW_AT, and DW_FORM
values that aren't handled.

Issue: https://github.com/flutter/flutter/pull/101586
Change-Id: Ief5edc275ccd1192669252140d128136cd2bed26
Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-nnbd-mac-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252821
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-08-25 13:27:24 +00:00
Tess Strickland 24683da915 [vm] Add OS and architecture to non-symbolic stack traces.
Examples of the new line added to non-symbolic stack traces:

os: linux arch: x64 comp: yes sim: no
(Running on linux-x64c)

os: macos arch: arm64 comp: no sim: yes
(Running on mac-simarm64)

This CL also abstracts out the separate hardcoded strings across
the codebase for host and target OS and architecture into
definitions in platform/globals.h to ensure that they stay
in sync across different uses.

TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag

Issue: https://github.com/flutter/flutter/pull/101586
Change-Id: Ifdfea5138dd1003f561da0174e89aebc165bf9b0
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253283
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2022-08-05 11:53:37 +00:00
Zach Anderson 3b8817e7fc Revert "[vm/compiler] Add symbols for read-only data when requested."
This reverts commit 286326f834.

Reason for revert: Reverting for the regressions mentioned in https://github.com/flutter/flutter/issues/108378

Original change's description:
> [vm/compiler] Add symbols for read-only data when requested.
>
> Symbols for non-clustered objects in the read-only data section are
> now added to the static symbol tables for unstripped snapshots and
> separate debugging information.
>
> In DEBUG mode, the name for a non-String read-only data object also
> includes the name of the parent object.
>
> TEST=vm/dart{,_2}/readonly_data_symbols
>
> Change-Id: I623b023138aeca0580bc76392882eac5686f8f50
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251104
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: I82bdabf07c137fbabe7b4c45bdf23011350c3d87
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252801
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2022-07-27 13:30:28 +00:00
Tess Strickland 286326f834 [vm/compiler] Add symbols for read-only data when requested.
Symbols for non-clustered objects in the read-only data section are
now added to the static symbol tables for unstripped snapshots and
separate debugging information.

In DEBUG mode, the name for a non-String read-only data object also
includes the name of the parent object.

TEST=vm/dart{,_2}/readonly_data_symbols

Change-Id: I623b023138aeca0580bc76392882eac5686f8f50
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251104
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-07-25 11:03:29 +00:00
Tess Strickland 27f6c6d660 Reland "[pkg/native_stack_traces] Support Mach-O dSYM debugging information."
This is a reland of commit 08c13f173c

Fixes test failures on non-x64 architectures, both in the test
harness and due to DWARF5 line number program headers having a
non-backwards compatible format. (We generate DWARF2 in the
ELF snapshot writer, but the assembler used for assembly snapshots
may generate DWARF5.)

TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag

Original change's description:
> [pkg/native_stack_traces] Support Mach-O dSYM debugging information.
>
> TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag
>
> Bug: https://github.com/dart-lang/sdk/issues/43612
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
> Change-Id: Icda21bb14dcc0cf4784cea118e6ba7dd4edd35aa
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250381
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>

Bug: https://github.com/dart-lang/sdk/issues/43612
Change-Id: I8a9cb70e78bc8594bcae004809c5a1be778d691d
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-mac-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251464
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-07-22 07:05:26 +00:00
Ben Konyi a75e9cd3ee Revert "[pkg/native_stack_traces] Support Mach-O dSYM debugging information."
This reverts commit 08c13f173c.

Reason for revert: Causing failures on simarm, simarm64, etc

Original change's description:
> [pkg/native_stack_traces] Support Mach-O dSYM debugging information.
>
> TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag
>
> Bug: https://github.com/dart-lang/sdk/issues/43612
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
> Change-Id: Icda21bb14dcc0cf4784cea118e6ba7dd4edd35aa
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250381
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: https://github.com/dart-lang/sdk/issues/43612
Change-Id: I020c29f7329e9b53a8fe0f4f4a4de4070fca0ec3
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251242
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2022-07-13 16:15:45 +00:00
Tess Strickland 08c13f173c [pkg/native_stack_traces] Support Mach-O dSYM debugging information.
TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag

Bug: https://github.com/dart-lang/sdk/issues/43612
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Change-Id: Icda21bb14dcc0cf4784cea118e6ba7dd4edd35aa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250381
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2022-07-12 11:10:02 +00:00
pq bb47e41ab2 rename local functions with _s
These will be flagged by the next linter release which updates `non_constant_identifier_names` to flag local functions.

Change-Id: I873400f61748a145f3c2a6630a21bcd1975c309c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242503
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Phil Quitslund <pquitslund@google.com>
2022-04-27 00:29:34 +00:00
pq 47ac47fe90 lints 2.0 fixes
In anticipation of lints v 2.0.

(Note the ignores -- I was leery of making API changes but happy to with some guidance.)

See: https://dart-review.googlesource.com/c/sdk/+/237746

Change-Id: I93323e912911bbd62a583b379f0f8140a8ca448d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/237764
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Phil Quitslund <pquitslund@google.com>
2022-03-17 23:41:27 +00:00
Yogafire 098d670483 Expand the range of trace line numbers
00~999.
Solve the problems that more than 100 lines cannot be translated.

Closes https://github.com/dart-lang/sdk/pull/47947
https://github.com/dart-lang/sdk/pull/47947

GitOrigin-RevId: 540aeff9bb35dfcca4cdfd514928047f5f2ade57
Change-Id: I94625ad4b1cc59d9ecf186f1dd16fcb2e9fb7b29
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/224560
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-01-03 11:45:44 +00:00
Kevin Moore 3816e247c5 pkg:native_stack_trace - enable and fix a few more lints
Change-Id: I1c0c572bc87480a8f935c51965423f0d5fbe6d0f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216692
Auto-Submit: Kevin Moore <kevmoo@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-10-14 07:35:43 +00:00
Kevin Moore b0874f26b7 pkg:native_stack_traces - use Object.hash
Require Dart 2.14

Change-Id: Iec3e1b0563c6402840bfbcb66bd68478b40063c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216691
Auto-Submit: Kevin Moore <kevmoo@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-10-14 07:35:43 +00:00
Kevin Moore 0fb4f647cc pkg:native_stack_traces - Migrate to pkg:lints, fix new failures
Change-Id: I54aa779471c3cfb723be99238650bc786cf84643
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216690
Auto-Submit: Kevin Moore <kevmoo@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-10-14 07:35:43 +00:00
Kevin Moore ca00a4b27e pkg:native_stack_traces - fix existing lints
And validate lints as part of test matrix

Change-Id: Ia074c56f0c7a7f9a674202667ac4729d06993302
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216689
Auto-Submit: Kevin Moore <kevmoo@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-10-14 07:35:43 +00:00
Tess Strickland d16ad3d64b [vm/elf] Reorder non-NOBITS sections before NOBITS sections.
This ensures that the relocated addresses of sections with file contents
are contained in the loaded segment.

Fixes https://github.com/dart-lang/sdk/issues/47289

TEST=vm/dart{,_2}/use_save_debugging_info_flag

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: I6f7f94900ab1f9f0cb5ead4b0dd63bd2402e5a19
Fixed: 47289
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/214800
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-09-28 17:23:32 +00:00
Tess Strickland 010523ee4a [vm] Fix handling of WeakSerializationReferences in v8 profile.
Instead of trying to lazily create artificial nodes when needed for
WeakSerializationReference (WSR) targets in all cases, create them
eagerly for targets in reachable WSRs, since those are guaranteed
to be needed.

Those for unreachable WSRs are still lazily created as needed, since the
WSR won't even be accessed by the clustered snapshot writer unless the
unreachable WSR is part of another unreachable object that has an
artificial node created.

This rework avoids some issues seen on upcoming CLs where the artificial
nodes for WSR targets weren't getting correctly generated.

-----

Also extend the v8 snapshot profile writer tests to check the sizes of
the text and data sections in ELF snapshots. That means the v8 snapshot
profile writer tests check up to three different measures, from most
precise to least precise, depending on the output mode:

* If writing an ELF snapshot directly: the sum of the sizes attributed
  to the text and data section symbols are checked to be exactly the
  same as the sum of the sizes of the objects in the profile.

* If writing an ELF snapshot either directly or via assembly: the sum
  of the sizes of the text and data sections are checked to be the same
  as the sum of the sizes of the objects in the profile. If using an
  assembler that merges text and data sections, then account for the
  padding between the sections using an approximate check.

* For all: Check that the size of the snapshot as a whole is
  approximately equal to the sum of the sizes of the objects in the
  profile, accounting both for possible padding and for the global
  header information and non-data/non-text sections like the dynamic
  symbol section.

TEST=vm/data{,_2}/v8_snapshot_profile_writer_test

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-mac-release-simarm64-try
Change-Id: I66e0e7fdb5bb98045621bf516f150a4723e08147
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198942
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-05-11 11:21:44 +00:00
Alexander Markov a98500a139 [vm/aot] Discard Code objects of invisible functions
Previously, Code objects of invisible functions were retained
in order to omit frames corresponding to such functions from stack
traces when stack trace is printed.

This change drops Code objects of invisible functions. That also
means that frames corresponding to such functions are no longer
skipped in binary DWARF stack traces.

In order to account for that, DW_AT_artificial attribute is added to
generated DWARF debug information to mark invisible functions.
Stack trace decoding now looks at this attribute and skips those
frames when symbolizing stack trace.

Flutter gallery in release-sizeopt mode:
Heap size of snapshot objects -4.2% (arm), -4.4% (arm64).

A large application in --dwarf_stack_traces mode:
Number of discarded Code objects increased from 72.4% to 83.7%
(out of all Code objects).
Heap size of Code objects -37.4%.
Heap size of all snapshot objects -5%.

TEST=tests/standalone/dwarf_stack_trace_invisible_functions_test.dart
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: Ib804852aba1e083670f1d9b9d66cbaab7dcdcff9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196583
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-04-27 23:31:02 +00:00
Tess Strickland 927c59104f [vm] Re-tighten requirements that the node type be set before writing.
Fixes the case where the Image name and type was not written
to the v8 snapshot profile in non-precompiled mode.

Adds core-jit snapshot tests to the v8 snapshot profile writer test.

In core-jit and app-aot-elf modes, verify that all bytes in the
snapshot sections are accounted for in the profile.

Bug: https://github.com/dart-lang/sdk/issues/45787

TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try
Change-Id: I88be309cdea6293d768e2fa458492387d70fde22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196323
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-04-22 15:27:38 +00:00
Tess Strickland a8b42efa1a [pkg/native_stack_traces] Create stable null safety version.
Also remove assert no longer needed due to null safety.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: I1b19316801db7cb586d16e6bdc1f00014be93bdb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186286
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-02-23 10:27:39 +00:00
Tess Strickland d5649fc9f6 [vm] Add build ID to non-symbolic stack traces.
Since we've run out of room for more fields in the Image object header
on 64-bit architectures, the serializer instead creates an ImageHeader
object for precompiled snapshots that is placed at the start of text
segments. The new ImageHeader object contains the following information:

* The offset of the BSS segment from the text segment, previously
  stored in the Image object header.

* The relocated address of the text segment in the dynamic shared
  object. Due to restrictions when generating assembly snapshots, this
  field is only set for ELF snapshots, and so it can also be used to
  detect whether a snapshot was compiled to assembly or ELF.

* The offset of the build ID description field from the text segment.

* The length of the build ID description field.

We replace the BSS offset in the Image object header with the offset of
the ImageHeader object within the text segment, so that we can detect
when a given Image has an ImageHeader object available.

There are no methods available on ImageHeader objects, but instead the
Image itself controls access to the information. In particular, the
relocated address method either returns the relocated address
information from the ImageHeader object or from the initialized BSS
depending on the type of snapshot, so the caller need not do this work.
Also, instead of returning the raw offset to the BSS section and having
the caller turn that into an appropriate pointer, the method for
accessing the BSS segment now returns a pointer to the segment.

Bug: https://github.com/dart-lang/sdk/issues/43274
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try
Change-Id: I15eae4ad0a088260b127f3d07da79374215b7f56
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/163207
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2020-09-22 17:14:44 +00:00
Tess Strickland 37cfae173c [pkg/native_stack_traces] Make package NNBD compatible.
Bug: https://github.com/dart-lang/sdk/issues/43304
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: Iaf451047ce74add6520bd514bafae8b0e3c5fa59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161704
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-09-22 11:33:13 +00:00
Tess Strickland 3e40abbc09 [vm/aot] Reland "Keep column information when possible for precompiled mode."
Changes:

Doing this always in precompiled mode meant increased data segment sizes
when CodeSourceMaps are stored, since encoded line/column information is
larger in the LEB-like encoding used. Now we only store column
information when we produce non-symbolic stacks, since the increased
space needed to store the columns is instead in DWARF sections and can
be stripped or elided.

Original description:

Previously, we passed line number information to the stack trace printer
and to DWARF by changing the non-special positions in the CodeSourceMap
to line numbers in precompiled mode. However, doing this lost column
information.

We get the column information back in the majority of cases by encoding
the line number and column information when neither is too large to pack
together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
Otherwise, we just store the line information as before, though due to
using a bit to encode whether column info exists, it's reduced to 30
bits. If the line info is too big for that, we just return kNoSourcePos.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Ia8baee71468da6100a170fa305d03059ffd17f78
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151822
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-06-19 13:50:03 +00:00
Martin Kustermann db8a105de7 Revert "[vm/aot] Keep column information when possible for precompiled mode."
This reverts commit 7c34a80c11.

Reason for revert: This change seems to cause >2% code size regression
in flutter-release mode.

Original change's description:
> [vm/aot] Keep column information when possible for precompiled mode.
> 
> Previously, we passed line number information to the stack trace printer
> and to DWARF by changing the non-special positions in the CodeSourceMap
> to line numbers in precompiled mode. However, doing this lost column
> information.
> 
> We get the column information back in the majority of cases by encoding
> the line number and column information when neither is too large to pack
> together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
> Otherwise, we just store the line information as before, though due to
> using a bit to encode whether column info exists, it's reduced to 30
> bits. If the line info is too big for that, we just return kNoSourcePos.
> 
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
> Change-Id: Id1c826f10871e2f304fa40a59d8b704404d3a2c9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151507
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com,sstrickl@google.com

Change-Id: I5cf97543d1ac2731bb27bdb58ae97af6f22f2cfb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151820
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-06-18 19:57:44 +00:00
Tess Strickland 7c34a80c11 [vm/aot] Keep column information when possible for precompiled mode.
Previously, we passed line number information to the stack trace printer
and to DWARF by changing the non-special positions in the CodeSourceMap
to line numbers in precompiled mode. However, doing this lost column
information.

We get the column information back in the majority of cases by encoding
the line number and column information when neither is too large to pack
together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
Otherwise, we just store the line information as before, though due to
using a bit to encode whether column info exists, it's reduced to 30
bits. If the line info is too big for that, we just return kNoSourcePos.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Id1c826f10871e2f304fa40a59d8b704404d3a2c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151507
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-06-18 10:45:33 +00:00
Tess Strickland 6f83a5ff9b [vm/aot] Add a GNU build ID to direct-to-ELF snapshots.
For proper crashpad integration, we need to generate a build ID, as the
build ID generated by crashpad if there is not one will be a simple XOR
of the first text page, which rarely changes for Dart snapshots.
Assembly snapshots already have a build ID included by the assembler, so
we currently only do this for ELF snapshots.

Currently the build ID is a 128-bit hash value that is four separate
32-bit hash values concatenated together. Those hash values come from
the contents of the VM and isolate .text and .rodata sections.

This change also contains work to separate out the concepts of sections
and segments in the ELF builder. Now, consecutive allocated sections
with the same write and execute flags are combined into a single PT_LOAD
segment when possible, which reduces the padding needed to ensure that
segments start on page boundaries in ELF snapshots.

Bug: https://github.com/dart-lang/sdk/issues/42020
Change-Id: I42a837dae665a3902d881b8d151b49ede87d6c67
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150625
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-06-16 12:46:24 +00:00
Tess Strickland 71a8a3c11a [vm] Print DSO base of isolate when available.
Bug: https://github.com/dart-lang/sdk/issues/41880

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie68248a81167dfe05cbe65960fac8a8e029b9db3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147907
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-05-20 08:14:52 +00:00
Tess Strickland a6d06b59ec Reland "[vm] Mark snapshots that were compiled directly to ELF.
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF."

Storing the relocated address as an extra field in the Image header,
which requires increasing the Image header size on 64-bit platforms,
means Image pages cannot be used reliably as HeapPages as objects no
longer start after kMaxObjectAlignment bytes.

Instead, we return to an older design that just uses the lowest bit
in the BSS offset to store whether the instructions in an Image were
compiled directly to ELF.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Change-Id: I3819b0dc2719d69f5e8764ca8be8c6ae7171a7bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146560
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-05-11 08:41:17 +00:00
Tess Strickland 049949fdf4 Revert "[vm] Mark snapshots that were compiled directly to ELF."
This reverts commit 250b84987d.

Reason for revert: Failures on FFI CI tests.

Original change's description:
> [vm] Mark snapshots that were compiled directly to ELF.
> 
> Since the virtual addresses in ELF snapshots are the same as in
> separately saved debugging information, print the virtual address in
> non-symbolic frames again when running from a snapshot compiled directly
> to ELF.
> 
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
> Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

TBR=vegorov@google.com,rmacnak@google.com,dacoharkes@google.com,sstrickl@google.com

Change-Id: I1c5f58e9d9e11b994c1016970839f95b92885d43
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146220
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-05-04 09:52:25 +00:00
Tess Strickland 250b84987d [vm] Mark snapshots that were compiled directly to ELF.
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-05-04 09:19:23 +00:00
Tess Strickland 2d865dc69b [vm] Fix non-symbolic stack traces with --lazy-async-stacks.
Symbolic and non-symbolic stack traces had separate ToCString
implementations with the same general structure, but those
implementations diverged enough to cause issues in certain async stack
modes. They are now merged, with the --dwarf-stack-traces-mode flag
checked where appropriate within the merged method.

Also, now non-symbolic stack traces do not include frames for Code with
invisible Function owners unless the flag --show-invisible-frames is
enabled. With this change, pkg/native_stack_traces no longer needs to
guess at which frames corresponding to Dart code should be treated as
internal.

Tested by adding --dwarf-stack-traces version of the tests
in runtime/tests/vm/dart/causal_stacks.

Fixes https://github.com/dart-lang/sdk/issues/41578

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Change-Id: I41a887129616c88acd7729492addf7364d95df33
Bug: 41578
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143816
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-04-23 09:31:08 +00:00