The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Factor out the .eh_frame section content generation from ElfWriter,
putting in as a static method in Dwarf instead.
Create a __eh_frame section in the __TEXT section of non-MacOS Mach-O
snapshots with the same information as .eh_frame in ELF snapshots.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/60307
Change-Id: I4e123f586e59e1263385c838b755c1575a6330eb
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-mac-release-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-fuchsia-release-x64-try,vm-fuchsia-release-arm64-try,vm-linux-debug-ia32-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv32-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-release-simarm_x64-try,vm-gcc-linux-try,vm-ubsan-linux-release-arm64-try,vm-aot-win-release-arm64-try,vm-aot-win-release-x64-try,vm-win-release-x64-try,vm-win-release-arm64-try,vm-aot-win-debug-arm64-try,vm-win-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/431600
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Fix the following compile-time errors:
C:\b\s\w\ir\cache\builder\sdk\runtime\vm\elf.cc(601) : error C2220: the following warning is treated as an error
C:\b\s\w\ir\cache\builder\sdk\runtime\vm\elf.cc(601) : warning C4715: 'dart::ElfSymbolTable::ElfSymbolType': not all control paths return a value
../../runtime/vm/elf.cc: In static member function ‘static intptr_t dart::ElfSymbolTable::ElfSymbolType(dart::SharedObjectWriter::SymbolData::Type)’:
../../runtime/vm/elf.cc:601:3: error: control reaches end of non-void function [-Werror=return-type]
601 | }
| ^
TEST=ci
Change-Id: Ib06339405b406ea4a6f6d395a04a16573d92c704
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430321
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
This CL pulls out the refactorings used to support the new MachOWriter
in a followup CL to allow them to be reviewed separately.
Rename Elf -> ElfWriter. Also rename model classes used by ElfWriter
for concepts that exist both in ELF and Mach-O to ElfX. For example,
the old ELF-specific SymbolTable is renamed to ElfSymbolTable.
Adds SharedObjectWriter to serve as a base class for both ElfWriter
and the upcoming MachOWriter.
Adds a new AbstractWriteStream that serves as a common superclass
of both BaseWriteStream and SharedObjectWriter::WriteStream and
allows the creation of fully delegating WriteStreams that do not
maintain a local buffer.
Abstract the old Elf::SymbolData class into
SharedObjectWriter::SymbolData, which stores an enum value as the type
of the symbol instead of storing the ELF encoding of the type.
Rename the DwarfElfStream (which actually wasn't ELF specific, as
all the ELF-specific DWARF information is handled by ElfWriter) to
DwarfSharedObjectStream and put it in a separate header file.
Rename Image::compiled_to_elf() to Image::compiled_to_shared_object()
and add a separate Image::compiled_to_elf() that checks for the ELF
magic value at the DSO base. Also add Image::shared_object_start()
and Image::build_id_start() to return pointers to the DSO base and
the build ID note, respectively.
Refactor Image::build_id() and Image::build_id_length() to check
compiled_for_elf() prior to decoding the data pointed to by
build_id_start() as an ELF note section.
Create an AOTSnapshotType enum to specific the snapshot writer to use
in CreateAppAOTSnapshot instead of using an as_elf boolean.
TEST=refactorings, so existing tests on ci
Change-Id: Ia3ab37a4dff93b6e00390b123753be5a51fbdaaa
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/421301
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This is a reland of commit e6d55c6c2f
Reworks guards for 64-bit Windows unwinding support code so that
it is not included in any 64-bit Windows gen_snapshot executable that
targets a non-64-bit Windows platform.
TEST=ci, build only
Original change's description:
> Add gen_snapshot binaries, producing Linux ARM64/x64 snapshots
>
> Inspired by `//runtime/bin:gen_snapshot_fuchsia`, required for go/dart-cross-compile.
>
> TEST=ci, build only
>
> Change-Id: Ie521c984a8f44d25f3f78d946af9416582a572dc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/410402
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
Change-Id: I4f1323ac8f7ab215c2dffeef188f70c466fb62ea
Cq-Include-Trybots: luci.dart.try:vm-win-release-arm64-try,vm-win-release-ia32-try,vm-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/413402
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
The algorithm for ordering sections in `CreateProgramTable` permitted
skew between the order of sections in snapshot and corresponding
debug info. This skew can lead to inconsistencies in `.text` section
memory offset between two generated ELF files making debug info
invalid.
This skew was left unnoticed for a long time because executable
segment has a huge alignment requirement (64K) while the effect
of the skew was usually rather small and thus hidden by realignment.
This CL adds a strict verification pass (always enabled) which
guards against such skew in the future.
A better architecture would be to ditch parallel generation of
snapshot and corresponding ELF files altogether and other implement
splitting as a post processing path over generated ELF or
rely on external tooling. But this is a much bigger change
than I am willing to invest time into right now.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try
Change-Id: I0881783237faa6534eb3c076b6cae472bbdc7243
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384742
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Dart VM changes:
Note that the following changes are backwards compatible in the
case that a Dart program has no deferred loading units (i.e., the
Dart program is contained in a single shared object snapshot).
When there are non-root loading units, the non-symbol stack trace
header now includes information about loading units as follows:
loading_unit: N, build_id: S, dso_base: A, instructions: A
where N is an integer, S is a string of hex digits (0-9a-f), and A
is a word-sized address printed as a hex string (without prefix).
In addition, all non-symbolic stack frames for isolate instructions
include a unit field, including those for the root loading unit, e.g.,
#NN abs <address> unit <id> virt <address> <symbol>+<offset>
If there are no non-root loading units, then the non-symbolic stack
trace is unchanged from its previous format.
Adds a build ID to split deferred loading unit snapshots.
Fixes: https://github.com/dart-lang/sdk/issues/43516
If separate debugging information is requested, the loading unit
manifest includes a 'debugPath' field for each loading unit,
which contains the path to its separate debugging information.
Removes the attempt to store the relocated address of the instructions
section when running from an assembled snapshot in the initialized BSS.
Adds OS::GetAppDSOBase, which takes a pointer to the instructions
section and returns a pointer to its loaded shared object in memory.
For compiled-to-ELF snapshots, it does this using the relocated address
of the instructions in the Image, and for assembled snapshots, it
delegates to NativeSymbolResolver::LookupSharedObject.
-----
Changes to package:native_stack_traces:
PCOffset now has two new fields:
* int? unitId: the unit ID of the loading unit, when available.
* String? buildId: the build ID of the loading unit, when available.
For PCOffsets in the VM section, the unitId and buildId are those of
the root loading unit.
The constructor for the DwarfStackTraceDecoder now takes two
additional optional named arguments:
* Map<int, Dwarf>? dwarfByUnitId: A map associating loading unit IDs
with the appropriate Dwarf object. May or may not contain an entry
for the root loading unit.
* Iterable<Dwarf>? unitDwarfs: An iterable container holding Dwarf
objects. May or may not contain an entry for the root loading unit.
The Dwarf object that is passed to the DwarfStackTraceDecoder as a
positional argument is used for all lookups within the root loading
unit. If the dwarfByUnitId or unitDwarfs arguments contain an entry
for the root loading unit, it should be the same as the positional
argument.
When decoding a non-symbolic stack frame with a non-root loading unit
id, the decoder first looks in the map for the appropriate Dwarf object.
If one is not found, the decoder uses the build ID for the loading unit
to find the appropriate Dwarf object in the iterable container. If an
appropriate Dwarf object cannot be found in either manner, the
non-symbolic stack frame is emitted without change.
The native_stack_traces:decode executable now takes two additional
multi-options for the translate command:
* -u, --unit_debug: Takes a path to the associated DWARF information.
* --unit_id_debug: Takes N=FILE, where N is the loading unit ID and
FILE is a path to the associated DWARF information.
The arguments to -u are collected into an iterable container to be
passed as the unitDwarfs argument to the DwarfStackTraceDecoder, and
the arguments to --unit-id-debug are collected into a map to be passed
as the dwarfByUnitId argument.
TEST=vm/dart/use_dwarf_stack_traces_flag_deferred
Issue: https://github.com/dart-lang/sdk/issues/53902
Change-Id: I210d4f69e4ae9fd37275a96beb1aac55c5e9d080
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-product-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This is the same metadata system that will support async callbacks. The
main thing this does is take the trampolines that used to be JIT only,
and use them in AOT too. This is partly a safety thing (there's some
extra checks that used to be skipped on AOT), but mostly just so that
the metadata system is unified between the sync and async callbacks.
More details about the design:
https://docs.google.com/document/d/1QDjyY_6wOTOgURwpeYMKU9qEz0gKxx2MUrdruC6Kp6c/edit?usp=sharing
I split this off the async CL. Some of the comments refer to async
stuff that doesn't exist yet, but it's coming immediately after this so
I didn't update them.
Change-Id: Icd5e86934ee9ae34c2c0e2ed2bbd1b928a7184ac
TEST=ffi_callback_metadata_test.cc and CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302903
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Instead of trying to make names assembler-friendly, instead just
quote the symbols names in the assembly output and use user visible
names for function and object symbols.
More specifically:
* This CL changes the symbol name from an assembler-safe version of the
scrubbed name to an assembler-unsafe version of the user visible name
that is then quoted when used in assembly output.
* For code symbols, regular stubs are prefixed with `stub`, allocation
stubs are prefixed with `new` and type testing stubs are prefixed with
`assert type is` (e.g., `stub Await`, `new RegExp`,
`assert type is List<Int>`). Function symbols have no prefix.
* Readonly data symbols begin with the type of the readonly data. If
there's any additional information (e.g., the function name for
objects like `PcDescriptors` when `SNAPSHOT_BACKTRACE` is defined),
that follows the type in parentheses.
* For direct-to-ELF snapshots, we allow symbols to have duplicate names.
Thus, internally we create unique ids for all created symbols and
refactor the ELF and DWARF subsystems to pass and store these ids
instead of the symbol names.
* For assembly output, symbols must have unique names. The namer now
has a `CStringIntMap` that keeps counts of the number of times a
given symbol name has been generated for assembly output. If the
symbol name is in the `CStringIntMap`, then the new count in the map
is added as a suffix to ensure uniqueness.
For example, the first symbol created using the name `_StringBase.[]`
has no suffix. If a second symbol is created using the same name,
the second symbol is instead named `_StringBase.[] (#2)`.
This also happens for read-only objects with additional information if
that additional information is not sufficient to make it unique, and
is added separately from that additional information.
* A further restriction for assembly output is that symbols that begin
with 'L' have special meaning to the assembler. Thus, we keep the
old system of prepending such symbol names with an underscore to
avoid this behavior.
TEST=vm/dart{,_2}/use_add_readonly_data_symbol_flag
Bug: b/248012965
Change-Id: I1b6e876f4fb4f42f4a3b90e03110a34a87a03a7c
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262242
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Even if the main program disables executable stacks, dlopen'ing a library that doesn't itself disable executable stack will switch the stack to executable. (Presumably in the name of compatability with GNU nested functions.)
Dart does not need executable stacks, and executable stacks are an additional attack surface.
TEST=readelf
Bug: b/229648756
Change-Id: Ia8c234ebc6178a26528d37b1359a10dd42039a9b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241540
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
We used to emit an unwinding rule for caller's SP, however
this does not really work as we expected it to work. Instead there is
an unwritten rule that caller's SP is equal to the CFA which overwrites
our unwinding rule (at least in libunwindstack library used by
Android).
This leads to unwinding issues on the boundary between C++ and Dart:
when we unwind through the InvokeDartCode stub frame we arrive to
C++ code with SP equal to callee's CFA, which is set to FP by our
unwinding rules. In this case if C++ code does not use FP based frames
we will fail to unwind C++ frame property - because SP will be
incorrect. This manifest in simpleperf profiles as stacks which
abruptly truncate at DartEntry::InvokeCode.
This CL rewrites unwinding rules to fix this: instead of using
FP as CFA we define CFA to be caller's SP (FP+2*kWordSize). This ensures
that unwinding on the boundary between C++ and Dart works correctly:
as CFA will always be consistently set to caller's SP.
Additionally this CL fixes InvokeDartCode stub to remove a value
that this stub pushes above saved return address in PRODUCT mode.
This value acts as a profiler marker used by builtin profiler
which is only included in non-PRODUCT builds - but the presence of this
value interfers with unwinding (caller's SP will be off by 1 due the
presence of this value).
We could produce custom unwinding information for this stub instead, but
just removing this value in PRODUCT builds should solve the problem
as we usually only profile Flutter release builds with simpleperf.
Fixes b/220804295
TEST=manually profiled a Flutter app and confirmed correct flamegraph.
Change-Id: I2094afaab6e54e89625c0b5a89aebc6b9823d67c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/235226
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.
The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.
Quirks:
- RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
- There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
- The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
- For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
- Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
- There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
- The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
- We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.
Unimplemented:
- non-trivial FFI cases
- Compressed pointers - No intention to implement.
- Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
- BigInt intrinsics
TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This ensures that the relocated addresses of sections with file contents
are contained in the loaded segment.
Fixes https://github.com/dart-lang/sdk/issues/47289
TEST=vm/dart{,_2}/use_save_debugging_info_flag
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: I6f7f94900ab1f9f0cb5ead4b0dd63bd2402e5a19
Fixed: 47289
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/214800
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
On Android native libraries can be mapped directly from an APK
they are stored uncompressed in it. In such situations the name
of the mapping no longer provides enough information for libunwindstack
to find the original ELF file and instead it has to rely on heuristics
to locate program header table. These heuristics currently assume that
program header table will be located in the RO mapping which precedes
RX mapping. This implies the following order of segments: RO (program
header, build id section if present, .rodata), RX (.text), RW (.dynamic
and .bss).
TEST=ci, manual
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: I3fa4de75c5d8841a90fd4cae1ab230d02c554f94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206620
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Previously in Elf::Relocation, symbol names had this mapping:
* nullptr: relative to the start of the snapshot
* ".": relative to the position of the relocation
* otherwise, relative to the given static symbol value
Change this as follows:
* nullptr: relative to the position of the relocation
* "": relative to the start of the snapshot
* otherwise, relative to the given static symbol value
This uses the fact that in ELF, symbol tables always have an initial
symbol with a name of "" that has a value of 0, so now we only have to
handle the nullptr case specially there, instead of having to handle
three different cases.
TEST=Refactoring, so existing tests of snapshots.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: Icb428551091e5fcf04facd34a5bf57ea0d664fa4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206544
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
To do this, BitsContainer is changed to be a more rope-like
representation of section portions. In addition to containing
most of the same information stored per-section previously, each
portion also has a section-relative offset that is calculated when
it is added. Thus, merging two compatible BitsContainer sections
is just adding the portions from the second to the first, tweaking
the section-relative offset for each.
Other changes in this CL:
* Create PseudoSections subclasses for the elf header, program
header table, and section header table, so we can treat them
more uniformly with the other parts of the ELF snapshot.
* We now only allocate as much BSS space in the snapshot as is needed
for any text sections in the snapshot, instead of always allocating
a big enough BSS space for both VM and isolate, even for deferred
snapshots where there is no VM isolate.
* We already separated segment and section alignment in previous CLs,
so the fact that our own ELF loader needs load segments to be
page-aligned no longer means that the sections within those segments
also needs to be. Thus, we align individual instructions sections to
kMaxObjectAlignment, like readonly data sections, since both hold a
single Image object. This removes unnecessary intra-section padding.
TEST=Tests that check DWARF information and trybots that use ELF
snapshots.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: If0315c8b7b0f31481b676a8901f49cd3a44b5561
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206365
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Create all symbol table-related sections after all other sections, right
before section re-ordering. Also remove the need to keep fields in the
Elf object for most of these sections, only keeping fields for the
static symbol table and the dynamic table, which handles updates
and finalization for the dynamic symbol table.
Reworks symbols so that they are stored in a growable array in the
symbol table instead of as separately allocated objects. Since this
disallows constant fields in Symbol objects due to the need for a copy
constructor, initialize the offset of symbols with the section-relative
offset. This can be adjusted to a snapshot-relative offset after
section memory offsets are computed.
Instead of updating indices and offsets by individually looking up
symbols via their name, just create a map from section indices to new
section indices (for index updates) or to memory offsets (for offset
adjustment) and iterate over all the symbols, applying the map
appropriately.
Move non-PT_LOAD, non-PT_PHDR segment creation into
OrderSectionsAndCreateSegments.
TEST=Refactoring, so existing ELF-based tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: Ic22f1bf3ab0b00ff3e73c431f92209526e787927
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206220
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
It's not used outside of elf.cc and rarely used as a type outside
of SymbolTable, so nest it to avoid defining dart::Symbol.
TEST=Refactoring, so existing tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: Idc8895b95b021edf831bc310aff0cf3106de8eb1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206367
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Previous CLs made it so that we don't need to calculate memory offsets
until finalization. This means that we can now reorder sections as
desired and then calculate the memory offsets at the same time as the
file offsets.
Thus, we reorder them so that writable, non-executable allocated
sections come first (due to requirements by some loaders to not sandwich
writable segments between non-writable ones), followed by non-writable,
non-executable allocated sections, followed by non-writable, executable
sections, and finally unallocated sections (to avoid differences in
memory offsets between snapshots and separate debugging information).
This ordering minimizes the number of segments that need to be created
(5 or 6, depending on whether a build ID exists).
Since we can reorder, we also delay the creation of the build ID (if
generated) and BSS sections until finalization, instead of creating them
up front in the constructor.
TEST=Further refactorings, so existing tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: Id4142a021c2bb800938e8bc711b1b8a7529bff51
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205780
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This CL splits the creation of ELF symbols into two parts: an
initialization that creates the symbol with all information but
the offset, and a finalization that sets the offset appropriately.
The initialization happens eagerly when adding sections, and the
finalization happens just prior to writing out the ELF contents,
after all memory and file offsets have been calculated.
To ensure safety, retrieving the offset from a symbol checks that
the symbol has been properly finalized.
This removes another obstacle to reordering sections and segments.
TEST=Further refactorings, so existing tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: I5dcb0d2c685fb341478e9a7d90c67fc4649a4e75
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205481
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This removes the need for the image snapshot writer to know the
starting memory offset of any ELF sections. Instead, relocation
and symbol information are recorded in the ELF writer to be
resolved during finalization.
This is a step towards being able to reorder sections in the
ELF memory space for better packing into segments. Delaying
relocation and symbol resolution also moves towards merging
multiple text or data sections into a single section if desired.
In addition, this CL removes redundant rows from the DWARF line
number program matrix to reduce the size of debugging information.
Originally, rows were written for every PC offset visited by the
CodeSourceMap. However, no row is needed if the source information
(file, line, column) is the same as the previously written row.
There are also fixes to Utils::{IsInt,IsUint,IsAbsoluteUint} to
allow N >= the bit size of the value.
TEST=Refactoring, so existing tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: I09f87cea214bca06b6fca60cd66138dae6da9e83
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203766
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
* Make sure to emit .eh_frame before .dynamic section to satisfy
a single non-writable segment requirement (see Elf::WriteProgramTable)
* Use compiler::target::kWordSize instead of kWordSize when aligning
CIE/FDE and for data alignment factor.
TBR=aam@google.com
TEST=dartkp-linux-debug-simarm-crossword
Fixes https://github.com/dart-lang/sdk/issues/46403
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try
Change-Id: Ieb73b7b90edcce91709201dc92caf0bb74ac1189
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204201
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
TEST=tested manually
This is a reland of ddfbe05eee
Original change's description:
> [vm/aot] Emit simple .eh_frame in ARM/ARM64 ELF snapshots.
>
> This allows simpleperf to unwind through AOT compiled frames and
> consequently allows Android tooling to produce useful
> timelines and flamegraphs.
>
> Note: emitted .eh_frame is rather primitive and does not cover all possible
> asynchronous unwinding cases (e.g. tick might arrive before Dart frame was
> fully setup). We will address this in a separate CL.
>
> Additionally, this CL tweaks .eh_frame emitted in assembly snapshots to
> workaround Android-specific unwinding bug in libunwindstack (b/191113792):
> current versions of libunwindstack don't push CFA value when evaluating
> DW_CFA_expression.
>
> TEST=tested manually
>
> Bug: b/184102680,b/190248517
> Change-Id: Iea209b2354cca0753385093f165b0320857104b1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203771
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
Bug: b/184102680,b/190248517
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try
Change-Id: Id888f906307ec3b2e73deec3cf1c2ead5672a2af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204100
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This reverts commit ddfbe05eee.
Reason for revert: broke ARM32 builds
Original change's description:
> [vm/aot] Emit simple .eh_frame in ARM/ARM64 ELF snapshots.
>
> This allows simpleperf to unwind through AOT compiled frames and
> consequently allows Android tooling to produce useful
> timelines and flamegraphs.
>
> Note: emitted .eh_frame is rather primitive and does not cover all possible
> asynchronous unwinding cases (e.g. tick might arrive before Dart frame was
> fully setup). We will address this in a separate CL.
>
> Additionally, this CL tweaks .eh_frame emitted in assembly snapshots to
> workaround Android-specific unwinding bug in libunwindstack (b/191113792):
> current versions of libunwindstack don't push CFA value when evaluating
> DW_CFA_expression.
>
> TEST=tested manually
>
> Bug: b/184102680,b/190248517
> Change-Id: Iea209b2354cca0753385093f165b0320857104b1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203771
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
TBR=vegorov@google.com,sstrickl@google.com
Change-Id: I57ab5df1d567bb4ddea813024245ef606712a79c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: b/184102680,b/190248517
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203778
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This allows simpleperf to unwind through AOT compiled frames and
consequently allows Android tooling to produce useful
timelines and flamegraphs.
Note: emitted .eh_frame is rather primitive and does not cover all possible
asynchronous unwinding cases (e.g. tick might arrive before Dart frame was
fully setup). We will address this in a separate CL.
Additionally, this CL tweaks .eh_frame emitted in assembly snapshots to
workaround Android-specific unwinding bug in libunwindstack (b/191113792):
current versions of libunwindstack don't push CFA value when evaluating
DW_CFA_expression.
TEST=tested manually
Bug: b/184102680,b/190248517
Change-Id: Iea209b2354cca0753385093f165b0320857104b1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203771
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This reverts commit 80102c981b.
Reason for revert: b/186359854
Original change's description:
> [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
>
> This reduces the number of variable-length integer encodings in our
> database from 2 to 1, and chooses the more standard one.
>
> TEST=Existing tests, in particular any that involve snapshots.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
> Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Idf32bdd879cf8bb7407f6dae764312140ad6eeb2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196920
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
This reduces the number of variable-length integer encodings in our
database from 2 to 1, and chooses the more standard one.
TEST=Existing tests, in particular any that involve snapshots.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is a reland of 5909fd111d
Does a large refactoring on the V8 snapshot profile writer
to clean things up, add more debugging support, and to fix
the problems that surfaced during the original landing.
Other changes:
Changes Serializer::CreateArtificialNodeIfNeeded() to create
artificial nodes for Code objects and immutable arrays.
Fixes CodeSerializationCluster::Trace() to only push needed parts of
discarded code objects, instead of tracing them like full code objects.
Adds test cases to v8_snapshot_profile_writer_test that exercise
the following situations (both separately and together):
* Non-symbolic stack traces are enabled and code and function objects
are dropped when not needed at runtime.
* Creation of the dispatch table is disabled.
TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test
Original change's description:
> [vm] Fix V8 snapshot profile handling of the dispatch table.
>
> Fixes https://github.com/dart-lang/sdk/issues/45702.
>
> TEST=Tests listed in the issue above.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
> Change-Id: Ibf5e3ccf3828c01f9dda47de360314dabe8cb8a9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195272
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I8e7030267fe190079a8f68d00fe20bf7170e5719
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195513
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Ryan Macnak