- Declare OS::SafeReadMemory in os.h
- Implement using pread64(/proc/self/mem) on Linux and Android
- Add _readNativeMemory VM Service RPC with pre-checks for null
and address overflow
TEST=runtime/vm/service_test.cc
Cq-Include-Trybots: dart/try:vm-asan-linux-release-x64-try,vm-asan-mac-release-arm64-try,vm-asan-win-release-x64-try,vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-arm64-try,vm-linux-debug-ia32-try,vm-linux-debug-simriscv32-try,vm-linux-debug-simriscv64-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-linux-release-simarm-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-tsan-linux-release-x64-try,vm-tsan-mac-release-arm64-try,vm-ubsan-linux-release-x64-try,vm-ubsan-mac-release-arm64-try,vm-ubsan-win-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try
Change-Id: Id15a82bf478bc4822c08d7fdf0a5c8bfd71a1fe0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505781
Auto-Submit: Nourhan Hasan <nourhan.m.hasan@gmail.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This CL makes it possible for the Dart VM to start or stop `perf` while
doing GC, which gives a significantly more stable output.
Example usage:
Bash script that creates the correct file handles and calls dart under
`perf stat` based on the arguments given to the script:
```
DART=$1
shift
AOTSNAPSHOT=$1
shift
ctl_dir=/tmp/
ctl_fifo=${ctl_dir}perf_ctl.fifo
test -p ${ctl_fifo} && unlink ${ctl_fifo}
mkfifo ${ctl_fifo}
exec {ctl_fd}<>${ctl_fifo}
ctl_ack_fifo=${ctl_dir}perf_ctl_ack.fifo
test -p ${ctl_ack_fifo} && unlink ${ctl_ack_fifo}
mkfifo ${ctl_ack_fifo}
exec {ctl_fd_ack}<>${ctl_ack_fifo}
perf_ctl_fd=$ctl_fd perf_ctl_fd_ack=$ctl_fd_ack perf stat --delay=-1 --control fd:${ctl_fd},${ctl_fd_ack} -B -e "task-clock:u,context-switches:u,cpu-migrations:u,page-faults:u,cycles:u,instructions:u,branch-misses:u" $DART --perf_ctl_fd=${ctl_fd} --perf_ctl_fd_ack=${ctl_fd_ack} --perf_ctl_usage=1 --deterministic $AOTSNAPSHOT $@
exec {ctl_fd_ack}>&-
unlink ${ctl_ack_fifo}
exec {ctl_fd}>&-
unlink ${ctl_fifo}
```
This will start `perf stat` paused (which would require the run dart
aot-compiled script to start it when it wants to) and where the VM
pauses `perf` while doing GC.
In practise, looking at instruction counts reported by `perf stat`, I've
seen the difference between runs go from 2+ mio (and in some instances
23+ mio) to around 30,000 (!) on runs of the analyzer (tool
"stable_analysis").
TEST=manually
Change-Id: Iab955a5dd35e47f22c4f693ae50effc8ee633897
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/468260
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
The command line flag allows the user to specify the minimum MacOS or
iOS version used in the build version load command for the Mach-O
dynamic library snapshot. If not specified, the minimum OS version
specified in macOS Mach-O snapshots is 10.15 (Catalina) and the minimum
OS version specified in iOS Mach-O snapshots is 13.
This CL also changes the targeted OS version in the build version load
command to be the same as the minimum OS version.
Adds support to parsing build version load commands to
pkg/native_stack_traces's Mach-O parser in order to test the
existence and contents of build version load commands in
vm/dart/unobfuscated_static_symbols.
TEST=vm/dart/unobfuscated_static_symbols
Issue: https://github.com/dart-lang/sdk/issues/60307
Change-Id: I3ee3ba34297d3261be7e3a1d2fb3c1da1ef0ef05
Cq-Include-Trybots: luci.dart.try:vm-aot-mac-debug-x64-try,vm-aot-mac-debug-arm64-try,vm-aot-linux-debug-arm64-try,vm-aot-linux-debug-x64-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-linux-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/438901
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Dart VM changes:
Note that the following changes are backwards compatible in the
case that a Dart program has no deferred loading units (i.e., the
Dart program is contained in a single shared object snapshot).
When there are non-root loading units, the non-symbol stack trace
header now includes information about loading units as follows:
loading_unit: N, build_id: S, dso_base: A, instructions: A
where N is an integer, S is a string of hex digits (0-9a-f), and A
is a word-sized address printed as a hex string (without prefix).
In addition, all non-symbolic stack frames for isolate instructions
include a unit field, including those for the root loading unit, e.g.,
#NN abs <address> unit <id> virt <address> <symbol>+<offset>
If there are no non-root loading units, then the non-symbolic stack
trace is unchanged from its previous format.
Adds a build ID to split deferred loading unit snapshots.
Fixes: https://github.com/dart-lang/sdk/issues/43516
If separate debugging information is requested, the loading unit
manifest includes a 'debugPath' field for each loading unit,
which contains the path to its separate debugging information.
Removes the attempt to store the relocated address of the instructions
section when running from an assembled snapshot in the initialized BSS.
Adds OS::GetAppDSOBase, which takes a pointer to the instructions
section and returns a pointer to its loaded shared object in memory.
For compiled-to-ELF snapshots, it does this using the relocated address
of the instructions in the Image, and for assembled snapshots, it
delegates to NativeSymbolResolver::LookupSharedObject.
-----
Changes to package:native_stack_traces:
PCOffset now has two new fields:
* int? unitId: the unit ID of the loading unit, when available.
* String? buildId: the build ID of the loading unit, when available.
For PCOffsets in the VM section, the unitId and buildId are those of
the root loading unit.
The constructor for the DwarfStackTraceDecoder now takes two
additional optional named arguments:
* Map<int, Dwarf>? dwarfByUnitId: A map associating loading unit IDs
with the appropriate Dwarf object. May or may not contain an entry
for the root loading unit.
* Iterable<Dwarf>? unitDwarfs: An iterable container holding Dwarf
objects. May or may not contain an entry for the root loading unit.
The Dwarf object that is passed to the DwarfStackTraceDecoder as a
positional argument is used for all lookups within the root loading
unit. If the dwarfByUnitId or unitDwarfs arguments contain an entry
for the root loading unit, it should be the same as the positional
argument.
When decoding a non-symbolic stack frame with a non-root loading unit
id, the decoder first looks in the map for the appropriate Dwarf object.
If one is not found, the decoder uses the build ID for the loading unit
to find the appropriate Dwarf object in the iterable container. If an
appropriate Dwarf object cannot be found in either manner, the
non-symbolic stack frame is emitted without change.
The native_stack_traces:decode executable now takes two additional
multi-options for the translate command:
* -u, --unit_debug: Takes a path to the associated DWARF information.
* --unit_id_debug: Takes N=FILE, where N is the loading unit ID and
FILE is a path to the associated DWARF information.
The arguments to -u are collected into an iterable container to be
passed as the unitDwarfs argument to the DwarfStackTraceDecoder, and
the arguments to --unit-id-debug are collected into a map to be passed
as the dwarfByUnitId argument.
TEST=vm/dart/use_dwarf_stack_traces_flag_deferred
Issue: https://github.com/dart-lang/sdk/issues/53902
Change-Id: I210d4f69e4ae9fd37275a96beb1aac55c5e9d080
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-product-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This reverts commit 6027bcb1e2.
Reason for revert: We are seeing TSAN failures in the bot runs
Original change's description:
> Call tzset() before localtime_r().
>
> POSIX and glibc do not guarantee that tzset() is called by localtime_r(). tzset sets the timezone name, UTC offset and whether or not it is daylight savings time.
>
> Android <= 7.1.1 did *not* call tzset() in localtime_r(), which means that users will see their timezone as "GMT".
>
> macOS/iOS seem to guarantee that tzset() is called by localtime_r() but the wording is vague:
>
> localtime_r() and gmtime_r() functions provide the same functionality
> as localtime() and gmtime()...
>
> N.B.: localtime() is guaranteed to call tzset().
>
> tzset() must be called before each localtime_r() to catch the case where the user changes the timezone while the application is running.
>
> Bug:https://github.com/dart-lang/sdk/issues/53276
> Change-Id: I0503a0a109aa6c281c9a3aefe8ba0b54841a42a7
> Tested: manually tested on Android 7.1.1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336240
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Brian Quinlan <bquinlan@google.com>
Bug: https://github.com/dart-lang/sdk/issues/53276
Change-Id: I4bbadf2b69eeac583b7da605c0686bcc6f72081f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336721
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
POSIX and glibc do not guarantee that tzset() is called by localtime_r(). tzset sets the timezone name, UTC offset and whether or not it is daylight savings time.
Android <= 7.1.1 did *not* call tzset() in localtime_r(), which means that users will see their timezone as "GMT".
macOS/iOS seem to guarantee that tzset() is called by localtime_r() but the wording is vague:
localtime_r() and gmtime_r() functions provide the same functionality
as localtime() and gmtime()...
N.B.: localtime() is guaranteed to call tzset().
tzset() must be called before each localtime_r() to catch the case where the user changes the timezone while the application is running.
Bug:https://github.com/dart-lang/sdk/issues/53276
Change-Id: I0503a0a109aa6c281c9a3aefe8ba0b54841a42a7
Tested: manually tested on Android 7.1.1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336240
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Brian Quinlan <bquinlan@google.com>
For direct-to-ELF snapshots, the story remains the same as before,
as we use the information from the Image header if available.
If it isn't, then we fall back to dladdr to get the dynamic shared
object containing the app snapshot and then walk the ELF or Mach-O
headers to find the build ID or UUID information.
TEST=vm/dart/use_dwarf_stack_traces_flag
Issue: https://github.com/dart-lang/sdk/issues/51941
Change-Id: I3705ed244d1b4a1255e75fffd238a29fc2a60800
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-mac-product-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-win-release-x64-try,vm-aot-win-product-x64-try,vm-aot-win-debug-x64c-try,vm-aot-android-release-arm_x64-try,vm-aot-android-release-arm64c-try,vm-fuchsia-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306640
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Querying the thread clock takes the overwhelming majority of time when using the VM-internal recorders (ring, startup, endless, file). Including the thread clocks in the timeline allowed visualizing roughly how much an event got scheduled out, but not when. This is a high price to pay for such fuzzy information. Precise scheduling information is available from Fuchsia tracing, systrace or Signposts.
```
import "dart:developer";
main() {
var n = 1000000;
var watch = new Stopwatch()..start();
for (var i = 0; i < n; i++) {
Timeline.timeSync("EventName", () {});
}
print("${watch.elapsedMicroseconds.toDouble() / n} us/event");
}
```
--timeline_recorder=ring 2.751822 -> 0.487936 us/event (5.64x)
--timeline_recorder=systrace 0.440439 -> 0.435997 us/event
TEST=ci
Bug: https://github.com/flutter/flutter/issues/121372
Change-Id: Ib6d625f28384a4e87df2a3305483f0f6facd08ff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286661
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Examples of the new line added to non-symbolic stack traces:
os: linux arch: x64 comp: yes sim: no
(Running on linux-x64c)
os: macos arch: arm64 comp: no sim: yes
(Running on mac-simarm64)
This CL also abstracts out the separate hardcoded strings across
the codebase for host and target OS and architecture into
definitions in platform/globals.h to ensure that they stay
in sync across different uses.
TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag
Issue: https://github.com/flutter/flutter/pull/101586
Change-Id: Ifdfea5138dd1003f561da0174e89aebc165bf9b0
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253283
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.
The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.
Quirks:
- RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
- There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
- The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
- For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
- Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
- There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
- The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
- We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.
Unimplemented:
- non-trivial FFI cases
- Compressed pointers - No intention to implement.
- Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
- BigInt intrinsics
TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This relands https://dart-review.googlesource.com/c/sdk/+/205633
but without renaming TARGET_OS_IPHONE to DART_TARGET_OS_IPHONE.
It also changes uses of TARGET_OS_IOS to
DART_TARGET_OS_MACOS_IOS to be consistent with the rest of the
VM.
TargetConditionals.h for XCode 13 defines several
TARGET_OS_* preprocessor symbols that confuse the
Dart build. There is probably a more targeted fix
for this, but renaming the symbols that Dart uses
will also prevent this problem if more symbols
are added to the platform headers in the future.
See: https://github.com/dart-lang/sdk/issues/46499
TEST=It builds.
Change-Id: Ie775c19dd23cfdf5f65e5ebc6ee4ec3a561676fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205860
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
TargetConditionals.h for XCode 13 defines several
TARGET_OS_* preprocessor symbols that confuse the
Dart build. There is probably a more targeted fix
for this, but renaming the symbols that Dart uses
will also prevent this problem if more symbols
are added to the platform headers in the future.
See: https://github.com/dart-lang/sdk/issues/46499
TEST=It builds.
Change-Id: I3b33a03b4a9a14b76d55fe12f8cdefec4b3c3664
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205633
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Option --write_code_comments_as_synthetic_source_to=file tells AOT
compiler to create a file which contains code comments for all code
objects written into an ELF snapshot and then emit a DWARF line number
program into .debug_line section which attributes code to code comments
in a way similar to how our own disassembler does it.
This enables tools like objdump to display our code comments as part of
the disassembly.
This CL also tweaks ifdefs in such a way that IL printer and code comments
facilities is now included into PRODUCT gen_snapshot binary.
TEST=manually run product build with --print-flow-graph and --write_code_comments_as_synthetic_source_to
Change-Id: Id6741013d43e1733b4ddeb34891a4d2fc06b9313
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181380
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
When system tracing is enabled on the Fuchsia platform, avoid calling
|OS::GetCurrentThreadCPUMicros|. The thread timestamp values are
discarded later, so querying them is wasteful w.r.t. both time and
trace buffer space.
Change-Id: I400075c89e69ddde6b032808859826c95f029008
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149395
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Nathan Rogers <nathanrogers@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Nourhan Hasan