When the unoptimized code of a function exceeds 1 MB, start its
optimized compile with far_branch_level=1 instead of 0. Without this,
the assembler fails on out-of-range conditional branches at level=0,
longjmps out of the optimizing pipeline, and the whole pipeline reruns
at level=1 from scratch.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted: JIT compile drops from ~26.6s to ~13.2s (about 50%).
Work towards https://github.com/dart-lang/sdk/issues/63230
TEST=ci
Change-Id: I2608a23ef899603a46b401b71be7322b7a8dd44b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506120
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
- Reduce header surface area.
- Remove runtime_entry_<arch>.cc, which no longer include code generation.
- Fix signature of DLRT_Enter/ExitSafepoint.
- Give non-leaf runtime functions C linkage, as C++ does not have a defined ABI.
TEST=ci
Change-Id: I21e21a50ad9e4f9b32b1204d755bfa69a2ae9b17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430981
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Static (direct) calls in the optimized JIT are performed through Code
objects without passing Function objects. This is not compatible with
bytecode interpreter calling conventions, so optimized JIT should
avoid static calls to bytecode functions.
This change disables devirtualized and guarded static calls to
functions declared in bytecode.
TEST=ci
Closes https://github.com/dart-lang/sdk/issues/60716
Change-Id: I95e4fa0706f376355f4fa7aea7fddf44f303e0a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/428340
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Previously catch blocks were hanging off the function entry blocks being effectively an alternative entry-point into the function. Now catch blocks are hanging off the try-entry blocks.
Previously all the variables that are used in the catch block and beyond were declared as parameters to catch block. Now only those that have their definitions not dominating catch entry will become parameters (for example, if a variable is assigned in the try-block, it becomes a parameter to catch block).
During OSR, if OSR target entry point is inside some try-blocks, then all of corresponding try-entry/catch-blocks are pulled up to the OSR entry forming a chain that ends with a jump to the OSR target entry.
TEST=vm/dart/trycatch*, ci
Change-Id: Iae20c6548d5d65c63be6d7a53c1b0d2adac7ac31
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356311
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This is a preparation to detach code generation from flow graph
building and optimization passes.
Extra inlining info (inline_id_to_function, inline_id_to_token_pos and
caller_inline_id) is now added to FlowGraph and can be serialized and
deserialized.
ic_data_array is not needed for optimized compilation and is no longer
passed.
A separate CompilerPassState is created for code generation.
TEST=ci (refactoring)
Change-Id: Ib61af47c2ddde1353b9a0fe24995d29d6e85acad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/399883
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
No client of the VM uses this flag, only tests, and this flag was always
set to false in AOT mode. Thus, remove uses of this flag and instead
always lazily create dispatchers as needed when resolving method names
in JIT mode.
Remove the implicit value of `allow_add` for some Resolver
static methods. For callers that previously depended on the implicit
`true` value (which includes the AOT precompilier), pass `true` for
uses in the compiler and pass `!FLAG_precompiled_mode` for uses in the
runtime. Assert that `allow_add` is false when these methods are invoked
from the precompiled runtime.
Remove Resolver static methods that are no longer used.
TEST=ci
Change-Id: Ib6a7354f7a859e86743c381513a4129c14895753
Cq-Include-Trybots: luci.dart.try:vm-linux-debug-x64-try,vm-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-mac-debug-arm64-try,vm-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/366668
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Specialized compiler pipeline for force-optimized functions is not
needed (force-optimized functions can be inlined into the arbitrary
functions and optimized using ordinary pipeline anyway) and error-prone
(certain optimizations could be missing by mistake).
This change unifies the pipelines by using ordinary pipeline for
force-optimized functions.
TEST=ci
Change-Id: Ia315abded61954e06acfc209d31ccce45c286437
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360960
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This reverts commit eebc219bab.
Reason for revert: performance regressions
Original change's description:
> [vm/compiler] Move handling of recognized methods from inliner to call specializer
>
> This refactoring reduces number of places where recognized methods are
> handled and cleans up general-purpose inliner pass.
>
> The code is mostly moved as is to simplify reviewing, with a notable
> exception: in a few places handling of recognized method calls is
> removed as it duplicates handling performed during call specialization
> passes (ApplyClassIds and ApplyICData) which run both before and after
> inlining.
>
> TEST=ci
>
> Change-Id: I8d53d23587beb09e6edf64ee95524f6ad3e060de
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358221
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
Change-Id: I9576bc4b385b5d0e0b728556defedb4962ec0a8a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359842
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
This refactoring reduces number of places where recognized methods are
handled and cleans up general-purpose inliner pass.
The code is mostly moved as is to simplify reviewing, with a notable
exception: in a few places handling of recognized method calls is
removed as it duplicates handling performed during call specialization
passes (ApplyClassIds and ApplyICData) which run both before and after
inlining.
TEST=ci
Change-Id: I8d53d23587beb09e6edf64ee95524f6ad3e060de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358221
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Make FlowGraph constructor compute whether we are planning to reorder
blocks before code generation or not. This makes the state of the
graph at the end of the compilation more clear.
This is just a refactoring without any functional changes.
TEST=ci
Change-Id: Ieefb02237cc1ebd69d5d2b217bdc8ebbfdbf15c7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358446
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This change is almost trivial. The closure is stored on the callback's
RawReceivePort, not in the VM. So we can basically just remove the CFE
check and it pretty much works. The only problem is that we can't set
function.FfiCallbackTarget anymore, so most of the CL is dealing with
that.
A few places were deciding whether an FFI trampoline was a call or a
callback based on whether function.FfiCallbackTarget() was null. But
now the target will be null for async callbacks. So instead I've added
a new value to the FfiCallbackKind enum (and renamed it), and changed
those checks.
Sync callback closures will be a separate CL, because they're more
complicated.
Bug: https://github.com/dart-lang/sdk/issues/52689
Change-Id: I8e5dfb557362e679f66195b735c3c382e6792840
TEST=async_void_function_callbacks_test.dart
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316160
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Force-optimized functions cannot deoptimize & reoptimize. Compiling them
always produces the same code.
If two mutators trigger lazy compilation of the same force-optimized
function we can let the first one win and let all others skip the
installation of code (detected in the installation phase where all
other mutators are stopped at safepoint).
This will avoid having multiple [Code] objects for the same
force-optimized function in normal circumstances.
Issue https://github.com/dart-lang/sdk/issues/52512
TEST=ci
Change-Id: I922728795943f006406feeca2e7a9f899859e800
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305920
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The ffi-callback related information on the [Thread] object is metadata
corresponding to ffi-callback-trampoline [Function] objects. There is
nothing thread or isolate specific about it.
Moving it away from [Thread] is needed because an [Isolate] can
have different [Thread] objects across its lifetime (see [0]): When
the stack of an isolate is empty, we reserve now the right to
re-cycle the [Thread]. If the isolate later runs again, it may
get a new [Thread] object.
This CL moves this information from [Thread] to the [ObjectStore]. In
addition we make the compiler be responsible for populating this
metadata - instead of doing this per-call site of
`Pointer.fromFunction()`. It will be preserved across snapshot writing
& snapshot reading (for AppJIT as well as AppAOT).
Similarly the JIT trampolines that are on Isolate aren't isolate
specific and can go to [IsolateGroup]. This simplifies doing the above
as the compiler can allocate those as well.
The effect is that [Thread] object gets smaller, GC doesn't have to
visit the 2 slots per-thread. It comes at expense of 2 more loads
when invoking the callback.
[0] https://dart-review.googlesource.com/c/sdk/+/297920
TEST=Regression test is vm/ffi{,_2}/invoke_callback_after_suspension_test
Change-Id: Ifde46a9f6e79819b5c0e359c3d3998d1d93b9b1e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303700
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Some cleanups factored out of a larger CL (which refactors enter/exit of threads):
* remove unused `#include "vm/thread_registry.h"`
* remove unused/unnecessary fields from [Thread] object
* rename IsMutator() -> IsDartMutator()
* make tests using setjmp() drain the sticky error
=> to ensure there's no sticky error on isolate shutdown
TEST=ci
Change-Id: I53935e8bd0628ab3768627d6d5e01c3f0e3a57ad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296582
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
During safepoint we can distinguish between
* owner of the safepoint operation (which is running code)
* everyone else (which are all blocked
Currently `Thread::IsAtSafepoint()` will return true for both. Since the
thread owning the safepoint operation is running, it's not actually
guaranteed that it's at "safe" point (e.g. to GC or to deopt) - it
really depends on what it's doing.
=> This CL will change it so that only actually parked threads will
have `Thread::IsAtSafepoint()`.
In order to do that we change varrious usages of `IsAtSafepoint()` to be
more precise:
* `Thread::OwnsSafepoint()`: True if this thread owns the
active safepoint. The thread is running.
* `Thread::OwnsGCSafepoint()`: True if the active safepoint is a GC
(or Deopt) safepoint and this thread owns it. The thread is running.
* `Thread::OwnsDeoptSafepoint()`: True if the active safepoint is a
Deopt safepoint and this thread owns it. The thread is running.
* `Thread::CanAcquireSafepointLocks()`: True if the thread is allowed
to acquire safepoint locks.
* `Thread::IsAtSafepoint()`: true if this thread is parked at a
safepoint
TEST=ci
Change-Id: I1a5a6727e84843ae79e0a344c438da19b7d6d916
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/295781
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Our compiler shouldn't depend on current isolate, since it can
run on any isolate within an IG.
Doing this change, reveals two existing dependencies on current
isolate from compiler
- resolving native symbols in unoptimized compilations
- issuing of debug events for breakpoints
For the former we'll re-enter the currently active isolate that
triggered unoptimized compilation.
=> We may want to change that embedder API to not be based on
handles and instead give embedder a simple `const char*`.
For the ladder we'll enter the isolate corresponding to the
breakpoint debug event to be issued. We are at place where
all mutators are stopped, so that does seem okish.
=> Future could remove this by making Object Id Ring per-IG
Issue https://github.com/dart-lang/sdk/issues/48523
TEST=service_2/break_on_function_many_child_isolates_test/dds
Change-Id: Id246db5972ae505e82f637ce04bb2302bed76257
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278901
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
We have many tests that explicitly use --optimization-counter-threshold=
flag. We also have a CI builder that uses this flag for all tests.
This is an issue for cases where `kernel-isolate` is not AppJIT'ed,
which is the case for simulators and ia32. Especially in debug builds
this causes a very-very slow time-to main due to JIT'ing the CFE code in
`kernel-isolate` and running flow graph checker etc (in debug mode).
This causes various tests to sporadically hit the timeout limit, become
flaky and require gardening attention.
As workarounds: the actual threshold was modified in tests, status files
were updated to mark tests as Pass,Slow etc.
=> This Cl will make `kernel-isolate` no longer be affected by the
`--optimization-counter-threshold`
=> This should make the cycle times faster on those modes and avoid
flaky timeouts that gardeners constantly have to pay attention to.
TEST=ci
Change-Id: Ia58e807b22f69f924315a43c6764427afe398ee6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266683
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Certain optimizations are disabled when compiling huge functions.
Also, compiling large functions may cause out of memory in compiler.
This change adds a warning in order to make it easier for users to
find out about huge functions in their programs and diagnose OOM
during compilation.
TEST=ci
Issue: https://github.com/flutter/flutter/issues/94461
Change-Id: Ibc7fb69df21fa96b8159d24c1b71ef8d690b23df
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/230160
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.
The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.
Quirks:
- RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
- There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
- The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
- For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
- Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
- There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
- The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
- We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.
Unimplemented:
- non-trivial FFI cases
- Compressed pointers - No intention to implement.
- Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
- BigInt intrinsics
TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This changes Profiler::DumpStackTrace to include information available
in the CompilerState attached to the thread into the crash report.
This includes the name of the function being compiled, compilation mode,
current phase, flow graph, instruction being compiled.
TEST=manually by causing compiler to crash
Change-Id: I310629f032a86781e30f0a5e218d21df7d27528e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220001
Auto-Submit: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Modestas Valauskas