Commit Graph

422 Commits

Author SHA1 Message Date
Daco Harkes 6c21348008 [vm] Introduce HAS_SMI_63_BITS
TEST=This is a refactoring, existing tests cover this.

Change-Id: I19c062bcf3d468814e2c2ecfacc9a65050925fc5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208650
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-08-03 13:19:01 +00:00
Alexander Aprelev 4f859fd133 [vm/concurrency] Notify tsan of load-acquire/store-release from generated code.
TEST=IsolateSpawn on tsan

Fixes https://github.com/dart-lang/sdk/issues/46493

Change-Id: I0134d7801e3cdb38f44f701fd40091fe13de7b77
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206664
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-07-15 13:26:56 +00:00
Ryan Macnak 3e83023d72 [vm] Fix vm/cc/AllocationSinking_Arrays and StoreIntoObject with compressed pointers.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/46468
Change-Id: I8bceea667cc2b1ed08dbccd93359387009eefb80
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206567
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-07-12 21:10:37 +00:00
Ryan Macnak 1bfc46ec2b [vm, compiler] Fix TestSmiInstr on X64 with compressed pointers.
Fixes various utf8 decoding tests.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/46468
Change-Id: I5ddc03cf72043da64a1107b59ef350f995e4a00b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206323
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-07-08 22:33:34 +00:00
Tess Strickland f709bf12f0 Reland "[platform] Fix Utils::IsAbsoluteUint and rename to MagnitudeIsUint."
datastream.h is included in places where we cannot import the compiler
namespace, so we can't make compiler::target::word the type of
WriteTargetWord. That also would possibly silently truncate a
host-word sized value that was passed in instead of alerting to a
possible issue if those bits were important.

However, just using IsInt fails if the value being passed in is
a compiler::target::uword that is larger than the max value
that fits in a compiler::target::word. Instead, check for
truncation by checking the bit length of the value, which works
for both signed and unsigned target word values.

Original description:

Also remove previous (incorrect) uses in datastream.cc and
image_snapshot.cc, now that Utils::IsInt<T>(N, value) can now
be run for values of N >= sizeof(T).

Fixes https://github.com/dart-lang/sdk/issues/46572

TEST=language/generic/super_bounded_types_test passes on NNBD simarm,
     added value that triggered failure to vm/cc/MagnitudeIsUint

Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-precomp-linux-debug-simarm_x64-try
Change-Id: Idbfdda9f28243d206d482a1d9ac7ae76a5022ad2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206201
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-07-08 14:22:54 +00:00
Liam Appelbe ecb6354a79 Revert "[platform] Fix Utils::IsAbsoluteUint and rename to MagnitudeIsUint."
This reverts commit 2ea5507fc3.

Reason for revert: Broke vm-kernel-precomp-linux-debug-simarm_x64

Original change's description:
> [platform] Fix Utils::IsAbsoluteUint and rename to MagnitudeIsUint.
>
> Also remove previous (incorrect) uses in datastream.cc and
> image_snapshot.cc, now that Utils::IsInt<T>(N, value) can now
> be run for values of N >= sizeof(T).
>
> Fixes https://github.com/dart-lang/sdk/issues/46572
>
> TEST=language/generic/super_bounded_types_test passes on NNBD simarm,
>      added value that triggered failure to vm/cc/MagnitudeIsUint
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-linux-release-simarm-try
> Change-Id: Ibc2d2bb5037a8fdee11fc26fa2a313149d3ca274
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206083
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Daco Harkes <dacoharkes@google.com>

TBR=vegorov@google.com,dacoharkes@google.com,sstrickl@google.com

Change-Id: I5194ba777d8811bb5557338c6b071ea152b93d44
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-linux-release-simarm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206143
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2021-07-07 20:14:55 +00:00
Tess Strickland 2ea5507fc3 [platform] Fix Utils::IsAbsoluteUint and rename to MagnitudeIsUint.
Also remove previous (incorrect) uses in datastream.cc and
image_snapshot.cc, now that Utils::IsInt<T>(N, value) can now
be run for values of N >= sizeof(T).

Fixes https://github.com/dart-lang/sdk/issues/46572

TEST=language/generic/super_bounded_types_test passes on NNBD simarm,
     added value that triggered failure to vm/cc/MagnitudeIsUint

Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-linux-release-simarm-try
Change-Id: Ibc2d2bb5037a8fdee11fc26fa2a313149d3ca274
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206083
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-07-07 12:39:37 +00:00
Zach Anderson f407419d0a [vm] Reland: Prefix HOST_OS_* and TARGET_OS_* with DART_
This relands https://dart-review.googlesource.com/c/sdk/+/205633
but without renaming TARGET_OS_IPHONE to DART_TARGET_OS_IPHONE.
It also changes uses of TARGET_OS_IOS to
DART_TARGET_OS_MACOS_IOS to be consistent with the rest of the
VM.

TargetConditionals.h for XCode 13 defines several
TARGET_OS_* preprocessor symbols that confuse the
Dart build. There is probably a more targeted fix
for this, but renaming the symbols that Dart uses
will also prevent this problem if more symbols
are added to the platform headers in the future.

See: https://github.com/dart-lang/sdk/issues/46499

TEST=It builds.

Change-Id: Ie775c19dd23cfdf5f65e5ebc6ee4ec3a561676fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205860
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-07-02 19:06:45 +00:00
Slava Egorov 42164cc140 Revert "[vm] Prefix HOST_OS_* and TARGET_OS_* with DART_"
This reverts commit aa9201b76b.

Reason for revert: blocks G3 roll (b/192627187)

Original change's description:
> [vm] Prefix HOST_OS_* and TARGET_OS_* with DART_
>
> TargetConditionals.h for XCode 13 defines several
> TARGET_OS_* preprocessor symbols that confuse the
> Dart build. There is probably a more targeted fix
> for this, but renaming the symbols that Dart uses
> will also prevent this problem if more symbols
> are added to the platform headers in the future.
>
> See: https://github.com/dart-lang/sdk/issues/46499
>
> TEST=It builds.
> Change-Id: I3b33a03b4a9a14b76d55fe12f8cdefec4b3c3664
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205633
> Commit-Queue: Zach Anderson <zra@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=rmacnak@google.com,zra@google.com,asiva@google.com

Change-Id: Ib06ca418c7e9d3b4df62c72c033cd39f462f7667
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205790
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2021-07-02 14:02:02 +00:00
Zach Anderson aa9201b76b [vm] Prefix HOST_OS_* and TARGET_OS_* with DART_
TargetConditionals.h for XCode 13 defines several
TARGET_OS_* preprocessor symbols that confuse the
Dart build. There is probably a more targeted fix
for this, but renaming the symbols that Dart uses
will also prevent this problem if more symbols
are added to the platform headers in the future.

See: https://github.com/dart-lang/sdk/issues/46499

TEST=It builds.
Change-Id: I3b33a03b4a9a14b76d55fe12f8cdefec4b3c3664
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205633
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-07-02 06:02:48 +00:00
Ryan Macnak 46222a1743 [vm] Fix gcc build.
TEST=local build
Change-Id: Ie1aa62cbcd73d2ab9a11286ddb546b18f42a25a9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205801
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-07-02 00:45:53 +00:00
Ryan Macnak 43311ae7ec [vm, compiler] Fix LoadCompressed[Smi]FromOffset on ARM64 with large offsets.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/46468
Change-Id: I3e03d371059a88307d6d6412120aa30dd54d4756
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205632
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-07-01 00:10:54 +00:00
Liam Appelbe 155ac440f7 [vm] Migrate Arrays and Instances to compressed pointers
Change-Id: I8264b340ee91d883afc52a52e83cda504f552e37
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191302
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-06-24 17:31:12 +00:00
Tess Strickland b8ecf06943 [vm/compiler] Add kUnboxedUint16 representation.
Adding this representation allows us to create slots for storing
and retrieving values from uint16_t-typed object fields in the runtime.

Also generalize BoxUint8Instr to BoxSmallIntInstr, which holds any
unboxed integer representation where the value is guaranteed to fit
into a Smi.

To unify the implementation of BoxSmallIntInstr across architectures,
we add ExtendValue(Register dst, Register src, OperandSize sz), which
extends the sz-sized value in src to a full register size in dst. If
sz is a signed OperandSize, then the resulting value is sign
extended appropriately.

We also add MoveAndSmiTagRegister / ExtendAndSmiTagValue, which is like
the base operation but also tags the value as a Smi. On some
architectures, this can be performed using a single instruction.

Also adds sbfx/ubfx on ARM for use in ExtendValue.

TEST=Existing tests for BoxSmallIntInstr, followup CL for the new
     representation.

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Change-Id: I9c0b3a23f5e28480e0bfbde55b9494f7b822dcfa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203761
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-06-24 12:15:48 +00:00
Ryan Macnak e5bc0f0b86 [vm] Make various arrays const.
TEST=build
Change-Id: I8d3445b87caa979472c9a37df62507f152a4aefb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203202
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-06-14 20:45:18 +00:00
Tess Strickland faa1c9ff98 [vm] Move context setting into AllocateClosure stub.
With this change, allocation instructions can now be direct inputs to
other allocation instructions, so the redundancy eliminator is extended
to handle this possibility.

Methods for working with instruction input-related slots are added to
subclasses of AllocationInstr, so that the redundancy eliminator can be
written more generically in places, instead of needing to add cases for
new allocation instructions and/or instruction inputs.

Code size different in Flutter gallery (release-sizeopt):
* ARM7: Total -0.30%, instructions -0.41%
* ARM8: Total -0.31%, instructions -0.47%

TEST=Current test suite.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-x64-try,vm-kernel-linux-debug-x64c-try
Change-Id: Idc1aa2a1cb8c0c62f0bcb64aee89a7525dd3d1e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198406
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-06-02 18:10:38 +00:00
Martin Kustermann 4d062c9e03 [vm/compiler] Avoid TTS running into instruction encoding limits
When TTSs access type argument vectors we should avoid using
Address/FieldAddress - which only work if offsets are guaranteed to be
within certain instruction encoding limits.

It also fixes Assembler::LoadCompressedFieldFromOffset on ARM64
to not assume (base, offset) is instruction encode-able.

TEST=vm/dart{,_2}/flutter_regress_82278_2_test

Change-Id: I9be316a068f222a6c1897c323111c4b18adb1e7a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/200222
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-05-18 07:19:53 +00:00
Martin Kustermann aa91e8c433 [vm/compiler] Ensure TTS stubs can handle types where the tav is at large offset
If the type arguments vector gets introduced in subclasses where the
base classes have already many fields we may not be able to load the TAV
in one instruction on ARM64.

Issue https://github.com/flutter/flutter/issues/82278

TEST=vm/dart{,_2}/flutter_regress_82278_test

Change-Id: I164ef42af3afe8267fe23a8a11af9401776eccdb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/199481
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-05-17 11:03:20 +00:00
Ryan Macnak b4c64a1787 [vm, x64] Shorter UnboxInt64Instr with compressed Smis.
TEST=ci
Change-Id: Ib80b9cd80f3656ea6c5f31e0dbc7229632d8783d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198140
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-05-12 17:02:10 +00:00
Ryan Macnak 3fa5667c1a [vm, arm64] Combine BARRIER_MASK and HEAP_BASE into one register.
BARRIER_MASK requires 8 low bits and HEAP_BASE requires 32 high bits. Combine them into a single register HEAP_BITS, and use ARM's shifted operands to continue accessing the relevant bits in a single instruction.

TEST=ci
Change-Id: I5a4cd43fec2d19615239ec5cec2ac282d17c461c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198100
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-05-12 15:41:40 +00:00
Tess Strickland fd31aa3f33 [vm] Move function stores into the AllocateClosure stub.
Now AllocateClosureInstr takes the closure function as a Value* instead
of as a const Function&, a new kFunctionReg is added to
AllocateClosureABI, and the AllocateClosure stub sets the closure
function internally instead of initializing it to null and requiring it
to be set post-allocation.

A convenience method known_function() is added to AllocateClosureInstr
which returns either the closure function, if known at compile time,
or null otherwise.

TEST=More refactorings, so just passing current tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-x64-try,vm-kernel-linux-debug-x64c-try
Change-Id: If843b401e0f282b191dd60b1c6c73c01f1d5bc70
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198285
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-05-11 16:54:14 +00:00
Tess Strickland 02a8dd90e1 [vm] Add a bare-bones AllocateClosure stub.
Also create a subclass of AllocationInstr, AllocateClosureInstr,
which is used when allocating closures.

Followup CLs add inputs to this instruction and to the AllocateClosure
stub, starting with the closure function. Adding these inputs allows
the related field to be set within the stub, instead of using
StoreInstanceField manually at each closure allocation point.

Since this CL only adds the initial stub/instruction implementation,
the overhead on snapshots is minimal: just the addition of the new
stub to each isolate.

-----

This CL also adds virtual and non-virtual methods to assembler_base.h
revolving around field loads and stores and attempted inline object
allocation, to ensure all architectures have these methods.

It also adds LoadFromSlot/StoreToSlot/StoreToSlotNoBarrier, which
appropriately calls one of the other methods based on whether the slot
is compressed or not and whether it is a boxed or unboxed field.

With these additions, the AllocateClosure stub generator can be defined
in an architecture-independent way.

TEST=Basically a refactoring, so check using current test suites.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-x64-try,vm-kernel-linux-debug-x64c-try
Change-Id: I71f5691307679f8d5e3604007699de4706f86eb8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198284
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-05-11 11:22:44 +00:00
Martin Kustermann 67ffc84c05 [vm] Share BoxAllocationSlowPath across architectures and use BoxABI::kResultReg
Slightly related to
Issue https://github.com/dart-lang/sdk/issues/45213

TEST=Existing test coverage.

Change-Id: I59953c9734b1e8cdba13a03c5734d084fc333130
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196929
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-05-10 12:08:49 +00:00
Martin Kustermann 8a4cba83a2 [vm/concurrency] Make box allocation slow paths go to separate runtime entry
The optimizing compiler can insert box instructions at various places
during the optimization pipeline. There is no guarantee that a box
instruction will get a proper (deopt-id, deopt-env) assigned to it - in
many places it will not.

Furthermore StoreInstanceField's into "unboxed fields" might cause
box allocations to happen. Those stores might also not have deopt
information associated with them.

In general we require stackmaps when allocation slow paths go to runtime
and cause GC. If GC throws OOM we theoretically would also need deopt
information in order to populate correct catch-entry state. Though OOM
is uncommon and the VM could decide to ignore the top-frame - if it's
missing deopt information - since box allocations aren't something
users are in control of (as opposed to user-allocated objects inside
try/catch).

=> This CL makes box allocations go to a runtime entry that doesn't
   support lazy-deopt. While being in those runtime entries the mutators
   will also not participate in "deopt safepoint operation" requests.

Issue https://github.com/dart-lang/sdk/issues/45213

TEST=Existing test suite.

Change-Id: I1b61f77e3166da82efad08bb49bc1756576d220c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196928
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-05-10 10:34:49 +00:00
Martin Kustermann 3c81d992ef [vm/concurrency] Distinguish "gc safepoint operations" from "deopt safepoint operations"
This extends the existing safepoint operation mechanism by allowing to
perform two different operations:

  * "gc safepoint operations": All mutators are stopped at places where
    it's safe to GC. It therefore requires stackmaps to be available for
    all optimized mutator frames.

  * "deopt safepoint operations": All mutators are stopped at places
    where it's safe to GC, but also safe to lazy-deopt mutator frames.
    It therefore requires deopt-id/deopt-info to be available for all
    optimized mutator frames.

Mutators can be asked to block for any of those two safepoint operations.
If a mutator is at a place where its safe to GC it will respond to "gc
safepoint operations" requests, if a mutator is additionally at a place
where it's also safe to lazy-deopt it will respond to "deopt safepoint
operation" requests.

Depending on how the runtime was entered (which is tracked via the
[Thread::runtime_call_deopt_ability_] value) - the mutator might
participate in both or only in gc safepoint operations.

During the start of a "deopt safepoint operation", the safepoint handler
will request all threads to stop at a "deopt safepoint". Some threads
might first want to initiate their own "gc safepoint operation"
(e.g. due to allocation failure) before they reach a "deopt safepoint".

We do allow this by letting the safepoint handler own a "deopt safepoint
operation" but still participate in other thread's "gc safepoint
operation" requests until all mutators are checked into places where
it's safe to lazy-deopt at which point the "deopt safepoint operation"
also owns a "gc safepoint operation".

In order to facilitate this, the Thread's safepoint_state will be
extended to consist of the following bits:

  * AtSafepoint
  * SafepointRequested
  * AtDeoptSafepoint
  * DeoptSafepointRequested
  * BlockedForSafepoint

Issue https://github.com/dart-lang/sdk/issues/45213

TEST=vm/cc/SafepointOperation_*

Change-Id: Icdc2827718f6780818f99b829a5e806d6bb5b130
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196927
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-05-10 09:13:09 +00:00
Tess Strickland 5d636c6261 [vm/compiler] Handle native unboxed fields in StoreInstanceField.
Also creates LoadFromOffset/StoreToOffset variants across all
architectures (was missing in some) that take the register to
load to/store from, the memory address, and the size to load/store.

Fixes LoadFromOffset on X64 to use MOVSXD when the size is kFourBytes.

Creates slots for the data field of TypedDataBase/Pointer objects
and replaces the StoreUntagged instruction with uses of
StoreInstanceField instead to create uses of the new version of
the instruction.

BUG=https://github.com/dart-lang/sdk/issues/42793

TEST=Tests that involve FFI pointers or TypedData constructors.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-x64-try,vm-kernel-linux-debug-x64c-try
Change-Id: I2c96e83bb086aa93c56b834e809e7141a32cfc35
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196924
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-05-03 16:22:19 +00:00
Ryan Macnak 7e1d1963e5 [vm] Switch TypeArguments to compressed pointers.
TEST=ci
Change-Id: Ic3162ef1558fb803ab68749f1f1a4d820645ec7e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194017
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-04-21 00:06:58 +00:00
Ryan Macnak a3f1dee0a0 [vm, arm64] Fix Address::OperandSizeFor.
TEST=ci
Change-Id: If9fa591e0da7f09ecdc408f1b181a50cc365e7d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196021
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-04-19 21:44:28 +00:00
Tess Strickland 0f8861e3ba [vm] Standardize hashing instance methods in C++.
In the codebase, we have several different interfaces for instance
methods that return a hash:

- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;

This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.

TEST=Existing test suites, as this is an internal refactoring.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-04-14 10:04:48 +00:00
Ryan Macnak a6ffc74a4a [vm] Switch most Instances whose layout is determined in C++ to compressed pointers.
Instances whose layout must agree with a layout determined by Dart code are not switched: Array, GrowableObjectArray, Closure, LinkedHashMap.

TEST=ci
Change-Id: I19baf6473a29dd14f84aa1e087eee16b0e6e9d77
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193442
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-04-05 17:43:16 +00:00
Tess Strickland e7d3a6027c [vm] Create more Function helper methods for code clarity.
Creates HasParent() as a predicate for Function objects with a non-null
parent function and makes IsLocalFunction() specific to local closures
(i.e. explicit closures with non-null parent functions).

Creates HasSavedArgumentsDescriptor() as a predicate to determine when
saved_args_desc() can be used, to avoid the caller having to do explicit
predicate checks for each dispatcher type that contains one.

Also simplifies Function::PrintName, creates different suffixes for a
SyncGenClosureMaker and its associateed SyncGenClosure, adds suffixes
for skipping more than one generated body when printing the parent name,
and changes ArgumentsDescriptor::PrintTo to match the compact syntax
Function::PrintName used for signifying the arguments descriptor for
certain dispatchers.

TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test

Cq-Include-Trybots: luci.dart.try:app-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,pkg-linux-release-try
Change-Id: I0ac1226d1ffe1d81743b5c6788da170d5a4010c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191560
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-03-23 12:21:26 +00:00
Tess Strickland d504084f42 [vm] Fix uses of NullIsAssignableTo at runtime.
NullIsAssignableTo only takes the type to check for nullability.
However, while a type parameter may itself be non-nullable, its
instantiation with a nullable type is nullable.

Thus, this CL adds type instantiation before runtime uses of
NullIsAssignableTo in Function::DoArgumentTypesMatch.

Fixes https://github.com/dart-lang/sdk/issues/45270

TEST=vm/dart/regress_45270

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Iaefffe9ab6d4d3ad889c4962489b2153d2bbaee1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190443
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2021-03-19 12:29:27 +00:00
Ryan Macnak 19e5899a02 [vm] Switch most non-Instances to compressed pointers.
The following are not compressed: Code, ObjectPool, Context, CallSiteData and subclasses.

Code::instructions_ is uncompressible because the VM doesn't control the address of Instructions when they are loaded via dynamic library.

Loads from the ObjectPool are a frequent occurrence in generated code, and shrinking ObjectPool entries would be offset by longer code sequences to decompress them. Also, some ObjectPool entries contain uncompressible pointers to C functions.

Context is not compressed based on reports from earlier work on compressed pointers that compressing them is disproporately bad for performance. The earlier work used a different, more-expensive compression scheme, so we should reinvestigate this.

CallSiteData et al. are on the hot path for unoptimized code but make up a comparatively small portion of the heap size.

TEST=ci
Change-Id: I9a425fa4e12113c9fd0b51999525d32a7e589a90
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190600
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-03-18 22:52:04 +00:00
Martin Kustermann 26f0eba03b [vm/compiler] Make AssertSubtypeInstr always call stub - including on ia32
This simplifies the code and makes it easier for later work
related to lazy-deopt of the instruction when returning from
runtime call.

Issue https://github.com/dart-lang/sdk/issues/45213

TEST=Existing test suite.

Change-Id: I056baac434532d3ed034ba488252ed0ff77eb488
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191041
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-03-17 13:16:49 +00:00
Ryan Macnak 8db0a975c1 [vm] Allow garbage in the upper-half of compressed Smis.
Allows removing a branch when decompressing pointers and removing fixup sign-extension after Smi ops that use 32-bit ops to detect overflow. Increases the code size of indexed and stores for some widths due to addressing mode limitations.

TEST=ci
Change-Id: Ia48353cccbbc586dd0d2e055a843c65e37c63a30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185660
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-03-11 19:39:01 +00:00
Ryan Macnak 8c1cb52c5a [vm, arm64] Fix encoding of immediates in some 32-bit operations.
TEST=ci
Change-Id: I53333355f04636ed49e9a05aeaf1378ea2dd35a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188601
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-03 23:54:45 +00:00
Ryan Macnak 1a83a454dd [vm, arm64] Fix simulation of 32-bit clz.
TEST=ci
Change-Id: Ie9cd4d39ed95ec3c770b94c40362c5827ceb4a76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186460
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-02-24 20:18:44 +00:00
Alexander Markov 7f8001218b [vm] Support unsigned shift operator int.>>>
TEST=runtime/tests/vm/dart/unsigned_shift_right_test.dart and other
tests for >>>.

Issue https://github.com/dart-lang/sdk/issues/30892
Issue https://github.com/dart-lang/sdk/issues/44914

Change-Id: I7f0a9e1e486be06576c5d9bff3659b0aa6d68c23
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184921
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2021-02-24 00:14:57 +00:00
Ryan Macnak a32b550034 [vm, arm64] Fix simulation of 32-bit sbfm.
Use the preferred disassembly for asr/lsr/lsl aliases of sbfm/ubfm.

TEST=ci
Change-Id: I80a16c3faa06885354b786bcbd078d0c1e39e387
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186080
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-02-24 00:02:58 +00:00
Alexander Markov ddfc98337b Reland "[vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces"
This is a reland of b6dc4dad4d

TEST=ci

Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
>   fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
>   now avoids using Code objects.
>   In order to find CompressedStackMaps it now calls
>   ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
>   a replacement for Code objects in stack traces. It has
>   PayloadStart() == 0 and Size() == kUwordMax so it includes
>   arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
>   frames are replaced with StubCode::UnknownDartCode().
>   This simulates future behavior of ReversePc::Lookup when Code objects
>   will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

Change-Id: Ia2fc4672a085cd963b7fc103369851df3603590c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186202
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-22 10:46:28 +00:00
Ryan Macnak 109d963bf9 [vm] Use 31-bit Smis in compressed pointer mode.
The range of Smis must be reduced so that pointer compression is not lossy.

Smis are kept in sign-extended form. In the future, we might track down all places that perform Smi comparisons or otherwise assume sign-extension. Allowing garbage in the upper half would make it safe to unconditionally add the heap base during pointer decompression.

Based on cf78da8a48.

TEST=ci
Change-Id: If8c76da3166c170618e5b3f3991353bab5c84e6f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181760
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-02-18 20:40:20 +00:00
Alexander Markov fde0ed2e93 Revert "[vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces"
This reverts commit b6dc4dad4d.

Reason for revert: broke package:vm_snapshot_analysis in Flutter
(https://github.com/flutter/flutter/issues/76313).

Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
>   fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
>   now avoids using Code objects.
>   In order to find CompressedStackMaps it now calls
>   ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
>   a replacement for Code objects in stack traces. It has
>   PayloadStart() == 0 and Size() == kUwordMax so it includes
>   arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
>   frames are replaced with StubCode::UnknownDartCode().
>   This simulates future behavior of ReversePc::Lookup when Code objects
>   will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I6f66171eecf1133363a7ce56193e782e43a20baf
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185488
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-02-18 18:04:29 +00:00
Ryan Macnak 2905d0835e [vm, arm64] Reserve a register for pointer decompression.
TEST=ci
Change-Id: I661cffec3de58fbbba70e13e9b4ad612574f494b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185200
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-02-18 17:47:37 +00:00
Alexander Markov b6dc4dad4d [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
The following changes are done in preparation for the removal of Code
objects in AOT with --dwarf-stack-traces:

* Stack trace objects are extended to hold uword PCs (which may not
  fit into Smi range).

* Scanning stack frames in GC (StackFrame::VisitObjectPointers)
  now avoids using Code objects.
  In order to find CompressedStackMaps it now calls
  ReversePc::FindCompressedStackMaps.

* Singleton Code object (StubCode::UnknownDartCode()) is prepared as
  a replacement for Code objects in stack traces. It has
  PayloadStart() == 0 and Size() == kUwordMax so it includes
  arbitrary PCs.

* In --dwarf-stack-traces mode, most Code objects obtained from stack
  frames are replaced with StubCode::UnknownDartCode().
  This simulates future behavior of ReversePc::Lookup when Code objects
  will be removed.

Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
TEST=ci
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-02-17 20:12:32 +00:00
Martin Kustermann b80ab44a53 [vm/concurrency] Move class registration from Isolate to IsolateGroup
We share the program across all isolates within a group. So hot-reload
should not depend on a particular isolate. This CL moves class
registration (which hot reload triggers) from Isolate to IsolateGroup.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing implementation.

Change-Id: Ibdb4e67a095c7f2f3e3c1a8f8194bd0ca936b140
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184266
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-02-12 07:53:46 +00:00
Ryan Macnak 34ce22dcc1 [vm] Remove "ctx" register name from ARM and ARM64.
We no longer have a fixed CTX register.

TEST=ci
Change-Id: Ie15a66077f69444e834bf14576dd7422512c1552
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184207
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2021-02-11 16:33:43 +00:00
Martin Kustermann 0fec35f7df [vm/concurrency] Remove redundant Isolate::shared_class_table_
Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing implementation.

Change-Id: I4e875ed0996f86a73c712c67da3d261949603512
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184264
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-02-10 20:48:05 +00:00
Vyacheslav Egorov d7a57a5df4 [vm] Add feature to connect AOT code to code comments via .debug_line
Option --write_code_comments_as_synthetic_source_to=file tells AOT
compiler to create a file which contains code comments for all code
objects written into an ELF snapshot and then emit a DWARF line number
program into .debug_line section which attributes code to code comments
in a way similar to how our own disassembler does it.

This enables tools like objdump to display our code comments as part of
the disassembly.

This CL also tweaks ifdefs in such a way that IL printer and code comments
facilities is now included into PRODUCT gen_snapshot binary.

TEST=manually run product build with --print-flow-graph and --write_code_comments_as_synthetic_source_to

Change-Id: Id6741013d43e1733b4ddeb34891a4d2fc06b9313
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181380
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-02 11:35:00 +00:00
Ryan Macnak 378dfc241d [vm, arm64] Add double-to-int32 conversion.
For compressed Smis. Ported from cf78da8a48.

TEST=ci
Change-Id: Iabdaf8ac0b7836c7656e60f5a3ddf19836fee8c7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181021
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-01-26 20:07:39 +00:00
Ryan Macnak b1c09ecd8f [vm] Make naming more consistent when converting between handles, tagged and untagged pointers.
Currently we have things called XPtr which are not what you get from ptr().

Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)

After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*

New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*

TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-15 23:32:02 +00:00