Commit Graph

418 Commits

Author SHA1 Message Date
Chase Latta dfb009d70a [fuchsia] allow injecting the inspect node
This CL allows users to inject the inspect node used inside the
os_fuchsia.cc file instead of having it get created automatically.
This allows us to use inspect in other areas of the flutter/dart
runners. This code needs to be soft transitioned so there will be
follow up CLs after the runners are migrated.

TEST=This was tested against the current version of the runner that does
not use these new codepaths and against a version that does.

Change-Id: I2fff36223aa4021da9cd2051daf6312d2b115492
Bug: fxbug.dev/69558
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190960
Commit-Queue: Chase Latta <chaselatta@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-22 17:38:24 +00:00
Tess Strickland 6d1ea68cc9 [build] Move from the C++11 to the C++17 standard.
Also includes some example changes using C++14/C++17 features.

TEST=vm/cc/BitField_Assert

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-linux-release-x64-try,dart-sdk-win-try,vm-kernel-win-release-x64-try,vm-kernel-win-release-ia32-try,dart-sdk-mac-try,vm-kernel-mac-release-x64-try
Change-Id: Icf5c5267431daf9cea2e61b67131021557675f3c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192183
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-03-19 15:29:57 +00:00
Tess Strickland 5fc2ba5fc4 [platform] Adjust Utils::StringHash() for misaligned starts.
Fixes https://github.com/dart-lang/sdk/issues/45290

TEST=vm/cc/StringHash

Cq-Include-Trybots: luci.dart.try:vm-kernel-ubsan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try
Change-Id: I7c1ef59cece77c6c9b705116e956e945bfcb55d4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190882
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-18 17:42:53 +00:00
Vyacheslav Egorov 69ec09825a [vm] On Android set abort message before aborting
Android L and newer (API level 21) provides a way to
specify abort message which will be included into
crash report created by debuggerd.

Make sure our DynamicAssertionHelper uses this functionality
if available.

TEST=tested manually

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-android-release-arm64-try,vm-kernel-mac-release-x64-try
Change-Id: If25cd168d43677e0a0fe9b63e21a017415686ab7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191140
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-15 15:35:10 +00:00
Vyacheslav Egorov d7a57a5df4 [vm] Add feature to connect AOT code to code comments via .debug_line
Option --write_code_comments_as_synthetic_source_to=file tells AOT
compiler to create a file which contains code comments for all code
objects written into an ELF snapshot and then emit a DWARF line number
program into .debug_line section which attributes code to code comments
in a way similar to how our own disassembler does it.

This enables tools like objdump to display our code comments as part of
the disassembly.

This CL also tweaks ifdefs in such a way that IL printer and code comments
facilities is now included into PRODUCT gen_snapshot binary.

TEST=manually run product build with --print-flow-graph and --write_code_comments_as_synthetic_source_to

Change-Id: Id6741013d43e1733b4ddeb34891a4d2fc06b9313
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181380
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-02 11:35:00 +00:00
Tess Strickland d017b8eee8 [vm] Compress CodeSourceMaps by folding opcodes and arguments.
Also delta encode the position/line argument for ChangePosition.

Code size changes on Flutter gallery in release mode from compression:

* ARM7: total -1.39%, readonly -6.47%
* ARM8: total -1.28%, readonly -4.81%

Additional code size changes on Flutter gallery in release mode from
delta encoding position/line:

* ARM7: total -0.48%, readonly -2.37%
* ARM8: total -0.54%, readonly -2.09%

TEST=Existing test on trybots, including stack trace tests and
DWARF decoding tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-ia32-try
Change-Id: Ibaf96b1d6c1916ce2f7c71942e333ca7a0b86c3e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175725
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-12-18 00:42:14 +00:00
Tess Strickland bb7081c6c7 [vm] Avoid undefined wraparound behavior for token position deltas.
TEST=Ran vm/cc/DescriptorList_TokenPositions on UBSAN trybot to verify.

Cq-Include-Trybots: luci.dart.try:vm-kernel-ubsan-linux-release-x64-try
Change-Id: I2fd5314a2420fd76ca4dacd85851060589e42a4a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176520
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
Auto-Submit: Tess Strickland <sstrickl@google.com>
2020-12-17 13:13:03 +00:00
Tess Strickland 66646f312c [vm] Signal bad token positions and lines in KernelLineStartsReader.
First, change LocationForPosition and TokenRangeAtLine to return
whether they successfully located the offset/line. Also change the
wrappers in Script (GetTokenLocation and TokenRangeAtLine) to return
whether or not any information was successfully located.

These methods now only change the out parameters in the case that
appropriate information was found, so any unconditional uses of the out
parameters need the out parameters to be appropriately initialized.

We now allow negative lines to be requested in TokenRangeAtLine (with
the failure behavior described above) so the line returned by
LocationAtPosition can be fed into TokenRangeAtLine without intermediate
checking.

The calculation of the token length, which uses the script source and
not the line starts array, has been moved to a new method,
Script::GetTokenLength.  The new method returns the length (or a
negative value if the token length could not be determined) instead of
using an out parameter.

TEST=Existing tests on trybots, to ensure tests using current
line/number info aren't affected.

Bug: https://github.com/dart-lang/sdk/issues/44436

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try
Change-Id: Ibc048a226d11ff9a340a8d249654d07720fdf115
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175365
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2020-12-16 11:14:12 +00:00
Martin Kustermann 868cb26a2d [vm] Make event handler use appropriate data structure for timers
Right now each isolate can have one port registered at the eventhandler.
When the timer expires the isolate will be notified. The isolate can
also cancel it's timer.

The timer implementation in the eventhandler is implemented as a linked
list, sorted by expiration date. Removing an entry for a given port will
perform a linear search. This O(N) search can be very problematic as the
number of isolates scales, effectively leading to O(N^2) behavior if N
isolates register/cancel N timers.

The priority_heap.h/priority_heap_test.cc was imported from the
implementation in "github.com/dartino/sdk".

Issue https://github.com/dart-lang/sdk/issues/44457

TEST=runtime/bin/priority_heap_test.cc

Change-Id: I657c1c1ac0c68ade5295b2569e3fb1a3478325ec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175729
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2020-12-16 10:55:12 +00:00
Alexander Markov 4ef857e5d8 [vm] Add buffering of --trace_precompiler_to output in memory
Under certain circumstances we see that gen_snapshot spends significant
portion of time in kernel. This happens with --trace_precompiler_to
option specified on the command line.

This change reduces number of I/O operations required to write
--trace_precompiler_to output by adding a memory buffer for the whole
output.

TEST=pkg/vm_snapshot_analysis/test/precompiler_trace_test.dart

Change-Id: I7c2b3fbf1f6b6bee950c2d23809e68e4698f84f4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175780
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2020-12-11 18:19:39 +00:00
Alexander Aprelev 16f163157a [vm/slot] Use seq_cst memory order for failure-branch of compare_exchange.
This fixes flutter engine test failure on Windows where it complained about invalid load memory order for failure branch(default=acq_rel) for this operation.
acq_rel is not allowed for failed comparison test memory order(per https://en.cppreference.com/w/cpp/atomic/atomic/compare_exchange)

TEST=flutter/engine runtime_unittests on Windows

Change-Id: I59dd3aed04f483b2e2a82721e337b78b8e3190d3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175101
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2020-12-04 20:38:52 +00:00
Daco Harkes 9f96aeda6f [test/ffi] Add C++ unit tests for all target ABIs
This CL introduces unit tests for the Native* classes in compiler/ffi
that can run for all supported target ABIs on any host architecture.

The unit tests are compiled for all target ABIs with
`tools/build.py run_ffi_unit_tests` and run for all target ABIs with
`tools/test.py ffi_unit`.

The unit test and tested code do not conceptually depend on having a
DartVM. The tests are compiled with a custom `dart::Zone` and
`platform/`. This enables compiling for all `TARGET_ARCH_*` and
`TARGET_OS_*` on any host, and running unit tests for all target ABIs
on any host.

Because the `run_ffi_unit_tests` executables do not include the DartVM
their build is quick (<10seconds) and they are small (~6MB) when
compared to `run_vm_tests` (~250MB).

The tests are added to the existing FFI QEMU bot to prevent adding an
extra bot which would add checkout overhead.

The unit tests themselves are set up to be fairly similar to vm/cc
tests. The only difference is the NativeCallingConvention tests which
are set up with `.expect` files for easy inspection and updating.

TEST=runtime/vm/compiler/ffi/native_calling_convention_test.cc
TEST=runtime/vm/compiler/ffi/native_location_test.cc
TEST=runtime/vm/compiler/ffi/native_type_test.cc

Change-Id: I7b8bf4de9ef070e7546472217e571a60362b9639
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171725
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2020-11-16 16:10:55 +00:00
Ryan Macnak b9c86f07dc [vm] Produce clearer error messages for malloc/realloc failures.
TEST=bots
Bug: https://github.com/dart-lang/sdk/issues/43642
Change-Id: I7bc2b3b5231413967f9b1f608c957326ff6c6aaa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171680
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-11-13 22:10:54 +00:00
Daco Harkes 00fe513296 [vm] Globals cleanup TARGET_ARCH_IS_*
There were two sets of defines of `TARGET_ARCH_IS_*` in `globals.h`.

`IS_SIMARM_X64` implies `TARGET_ARCH_ARM` implies
`TARGET_ARCH_IS_32_BIT`, so the second set of defines can be removed.

Closes https://github.com/dart-lang/sdk/issues/44177.

TEST=The build on all configurations.

Change-Id: I9f8d5ce9ebc7186c4fb36a7d8acbc6c0165c80d1
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-linux-debug-ia32-try,vm-ffi-android-debug-arm64-try,vm-ffi-android-debug-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171884
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2020-11-13 21:59:24 +00:00
Ryan Macnak 1d05591fbb [vm] Produce clearer error messages when operator new fails.
TEST=bots
Bug: https://github.com/dart-lang/sdk/issues/43642
Bug: https://github.com/dart-lang/sdk/issues/43862
Change-Id: I598a26a56762c141a1aef972f7d32f2c9594b244
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171802
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-11-13 19:07:20 +00:00
Tess Strickland 45a46ca2b8 [vm] Add (S)LEB128 encoding/decoding to BaseWriteStream.
Unlike cfc8e6de, this does _not_ replace the default variable length
encoding for {Read,Write}Streams, but insteads adds separate
{Read,Write}{S,}LEB128 methods to the appropriate classes. If we later
find the cause of the issues that led to the revert of cfc8e6de, it'll
be easy to switch over then.

Note that WriteLEB128 asserts that the value is non-negative if used
with a signed type (since negative values suggests that SLEB128 should
be used instead for minimal encoding).

Also removes the various other encoding and decoding methods for
(S)LEB128 across the codebase and changes those clients to use
{Read,Write}Streams instead.

Other cleanups:

* Various constant-related cleanups in datastream.h.

* Adds DART_FORCE_INLINE to ReadStream::ReadByte and uses it in the
  default variable length decoding methods for retrieving bytes
  from the stream instead of managing current_ by hand.

* Creates a canonical empty CompressedStackMaps instance and uses
  that instead of the null CompressedStackMaps instance in most cases.
  The only remaining (expected) use of the null CompressedStackMaps
  instance is for the global table in the object store when no global
  table exists (e.g., in JIT mode before any snapshotting).

* Moves CompressedStackMapsIterator from code_descriptors.h to an
  Iterator class within CompressedStackMaps in object.h (similar to
  PcDescriptors::Iterator), to limit friend declarations and because it
  conceptually makes more sense as part of CompressedStackMaps.

* Removed CompressedStackMaps::PayloadByte, since existing clients
  (CompressedStackMaps::Iterator, StackMapEntry in program_visitor.cc)
  are better served by just operating on the payload buffer directly
  (with appropriate NoSafepointScopes).

* WriteStreams no longer allocate their initial space on construction,
  but rather on the first write, so no allocation is performed by
  constructing a never-used WriteStream.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-ubsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: Ice63321abaa79157fbe9f230a864c8bba0e6dea9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166421
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-10-09 10:08:16 +00:00
Tess Strickland 1b50200abd [vm] Simplify symbol naming/address retrieval.
Before, picking a symbol name for a Code object when generating
assembly required adding a placeholder entry in the Dwarf object,
because the Dwarf object separately used a SnapshotTextObjectNamer
to determine the appropriate symbol name to use in relocations.

Instead, now the ImageWriter subclasses are entirely responsible for
creating the symbol name for a given Code object, and that name is
passed when adding the Code object to the Dwarf object.

For Elf output, we eagerly create symbols for sections and Code objects
again in the appropriate symbol table, and so looking up the start of a
given section or Code object can be done simply by asking for the
corresponding symbol's address, removing special case methods and other
workarounds.

We also clean up the generated static and dynamic symbol tables by
only generating local symbols for Code objects and BSS sections (these
are local in assembly output also), and only adding global symbols to
the dynamic symbol table.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie40c0c836681e98f345483136bb6860e5588ef30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166002
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-10-06 17:10:05 +00:00
Tess Strickland fca36cae39 [vm] Merge ImageHeader into InstructionsSection.
Before, we only wrapped the Instructions payloads in an
InstructionsSection in precompiled snapshots with bare instructions mode
enabled. This CL changes that to always add an InstructionsSection
wrapper in precompiled snapshots, even when not in bare instructions
mode, and moves all the information from the old ImageHeader object into
the InstructionsSection object instead. Note that in non-bare mode, the
InstructionsSection has no payload, and the Instructions objects follow
the InstructionsSection object instead of being contained by it.

This CL also wraps all implementations of AssemblyImageWriter and
SnapshotTextObjectNamer methods in a single DART_PRECOMPILER ifdef
block, since we only generate assembly in precompiled mode.  This
removes the need for the smaller DART_PRECOMPILER ifdef blocks scattered
across the AssemblyImageWriter method implementations.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: I3676fb5c6675845463707acbbb4759a465cfa342
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165563
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-10-06 17:10:05 +00:00
Tess Strickland 72cfc5c638 [vm] Clean up handling of payload-containing objects.
This CL adds a new PAYLOAD_SIZEOF specification to
runtime_offsets_list.h which defines InstanceSize methods given an
method name to invoke to get the header size (i.e., the size of
the object portion before the payload).

It uses this new specification to create appropriate InstanceSize()
methods for objects written to read-only sections of snapshots,
instead of needing separate size calculations for SIMARM_X64.

It adds more methods to Instructions to avoid special casing for bare
instructions mode. It also removes the special casing for SIMARM_X64,
serializing all read-only objects in the same manner even when not
in a crossword situation.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie3e4009f4bc03688998c32281e42fa22a255731d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165501
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-10-06 07:59:33 +00:00
Andrei Constantinescu e72adbc6ca [util] Allow USE() template to be used with Dart heap objects
Currently, Dart heap objects are non-copyable. For this reason, code such as

> const Function& f = ...
> USE(f);

would not compile. Doing this can be useful when doing initial testing runs of one's code. Of course, the problem generalises to other kinds of objects.

This CL fixes the issue by changing the template function to use a universal reference.
The other beneficial aspect of this change is that when a copyable class is passed to USE, it will no longer be copied.

Tested: all trybots, see results.

Change-Id: Ic1497067e2228b1b59806ddad8df9e85d87f4c27
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162502
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Andrei Constantinescu <andreicostin@google.com>
2020-09-15 09:50:02 +00:00
Tess Strickland 09ddcece30 [vm] Make BufferFormatter also a subclass of BaseTextBuffer.
Change-Id: I4d2759ffa80c0106838ad0ddbc6dd086fddda1dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157742
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-08-07 21:40:45 +00:00
Tess Strickland fa03ee7782 [vm] Create a common base class for TextBuffer/ZoneTextBuffer.
Generally, methods that take a *TextBuffer pointer do not care how the
internal buffer is allocated, and so they could be used for either if
both were subclasses of a base class that contained the printing
methods.  This CL makes that base class, and now TextBuffer and
ZoneTextBuffer now share the exact same set of methods for printing to
the internal buffer.

Since the base class is in platform, this does mean dropping the
overload of AddString for Dart String objects that was part of
ZoneTextBuffer.  Instead, this CL just adds an intermediate call to
ToCString() for the small number of callers that used the overload,
keeping the printing interface the same for both.

In addition, one use of TextBuffer that then re-allocated the buffer
contents into the zone manually has been replaced with a ZoneTextBuffer
instead.

Change-Id: I438a085e7e20d55d93987fd7f36afd636f95955f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157741
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-08-07 18:09:35 +00:00
Martin Kustermann d215ab6d73 [vm] Assign top-level classes cids outside 16-bit range
Right now we assign class ids to top-level classes, abstract classes as
well as concrete classes. All of them have allocated from a 16-bit pool
of ids. The VM FATAL()s once it hits that limit.

Customers who run very large programs (significant amount of generated
code) on the Dart VM have started to hit this 16-bit class limit.

Concrete classes can have instances in the heap. Our current heap layout
only allows 16-bit class ids to be encoded in the header word. To avoid
increasing the size of heap objects or shrinking the size of the identity
hash code to 16-bit we keep class ids in object headers to be 16-bit.

Abstract classes cannot have instances in the heap. Though their class
ids are encoded in type objects. Furthermore we sort classes in
AOT/AppJIT mode to perform fast class-id range checks. To avoid impacting
this optimization we treat abstract classes the same way as concrete
classes.

Top-level classes cannot have instances in the heap. Their class ids are
only used in the runtime code, for example for hot-reload as well as
part of the service protocol.

=> We can allocate class ids outside the 16-bit range for top-level
classes, thereby freeing a significant amount of space in the 16-bit
range.

This CL does exactly that: We change classid_t to be int32_t. The
ClassLayout::id_ can now be assigned ids outside 16-bit range for
top-level classes. To do this we keep dart classes and top level classes
as separate arrays in the ClassTable.

Issue https://github.com/dart-lang/sdk/issues/42533

See also b/160229360

Change-Id: I6710a644e7b0ab2d4f4c792bef8e1f91cb117421
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153607
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-07-09 18:33:32 +00:00
David Morgan e5086f9eac Support google3 DEBUG/NDEBUG handling.
Change-Id: I04beeb6c24969610255d52a1816e444e57dfe51f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153616
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: David Morgan <davidmorgan@google.com>
2020-07-09 09:51:31 +00:00
Aske Simon Christensen 3c3a00f331 [vm] Adjust internal UTF-8 encoder to match dart:convert.
The breaking change https://github.com/dart-lang/sdk/issues/41100
changed the UTF-8 encoder to encode unpaired surrogates as replacement
characters.

However, the VM contains its own, internal UTF-8 encoder, which is used
for printing and for the Dart_StringToUTF8 function in the Dart API.
Here, this encoder is changed to also encode unpaired surrogates as
replacement characters.

Fixes https://github.com/dart-lang/sdk/issues/42094

Change-Id: I9d55168f67d124dbc7987fb759696a98e7526c29
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149292
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2020-06-29 16:10:13 +00:00
Alexander Aprelev d33e51df3c [build] Support --clang when building dart sdk on Windows.
It relies on flutter copy of clang distribution, same one that is used to build flutter/engine.

It addressed several deprecated warnings from clang compiler for functions like strdup, unlink, etc.

It allows few warnings still since they are triggered in third_party sources.

Change-Id: Ieb13792c011438d46dbbc0fa030e1b5e4ea14315
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142704
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-06-25 03:36:41 +00:00
Liam Appelbe 764e72800f Reland "Revert "Add an --os=fuchsia option to build.py:""
This reverts commit d39d4d0d55.

Reason for revert: Relanding with fix

Change-Id: I6d6911b6cdc578a3b576b237db94c7f1db9824b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151600
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2020-06-18 00:24:31 +00:00
Tess Strickland 6f83a5ff9b [vm/aot] Add a GNU build ID to direct-to-ELF snapshots.
For proper crashpad integration, we need to generate a build ID, as the
build ID generated by crashpad if there is not one will be a simple XOR
of the first text page, which rarely changes for Dart snapshots.
Assembly snapshots already have a build ID included by the assembler, so
we currently only do this for ELF snapshots.

Currently the build ID is a 128-bit hash value that is four separate
32-bit hash values concatenated together. Those hash values come from
the contents of the VM and isolate .text and .rodata sections.

This change also contains work to separate out the concepts of sections
and segments in the ELF builder. Now, consecutive allocated sections
with the same write and execute flags are combined into a single PT_LOAD
segment when possible, which reduces the padding needed to ensure that
segments start on page boundaries in ELF snapshots.

Bug: https://github.com/dart-lang/sdk/issues/42020
Change-Id: I42a837dae665a3902d881b8d151b49ede87d6c67
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150625
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-06-16 12:46:24 +00:00
George Wright d39d4d0d55 Revert "Add an --os=fuchsia option to build.py:"
This reverts commit 4ce18ab931.

Reason for revert: bad merge, caused build breakages in flutter/engine

Original change's description:
> Add an --os=fuchsia option to build.py:
> 
> tools/build.py --os=fuchsia runtime create_sdk
> 
> This is analogous to --os=android. It cross compiles from Linux x64 to
> Fuchsia.
> 
> A lot of the build rules are just slightly different between the existing
> Fuchsia build rules used by Flutter, and the ones added by GN SDK. For
> example "$fuchsia_sdk_root/pkg:fdio" is now "$fuchsia_sdk_root/pkg/fdio".
> So to support this I had to add a new variable, using_fuchsia_gn_sdk,
> analogous to using_fuchsia_sdk. Flutter will need to set this to false.
> 
> Change-Id: Ief275d65f30a42a801607de93cf2d27a1fe825dd
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150689
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Liam Appelbe <liama@google.com>

TBR=rmacnak@google.com,asiva@google.com,liama@google.com,kaushikiska@google.com,matthewcarroll@google.com

Change-Id: If01ee34eba906c55f2d56ba650748a86e81e701c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151321
Auto-Submit: George Wright <wrightgeorge@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2020-06-15 23:01:53 +00:00
Liam Appelbe 4ce18ab931 Add an --os=fuchsia option to build.py:
tools/build.py --os=fuchsia runtime create_sdk

This is analogous to --os=android. It cross compiles from Linux x64 to
Fuchsia.

A lot of the build rules are just slightly different between the existing
Fuchsia build rules used by Flutter, and the ones added by GN SDK. For
example "$fuchsia_sdk_root/pkg:fdio" is now "$fuchsia_sdk_root/pkg/fdio".
So to support this I had to add a new variable, using_fuchsia_gn_sdk,
analogous to using_fuchsia_sdk. Flutter will need to set this to false.

Change-Id: Ief275d65f30a42a801607de93cf2d27a1fe825dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150689
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2020-06-15 20:05:16 +00:00
Martin Kustermann cfe45b6e3e [vm] Add opt-in flag to set priority for worker threads created by the VM
The newly added --worker-thread-priority can be used by users as an
opt-in to set the thread priority / nice value.

It is the responsibility of the user to ensure that the process has the
right to change the priority to the given value. Failure to set the
priority will be fatal.

See b/154918152

Change-Id: I3b3791e88b6ddf4fa6e39e4587f7fe1ab4d2312f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150560
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-06-10 19:57:02 +00:00
Tess Strickland f89d930e38 [vm/aot] Separate out stream methods from the Elf class.
Introduce an internal StreamingWriteStream wrapper and use that instead.

Also add more ELF-defined constants and use those appropriately.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-product-x64-try
Change-Id: I2bd34ebac60e789aea82c8f064c59351e2c55dd3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149291
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-05-29 07:47:08 +00:00
Aske Simon Christensen 81ad24322c [vm/compiler] Check BoxInt64 overflow via the overflow flag on ARM64.
This saves one instruction for every BoxInt64 overflow check, reducing
the instructions size for Flutter benchmarks by between 0.26% and 1.03%
(0.26% for Flutter Gallery).

Also add OVERFLOW and NO_OVERFLOW condition aliases to arm and arm64.

Change-Id: I82990419b448f21a22ea2cc7a15a9497d3275943
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145860
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-05-20 14:05:40 +00:00
Ryan Macnak 40d876382a [vm] Fix various UBSan failures: AOT.
Bug: https://github.com/dart-lang/sdk/issues/39427
Change-Id: I2594e28a277f1d9f838aa93d546714d4f300b5b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147682
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-05-12 20:28:11 +00:00
Ryan Macnak 17f3bf9183 [vm] Fix various UBSan failures: ARM.
Bug: https://github.com/dart-lang/sdk/issues/39427
Change-Id: Ie6ba61d699c1ea51310e15b924fb58cb02cf6e1b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146989
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-05-12 19:25:11 +00:00
Ryan Macnak 16253c0a54 [vm] Fix LSAN when used without ASAN.
Mostly interesting on Mac, where ASAN does not include LSAN.

Bug: https://github.com/dart-lang/sdk/issues/41811
Change-Id: Icbbe57946a6d3b3406a9e3a6541b6c4958cbfa91
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147685
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-05-12 17:38:19 +00:00
Ryan Macnak 2707880f1b [vm] Fix various UBSan failures.
Bug: https://github.com/dart-lang/sdk/issues/39427
Change-Id: I74e0eee623d88005fb2893d03e284a87daa09260
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146696
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-05-07 19:40:18 +00:00
Ryan Macnak 14dfa1b9ee [vm] Fix gcc build.
Change-Id: I6634dc10fdc8d7523562c0fcc20e3561eb580acf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146023
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-05-04 21:15:27 +00:00
Filip Filmar b161b52331 Use new call to get ComponentContext
Fuchsia OS is making a change in the serving sequence for component
startup, to avoid a race condition in which clients attempt to connect
to services that are not yet exposed but will be.

This change is needed for a smooth transition to that new API.  The
Fuchsia SDK change has already rolled to Dart.

See: https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=50758
Change-Id: Ib1249dcf771e99f542343328918ae55497ddaf71
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145700
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Filip Filmar <fmil@google.com>
2020-05-04 17:35:24 +00:00
Filip Filmar 595cd384d3 Add dart::ComponentContext() for Fuchsia
This allows the program to get at the singleton component context
for the currently running component.  The component context can
be meaningfully instantiated only once; and since dart doesn't
have a predefined way to wire platform-specific values into
platform-specific code, we instantiate a singleton here, so all
code that is interested can use it.

The users of this functionality will be the dart and flutter
fuchsia runners.  See issue
https://github.com/dart-lang/sdk/issues/41523 for details.

Tested:
  manual compile for fuchsia, check behavior.

Change-Id: Ib4e15af376ecb932c2d2b28d6f099383d53ccc84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136524
Commit-Queue: Filip Filmar <fmil@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-04-21 19:09:25 +00:00
Victor Agnez Lima 76bff1e198 Fix undefined behavior in UnboxedFieldBitmap
UnboxedFieldBitmap was relying that shifting operations would
result in 0 if `position` is larger than the number of bits in the
bitmap, but this is UB.

Closes https://github.com/dart-lang/sdk/pull/41422
https://github.com/dart-lang/sdk/pull/41422

GitOrigin-RevId: 7b6db75ee9247cee54345add6d155f55a501e79e
Change-Id: I27c6fcadb2149826de86067c2378cef57ee3367c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143000
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-04-14 14:58:29 +00:00
Ryan Macnak 4978dcc928 Reland "[vm, gc] Parallel scavenge."
Fix ScavengeVisitorBase::HasWork to account for undone allocations.

Bug: b/153524644
Change-Id: Ifc30017a411c5b65716d4e38752f2edeeb80a77e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142908
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-04-10 19:27:04 +00:00
Martin Kustermann 13c84d8794 Revert "Reland "[vm, gc] Parallel scavenge.""
This reverts commit 898b080d09.

Reason for revert: Seems to cause crashes in g3, see b/153524644.

Original change's description:
> Reland "[vm, gc] Parallel scavenge."
> 
> Remove header access from typed data case in RawObject::HeapSizeFromClass.
> 
> Address data race between visiting remembered cards and allocating new large pages for promotion.
> 
> Change-Id: I92cc4604ed38ab8f42b87b140a26a8118e7030d4
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142565
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

TBR=kustermann@google.com,aam@google.com,rmacnak@google.com,asiva@google.com

Change-Id: I3ef7bd5c3d0ca575f33cf5f712f58476037d2d90
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142861
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-04-08 14:21:32 +00:00
Ryan Macnak 898b080d09 Reland "[vm, gc] Parallel scavenge."
Remove header access from typed data case in RawObject::HeapSizeFromClass.

Address data race between visiting remembered cards and allocating new large pages for promotion.

Change-Id: I92cc4604ed38ab8f42b87b140a26a8118e7030d4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142565
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-04-07 17:07:06 +00:00
Ryan Macnak bb95f88139 Revert "[vm, gc] Parallel scavenge."
This reverts commit 9209f522d6.

Reason for revert: Flaky analyzer crashes

Original change's description:
> [vm, gc] Parallel scavenge.
> 
> Run N tasks that compete to copy or promote survivors. Installation of the forwarding pointer uses a CAS.
> Tasks do not share the list of copied objects to be processed.
> Tasks do share the list of promoted objects to be processed.
> Divide the freelist in N copies. Each scavenger task uses one without locking. The sweeper inserts into the freelists round-robin.
> 
> Change-Id: I54c7839ac5f15829462b3be7cf8c9635c01b785f
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135962
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,aam@google.com,rmacnak@google.com,asiva@google.com

Change-Id: I0d97b8d9148b2a81cabafde4602512e423eba223
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142554
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-04-06 22:51:08 +00:00
Ryan Macnak 9209f522d6 [vm, gc] Parallel scavenge.
Run N tasks that compete to copy or promote survivors. Installation of the forwarding pointer uses a CAS.
Tasks do not share the list of copied objects to be processed.
Tasks do share the list of promoted objects to be processed.
Divide the freelist in N copies. Each scavenger task uses one without locking. The sweeper inserts into the freelists round-robin.

Change-Id: I54c7839ac5f15829462b3be7cf8c9635c01b785f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135962
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-04-06 20:14:54 +00:00
Ryan Macnak 23792e7816 [vm] Require explicit loads and stores when using AcqRelAtomic.
Change-Id: I19d64667cea7dd735c3bf7194285383289019aaa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139942
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-03-19 16:19:59 +00:00
Ryan Macnak 9a140bab87 [vm] Use acquire-release ordering to publish and consume the class table backing arrays.
Change-Id: I57a3109b3ba42e1addf97c15c5d8108cd0338266
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139860
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-03-18 17:18:55 +00:00
Ryan Macnak 95f87f321a [vm, io] Fix some MemorySanitizer issues.
Change-Id: Id6e085563b05861351c431a9c067387b4b01270b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137380
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-02-27 17:50:43 +00:00
Victor Lima 7eedaf192f [vm/aot] Fix broken benchmarks
This CL fix an assertion inside Serializer::WriteInstructions
that was always failing in debug mode AOT 32-bit platforms
which used to happen when Utils::IsUint was called with a
signed type value was being passed.

It also fixes a bug that happens on some architectures that do not
support atomic unaligned load operations of 64-bit values.

Change-Id: Ic26cdeabccb55fcda5257da00f13dc400fa37e9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136228
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-02-19 20:49:29 +00:00