Commit Graph

456 Commits

Author SHA1 Message Date
Alexander Markov 0e6b74543c Revert "[vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers"
This reverts commit ab6aeaa106.

Revert "[vm/compiler] Speed up the compiler part which deals with kernel reading up in DEBUG mode"

This reverts commit b316210d94.

Reason for revert: regression of snapshot sizes (DNO-599).

Original change's description:
> [vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers
>
> Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
> objects were referencing ExternalTypedData objects pointing into the middle of
> c-allocated memory. This made it impossible for the GC to determine when the last
> reference is gone.
>
> This CL ensures that the actual buffers are *always* made available via
> ExternalTypedData and any inner pointers into it are created via TypedDataViews.
>
> The embedder guarantees to the free kernel buffers it has provided to:
>     - Dart_CreateIsolateFromKernel
>     - Dart_LoadScriptFromKernel
>     - Dart_LoadLibraryFromKernel
>     - Dart_SetDartLibrarySourcesKernel
> on isolate shutdown.
>
> All other kernel buffers will get a finalizer attached, which ensures the
> kernel buffers get freed by the GC once they are no longer referenced:
>     - Kernel blobs for expression evaluation
>     - Kernel blobs for Hot-Reload
>     - Kernel blobs for cc tests
>
> Fixes https://github.com/dart-lang/sdk/issues/33973
> Fixes https://github.com/dart-lang/sdk/issues/36857
> Issue https://github.com/dart-lang/sdk/issues/37030
>
> Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,rmacnak@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: I49715d2400f4a5c8806b7d6a2912b7258f671a0a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104343
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
2019-05-31 22:15:51 +00:00
Martin Kustermann ab6aeaa106 [vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers
Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
objects were referencing ExternalTypedData objects pointing into the middle of
c-allocated memory. This made it impossible for the GC to determine when the last
reference is gone.

This CL ensures that the actual buffers are *always* made available via
ExternalTypedData and any inner pointers into it are created via TypedDataViews.

The embedder guarantees to the free kernel buffers it has provided to:
    - Dart_CreateIsolateFromKernel
    - Dart_LoadScriptFromKernel
    - Dart_LoadLibraryFromKernel
    - Dart_SetDartLibrarySourcesKernel
on isolate shutdown.

All other kernel buffers will get a finalizer attached, which ensures the
kernel buffers get freed by the GC once they are no longer referenced:
    - Kernel blobs for expression evaluation
    - Kernel blobs for Hot-Reload
    - Kernel blobs for cc tests

Fixes https://github.com/dart-lang/sdk/issues/33973
Fixes https://github.com/dart-lang/sdk/issues/36857
Issue https://github.com/dart-lang/sdk/issues/37030

Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-05-22 01:23:46 +00:00
Matthew Dempsky 6d5f763952 [vm] Eliminate Mutex/Monitor indirection where possible
In many cases, the Mutexes and Monitors have to be marked "mutable"
because they're used to synchronize const accessor methods.

Small text segment improvement for Product builds:

$ size dart.{arm,x64}.{before,after}
   text	   data	    bss	    dec	    hex	filename
19726069	 409960	 392332	20528361	1393ce9	dart.arm.before
19725525	 409960	 392332	20527817	1393ac9	dart.arm.after
22576021	 600376	1782824	24959221	17cd8f5	dart.x64.before
22574821	 600376	1782824	24958021	17cd445	dart.x64.after

Change-Id: I68f5cd5ad452044df8bfebd160910496036a3e6b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101745
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2019-05-13 21:17:51 +00:00
Matthew Dempsky 54a541706d [vm] Simplify IsolateSpawnState
Move SpawnCount logic into SpawnIsolateTask, where it logically makes
more sense. This eliminates some code and also prevents
IsolateSpawnState from having lingering pointers to the parent
Isolate's fields after it might have been reclaimed.

Change-Id: If8d827f299ee90dd54faf896609a7acdfbb87608
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101628
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
2019-05-11 00:12:08 +00:00
Matthew Dempsky 520ba89658 [vm] Remove IsolateSpawnState::package_root
Change-Id: I5a35f0e3c233353eafcb4e772abeeb6abc11daba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101626
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
2019-05-08 19:29:11 +00:00
Alexander Markov 6fe73e4544 [vm/bytecode] Support compact bytecode instructions in VM (part 2/3)
While adding support for new compact bytecode instructions,
VM also keeps support for old bytecode instructions to preserve
backwards compatibility and allow soft transition.

This change is separate from bytecode generator changes in order
to test VM with old bytecode generator.

Corresponding bytecode generator changes:
https://dart-review.googlesource.com/c/sdk/+/99400

Change-Id: Icf5ceee7d51f27ffe3f79d0eae81e0ddc0a7e855
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101062
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-05-08 17:33:15 +00:00
Matthew Dempsky a53c12d07a [vm] Use std::unique_ptr with Message
Message is a C++ type with a simple ownership model appropriate for
std::unique_ptr. This CL applies the following changes:

1. All uses of "new Message(...)" are replaced with
"Message::New(...)", which is effectively
"std::make_unique<Message>(...)". (The latter was only added in C++14,
but Dart still compiles in C++11 mode.)

2. All owning Message* are replaced with std::unique_ptr<Message>. The
notable exception is MessageQueue, which still uses raw Message*
internally to simplify the linked list handling.

3. All "delete message;" statements are removed.

4. Uses of "NULL" replaced with "nullptr" as necessary.

Change-Id: I05b5804289f2a225bfa05d3c1631129358fed373
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101222
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
2019-05-06 21:01:39 +00:00
Ryan Macnak e803d02227 [vm] Update Isolate to C++11 style.
Change-Id: Iff2ea5366a99a42394e663852dcfee25ab7109d9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100566
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2019-04-30 18:53:41 +00:00
Ryan Macnak 347b72c260 [vm] Bypass malloc for large Zone allocations to avoid jemalloc leaks.
Revert 9a07ad88f4.

Bug: https://github.com/flutter/flutter/issues/29007
Change-Id: I6a5f51f0c3a54d354ec5f8495677d46f94d8a1d3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100568
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2019-04-30 16:17:18 +00:00
Ryan Macnak 9a07ad88f4 [vm, irregexp] Cache the backtracking stack on the isolate to avoid memory leaks in jemalloc.
Bug: https://github.com/flutter/flutter/issues/29007
Change-Id: Ic08e6cdad087cb063350b0738a65f426e67864d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100500
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2019-04-25 20:08:52 +00:00
Ryan Macnak ec27b09e68 [vm, service] Fix deferral of service extension invocation to not be mislead by other OOB messages.
This was the behavior intended in 7052bd6bed.

Bug: https://github.com/dart-lang/sdk/issues/24137
Bug: https://github.com/dart-lang/sdk/issues/36612
Change-Id: I2d0712d89ff45cd480580e73c4fd55ac4cf2ca59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99360
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2019-04-17 17:34:46 +00:00
Samir Jindel a2709992a2 [vm] Support optcounter=1 and randomize it for testing.
Change-Id: I1d371ca9644a65f8fe9864ac708689c1469da17d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97242
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2019-04-05 10:15:44 +00:00
asiva dc9370a07d [vm/runtime] Create separate background compilation queues for optimized and unoptimized compilations.
- create separate compilation queues for optimized and compilation
queues
- remove option FLAG_background_compilation_stop_alot doesn't seem to be used in any format testing
- remove some class finalization checks before starting the background compiler

Change-Id: I98f1773987e33a43e7b43ace4c383d17bd0a1710
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98453
Reviewed-by: Zach Anderson <zra@google.com>
2019-04-03 00:58:43 +00:00
Siva Annamalai 057497e57b Revert "[vm/runtime] Create separate background compilation queues for optimized and unoptimized compilations."
This reverts commit bcb6df13b9.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> [vm/runtime] Create separate background compilation queues for optimized and unoptimized compilations.
> 
> - create separate compilation queues for optimized and compilation
> queues
> - remove option FLAG_background_compilation_stop_alot doesn't seem to be used in any format testing
> - remove some class finalization checks before starting the background compiler
> 
> Change-Id: I09feb500c2a5cc9c6e93347f63bfca41b83b35cf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98301
> Reviewed-by: Zach Anderson <zra@google.com>
> Commit-Queue: Siva Annamalai <asiva@google.com>

TBR=zra@google.com,asiva@google.com

Change-Id: I3faf74c95dce7741c75f8836cce30f9dba2e154c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98448
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2019-04-02 01:17:24 +00:00
asiva bcb6df13b9 [vm/runtime] Create separate background compilation queues for optimized and unoptimized compilations.
- create separate compilation queues for optimized and compilation
queues
- remove option FLAG_background_compilation_stop_alot doesn't seem to be used in any format testing
- remove some class finalization checks before starting the background compiler

Change-Id: I09feb500c2a5cc9c6e93347f63bfca41b83b35cf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98301
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2019-04-02 00:32:33 +00:00
Ben Konyi ac2c934563 Reland "[ VM / dart:isolate ] Added ability to set names for spawned isolates."
This reverts commit 638f13ff3d.

Change-Id: Ib5015539b1c93791c2469be6995789f8f0362806
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97845
Commit-Queue: Ben Konyi <bkonyi@google.com>
Auto-Submit: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2019-03-28 16:39:50 +00:00
Daco Harkes 638f13ff3d Revert "[ VM / dart:isolate ] Added ability to set names for spawned isolates."
This reverts commit 59525264e8.

Reason for revert: causes Flutter test observatory and protocol to deadlock.
Issue: https://github.com/dart-lang/sdk/issues/36232

Original change's description:
> [ VM / dart:isolate ] Added ability to set names for spawned isolates.
> 
> Fixes issue #34059
> 
> Change-Id: I315498b02edc184e9e408c93eddb78aa1a5a8a1d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/90341
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Sigmund Cherem <sigmund@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=bkonyi@google.com,rmacnak@google.com,asiva@google.com,sigmund@google.com

Change-Id: I5f2115a2ac394a8d4c7c175bc97f2b88b65fcb49
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97107
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2019-03-15 14:40:37 +00:00
Ben Konyi 59525264e8 [ VM / dart:isolate ] Added ability to set names for spawned isolates.
Fixes issue #34059

Change-Id: I315498b02edc184e9e408c93eddb78aa1a5a8a1d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/90341
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-03-14 22:23:09 +00:00
Ben Konyi 1efe7e2518 [ VM / Service ] Added getMemoryUsage RPC and MemoryUsage object to service protocol
Change-Id: If15e49d7aab93382c15529a738b492552faf5376
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/95489
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-03-14 21:53:49 +00:00
Ryan Macnak 57f4da318c [vm, gc] The mutator's TLAB and the interpreter's lookup cache must be visited even if the mutator is unscheduled.
Remove strange conditional the made this a no-op before heap verification.

Bug: TC-384
Change-Id: Ic5aee093c6ee37df5d61dc5c22c79097a73d43e2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96400
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2019-03-11 20:17:46 +00:00
Martin Kustermann 72548cde13 Revert "[vm, gc] The mutator's TLAB and the interpreter's lookup cache must be visited even if the mutator is unscheduled."
This reverts commit 253499fe68.

Reason for revert: This seems to have caused new heap verification failures, e.g.

python tools/test.py -n dartkp-linux-debug-x64 lib_2/convert/streamed_conversion_json_utf8_decode_test
...                                                                                                                                                                                                                                                              
stderr:                                                                                                                                                                                                                                                       
Verifying before Scavenge...../../runtime/vm/raw_object.cc: 36: error: New object missing kNewBit: f3f3f3f3                                                                                          
                                                                                                           
version=2.2.1-edge.253499fe68b1efaed5c2be699b4e59af3703c0f3 (Thu Mar 7 01:01:02 2019 +0000) on "linux_x64"
thread=219499, isolate=isolate(0x55efd3734900)                                                            
  [0x000055efd10532e6] dart::Profiler::DumpStackTrace(bool)                                           
  [0x000055efd10532e6] dart::Profiler::DumpStackTrace(bool)                                   
  [0x000055efd1053224] dart::Profiler::DumpStackTrace(void*)                                        
  [0x000055efd13a6805] Dart_DumpNativeStackTrace                                                                                                                                                                                                             
  [0x000055efd13b7313] dart::Assert::Fail(char const*, ...)                                                                                                                                                                                                   
  [0x000055efd1073ac8] dart::RawObject::Validate(dart::Isolate*) const                                                                                                                                                                                        
  [0x000055efd1379c51] dart::VerifyObjectVisitor::VisitObject(dart::RawObject*)                                                                                                                                                                              
  [0x000055efd1376a43] dart::Scavenger::VisitObjects(dart::ObjectVisitor*) const                                                                                                                                                                             
  [0x000055efd136098b] dart::Heap::VisitObjectsNoImagePages(dart::ObjectVisitor*) const                                                                                                                                                                      
  [0x000055efd13632f3] dart::Heap::CreateAllocatedObjectSet(dart::Zone*, dart::MarkExpectation) const                                                                                                                                                        
  [0x000055efd1363522] dart::Heap::VerifyGC(dart::MarkExpectation) const                                                                                                                                                                                      
  [0x000055efd1377144] dart::Scavenger::Scavenge()                                                                                                                                                                                                            
  [0x000055efd1361b06] dart::Heap::CollectNewSpaceGarbage(dart::Thread*, dart::Heap::GCReason)                                                                                                                                                                
  [0x000055efd1360330] dart::Heap::CollectMostGarbage(dart::Heap::GCReason)                                                                                                                                                                                   
  [0x000055efd135fff5] dart::Heap::AllocateOld(long, dart::HeapPage::PageType)                                                                                                                                                                               



Original change's description:
> [vm, gc] The mutator's TLAB and the interpreter's lookup cache must be visited even if the mutator is unscheduled.
> 
> Bug: TC-384
> Change-Id: Ic3dcc2fea3e80ef181f8891ccb00a8e2e33fc077
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/95670
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

TBR=aam@google.com,rmacnak@google.com,alexmarkov@google.com,regis@google.com

Change-Id: Ia1e2cbf7c059e22a2c71f28cb641f07ccd3dddab
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: TC-384
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/95821
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-03-07 16:56:20 +00:00
Ryan Macnak 253499fe68 [vm, gc] The mutator's TLAB and the interpreter's lookup cache must be visited even if the mutator is unscheduled.
Bug: TC-384
Change-Id: Ic3dcc2fea3e80ef181f8891ccb00a8e2e33fc077
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/95670
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2019-03-07 01:01:02 +00:00
Alexander Markov e83b309354 [vm] Remove single-stepping from PRODUCT
Change-Id: I2884cc1feff563d05ca7d31f6154fbfda7922544
Reviewed-on: https://dart-review.googlesource.com/c/92025
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2019-02-05 21:51:55 +00:00
Vyacheslav Egorov f496e538f4 [vm] Decouple assemblers from runtime.
This is the next step towards preventing compiler from directly peeking
into runtime and instead interact with runtime through a well defined
surface. The goal of the refactoring to locate all places where compiler
accesses some runtime information and partion those accesses into two
categories:

- creating objects in the host runtime (e.g. allocating strings, numbers, etc)
during compilation;
- accessing properties of the target runtime (e.g. offsets of fields) to
embed those into the generated code;

This change introduces dart::compiler and dart::compiler::target namespaces.

All code in the compiler will gradually be moved into dart::compiler namespace.
One of the motivations for this change is to be able to prevent access to
globally defined host constants like kWordSize by shadowing them in the
dart::compiler namespace.

The nested namespace dart::compiler::target hosts all information about
target runtime that compiler could access, e.g. compiler::target::kWordSize
defines word size of the target which will eventually be made different
from the host kWordSize (defined by dart::kWordSize).

The API for compiler to runtime interaction is placed into compiler_api.h.

Note that we still permit runtime to access compiler internals directly -
this is not going to be decoupled as part of this work.

Issue https://github.com/dart-lang/sdk/issues/31709

Change-Id: If4396d295879391becfa6c38d4802bbff81f5b20
Reviewed-on: https://dart-review.googlesource.com/c/90242
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-01-25 16:45:13 +00:00
Martin Kustermann f1f688215a [VM/AOT] Initialize global VM flags by reading them from vm-isolate snapshots
Issue https://github.com/dart-lang/sdk/issues/33274

Change-Id: I2dca1fb9a8807294fb756ea796bb7560bb6d17cc
Reviewed-on: https://dart-review.googlesource.com/c/90221
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-01-23 09:49:46 +00:00
Alexander Markov e64428a839 Revert "[vm] Cleanup Dart_Get/Set/HasStickyError API and use isolate's sticky error only as a backup for thread's sticky error"
This reverts commit b10f179608.

Reason for revert: failed service/* tests

Original change's description:
> [vm] Cleanup Dart_Get/Set/HasStickyError API and use isolate's sticky error only as a backup for thread's sticky error
> 
> Both thread and isolate have sticky error fields. Dart API was using
> isolate's sticky error, while Dart code was using thread's sticky error.
> There was a one-way move of a thread's sticky error into isolate
> when thread was unscheduled.
> 
> This causes a problem as error in the isolate may go unnoticed and
> repeated unscheduling/re-scheduling might end up overwriting the error
> in the isolate (which triggers the assertion).
> 
> To solve this problem, this CL:
> * Cleans up Dart API which manipulates isolate's sticky error, so
>   isolate's sticky error is never set directly.
> * When sceduling an isolate to a thread, sticky error is moved back from
>   isolate (if any).
> 
> With this changes, thread's sticky error is always used if thread is running,
> and isolate's sticky error is only used to hold sticky error while
> isolate has no thread.
> 
> Fixes https://github.com/dart-lang/sdk/issues/35590
> 
> Change-Id: I99b128cac363ca2df75f6e64c083b1ec36c866ce
> Reviewed-on: https://dart-review.googlesource.com/c/89442
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com,zra@google.com,asiva@google.com

Change-Id: I15874575b6b8ddca618741c59c21d4e692c4dcab
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/90127
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-01-18 01:41:18 +00:00
Alexander Markov b10f179608 [vm] Cleanup Dart_Get/Set/HasStickyError API and use isolate's sticky error only as a backup for thread's sticky error
Both thread and isolate have sticky error fields. Dart API was using
isolate's sticky error, while Dart code was using thread's sticky error.
There was a one-way move of a thread's sticky error into isolate
when thread was unscheduled.

This causes a problem as error in the isolate may go unnoticed and
repeated unscheduling/re-scheduling might end up overwriting the error
in the isolate (which triggers the assertion).

To solve this problem, this CL:
* Cleans up Dart API which manipulates isolate's sticky error, so
  isolate's sticky error is never set directly.
* When sceduling an isolate to a thread, sticky error is moved back from
  isolate (if any).

With this changes, thread's sticky error is always used if thread is running,
and isolate's sticky error is only used to hold sticky error while
isolate has no thread.

Fixes https://github.com/dart-lang/sdk/issues/35590

Change-Id: I99b128cac363ca2df75f6e64c083b1ec36c866ce
Reviewed-on: https://dart-review.googlesource.com/c/89442
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-01-17 23:27:13 +00:00
Vyacheslav Egorov a9ce969e53 [vm] Decouple growable_array.h and zone.h from thread.h
- Introduce a slimmed down version of thread.h, which just depends on the
Zone and StackResource.
- Introduce a layering check that would prevent the coupling in the future.

This is the first step towards decoupling compiler from runtime.

There are multiple reasons to introduce the decoupling but the main
reason currently is to introduce a controlled surface through which
compiler reaches into runtime to catch any places where runtime word size
might influence the compiler and then enable building compiler that
targets 32-bit runtime but is embedded into a 64-bit runtime.

Issue https://github.com/dart-lang/sdk/issues/31709

Change-Id: Id63ebbaddca55dd097298e51c90d957a73fa476e
Reviewed-on: https://dart-review.googlesource.com/c/87182
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-01-11 20:47:10 +00:00
Alexander Aprelev 7d14785115 This relands "Load isolate from parent's kernel in Isolate.spawn calls.".
Original PR was reverted because it broke hot-reload bots. This CL adds steps to initialize kernel_service compiler for child isolate. Compiler is needed to support hot-reload request. Child isolate's compiler is created from full kernel file produced by main isolate's compiler. Changes since original PR are pkg/vm/bin/kernel_service.dart in pkg/vm/lib/incremental_compiler.dart.

Further this CL changes kernel fingerprint calculation for interface types so it calculates the hash of the canonical names themselves, rather than indices(that might change from one compilation to another).

This reverts commit 63fd8f63e6.

Change-Id: I6fe5b2ef99f209b32cd4087dfd1c8cac229c2d8b
Reviewed-on: https://dart-review.googlesource.com/c/87265
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2018-12-14 23:45:57 +00:00
Martin Kustermann f205292227 [VM] Bare instructions - Part 4: Add --use-bare-instructions flag to AOT compiler & runtime
This is the final CL which adds a new --use-bare-instructions flag to
the VM.

If this flag is set during AOT compilation, we will:

  * Build one global object pool (abbr: GOP) which all code objects
    share. This gop will be stored in the object store.  The PP register
    is populated in the enter dart stub and it is restored when
    returning from native calls.

  * Gets rid of the CODE_REG/PP slots from the dart frames. Instead the
    compiled code uses the global object pool, which is always in PP.

  * Starts emitting pc-relative calls for calls between two dart
    functions or when invoking a stub.
    Limitation: We only emit pc-relative calls between two code objects
    in the same isolate (this is because the image writer is writing
    instruction objects for vm-isolate/main-isolate seperately)

  * We do compile-time relocation of those static calls after the
    precompiler has finished its work, but before writing the snapshot.
    This patches all the instruction objects with pc-relative calls to
    have the right .text distance.

  * We emit a sorted list of code objects in ObjectStore::reverse_code_table,
    which will be used by the AOT runtime to go back from PC to Code
    objects (where all metadata, e.g. stack maps, catch entry moves, pc
    descriptors are available).

Issue https://github.com/dart-lang/sdk/issues/33274

Change-Id: I6c5dd2b1571e3a889b27e804a24c2986c71e03b6
Reviewed-on: https://dart-review.googlesource.com/c/85769
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-12-14 16:03:04 +00:00
Ryan Macnak 3e8b335af8 [vm] Adjust concurrent marker to re-enable code W^X.
Take 2: account for encountering Instructions in the write barrier.

Bug: https://github.com/dart-lang/sdk/issues/34002
Change-Id: I31a8fc8965042678a861a05b09070d52fa07e4a5
Reviewed-on: https://dart-review.googlesource.com/c/87082
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-12-14 02:02:54 +00:00
Martin Kustermann 2440a32461 [VM] Bare instructions - Part 3: Add support for building a PC -> Code mapping table
This CL adds a [ReversePcLookupCache] which, based on a list of code
objects, builds up a binary searchable table for mapping PCs to Code
objects (where all the metadata is available, like stackmaps, ...).

This CL also adds stack walking support for "bare instruction" frames.

In a later part we will start emitting the sorted list of code objects
(via the new field in the object store) under a flag.

Issue https://github.com/dart-lang/sdk/issues/33274

Change-Id: I3c06c12bc0fb266dc1bd843a4a11b5208773151d
Reviewed-on: https://dart-review.googlesource.com/c/85746
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-12-13 17:45:02 +00:00
Ryan Macnak 679a1a7867 [vm] Clean up sticky error accessors.
Bug: https://github.com/dart-lang/sdk/issues/35224
Change-Id: I867b47a95d7b38e511259b06ef24424d8809096d
Reviewed-on: https://dart-review.googlesource.com/c/85861
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-12-05 18:39:41 +00:00
Liam Appelbe 63fd8f63e6 Revert "Load isolate from parent's kernel in Isolate.spawn calls."
This reverts commit e143a52531.

Reason for revert: Broke all the reload-kernel bots.

Original change's description:
> Load isolate from parent's kernel in Isolate.spawn calls.
> 
> Bug: https://github.com/dart-lang/sdk/issues/6610
> Change-Id: Icd6e1c5d6d4b64b611fc58333c051a8a9a50679d
> Reviewed-on: https://dart-review.googlesource.com/c/85724
> Commit-Queue: Liam Appelbe <liama@google.com>
> Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=vegorov@google.com,rmacnak@google.com,asiva@google.com,liama@google.com

Change-Id: Ie9c305256da9b6478153b99502d0c63b0f43a3e6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/6610
Reviewed-on: https://dart-review.googlesource.com/c/86082
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2018-12-04 20:26:24 +00:00
Liam Appelbe e143a52531 Load isolate from parent's kernel in Isolate.spawn calls.
Bug: https://github.com/dart-lang/sdk/issues/6610
Change-Id: Icd6e1c5d6d4b64b611fc58333c051a8a9a50679d
Reviewed-on: https://dart-review.googlesource.com/c/85724
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-12-04 18:02:55 +00:00
Martin Kustermann 9ca6369ef5 [VM] Add locking around accessing constants arrays on scripts
Both the mutator as well as the background compiler are accessing constants
arrays attached to [Script] objects, which can cause races where mutator is
modifying the hashmap while the background compiler is reading.

This CL adds locking around those accesses.

Change-Id: Iac7bbbe242d13b964698ae95fe1653fb4ce28a36
Reviewed-on: https://dart-review.googlesource.com/c/84903
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-11-20 21:46:08 +00:00
asiva 0ca1c57593 Reland "[VM] Remove unused flags"
This is a reland of 1a4cff1bd4

Original change's description:
> [VM] Remove unused flags
> 
> Remove the following flags which are always ON
> --strong
> --reify_generic_functions
> 
> and these flags which are not applicable anymore
> --error-on-bad-override
> --error-on-bad-type
> --enable-type-checks
> 
> Change-Id: I6e1aeb68e663953f4ae49ca94ea0daa87b661900
> Reviewed-on: https://dart-review.googlesource.com/c/79431
> Commit-Queue: Siva Annamalai <asiva@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>

Change-Id: Ib4452c22fb05eed38820708abb648939e03b1df0
Reviewed-on: https://dart-review.googlesource.com/c/84923
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-11-20 17:25:42 +00:00
Siva Annamalai f6c41ac3bc Revert "[VM] Remove unused flags"
This reverts commit 1a4cff1bd4.

Reason for revert: The precompiled arm bot failed a test.

Original change's description:
> [VM] Remove unused flags
> 
> Remove the following flags which are always ON
> --strong
> --reify_generic_functions
> 
> and these flags which are not applicable anymore
> --error-on-bad-override
> --error-on-bad-type
> --enable-type-checks
> 
> Change-Id: I6e1aeb68e663953f4ae49ca94ea0daa87b661900
> Reviewed-on: https://dart-review.googlesource.com/c/79431
> Commit-Queue: Siva Annamalai <asiva@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com,asiva@google.com,regis@google.com

Change-Id: Ib0fe4f33c044047ef1cb31bca9d38591176b7d40
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/84883
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2018-11-20 07:32:11 +00:00
asiva 1a4cff1bd4 [VM] Remove unused flags
Remove the following flags which are always ON
--strong
--reify_generic_functions

and these flags which are not applicable anymore
--error-on-bad-override
--error-on-bad-type
--enable-type-checks

Change-Id: I6e1aeb68e663953f4ae49ca94ea0daa87b661900
Reviewed-on: https://dart-review.googlesource.com/c/79431
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2018-11-20 06:41:06 +00:00
Martin Kustermann d7ea36477f [VM] Invoke a new isolate notify callback on install if there are pending messages
This fixes an issue where service RPCs are not answered by the
vm-service.

Currently new isolates are visible via the service protocol immediately
after creation. This means a vm-service client can start doing service
RPCs to a particular isolate.  If the embedder has not-yet installed a message
handler the OOB messages will be queued up.

Once the embedder installs an isolate message notify handler, that
handler needs to be invoked if there are pending messages.

Change-Id: Ie75878daf55a3b380e2b2ec7930e15b002f3f520
Reviewed-on: https://dart-review.googlesource.com/c/83680
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2018-11-13 10:32:48 +00:00
Ryan Macnak 30b0b9b723 [vm] Code size reductions:
- Remove dead Isolate::IsTopLevelParsing.
 - Remove dead TraceFunctionEntry/Exit.
 - Migrate more tracing flags to flag_list.h to allow code elimination in non-debug modes.
 - Require an explicit Zone parameter for CheckedHandles.
 - Remove side-effects from RuntimeEntry constructor.

out/ProductX64/exe.stripped/dart_precompiled_runtime 4063464 -> 4046952 (-16k)

Change-Id: Iffd3de25a03d2354cdecf2d79aa761c33ab08bd3
Reviewed-on: https://dart-review.googlesource.com/c/83120
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-11-08 02:02:39 +00:00
Zach Anderson c4df463ba9 [vm] Reland kernel data library and class caches.
This is the same code as the previous commit, but it needs a fix
for issue #34953 to land first.

Change-Id: Id586f5e8a5ee498716e58ff44bb8a344822f73df
Reviewed-on: https://dart-review.googlesource.com/c/81840
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-11-01 20:25:59 +00:00
Ryan Macnak 7514ce941e [vm] Assert handles are only allocated when the thread is preventing safepoints.
Threads in the native or blocked states don't prevent safepoints, so they may run concurrently with a safepoint operation like GC. It is not safe for handles to be allocated while the GC is visiting them, so these threads must not allocate handles. Assert only threads in the VM or generated states, which prevent safepoints until they check in, may allocate handles. (Generated code does not allocate handles, but leaf runtime entries remain in the generated state.)

Bug: https://github.com/dart-lang/sdk/issues/34883
Change-Id: I1a211778f7ef96b53a2405f0ee9dde7871b122b6
Reviewed-on: https://dart-review.googlesource.com/c/81540
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-10-31 19:51:52 +00:00
Zach Anderson ae2441337a Revert "Reland: [vm] Add a library and class cache to KernelProgramInfo"
This reverts commit c1e6140323.

Reason for revert:

On Windows

.\out\DebugX64\run_vm_tests.exe --dfe=out\DebugX64\gen\kernel-service.dart.snapshot IsolateReload_TearOff_List_Set

fails with:

sdk\runtime\vm\raw_object.cc: 231: error: Size mismatch: -2459565876494606883 from class vs 112 from tags c0718

Original change's description:
> Reland: [vm] Add a library and class cache to KernelProgramInfo
> 
> Relands with fixed locking to protect the caches.
> 
> Original message:
> Also create fewer handles in the KernelLoader.
> 
> GenKernelKernelReadAllBytecode improves a further
> 15-20% after removing the timeline event argument
> in the bytecode reader.
> 
> fixes #34939
> 
> Change-Id: I88c57bdb673392a7677c5c6bbb16cfed8c95c0d7
> Reviewed-on: https://dart-review.googlesource.com/c/81620
> Commit-Queue: Zach Anderson <zra@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com,zra@google.com,asiva@google.com

Change-Id: I4f3ff1866db2990b512b3ee7e59bf66bdc480dcc
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/81745
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-10-27 07:53:51 +00:00
Zach Anderson c1e6140323 Reland: [vm] Add a library and class cache to KernelProgramInfo
Relands with fixed locking to protect the caches.

Original message:
Also create fewer handles in the KernelLoader.

GenKernelKernelReadAllBytecode improves a further
15-20% after removing the timeline event argument
in the bytecode reader.

fixes #34939

Change-Id: I88c57bdb673392a7677c5c6bbb16cfed8c95c0d7
Reviewed-on: https://dart-review.googlesource.com/c/81620
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2018-10-26 21:36:58 +00:00
Martin Kustermann 12fe255ff5 Revert "Reland: [vm] Add a library and class cache to KernelProgramInfo"
This reverts commits
  - 47c55605b8
  - c362e4862f.

Reason for revert:

Very likely the cause for flaky crashes where mutator and BG compiler are accessing the program info class cache concurrently without proper locking.

Issue https://github.com/dart-lang/sdk/issues/34939

Original change's description:
> Reland: [vm] Add a library and class cache to KernelProgramInfo
>
> Relands with locks to protect the caches.
>
> Original message:
> Also create fewer handles in the KernelLoader.
>
> GenKernelKernelReadAllBytecode improves a further
> 15-20% after removing the timeline event argument
> in the bytecode reader.
>
> Change-Id: I6d796986a7773938e528271033385465e4f13468
> Reviewed-on: https://dart-review.googlesource.com/c/79380
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Zach Anderson <zra@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com,zra@google.com,asiva@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Ib242ecc9ece8a233441fc4d5512e4a20ce3e0d11
Reviewed-on: https://dart-review.googlesource.com/c/81601
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-10-26 11:32:34 +00:00
Alexander Markov 781ea1be70 [vm/interpreter] Fix interpreter frame recognition in stack walking, v2
When GC is visiting pointers, it scans mutator thread even if it is not
active. It could happen if a thread has (temporarily) exitted isolate.
In such case, stack frame iteration would not find interpreter and
would not recognize interpreter frames. An attempt to visit interpreter
frame as non-interpreter causes crashes/assertions.

The fix is to move interpreter from Isolate to Thread. This way
interpreter is available for stack walking even if thread doesn't have
isolate.

This is the re-land of https://dart-review.googlesource.com/c/sdk/+/80760
with the fix for precompiler failures.

Change-Id: I1fa054d43b86aa49c48490d0951b876b7c7218d8
Reviewed-on: https://dart-review.googlesource.com/c/80782
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
2018-10-19 18:01:34 +00:00
Alexander Markov a474f49ada Revert "[vm/interpreter] Fix interpreter frame recognition in stack walking"
This reverts commit a0cc3e2684.

Reason for revert: broken precompiler bots

Original change's description:
> [vm/interpreter] Fix interpreter frame recognition in stack walking
> 
> When GC is visiting pointers, it scans mutator thread even if it is not
> active. It could happen if a thread has (temporarily) exitted isolate.
> In such case, stack frame iteration would not find interpreter and
> would not recognize interpreter frames. An attempt to visit interpreter
> frame as non-interpreter causes crashes/assertions.
> 
> The fix is to move interpreter from Isolate to Thread. This way
> interpreter is available for stack walking even if thread doesn't have
> isolate.
> 
> Change-Id: Iff12ce7391b19a6f87bd4fac94137cba5623dfb9
> Reviewed-on: https://dart-review.googlesource.com/c/80760
> Reviewed-by: Régis Crelier <regis@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com,zra@google.com,asiva@google.com,regis@google.com

Change-Id: Ic4d73536eadb6601ee323918c65f5f2b2b2b2418
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/80781
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-10-19 03:41:17 +00:00
Alexander Markov a0cc3e2684 [vm/interpreter] Fix interpreter frame recognition in stack walking
When GC is visiting pointers, it scans mutator thread even if it is not
active. It could happen if a thread has (temporarily) exitted isolate.
In such case, stack frame iteration would not find interpreter and
would not recognize interpreter frames. An attempt to visit interpreter
frame as non-interpreter causes crashes/assertions.

The fix is to move interpreter from Isolate to Thread. This way
interpreter is available for stack walking even if thread doesn't have
isolate.

Change-Id: Iff12ce7391b19a6f87bd4fac94137cba5623dfb9
Reviewed-on: https://dart-review.googlesource.com/c/80760
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-10-19 02:18:51 +00:00
Zach Anderson 47c55605b8 Reland: [vm] Add a library and class cache to KernelProgramInfo
Relands with locks to protect the caches.

Original message:
Also create fewer handles in the KernelLoader.

GenKernelKernelReadAllBytecode improves a further
15-20% after removing the timeline event argument
in the bytecode reader.

Change-Id: I6d796986a7773938e528271033385465e4f13468
Reviewed-on: https://dart-review.googlesource.com/c/79380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-10-12 14:54:48 +00:00