Commit Graph

1132 Commits

Author SHA1 Message Date
Clement Skau 3a5c166647 [VM] Makes the GCEventCallback VM-wide.
Bug: https://github.com/dart-lang/sdk/issues/43106
Change-Id: I35ba6875deb5239a8e0154fe3cc20c321392bcbe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162756
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2020-09-17 06:07:50 +00:00
Clement Skau 7a978c3a35 [API] Adds Dart_SetGCEventCallback for GC events in PRODUCT
Bug: https://github.com/dart-lang/sdk/issues/43106
Change-Id: Ied2201464eca387e0bc58ab54404041f7649e870
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160063
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2020-08-31 12:14:38 +00:00
Ryan Macnak ae37f48796 [vm] Fix Dart_GetNativeFieldsOfArgument when passed an object with no native fields.
Bug: https://github.com/flutter/flutter/issues/64598
Change-Id: I90325e5eecc7a7fca2cb76894ce1830c93c05388
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160185
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-08-26 18:03:05 +00:00
Alexander Aprelev 1c69c69650 [vm/resolve] Move class finalization out of lookup and resolve function methods.
This is to make sure that lookup can be done without getting a safepoint, which might be needed if class needs to be finalized.

Change-Id: I6ed52a56d53369327df0b85d7b04cf41a46ac690
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159188
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-08-21 02:57:07 +00:00
Ryan Macnak ba82a6023c Reland "[vm, gc] Protect various data in Heap under a safepoint operation."
Change order of set_tasks and SafepointOperationScope in CreateWithinExistingIsolateGroup.

Bug: https://github.com/dart-lang/sdk/issues/41580
Bug: https://github.com/dart-lang/sdk/issues/42812
Change-Id: I36d26b338fed2ac8333b3ef954d819619a89f21b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159164
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-08-18 22:07:30 +00:00
Ryan Macnak 56a4d7c5f8 [vm] Support --use_table_dispatch when splitting snapshots.
When reading a secondary snapshot, re-run the dispatch table initialization found in the root snapshot. Since each loading unit's contribution to the dispatch table is scattered, it would take more snapshot size to try to serialize their contributions separately. We'll revisit this when we can defer Class, Function and Code, which will prevent the root snapshot from referencing all Codes.

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Iefd2b98647b96ae59a7efe92897538f5cf8c2426
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158923
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-08-18 16:46:50 +00:00
Ryan Macnak 28ad17bee9 [gen_snapshot] Include library URIs in the loading unit manifest.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ifb7de4698563b5c1210d388bd3d069986fc0f541
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158725
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-08-18 00:28:56 +00:00
Ryan Macnak ca169d010e [vm, api] Add Dart_HintFreed to the embedding API.
Allows an embedder (or native extension) to inform the VM it suspects memory has become unreachable.

Bug: https://github.com/dart-lang/sdk/issues/42078
Change-Id: I977e14fbe760fd4b5a0cc68fd010561a66c71899
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150503
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-08-12 18:21:36 +00:00
Ryan Macnak 91ed534139 [vm] Support --use_bare_instructions when splitting snapshots.
- Don't use PC-relative calls when calling between loading units.
 - Sort the contents of the code cluster by loading unit, then by text offset.
 - Handle binding PC-relatives calls and inserting trampolines per loading unit.
 - Create one code order table per loading unit to implement PC -> Code lookup.
 - Read code order tables directly, instead of copying into malloc'd memory.

--use_table_dispatch still not yet supported.

This slightly shrinks non-split binaries (~2% clustered part, 0.4% total snapshot) due to the new delta encoding when Code references Instructions.

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I51052ebc7b4968b554897d1710135a6c41821302
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157820
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2020-08-12 17:23:14 +00:00
asiva cf96fa6721 [VM/C-API] - Fix errors reported by MSVC tool chain.
For details see https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8873403898611684880/+/steps/build_android_profile_gen_snapshot/0/stdout

Change-Id: I47fe33cab787595dff13b68c613fb216fb434f3b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157560
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-08-07 19:30:05 +00:00
Ryan Macnak 3f01c3e588 [vm] Handle all typed data element types in various C APIs.
Change-Id: Ie27989895c1bb4f6fd609523f9fac0a7d4f437a9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155360
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-07-27 22:40:46 +00:00
Daco Harkes 202e32afeb [vm] Native API: Introduce Dart_FinalizableHandle
Introduces Dart_NewFinalizableHandle which does auto delete itself,
but does not allow accessing the weak referenced object.

Issue: https://github.com/dart-lang/sdk/issues/42312

Change-Id: I24ea732925122c453213c4fa3f629761c352f838
Cq-Include-Trybots:dart/try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154695
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-07-23 11:30:39 +00:00
Tess Strickland bb24f76c72 [vm] Reland "Remove non-covariant checks from closure bodies (part 1)"
Also relands the followup CLs:
"Perform non-covariant checks when dynamically invoking callables."
"Use AreValidArguments so that names are checked as well."

Original description of first CL:

This change only affects compilation when running in non-precompiled
mode with --no-lazy-dispatchers enabled.

Instead of always compiling in non-covariant checks, even for closures
not called dynamically, remove the non-covariant checks from the closure
and instead do the non-covariant checks for dynamic calls during the
NoSuchMethodForCallStub fallback by calling
Function::DoArgumentTypesMatch.

Adds two overloads for Function::DoArgumentTypesMatch, one which takes a
function type argument vector and one which takes neither an
instantiator type argument vector or a function type argument vector.
For the versions that are not explicitly passed a type argument vector,
an appropriate one is constructed using the arguments. If there is not
enough information in the arguments, then we fall back to assuming the
empty type argument vector for the instantiator case and instantiating
to bounds in the function type argument case.

Fixes Function::DoArgumentTypesMatch to handle generic functions and to
check arguments appropriately according to the active null safety mode.
For generic functions, the provided or resulting function type vector
has non-covariant checks performed against the type parameter bounds.

This change uncovered one test that was incorrectly passing in strong
mode, see https://github.com/dart-lang/sdk/issues/42688 for details.

Original description of second CL:

The VM only does this when the callable function does not expect dynamic
invocations. Otherwise, performing the checks would be redundant, as the
function body already contains the appropriate non-covariant checks.

Third CL had no additional description.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-try,vm-kernel-reload-linux-release-x64-try, vm-kernel-reload-rollback-linux-debug-x64-try
Bug: https://github.com/dart-lang/sdk/issues/40813
Change-Id: I1a3e9c1865103a8d716e1cad814267caffaaadf2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154688
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-07-21 10:00:41 +00:00
Ryan Macnak c5a94db091 [vm] Initial implementation of deferred loading.
(Assignment of libraries to loading units is already done in the kernel generation step.)

After compiling and before serializing, we walk the program and for each Code we assign its Instructions, CodeSourceMap and CompressedStackMap to the loading unit of that Code's defining library. Deduplication may cause Instructions, CodeSourceMaps and CompressedStackMaps to belong to more than one loading unit; in this case the objects are assigned to the root loading unit. Later they can be more precisely assigned to the dominating loading unit.

All objects except some Instructions, CodeSourceMaps and CompressedStackMaps belong to the root loading unit's snapshot. This snapshot is written like an unsplit snapshot, except that when serializing Code, we will write a reference to a stub or null when the Code's Instructions, CodeSourceMap or CompressedStackMap belongs to a non-root loading unit. The snapshots of non-root loading units contain these deferred objects and references to the corresponding Code objects to patch. The types of objects we defer (Instructions, CodeSourceMaps and CompressedStackMaps) usually represent 70+% of the snapshot size.

Bare instructions mode must be disabled when splitting because we cannot have PC-relative calls between loading units. Later we can re-enable this for calls within loading units.

Broken: Compactor probably crashes we can now have an unbounded number of image pages and the compactor assumes a fixed number.

Embedder's guide:

At compile-time, gen_snapshot should be passed --loading_unit_manifest with a path, which will enable splitting and output a mapping from loading unit ids to snapshot output paths.

At runtime, sometime during isolate startup, an embedder should call Dart_SetDeferredLoadHandler, probably near an existing call to Dart_SetLibraryTagHandler. The callback is given a loading unit id, and should eventually call Dart_DeferredLoadComplete[Error].

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ib597eb87c8cd634416d5ee1f00629c5550aebb00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152427
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-07-17 19:29:47 +00:00
Alexander Aprelev d6eb87bbcc Revert "[vm] Remove non-covariant checks from closure bodies (part 1)" as it broke language_2/vm/no_such_args_error_message_vm_test.
This reverts commit 556f0fb5c8 and fixes merge conflict.

Revert "[vm] Perform non-covariant checks when dynamically invoking callables." as a collateral.

This reverts commit 7a6287bb92.

Revert "[vm] Use AreValidArguments so that names are checked as well." as a collateral

This reverts commit 90c683018f.

Change-Id: Iee2c5e91c39d19dca7a1e640c103443c95e6c4f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154705
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2020-07-16 23:25:54 +00:00
Tess Strickland 556f0fb5c8 [vm] Remove non-covariant checks from closure bodies (part 1)
This change only affects compilation when running in non-precompiled
mode with --no-lazy-dispatchers enabled.

Instead of always compiling in non-covariant checks, even for closures
not called dynamically, remove the non-covariant checks from the closure
and instead do the non-covariant checks for dynamic calls during the
NoSuchMethodForCallStub fallback by calling
Function::DoArgumentTypesMatch.

Adds two overloads for Function::DoArgumentTypesMatch, one which takes a
function type argument vector and one which takes neither an
instantiator type argument vector or a function type argument vector.
For the versions that are not explicitly passed a type argument vector,
an appropriate one is constructed using the arguments. If there is not
enough information in the arguments, then we fall back to assuming the
empty type argument vector for the instantiator case and instantiating
to bounds in the function type argument case.

Fixes Function::DoArgumentTypesMatch to handle generic functions and to
check arguments appropriately according to the active null safety mode.
For generic functions, the provided or resulting function type vector
has non-covariant checks performed against the type parameter bounds.

This change uncovered one test that was incorrectly passing in strong
mode, see https://github.com/dart-lang/sdk/issues/42688 for details.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-dartkb-linux-release-x64-try
Change-Id: I5030a46b356778448b51a243f16406eacf1c04bf
Bug: https://github.com/dart-lang/sdk/issues/40813
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153202
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-07-16 14:54:04 +00:00
Regis Crelier 994c6ba6cd [VM/nnbd] Class Future may be used for type normalization before it is cached in the object store.
Class Future is resolved on the fly if not already cached.
Note that postponing type normalization until type finalization does not help
and results in the same error.

Fixes issue https://github.com/dart-lang/sdk/issues/42643

Change-Id: Id4dbeba6a246dbc952cce4168a10137a49d3ce70
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153901
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
2020-07-14 00:42:19 +00:00
Alexander Aprelev eecf55fab7 [vm/bytecode] Ensure class is allocate-finalized before instantiated by bytecode runtime calls.
Also, updated ClassMirror::InvokeConstructor and Dart_New

This is follow-up to https://dart.googlesource.com/sdk/+/b387ebc29a761385ad8dc4034dae06bc990b9d98.

Fixes https://github.com/dart-lang/sdk/issues/42680

Change-Id: Id016a4f5f674e161b066cc98e7cc2f3590a80703
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154289
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-07-13 23:40:49 +00:00
Alexander Markov 0162e7c1f8 [vm] Enable fingerprints checking of recognized methods
Fixes https://github.com/dart-lang/sdk/issues/36376

Change-Id: I470b95a1f854ef6e3798422508a89bc61f71cd0c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153740
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2020-07-09 17:10:57 +00:00
Jens Johansen e8d3389eb5 [VM] Expression compilation when loaded from dill (part 2)
Previous version
(landed via https://dart-review.googlesource.com/c/sdk/+/135683)
only worked if the dill was loaded via
Dart_LoadScriptFromKernel where script_kernel_size was set,
and when the platform dill could be loaded from a file called the right
thing.

This CL makes it work for Dart_LoadLibraryFromKernel too and allows
the VM to send the platform along (explicitly or implicitly) and only
tries to load the platform from a file with the right name if no
platform is given in any of the inputs.

This should fix http://b/148776866

Change-Id: I62317400a932b7dcd9e126a5a88907d507f9d658
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153609
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2020-07-09 14:55:11 +00:00
Alexander Markov 6f6b1f8818 [vm] Rename --null-safety option to --sound-null-safety
Deprecated option --null-safety still remains in order to allow
graceful migration.

Fixes https://github.com/dart-lang/sdk/issues/41853

Change-Id: Ie47b1bebc9dd6532658a60743ecb85dc7fdc108c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153660
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2020-07-09 01:37:23 +00:00
Alexander Aprelev d33e51df3c [build] Support --clang when building dart sdk on Windows.
It relies on flutter copy of clang distribution, same one that is used to build flutter/engine.

It addressed several deprecated warnings from clang compiler for functions like strdup, unlink, etc.

It allows few warnings still since they are triggered in third_party sources.

Change-Id: Ieb13792c011438d46dbbc0fa030e1b5e4ea14315
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142704
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-06-25 03:36:41 +00:00
Siva Annamalai d46a696957 Revert "Reland : [VM/Runtime] Cleanup package config initialization code"
This reverts commit 03777dba31.

Reason for revert: Failures in couple of bots

Original change's description:
> Reland : [VM/Runtime] Cleanup package config initialization code
> 
> - Use _Init function for initialization of package config
> - Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,
> 
>   DartUtils::SetupPackageConfig
> - Rename Loader::InitForSnapshot to Loader::Init
> - Remove _setPackagesConfig, _setWorkingDirectory from builtin library
> - Rename _setPackagesMap to _setPackagesConfig
> Change-Id: I1e4a25e79706d7764247c9fbfcc8e8ce3231343c
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151633
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Siva Annamalai <asiva@google.com>

TBR=aam@google.com,asiva@google.com

Change-Id: Ibea8c35fe310ac7708f00bb9ffd58715b6924cb5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151869
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 08:02:53 +00:00
asiva 03777dba31 Reland : [VM/Runtime] Cleanup package config initialization code
- Use _Init function for initialization of package config
- Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,

  DartUtils::SetupPackageConfig
- Rename Loader::InitForSnapshot to Loader::Init
- Remove _setPackagesConfig, _setWorkingDirectory from builtin library
- Rename _setPackagesMap to _setPackagesConfig
Change-Id: I1e4a25e79706d7764247c9fbfcc8e8ce3231343c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151633
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 05:50:49 +00:00
asiva bbc8aedada [VM/Runtime] - Fix for issue https://github.com/dart-lang/sdk/issues/42421
Set global null safety flag based on the snapshot instead of trying to
detect it per isolate (was causing issues with the vm isolate loading).
Should fix https://github.com/dart-lang/sdk/issues/42421

Bug:42421
Change-Id: I9143560b76fedcb991e96522cbf5d820fde99f7f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151866
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 04:18:59 +00:00
asiva 4b9aa2bd7e Revert "[VM/Runtime] Cleanup package config initialization code"
This reverts commit 29e93bcdbd.

Reason for revert: Windows bots are failing with package resolution
errors. Looks like we are missing a windows path sanitization step.

Change-Id: Ib56f7e926b4f385fa3fde74c9c71947f64b8be1c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151469
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-06-17 00:03:19 +00:00
asiva 29e93bcdbd [VM/Runtime] Cleanup package config initialization code
- Use _Init function for initialization of package config
- Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,
  DartUtils::SetupPackageConfig
- Rename Loader::InitForSnapshot to Loader::Init
- Remove _setPackagesConfig, _setWorkingDirectory from builtin library
- Rename _setPackagesMap to _setPackagesConfig
Change-Id: I0b83574593fa95e1b948134fcd4a7b4858ea19c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151045
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-06-16 23:24:26 +00:00
Tess Strickland 2e939f215a [vm/aot] Move ELF section manipulation out of CreateAppAOTSnapshotAsElf.
Instead, we just always add a BSS section and we add the ROData
sections for ELF snapshots in BlobImageWriter::WriteText() (similar
to how AssemblyImageWriter::WriteText() handles it for assembly).

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I87ce64037821ce10f2344964e7f379376c4538df
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150937
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-06-16 12:46:24 +00:00
Tess Strickland 83d4f3af61 [vm/aot] Rework Dwarf object to be output-agnostic.
Previously, we created a separate DWARF object for each possible output.
For the most part, the information stored in each DWARF object was the
same, but the DWARF object was responsible for keeping track of abstract
origin addresses (in the non-assembly case) and for actually generating
the desired output (assembly or binary).

Instead, abstract out the output formatting into a DwarfWriteStream,
whose subclasses handle both the final output format and certain details
that differ between assembly and binary outputs. With this, the DWARF
class no longer depends on having direct access to the ELF file if being
used to generate ELF sections, and as such is output agnostic.

Now, if we have one or more outputs that contain DWARF information, we
only generate a single DWARF object that is shared across these outputs.
The only additional change required is in Dwarf::AddCode(), which now
handles being called multiple times for the same Code object.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I71de887a07a673164b3ccfa7fbbab2ccef80c576
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150302
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-06-11 08:57:59 +00:00
asiva a782dd82db [VM/runtime] - detect null safety before isolate initialization
- Detect null safety when not specified before isolate
      initialization
      - for source files by having CFE parse the source file
        for @dart annotations
      - for kernel files by sniffing the kernel file for
        compilation mode
      - for appJIT files by sniffing the feature string
      - for AOT snapshots by sniffing the feature string

    - Remove workaround of returning null safety to false during
      bootstrapping

    - Add a new Dart C API call for detecting null safety

Bug: 41766
Change-Id: Ia8cf264323a2d0d58c2855ce6491456aa6f1da07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150089
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-06-11 01:22:35 +00:00
Regis Crelier c0c66ab10f Reland "[VM/nnbd] Canonicalize TypeParameter objects in the VM."
This is a reland of 6a98e2719e

Original change's description:
> [VM/nnbd] Canonicalize TypeParameter objects in the VM.
> 
> Prior to this CL, type parameters were all assumed canonical, even duplicate ones.
> 
> Per a new convention introduced in this CL, the type parameter array in generic classes and generic functions contains canonical type parameters. As these type parameters get cloned with a different nullability, they are inserted in a new hash table of canonical type parameters.
> 
> This fixes performance issue https://github.com/dart-lang/sdk/issues/41421
> 
> Change-Id: I9086158fa6b6261e9997bb50edec6d7c54abbfa1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148223
> Commit-Queue: Régis Crelier <regis@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

Change-Id: I4a42d0f9d53c2244b1a638432ed2cd6a055fa72a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150266
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-06-11 00:44:05 +00:00
Ryan Macnak 0fab083e1c [vm, isolate] Merge heaps without evacuating new-space.
Change-Id: I03d65e866215f243f2ab6a4a1d9d41625be4d56f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148682
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-06-10 20:22:02 +00:00
Regis Crelier a5f8ddac99 [VM/runtime] Introduce a trail in IsSubtypeOf() to avoid cycles.
Clean up signatures of runtime functions in object.{cc,h} using the trail to avoid cycles. Move the trail to the last parameter of these functions.
Introduce a trail in IsSubtypeOf() to avoid cycles introduced by bounds of F-bounded types. Issue uncovered by the new normalization tests.

Change-Id: I3241c7e4023a09c122e1594b7aff90b5b103f4f4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150180
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-06-05 19:57:46 +00:00
Ryan Macnak 407ca1be87 [vm, api] Add Dart_UpdateExternalSize to the embedding API.
Allows an embedder (or native extension) to inform the VM when external memory is released before a weak handle is finalized.

Bug: https://github.com/dart-lang/sdk/issues/42078
Change-Id: Ifffd0c160e5305bc6e6752207a2315139f245e2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149245
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-06-02 21:29:03 +00:00
asiva 02bfe67840 [VM/Runtime] - Remove Dart_CompileSourcesToKernel from the Dart C API
Remove Dart_CompileSourcesToKernel from the Dart C API as it is not used
anywhere except in unit tests (unit_test.cc)

Change-Id: I314dafbe4cddbe20a81890505bc07c8bf2100eaa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149380
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-05-29 18:43:54 +00:00
Ben Konyi c4b7097e97 [ VM ] Migrate VM tests to support null safety and return errors from Dart_GetType and Dart_NewListOf when --null-safety is enabled
These methods return legacy types, which aren't valid with null safety
enabled.

Fixes https://github.com/dart-lang/sdk/issues/41154

Change-Id: I1181f0f3b9a8df156dea4dc4c82fef8afdf97ab9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148685
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-05-26 22:40:52 +00:00
Tess Strickland 826576dcfc [vm] Store relocated addresses for text sections in BSS.
The relocated addresses are populated during BSS::Initialize() for
natively loaded ELF snapshots, and during the non-native loader when it
is used. Putting this information in the BSS segment avoids having to
change the embedder interface, since we only need this information for
AOT snapshots. This also avoids depending on our ELF snapshot layout
to reverse-engineer the DSO base for ELF-compiled snapshots.

We now always print the DSO base for both the VM and isolate in
non-symbolic stack traces, not just for ELF-compiled snapshots. However,
we still only print the relocated addresses in individual stack frames
if we're guaranteed they match those in separately saved debugging
information.

Bug: https://github.com/dart-lang/sdk/issues/41880

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-mac-release-simarm64-try
Change-Id: I4837262f78e6e73a32eb7e24ef7a68ccb8ec2669
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148441
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-05-26 08:47:30 +00:00
Alexander Markov 85cb6eddf4 [vm/nnbd] Auto-detect null safety mode when bootstrapping from kernel
This change also enables auto-detection in gen_snapshot.

Issue: https://github.com/dart-lang/sdk/issues/41206
Change-Id: If047ab8457e8fa15dc24ec10c0dcfceaf836508a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148152
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2020-05-14 20:49:06 +00:00
Tess Strickland 245a574301 [vm] Make the ELF static symbol table a superset of the dynamic one.
Only the case for unstripped snapshots and debugging information.
Stripped snapshots still have no static symbol table, only the
dynamic one.

Fixes: https://github.com/dart-lang/sdk/issues/41783

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ibd82334597d34267a3b9e45a673d23c3f9e3fcb0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147182
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
2020-05-12 07:21:48 +00:00
Tess Strickland a6d06b59ec Reland "[vm] Mark snapshots that were compiled directly to ELF.
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF."

Storing the relocated address as an extra field in the Image header,
which requires increasing the Image header size on 64-bit platforms,
means Image pages cannot be used reliably as HeapPages as objects no
longer start after kMaxObjectAlignment bytes.

Instead, we return to an older design that just uses the lowest bit
in the BSS offset to store whether the instructions in an Image were
compiled directly to ELF.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Change-Id: I3819b0dc2719d69f5e8764ca8be8c6ae7171a7bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146560
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-05-11 08:41:17 +00:00
Martin Kustermann 518f097266 [vm] Make metrics API expose correct heap metrics in PRODUCT mode
See b/152430908

Change-Id: I89524489ec9868bf5fa40559292e4f82b352cfe3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147362
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-05-09 08:27:48 +00:00
Martin Kustermann c01f9fb8a9 [vm/concurrency] Make an isolate group use a custom ThreadPool with maximum size
Our TLAB sizes and maximum new space size constrain the number of
parallel mutator threads we can have. Having too many mutator threads
would cause constant races between threads to acquire TLABs.

In reality we should constrain the number of threads to be at most the
number of cores, since at most that many threads can run in parallel
(i.e. at the same time).

This CL extends the TreadPool implementation to be constrained by a
maximum size. Furthermore it makes each isolate group's have it's own
pool with constrained size and schedule all group member
mutator / message handler tasks on that pool.

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I095c749adad827ab892f33713a32be594d7606d1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145382
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-05-05 12:08:43 +00:00
Tess Strickland 049949fdf4 Revert "[vm] Mark snapshots that were compiled directly to ELF."
This reverts commit 250b84987d.

Reason for revert: Failures on FFI CI tests.

Original change's description:
> [vm] Mark snapshots that were compiled directly to ELF.
> 
> Since the virtual addresses in ELF snapshots are the same as in
> separately saved debugging information, print the virtual address in
> non-symbolic frames again when running from a snapshot compiled directly
> to ELF.
> 
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
> Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

TBR=vegorov@google.com,rmacnak@google.com,dacoharkes@google.com,sstrickl@google.com

Change-Id: I1c5f58e9d9e11b994c1016970839f95b92885d43
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146220
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-05-04 09:52:25 +00:00
Tess Strickland 250b84987d [vm] Mark snapshots that were compiled directly to ELF.
Since the virtual addresses in ELF snapshots are the same as in
separately saved debugging information, print the virtual address in
non-symbolic frames again when running from a snapshot compiled directly
to ELF.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie531953da7edd024cea02c0571e1e1dd7d7427eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145840
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-05-04 09:19:23 +00:00
Tess Strickland e0d722cf7b [vm/aot] Align virtual addresses in the snapshot and debug info.
Change-Id: I930fbed3f45f45c0fc63dfaecfd36a9f97368efb
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145588
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-05-01 07:06:34 +00:00
asiva c740bbc161 [VM] Set up Isolate specific null safety mode
- change the null_safety flag to be a tri state flag (unspecified, no
  null safety and null safety)
- added logic to set the null safety mode of an isolate based on the
  value specified in the kernel file (.dill file) if the null safety
  flag is not specified on the command line
- added logic to auto detect the null safety mode based on the language
  version API provided by CFE
- added logic to pass the correct null safety option to CFE when
  invoking it for compilation based on the null-safety flag setting
- Delete non-nullable-flag() function and adjust code that was using it.
- Remove 'nnbd-experiment' from the snapshot string.

https://github.com/dart-lang/sdk/issues/41206
https://github.com/dart-lang/sdk/issues/41207

Change-Id: I006bf3c9229980fc7986faac6a5850d3722aec92
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143160
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-05-01 04:03:04 +00:00
Ryan Macnak 6fe15f6df9 [vm] Represent tagged pointers as C++ value types instead of C++ pointer types.
This works around bugs in UndefinedBehaviorSanitizer and Clang.

Bug: b/28638298
Change-Id: I6be595f9664516019d28017d24559583a1ae3a21
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144354
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2020-04-25 05:21:27 +00:00
Ryan Macnak 160046f44c [vm] Move CID predicates out of RawObject.
When using C++ value types to represent tagged pointers, we cannot use forward declarations for tagged pointers. This helps to break include cycles when attempting to keep IsXYZ predicates as tagged pointer member functions.

class_id.h also seems like a more natural place for these predicates, which were written before there was a class_id.h.

Change-Id: I0677560a794ed084d10f844606e202feb0c3820a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144321
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-04-21 20:09:55 +00:00
Alexander Aprelev 9c4a322b08 Reland "[vm/isolates] Introduce fast isolate spawn in AOT."
This reverts commit 922ea3e9b6 in patchset 1, fix for assertion triggered in https://ci.chromium.org/b/8883214567628884960 in patchset 2, fix for deadlock around symbols table mutex in patchset 4.

Original commit description:

Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.

To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.

Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I655e337198214c9dfacbe76f7852b941b5a7e910
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143462
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-04-17 03:02:27 +00:00
Vyacheslav Egorov a41a1dc917 [vm/compiler] Split compiler sources out of normal VM sources.
Make them form its own source set (libdart_compiler) and completely exclude
them from AOT runtime targets.

Previously we had some inconsistencies, with some files were using
DART_PRECOMPILED_RUNTIME to fully or partially exclude either contents
from their headers or from implementation, while other files did nothing
and relied on linker to throw their contents away.

This change tries to address this inconsistency.

A follow up change would include a check in most compiler headers which
would prohibit to use them while building AOT runtime.

Change-Id: Ief11b11cbc518b301d3e93fce80580a31bbad151
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142993
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-04-16 22:59:03 +00:00