Commit Graph

261 Commits

Author SHA1 Message Date
Alexander Markov 1d6c1020c9 [VM] Fix OOM handling when allocating buffers in dart:io
* Test standalone_2/file_error_test is updated for Dart 2.0 fixed-size
  integers.

* This update revealed that certain dart:io native methods do not handle
  out-of-memory properly when I/O buffers are allocated.
  This CL fixes this bug.

Change-Id: I6a9018ab86da7b163d9797d745544835dfb0f15c
Reviewed-on: https://dart-review.googlesource.com/20582
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2017-11-14 18:26:34 +00:00
Zachary Anderson 2948f8c1b7 Reland: [dart:io] Moves Http code into a separate library.
This moves Http code into dart:_http. dart:io then imports and
re-exports dart:_http. This is the first stage of moving
dart:_http into its own pub package.

This CL was reverted due to a failure in the Flutter engine build
which happened due to an incomplete change in gen_snapshot.cc, and to
update sdk/lib/libraries.yaml and sdk/lib/libraries.json

Change-Id: Ie90c77ef631aea7a163774b58e8ccbaf71a24d3c
Reviewed-on: https://dart-review.googlesource.com/7588
Reviewed-by: Zach Anderson <zra@google.com>
2017-09-21 19:21:25 +00:00
Zach Anderson 64bf1b49b0 Revert "[dart:io] Moves Http code into a separate library."
Reverting for broken Flutter build and kernel platform compilation test. It looks like the new _http library is missing from various snapshots. Not sure why this didn't fail locally.

This reverts commit ac9ff3f3af.

Change-Id: I7fbc2f842cf1380793e7af9fb869cacd14682885
Reviewed-on: https://dart-review.googlesource.com/7600
Reviewed-by: Zach Anderson <zra@google.com>
2017-09-21 15:57:59 +00:00
Zachary Anderson ac9ff3f3af [dart:io] Moves Http code into a separate library.
This moves Http code into dart:_http. dart:io then imports and
re-exports dart:_http. This is the first stage of moving
dart:_http into its own pub package.

Change-Id: Icd33232a97b3d75511ef7ec88b01fd47cdcf5a23
Reviewed-on: https://dart-review.googlesource.com/6420
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Florian Loitsch <floitsch@google.com>
2017-09-21 15:10:22 +00:00
Zachary Anderson fc41a57a93 [dart:io] Allow an embedder to disallow certain operaions
Fuchsia's Dart and Flutter content handlers share the implementation of
dart:io with the command line embedder, but would like to disable
some operations that don't make sense for them, like exiting thoe whole
process, which may be hosting multiple unrelated Isolates.
Change-Id: I825ece49c685471ff85802873b1c0f974c507aed
Reviewed-on: https://dart-review.googlesource.com/3687
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2017-09-08 05:02:07 +00:00
Zachary Anderson d0295c873c [dart:io] Namespaces for file IO
Fuchsia requires the ability to sandbox Isolates w.r.t. file IO.
When a new Isolate starts, Fuchsia will pass the Isolate an object
called a namespace. We can translate the namespace object into a
file descriptor suitable for passing to the *at() family of
POSIX file system calls. The file system calls will then
have visibility only into the specified namespace.

We also plumb Namespaces through on all the other platforms as well to
make the change easier to test and so that in the future we can
implement e.g. per-isolate cwds.

This change adds a new internal class to dart:io called _Namespace,
which is implemented in a patch file. See:

sdk/lib/io/namespace_impl.dart
runtime/bin/namespace_patch.dart

The embedder can set up a non-default namespace by calling
_Namespace._setupNamespace during Isolate setup.

Instances of _Namespace have a native field that holds a pointer
to a native Namespace object. See:

runtime/bin/namespace.h

Calls from e.g. file_impl.dart are now also passed a
_Namespace object. The implementations in e.g. file.cc and
file_linux.cc then extract the namespace, and use it to compute a
file descriptor and path suitable for passing to e.g. openat().

related US-313

R=asiva@google.com, rmacnak@google.com

Review-Url: https://codereview.chromium.org/3007703002 .
2017-08-30 09:34:36 -07:00
Zachary Anderson 130f3be297 [dart:io] Move _getUriBaseClosure to dart:io
In a forthcoming change, this will allow Uri.base to be aware of
the isolate's current namespace.

R=rmacnak@google.com

Review-Url: https://codereview.chromium.org/2998373002 .
2017-08-24 20:06:52 -07:00
Ryan Macnak 7dfe62f70d [standalone] Automatically decompress gzip'd resources, including sources and script snapshots.
Closes #30315

R=asiva@google.com, zra@google.com

Review-Url: https://codereview.chromium.org/2991393002 .
2017-08-11 12:18:19 -07:00
Ryan Macnak 02ec4f494e Remove old dead tag handler.
R=asiva@google.com, zra@google.com

Review-Url: https://codereview.chromium.org/2991763002 .
2017-07-26 09:53:04 -07:00
Zachary Anderson 6cd8a79078 VM: Re-format to use at most one newline between functions
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2974233002 .
2017-07-13 08:08:37 -07:00
Siva Annamalai 286dd51948 Changes to make isolate reload functionality work with the --dfe option.
R=aam@google.com, rmacnak@google.com

Review-Url: https://codereview.chromium.org/2894773004 .
2017-05-25 15:40:44 -07:00
Ryan Macnak 877284947b Rename TARGET_OS_* to HOST_OS_*.
Like HOST_ARCH_*, HOST_OS_* describes the OS the VM is running on, which may be different from the OS the VM is generating code for during AOT compilation.

Currently we conflate the two when emitting AOT as assembly, and we get away with it because Flutter only uses assembly for targeting iOS and one can only target iOS from a Mac, but we expect to use assembly for Android as well so native tools can unwind Dart frames.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2750843003 .
2017-03-15 13:11:05 -07:00
Ryan Macnak b6a9be6e39 Teach gen_snapshot how to create script snapshots if given a core snapshot to create them against.
Require an explicit --snapshot_kind option instead of inferring the desired snapshot kind from the combination of output file options.

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2692223002 .
2017-02-15 14:28:35 -08:00
Vyacheslav Egorov eb38fb5d67 VM: Fix incorrect free in TryReadKernel.
SniffForMagicNumber updates buffer pointer to skip snapshot magic number.

Thus we have to preserve and pass original buffer pointer to the free(...) call.

R=kustermann@google.com
BUG=

Review URL: https://codereview.chromium.org/2503363002 .
2016-11-16 14:40:57 +01:00
Ryan Macnak 0d830711bf Fix Windows build.
Review URL: https://codereview.chromium.org/2481013005 .
2016-11-07 16:38:30 -08:00
Ryan Macnak 987165f022 Reload native extensions when starting from a snapshot.
Fixes #21180
Fixes #27574

R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2475523002 .
2016-11-07 16:17:47 -08:00
Zachary Anderson 479a97b129 clang-format runtime/bin
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2480793002 .
2016-11-04 12:30:56 -07:00
Vyacheslav Egorov 206daf356e Delete builtins that became unused after source loading was refactored.
BUG=
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2468993002 .
2016-11-02 15:02:24 +01:00
Vyacheslav Egorov f67ce21068 VM support for running Kernel binaries.
BUG=
R=asiva@google.com, fschneider@google.com

Review URL: https://codereview.chromium.org/2411823003 .
2016-10-15 22:48:46 +02:00
Ryan Macnak 3d00ecdc8f Reapply "Use a single file for app snapshots."
- Don't read script to check for an app snapshot if it is a pipe. Fixes loading source from a pipe.
 - Fix windows and fushsia builds.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2416973003 .
2016-10-14 09:57:54 -07:00
Ryan Macnak 6839a23240 Revert "Use a single file for app snapshots."
This reverts commit 4f8b16df3b.

Review URL: https://codereview.chromium.org/2410303008 .
2016-10-13 15:29:23 -07:00
Ryan Macnak 4f8b16df3b Use a single file for app snapshots.
For blobs, put all four pieces into a single file with a header describing their offsets.
For dynamic libraries, move the vm isolate and isolate pieces into the dynamic library as additional read-only sections.

Automatically detect if the script argument is an app snapshot and initialize the VM appropriately, similar to automatic detection of script snapshots.

R=asiva@google.com

Review URL: https://codereview.chromium.org/2405393002 .
2016-10-13 14:33:56 -07:00
Siva Annamalai 42c58c5c2d - fix some cases where we are using uninitialized memory
- use attribute 'no msan' on NativeEntry::ReturnValueIsError function
  as msan doesn't seem to track the return slot being set by a native
  function
- vsnprintf seems to have issues with msan so unpoison the allocated
  memory buffer everytime vsnprintf is used to suppress the error.

BUG=
R=fschneider@google.com

Review URL: https://codereview.chromium.org/2383293003 .
2016-10-05 16:54:18 -07:00
Todd Turnidge 1166f83bf6 Only reload libraries when they may have been modified.
We now check all of the scripts which make up a library.  If any
script is modified, then we consider that library to be modified.  We
also consider any library which imports a modified library to be
modified.  To propagate this info efficiently, we build and discard
the imported-by graph when beginning a reload.

The embedder must provide a FileModifiedCallback in order to support
the detection of modified scripts.

In order to detect changes to package: libraries, we have modified the
embedder interface to provide the resolved url when possible, so that
we don't need to re-resolve package uris when checking for reload.
This change is incompatible and all embedders will need to be updated.

We now support a "force" flag when reloading sources.  This causes all
libraries to be reloaded regardless of whether the underlying scripts
have been updated.

Closes #26919

R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2186423002 .
2016-07-29 11:23:18 -07:00
Siva Annamalai d41d5dc414 Do not use Dart_DefaultCanonicalizeUrl for resolving a script uri in the current working directory as that code has not been written to handle paths on windows.
Review URL: https://codereview.chromium.org/2081853004 .
2016-06-20 19:59:53 -07:00
Siva Annamalai aca3d3eedf Fix canonicalization of relative paths, Dart_DefaultCanonicalizeUrl expects the base)url to end with a '/' when a directory is passed in otherwise it assumes a file path is passed in.
Review URL: https://codereview.chromium.org/2087723002 .
2016-06-20 17:55:41 -07:00
Siva Annamalai 11682cd4c7 Use the new loader mechanism to resolve URIs and load script files in gen_snapshot. This should allow the package config resolution to work in gen_snapshot for flutter and dartium snapshot generation cases.
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2073173002 .
2016-06-20 13:56:06 -07:00
Siva Annamalai 72f7d1456c Change Dart_DefaultCanonicalizeUrl to accept the library url string instead of a library, this is useful to allow us to use this to generically resolve a Uri relative to another string.
BUG=
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2068833002 .
2016-06-14 17:15:51 -07:00
John McCutchan 6fd8fd7987 Rework standalone to use a synchronous loader that does not invoke Dart code
- [x] The first caller of the tag handler blocks, recursive callers queue work and exit.
- [x] Use a NativeMessageHandler to receive I/O results from the service isolate.
- [x] Preserve load error message format.
- [x] Move packages map into service isolate.
- [x] Wire up Todd's native URI code.

R=turnidge@google.com

Review URL: https://codereview.chromium.org/1998963003 .
2016-06-06 14:15:01 -07:00
John McCutchan d47866964c Fix canonicalization of --url_mapping URIs in gen_snapshot
When gen_snapshot encounters a url that has been remapped with the --url_mapping command line argument it rewrites the library's url to be the mapped url before canonicalization. The new C++ API only does canonicalization with respect to the library's url (as returned by Dart_LibraryUrl) which is insufficient for gen_snapshot. The correct long-term fix is to alter the new C++ API to allow for the base url to be passed in directly rather than indirectly via Dart_LibraryUrl.

This CL just restores some deleted code and has gen_snapshot work like it did before. The standalone embedder still uses the C++ API during loading.

Review URL: https://codereview.chromium.org/2031103002 .
2016-06-02 14:52:42 -07:00
Todd Turnidge a7d46eb5a6 Canonicalize uris in C++ instead of Dart for the standalone embedder.
Adds Dart_DefaultCanonicalizeUrl() to the dart embedding api.

Motivation:

As we try to get source reloading working for the standalone embedder, things get simpler if an isolate doesn't run Dart code while it is loading Dart code.  We intend to solve this by moving the embedder tag handler calls to the service isolate.  But making a blocking rpc into the service isolate whenever a url needs to be canonicalized during parsing seems like it would slow things down and make things complicated.  By moving canonicalization into C++, we avoid this.

R=ahe@google.com, fschneider@google.com, johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2011543002 .
2016-06-02 12:29:57 -07:00
Todd Turnidge b3a1066c8e Simplify the canonicalization of dart-ext uris.
We used to stuff the path of the canonicalizing library into the uri
so that we could retrieve it later.  That isn't necessary, as the
library is still available when we import later.

BUG=
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/2004933002 .
2016-05-23 10:25:25 -07:00
Ivan Posva 3884fd4e4b - Allow for loading dart:html and friends into the standalone
dart_bootstrap binary to evaluate startup and compilation
  performance.

BUG=
R=asiva@google.com

Review URL: https://codereview.chromium.org/1916793003 .
2016-05-03 20:33:14 -07:00
Ryan Macnak f262fad639 Option to output precompiled instructions as a blob for use with mmap instead of assembly for use in a shared library.
BUG=http://dartbug.com/26199
R=fschneider@google.com

Review URL: https://codereview.chromium.org/1915853004 .
2016-04-25 16:19:34 -07:00
Zachary Anderson 218545ba10 Fixes leak of native File objects.
Also employs the same reference counting technique as secure sockets to
avoid the IO Service touching dangling pointers.

R=iposva@google.com

Review URL: https://codereview.chromium.org/1892623002 .
2016-04-20 10:08:37 -07:00
Zachary Anderson 1336e6149f Search for a native extension for the host architecture
R=iposva@google.com

Review URL: https://codereview.chromium.org/1896203002 .
2016-04-20 08:02:16 -07:00
John McCutchan ad0878d1a8 Remove redundant timeline API
BUG=
R=rmacnak@google.com

Review URL: https://codereview.chromium.org/1848683002 .
2016-03-31 09:13:54 -07:00
Zachary Anderson 7308e58c3f Really remove io support when dart:io is unsupported.
Previously we would compile in implementaitions of native calls for
IO functions that would never be used. This CL provides implementations
that throw a Dart exception if they're called by mistake. It also uses
a DART_IO_DISABLED preprocessor define to clean up the build files and
check that we're not including code we shouldn't.

R=iposva@google.com, johnmccutchan@google.com

Review URL: https://codereview.chromium.org/1839463002 .
2016-03-29 11:22:03 -07:00
Zachary Anderson c75a817ee0 Fixes some memory leaks in //runtime/bin
Also some style cleanups.

Decided to fix these leaks by using Dart_ScopeAllocate
instead of malloc and new. Leaks are noted in the CL.

I haven't finished looking over all the code in
//runtime/bin yet, but this CL was getting big.

Review URL: https://codereview.chromium.org/1781883002 .
2016-03-14 11:08:52 -07:00
Siva Annamalai 1e5eab3a14 - reorganize DartUtils::PrepareForScriptLoading so that it does not have the wait for service load port code in it. This is needed so that it is possible to not wait for the service load port when running with a full application snapshot that requires no loading.
- cache builtin_lib in IsolateData so that the cached value can be used withouit having to call Builtin::LoadAndCheckLibrary which creates new string objects everytime it is called.

Review URL: https://codereview.chromium.org/1663963002 .
2016-02-04 09:18:31 -08:00
Ivan Posva 6d066c7e53 Add package config support to dart:isolate
- Add "static Future<Uri> get packageRoot;",
  "static Future<Uri> get packageConfig;" and
  "static Future<Uri> resolvePackageUri(Uri packageUri)" to Isolate class.
- Added "Uri packageRoot, Uri packageConfig, bool automaticPackageResolution: false" parameters to spawnUri.

BUG=
R=rmacnak@google.com

Review URL: https://codereview.chromium.org/1553233002 .
2016-01-12 18:19:04 -08:00
Florian Schneider fbdce6a59e VM: Const-correctness fixes.
Three types of fixes:
1. Remove redundant const_cast
2. Remove const_cast by adding const when appropiate.
3. Remove const_cast by removing const (e.g. places where we call free with it)

For now I only fixed places where the fix is local enough - i.e. does not require
changed a large amount of code.

BUG=
R=rmacnak@google.com

Review URL: https://codereview.chromium.org/1526123002 .
2015-12-15 19:24:40 +01:00
Florian Schneider 64e71a0b2e VM: Make more globals constant where easily possible.
Remove -fdata-section again from the Linux build configuration: It prevented
some optimizations in gcc and clang - fewer things were recognized as constants.

Add const to pointers/arrays in some places to help compilers with determining
what is a constant.  A common pattern is strlen of a string literal const char*,
or an array of function pointers.

Tested with gcc 4.8.4 / clang 3.4 and improves code qualtiy + reduces binary size.

BUG=
R=iposva@google.com, rmacnak@google.com

Review URL: https://codereview.chromium.org/1462953002 .
2015-11-20 10:52:07 +01:00
Ryan Macnak aaf3c393ab Clean up service isolate loading error check.
R=johnmccutchan@google.com

Review URL: https://codereview.chromium.org/1417963009 .
2015-10-30 09:11:13 -07:00
John McCutchan 2844eb86ca Service isolate requests Observatory assets from embedder
- Provide a new callback to Dart_Initialize that will be called when the service isolate needs the Observatory assets
- Move asset management code out of embedder and into common dart:_vmservice
- Use the tar format as the archive format
- Add build script to generate .cc file containing the archive
- Add a simple tar parser to service isolate

This makes it possible for embedders to not bake Observatory into their executable but instead load it off disk or over the network. The standalone embedder still links Observatory into the executable.

R=rmacnak@google.com

Review URL: https://codereview.chromium.org/1411853003 .
2015-10-26 07:21:23 -07:00
John McCutchan 8a231f797f Temporary fix for precompile_test
BUG=

Review URL: https://codereview.chromium.org/1408783003 .
2015-10-16 07:25:09 -07:00
John McCutchan 9013047425 Bring up the service isolate and use it for package resolution in gen_snapshot
- Link dart:io implementation into gen_snapshot.
- Start event handler in gen_snapshot.
- Build system updates for gypi and gn.

R=asiva@google.com

Review URL: https://codereview.chromium.org/1405183002 .
2015-10-16 06:49:53 -07:00
Ivan Posva 60eab65aa0 - Implement package map parameter when spawning isolate.
BUG=
R=lrn@google.com

Review URL: https://codereview.chromium.org/1403693002 .
2015-10-12 09:05:37 -07:00
Siva Annamalai 575fa10d39 Fix for issue 24487951 (heap buffer overflow problem)
BUG=
R=srdjan@google.com

Review URL: https://codereview.chromium.org//1376973002 .
2015-09-29 15:18:30 -07:00
Zachary Anderson 7093f2996b VM thread shutdown.
BUG=
R=iposva@google.com, turnidge@google.com

Review URL: https://codereview.chromium.org//1275353005 .
2015-09-15 12:49:52 -07:00