The class `VerifyDiagnosticsTest` is used to verify the correctness of
the values of the `documentation` keys in the `messages.yaml` files. As
a part of that, it verifies that the code snippets in the documentation
follow the rule that
- "Example" snippets generate one instance of the documented diagnostic
and no other diagnostics
- "Common fixed" snippets generate no diagnostics.
Snippets that fail to meet that criteria are not verified because
checking for them has been disabled. A common reason for failing to
meet the criteria is the production of one or more other diagnostics,
typically due to overreporting.
Not verifying them has some drawbacks.
- It's on an all-or-nothing basis; if a diagnostic is ignored all
snippets will be ignored even if only one snippet fails to follow the
rule.
- There's no indication if new diagnostics start being generated for
the snippet.
- There's no indication when the overreporting is fixed so that the
snippets can now be verified.
I propose we add a way to list the additional diagnostics to ignore and
remove these cases from the excluded diagnostics list. Specifically, I
want to add a new directive (in the test's sense, not in the language's
sense of that term) containing a list of expected other diagnostics.
I considered using an ignore comment, but errors can't be ignored this
way and sometimes the diagnostics that need to be ignored are errors.
This CL implements the proposal and removes one diagnostic from the
list as a sample of how this support will be used. If this approach
is adopted, other diagnostics can be removed from the list and the
snippets can be updated accordingly in one or more follow-on CLs.
This will _not_ remove the need for the list of ignored diagnostics
because that list has other use cases. However, I do think that we
should try to find solutions for those other use cases and eventually
remove the list entirely if possible.
Change-Id: I52b6b738e41b2163843caf915a6c0fe7f461ec44
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494620
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Brian Wilkerson <brianwilkerson@google.com>
Brian Wilkerson
bee0b56db8