Add a skill for finding obsolete issues in the Analysis Server open issues.

This is an initial commit. Plan to add - if repro is available, agent to test it out - add failing tests where possible Change-Id: I35c63a0e4183b7609643fb7d851f1b25a2025c74 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/503623 Reviewed-by: Brian Wilkerson <brianwilkerson@google.com> Commit-Queue: Keerti Parthasarathy <keertip@google.com>
2026-05-19 08:06:14 -07:00
parent e1c29b6381
commit c4ce9dbdfd
4 changed files with 230 additions and 0 deletions
@@ -0,0 +1,72 @@
+---
+name: closing-obsolete-issues
+description: Find obsolete, stale, or not reproducible analysis-server issues in the dart-lang/sdk repository.
+---
+
+# Closing Obsolete Issues
+
+Use this skill to find old, outdated issues in the `dart-lang/sdk` repository that have been fixed, are stale, obsolete, or not reproducible.
+
+## Instructions
+
+1. **Identify Target Issues**:
+   - Use the GitHub CLI (`gh`) to search for the oldest open issues.
+   - Use the label `area-devexp` to identify analysis server issues.
+   - Also use label `type-bug` and any other label that the user gives you.
+   - **Exclude Labeled Issues**: Exclude issues already labeled `verified-by-an-agent` or `closed-by-agent` by adding `-label:verified-by-an-agent -label:closed-by-agent` to the search query.
+   - Sort by creation date (`created-asc`) or last update (`updated-asc`) to find the most likely candidates for being outdated.
+   - Fetch at least 50 candidates.
+   - Example command (with label): `gh issue list --repo dart-lang/sdk --search "label:area-devexp is:open label:type-bug -label:verified-by-an-agent sort:created-asc" --limit 50 | cat`
+   - Example command (without label): `gh issue list --repo dart-lang/sdk --search "label:area-devexp is:open -label:verified-by-an-agent sort:created-asc" --limit 50 | cat`
+   - **Efficiency Filter**: Before investigating, read `references/investigated_issues.txt` (create it if it does not exist). Filter out and skip any candidate issue numbers that are already listed in this file.
+
+2. **Investigate Status**:
+   - **Pre-Qualification Guardrails**: Before evaluating any candidate against obsolete rationales, check the issue metadata. **Abort evaluation and keep the issue open** if any of the following are true:
+     1. *Recent Activity*: The issue has any comment or status change from a user within the last 365 days.
+     2. *Priority/Milestone*: The issue carries a high-priority label (e.g., `P0`, `P1`, `critical`) or is assigned to an active milestone.
+     3. *Corporate Interest*: The issue has active engagement or reproduction steps provided by a Dart/Flutter team member within the last 2 years.
+   - For candidates that pass pre-qualification, analyze their description and comments.
+   - Use the bundled script `scripts/fetch_issue_details.sh <number>` to get a comprehensive view of the issue and its comments.
+   - Compare the issue's request or reported bug and subsequent comments with the current state of the codebase.
+   - Refer to the data-driven checks in `references/rationale_templates.md` to verify if a rationale applies.
+   - **Safety Rule**: Do not assume a bug is fixed or obsolete just because the code has changed or the issue has not been updated for a long time. Verify if the specific bug behavior is still possible. Valid bugs or feature requests should not be closed as stale just because they are old or have no activity. Inactivity alone does not invalidate a feature request or bug report.
+
+3. **Draft and Review Comments (CRITICAL MANDATE)**:
+   - **For Candidates for Closure**: For issues identified as candidates for closure, draft a detailed comment for each explaining *why* it can be closed. Consult `references/rationale_templates.md` for wording inspiration. Each comment MUST end with: "If there is more work to do here, please let us know by filing a new issue with up to date information. Thanks!"
+   - **For Still-Valid Issues**: If you determine that a bug is still valid (reproducible on `HEAD`):
+     - Draft a comment confirming that the bug remains reproducible at `HEAD` on the latest SDK.
+     - Construct a minimal, self-contained Dart reproduction code example or a formal Dart unit test case (following existing testing patterns in the `test/` directory) demonstrating the issue.
+     - Include this reproduction or unit test in your drafted comment to assist developers in fixing the bug.
+     - **Footer Restriction**: Do NOT end comments on still-valid issues with the closure footer (about filing a new issue), as this issue is remaining open.
+   - **Required Output Format**: For each evaluated issue, generate your assessment in this structured JSON format so that a coordinator agent or human supervisor can easily parse, validate, and approve your findings:
+     ```json
+     {
+       "issue_number": 12345,
+       "issue_url": "https://github.com/dart-lang/sdk/issues/12345",
+       "eligible_for_close": true,
+       "matched_rationale_id": 1,
+       "confidence_score": 0.95,
+       "verification_finding": "Verified that standard LSP protocol capabilities natively handle the requested server configuration.",
+       "proposed_comment": "[Full proposed comment including the mandatory footer]"
+     }
+     ```
+   - **User Approval Required**: You MUST present both (a) the candidates for closure with their drafted comments, and (b) the still-valid issues with their drafted confirmation comments and reproductions, to the user and obtain explicit approval BEFORE running any command that comments on or closes an issue. You can present the JSON findings format directly to the user for review.
+
+4. **Iterate on Skill Knowledge (Learning Loop)**:
+   - If you discover a new, distinct category of closing rationale that is not covered in `references/rationale_templates.md`, **update the reference file** to include it.
+
+5. **Execute and Summarize**:
+   - **For Approved Closure Candidates**: Use `gh issue close` with the `-c` flag to post the comment and close the issue. Apply the `closed-by-agent` label to the issue.
+   - **For Approved Still-Valid Issues**: Use `gh issue comment <number> -b "<comment>"` to post the confirmation comment containing the minimal reproduction / test case. Apply the `verified-by-an-agent` label to the issue using `gh issue edit <number> --add-label "verified-by-an-agent"`.
+   - **Update Tracking File**: Append any investigated issue numbers that were determined to be STILL VALID (and thus left open) to `references/investigated_issues.txt`, one issue number per line. Do NOT track closed issues, as they are already filtered out by `is:open`.
+   - Provide the user with a clean bulleted list of closed issues and updated/commented valid issues.
+
+## Tips
+
+- Use available file and content search tools (such as `grep`, `ripgrep`, or environment-specific
+search tools) to check the current codebase for references to the issue or relevant code.
+- Look for related Gerrit CLs that might have fixed the issue but didn't close it automatically.
+- **Pro Tip**: Use the `read_gerrit_cl` skill ([SKILL.md](../../../../../.agents/skills/read_gerrit_cl/SKILL.md)) to inspect the patchset diffs and comments of open or merged Gerrit CLs.
+- **Always prioritize active verification on HEAD**: Regardless of how old an issue is or what version it specifies, always attempt to research and reproduce the reported issue against the current `HEAD` of the codebase before proposing closure. Never assume a bug is obsolete or fixed based solely on the passage of time or version discrepancies. If you cannot reproduce it, provide clear details of your reproduction attempt on the current codebase.
+
+
@@ -0,0 +1,75 @@
+# Issues investigated and confirmed still valid (leave open)
+23037
+24562
+25036
+25354
+25495
+25875
+26224
+26728
+26741
+57387
+28741
+28912
+29058
+29313
+29477
+57619
+31238
+31564
+31761
+31890
+31892
+32310
+32970
+33302
+33521
+33750
+33879
+57744
+34297
+57789
+57792
+35168
+35372
+35478
+35492
+35636
+57882
+57891
+57892
+35920
+35947
+57915
+36228
+36282
+36578
+36734
+57962
+37058
+37287
+37426
+37537
+37673
+38407
+37673
+58005
+38407
+58019
+38729
+38795
+38838
+39002
+39003
+42999
+43148
+43404
+43567
+43657
+43678
+43896
+43927
+44480
+44607
+45249
+45953
@@ -0,0 +1,53 @@
+# Common Obsolete Rationales for Analysis Server
+
+When evaluating open GitHub issues for the Analysis Server, Analyzer, or Linter components in the `dart-lang/sdk` repository, use this rationale matrix to determine eligibility for closure. For each rationale, you must perform the specified verification check before confirming.
+
+---
+
+## Rationale Evaluation Matrix
+
+### 1. Superseded by New Analysis Server Features
+*   **Criteria**: The request asks for a feature (e.g., a new command-line option, specialized diagnostic view, custom server capability, or specific formatting/plugin behavior) that is now natively supported by the modern Analysis Server or standard LSP protocol capabilities.
+*   **Verification**: Search the Analysis Server codebase or command-line configuration options (`lib/src/analysis_server.dart`, `lib/src/lsp/`, or server CLI options) to verify if the requested functionality is now natively supported or configurable.
+*   **Agent Comment Template**: "This feature request is now natively supported by modern Analysis Server capabilities (such as [Insert modern server option or capability]). Since the requested behavior is fulfilled by existing features, we are closing this. Thanks!"
+
+### 2. Superseded by Modern Dart Language Features
+*   **Criteria**: The issue requests an analysis fix or shorthand syntax that has since been natively solved by major Dart language upgrades (e.g., Extension Methods, Records, Patterns, Null Safety, Dot Shorthands, Primary Constructors, Private Named Parameters).
+*   **Verification**: Cross-reference the request with Dart language specifications added in Dart 2.x and 3.x. Note that the Analysis Server currently supports all Dart language versions down to Dart 2.12, meaning features like NNBD (Null Safety) are assumed.
+*   **Agent Comment Template**: "This request has been superseded by newer Dart language features (such as [insert feature, e.g., Records/Patterns]), which provide a robust, native solution to this pattern without requiring additional analyzer implementations. Thanks!"
+
+### 3. Stale Feature Requests
+*   **Criteria**: The issue is labeled `type-enhancement` or `type-feature`, is **greater than 3 years old**, and has **zero** community upvotes (thumbs up) or comments within the last 2 years.
+*   **Verification**: Calculate: `Current_Year - Issue_Creation_Year > 3` AND `Upvotes == 0` and `Comments in last 2 years == 0`.
+*   **Agent Comment Template**: "This feature request has seen no activity or community interest for several years. Because the Analysis Server priorities and architecture have evolved significantly since this was filed, we are closing this as stale. Thanks!"
+
+### 4. Untestable / Legacy SDK Version Bugs
+*   **Criteria**: The issue reports a bug or crash occurring exclusively on a highly outdated SDK version (compare against the baseline version found in `tools/VERSION`).
+*   **Verification**: The reported version must be **older than the last 4 stable minor releases**.
+*   **Agent Comment Template**: "This issue was reported on an outdated version of the Dart SDK and Analysis Server. Due to massive changes in the codebase, it is likely already resolved or no longer reproducible in the current stable release. Are you still experiencing this on the latest stable Dart SDK? If so, please reopen with updated reproduction steps. Thanks!"
+
+### 5. Insufficient Information (Dead End)
+*   **Criteria**: The issue lacks reproduction steps, code snippets, or logs, AND a maintainer requested info **greater than 90 days ago** with no response from the author.
+*   **Verification**: Check for a `needs-info` label or maintainer question followed by author silence exceeding 90 days.
+*   **Agent Comment Template**: "Without a minimal reproducible example or diagnostic logs, we are unable to investigate or debug this behavior further. As there has been no follow-up to our request for information, we are closing this issue. Please feel free to reopen if you can provide repro steps. Thanks!"
+
+### 6. Legacy JSON Protocol Specifics
+*   **Criteria**: The issue requests features or reports bugs specific to the legacy Analysis Server JSON protocol or legacy IDE integrations that do not apply to LSP.
+*   **Verification**: Verify the issue is tied to the old protocol and that the feature is natively handled or bypassed by the Language Server Protocol (LSP).
+*   **Safety Rule**: Do NOT blanket-close legacy protocol issues without evaluating their severity. While most editor clients have transitioned to LSP, critical or highly severe bugs on the legacy protocol must remain open and be addressed. Only propose closure if the behavior is resolved/handled standardly in LSP and the legacy issue is non-critical.
+*   **Agent Comment Template**: "This issue is specific to the legacy analysis server protocol which has been superseded by the Language Server Protocol (LSP). Modern editor clients have transitioned to LSP, where this behavior is standard or natively resolved. Thanks!"
+
+### 7. Deprecated or Removed Lint Rules / Diagnostics
+*   **Criteria**: The issue relates to a lint rule or analyzer diagnostic code that has been deprecated, retired, or merged into another rule.
+*   **Verification**: Search the repository's `pkg/linter` or `pkg/analyzer` directories to confirm the rule (`lint_name`) no longer exists or is explicitly marked deprecated.
+*   **Agent Comment Template**: "The lint rule or diagnostic referenced in this issue (`{lint_name}`) has been deprecated or completely removed from newer versions of the Dart SDK, rendering this request obsolete. Thanks!"
+
+### 8. Resolved Upstream / Outside SDK Repository
+*   **Criteria**: The root cause of the bug belongs to an IDE extension client wrapper (e.g., VS Code Dart/Flutter extension, IntelliJ Dart plugin) rather than the core SDK Analysis Server.
+*   **Verification**: Check if the issue description is entirely UI-dependent on a specific editor interface.
+*   **Agent Comment Template**: "This issue relates to the specific IDE client integration rather than the core underlying Analysis Server. Upstream updates in the [VS Code / IntelliJ] Dart extension have altered or resolved this behavior. Thanks!"
+
+### 9. Refactored Code Paths (Silent Fixes)
+*   **Criteria**: The bug targets a subsystem that underwent a complete architectural rewrite.
+*   **Verification**: Verify that the file paths or engine components mentioned in the original issue no longer exist in the current branch.
+*   **Agent Comment Template**: "Due to major internal refactorings and updates to the analyzer's core implementation since this issue was filed, the affected code paths have been completely replaced. The historical bug is no longer reproducible or applicable. Thanks!"
@@ -0,0 +1,30 @@
+#!/bin/bash
+
+# A script to fetch and format comprehensive issue details for investigation.
+# Usage: ./fetch_issue_details.sh <issue_number>
+
+ISSUE_NUMBER=$1
+
+if [ -z "$ISSUE_NUMBER" ]; then
+  echo "Usage: $0 <issue_number>"
+  exit 1
+fi
+
+echo "--- INVESTIGATION FOR ISSUE #$ISSUE_NUMBER ---"
+# Fetching all comments to ensure full context is captured.
+gh issue view "$ISSUE_NUMBER" --repo dart-lang/sdk --json number,title,author,createdAt,labels,body,comments -t '
+Title: {{.title}}
+Author: {{if .author}}{{.author.login}}{{else}}ghost{{end}}
+Created: {{.createdAt}}
+Labels: {{range .labels}}{{.name}}, {{end}}
+
+Description:
+{{.body}}
+
+--- ALL COMMENTS ---
+{{range .comments}}
+{{if .author}}{{.author.login}}{{else}}ghost{{end}} ({{.createdAt}}):
+{{.body}}
+------------------------------------------------------------
+{{end}}
+'