Repo Gardening: add logic to get label suggestions from OpenAI (#39152)

* Repo Gardening: add logic to get label suggestions from OpenAI

This will be triggered as part of the existing "Triage issues" task, only when an issue is opened.

It will fetch the list of "Feature" and "Feature Group" labels available in the repo, and then send the issue contents to OpenAI alongside with that list of labels.
OpenAI will respond with a list of labels that could match the issue.
We then apply those labels to the issue.

Internal reference: pdqkMK-1kt-p2

* Fix output format and error handling

* Clarify usage in readmes.

* Include issue title in text sent to OpenAI

* Update prompt

* Trigger the auto-labeling only on issues with a specific label

* During testing, comment to the issue explaining the label choices

* Update projects/github-actions/repo-gardening/src/tasks/triage-issues/index.js

Co-authored-by: Brad Jorsch <[email protected]>

* Update projects/github-actions/repo-gardening/action.yml

Co-authored-by: Brad Jorsch <[email protected]>

* Update projects/github-actions/repo-gardening/src/tasks/triage-issues/index.js

Co-authored-by: Brad Jorsch <[email protected]>

* Update projects/github-actions/repo-gardening/src/tasks/triage-issues/index.js

Co-authored-by: Brad Jorsch <[email protected]>

* Update projects/github-actions/repo-gardening/src/utils/labels/get-all-labels.js

Co-authored-by: Brad Jorsch <[email protected]>

* Switch to a different model, gpt-4o-mini

From OpenAI:

> GPT-4o mini (“o” for “omni”) is our most advanced model in the small models category, and our cheapest model yet. It is multimodal (accepting text or image inputs and outputting text), has higher intelligence than gpt-3.5-turbo but is just as fast. It is meant to be used for smaller tasks, including vision tasks.
>
> We recommend choosing gpt-4o-mini where you would have previously used gpt-3.5-turbo as this model is more capable and cheaper.

* Ensure labels are properly displayed

See #39152 (comment)

* Rename label getter

See #39152 (comment)

* Simplify label fetching / catching

See #39152 (comment)

* Adjust prompt to better separate each data point passed to OpenAI

See #39152 (comment)

* Always display OpenAI response

See #39152 (comment)

Co-authored-by: anomiex <[email protected]>

* Remove extra logging

See #39152 (comment)

Co-authored-by: anomiex <[email protected]>

* Try requesting a different format back from OpenAI

See #39152 (comment)

Co-authored-by: anomiex <[email protected]>

* Fail the job in case of OpenAI errors.

See #39152 (comment)

* Try a more strict response format

See #39152 (comment)

OpenAI's Structured Outputs does not appear to be available for our model, gpt-4o-mini, yet.

* Force a JSON Object response

* Tweak prompt

* Avoid shortcircuiting following logic when we get no results

* Add a label marking processed entries

This is necessary during the experiment, since we trigger the process on multiple actions (opened, labeled, reopened). We don't want the labeling to be duplicated if an issue is opened AND labeled.

* Simplify label checking logic

See #39152 (comment)

* customize prompt so it better understands the label list format.

See #39152 (comment)

* Update return type

Co-authored-by: Brad Jorsch <[email protected]>

* Be more precise in expected json object response

Co-authored-by: Brad Jorsch <[email protected]>

* Return early if no OpenAI key is set.

See #39152 (comment)

---------

Co-authored-by: Brad Jorsch <[email protected]>

.github/workflows/gardening.yml

@@ -67,3 +67,4 @@ jobs:
           slack_quality_channel: ${{ secrets.SLACK_QUALITY_CHANNEL }}
           triage_projects_token: ${{ secrets.PROJECT_AUTOMATION_TOKEN }}
           project_board_url: ${{ secrets.PROJECT_BOARD_URL }}
+          openai_api_key: ${{ secrets.OPENAI_API_KEY }}

projects/github-actions/repo-gardening/README.md

@@ -87,6 +87,7 @@ The action relies on the following parameters.
 - (Optional) `triage_projects_token` is a [personal access token](https://github.com/settings/tokens/new) with `repo` and `project` scopes. The token should be stored in a secret. This is required if you want to use the `updateBoard` task.
 - (Optional) `project_board_url` is the URL of a GitHub Project Board. We'll automate some of the work on that board in the `updateBoard` task.
 - (Optional) `labels_team_assignments` is a list of features you can provide, with matching team names, as specified in the "Team" field of your GitHub Project Board used for the `updateBoard` task, and lists of labels in use in your repository.
+- (Optional) `openai_api_key` is the API key for OpenAI. This is required if you want to use the `triageIssues` task to automatically add labels to your issues. **Note**: this option is only available for Automattic-hosted repositories.
 
 #### How to create a Slack bot and get your SLACK_TOKEN
 

projects/github-actions/repo-gardening/action.yml

@@ -56,6 +56,10 @@ inputs:
     description: "Mapping of team assignments for labels"
     required: false
     default: ""
+  openai_api_key:
+    description: "OpenAI API key"
+    required: false
+    default: ""
 runs:
   using: node20
   main: "dist/index.js"

projects/github-actions/repo-gardening/changelog/add-openai-auto-labeling-logic

@@ -0,0 +1,4 @@
+Significance: minor
+Type: added
+
+Issue triage: allow querying OpenAI to get a list of labels that could potentially be added to an issue, based off the issue body, when the issue is opened.

projects/github-actions/repo-gardening/package.json

@@ -18,7 +18,8 @@
 		"compare-versions": "3.6.0",
 		"glob": "10.4.1",
 		"moment": "2.29.4",
-		"node-fetch": "2.6.7"
+		"node-fetch": "2.6.7",
+		"openai": "4.56.1"
 	},
 	"devDependencies": {
 		"@vercel/ncc": "0.36.1"