Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: create automated dashboard ci #171

Draft
wants to merge 85 commits into
base: main
Choose a base branch
from
Draft
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
e7feeea
feat: create project-dashboard.yaml
rdash99 Oct 3, 2024
00f8d3a
refactor: remove erroneous id
rdash99 Oct 3, 2024
3b23654
refactor: switch to new token
rdash99 Oct 3, 2024
a4f1d8f
fix: use actual runners
rdash99 Oct 3, 2024
464c08c
test: try octokit import
rdash99 Oct 3, 2024
dfb4fd5
test: try long script
rdash99 Oct 3, 2024
579b4e0
test: disable code to find errant bracket
rdash99 Oct 3, 2024
0e055d7
fix: modify request
rdash99 Oct 3, 2024
409e98b
fix: modify request
rdash99 Oct 3, 2024
aa010d4
fix: modify request
rdash99 Oct 3, 2024
222f2f1
fix: modify request
rdash99 Oct 3, 2024
c0f93ad
fix: modify request based on a gist
rdash99 Oct 3, 2024
993a897
refactor: disable most code
rdash99 Oct 3, 2024
b1a1de2
refactor: add awaits
rdash99 Oct 4, 2024
635236d
fix: const vars
rdash99 Oct 4, 2024
7a8d9fd
fix: use built in pagination
rdash99 Oct 4, 2024
b2ef9d4
refactor: change runner group
rdash99 Oct 4, 2024
3330787
test: try adding plugin
rdash99 Oct 4, 2024
d0aacd1
test: try specifying octokit
rdash99 Oct 4, 2024
e3d32f2
test: try specifying github.graphql again
rdash99 Oct 4, 2024
f81c894
test: import with npm
rdash99 Oct 4, 2024
f8ae876
test: use built in plugin?
rdash99 Oct 4, 2024
446377a
fix: remove octokit reference
rdash99 Oct 4, 2024
16aaf23
test: add iterator
rdash99 Oct 4, 2024
52ba7b5
refactor: split jobs
rdash99 Oct 4, 2024
7251905
refactor: npm imports
rdash99 Oct 4, 2024
2ead9d4
test: change imports
rdash99 Oct 7, 2024
728537c
test: change imports
rdash99 Oct 7, 2024
e4e4c42
test: remove imports
rdash99 Oct 7, 2024
41941e7
test: try while loop
rdash99 Oct 8, 2024
5e7929a
test: try blank var
rdash99 Oct 8, 2024
a72e1c5
test: try modifying script
rdash99 Oct 8, 2024
8694d8f
test: change code
rdash99 Oct 8, 2024
87ffa11
test: search query
rdash99 Oct 8, 2024
58f2a9e
test: search query
rdash99 Oct 8, 2024
b536de2
test: update references to the actual correct thing
rdash99 Oct 8, 2024
b15e073
test: try map
rdash99 Oct 8, 2024
1c09ae7
test: try adding token
rdash99 Oct 8, 2024
018db92
test: try importing again
rdash99 Oct 8, 2024
5851228
test: remove vars
rdash99 Oct 8, 2024
9326b14
Update project-dashboard.yaml
rdash99 Oct 8, 2024
ccbfa69
fix: correct name
rdash99 Oct 8, 2024
bb1f251
test: update requires
rdash99 Oct 8, 2024
f3ab4d1
test: while loop again
rdash99 Oct 8, 2024
d4ea8f0
test: outputs
rdash99 Oct 8, 2024
a395dd0
refactor: remove unused steps and get all repos
rdash99 Oct 8, 2024
6c5e40a
test: change search query
rdash99 Oct 8, 2024
e84e8d1
test: up the limit
rdash99 Oct 8, 2024
d0b7550
test: add to query
rdash99 Oct 8, 2024
838cf7f
test: try getting private repos
rdash99 Oct 8, 2024
cd64edf
test: switch back to public
rdash99 Oct 8, 2024
5679cb2
test: update query
rdash99 Oct 8, 2024
99f6b70
fix: correct step name
rdash99 Oct 8, 2024
9cdae58
test: add isPrivate
rdash99 Oct 8, 2024
cbfa2a9
test: try adding undocumented token input
rdash99 Oct 10, 2024
cfcccdb
test: add board request to main job
rdash99 Oct 10, 2024
51db604
test: print board response
rdash99 Oct 10, 2024
8a931b2
test: attempt to print id
rdash99 Oct 10, 2024
28fef86
test: print id
rdash99 Oct 10, 2024
699c6d6
refactor: remove extra job, rename and switch to light runners
rdash99 Oct 10, 2024
2871834
feat: other queries and add to board
rdash99 Oct 10, 2024
98b27a4
fix: fix for loop
rdash99 Oct 10, 2024
52f404e
fix: fix query
rdash99 Oct 10, 2024
07dbf0b
refactor: remove commented out code
rdash99 Oct 10, 2024
f290874
fix: fix queries
rdash99 Oct 10, 2024
098ea61
fix: make repo name non-nullable
rdash99 Oct 10, 2024
993df8b
fix: remove author argument
rdash99 Oct 10, 2024
b3875cf
fix: missing bracket and vars
rdash99 Oct 10, 2024
a552862
refactor: remove old logging and change step name
rdash99 Oct 10, 2024
d8b4059
feat: add issues and expand PR scope
rdash99 Oct 10, 2024
bd24ab7
fix: variable declaration
rdash99 Oct 10, 2024
580fee3
test: try clearing the board at the start of every run
rdash99 Oct 11, 2024
69e67ae
test: only pull in open PRs to save space
rdash99 Oct 11, 2024
244feac
fix: don't redeclare var
rdash99 Oct 11, 2024
2d2abfc
refactor: scope error
rdash99 Oct 11, 2024
a25eff9
fix: brackets
rdash99 Oct 11, 2024
7708817
refactor: limit issues
rdash99 Oct 11, 2024
53f064c
fix: brackets
rdash99 Oct 11, 2024
e499a82
refactor: remove unused value
rdash99 Oct 11, 2024
56e8492
Merge branch 'main' into feat/project-board-collation-ci
rdash99 Oct 11, 2024
80eb13c
test: see how many points are used
rdash99 Oct 11, 2024
9d52418
test: print total limit
rdash99 Oct 11, 2024
a16cecb
Revert "test: print total limit"
rdash99 Oct 11, 2024
5d7d6d1
Revert "test: see how many points are used"
rdash99 Oct 11, 2024
622973c
refactor: add basic comments
rdash99 Oct 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
196 changes: 196 additions & 0 deletions .github/workflows/project-dashboard.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
name: Project Dashboard

on:
pull_request:
paths:
- '.github/workflows/project-dashboard.yaml'
workflow_dispatch:
schedule:
- cron: '0 * * * *'

concurrency:
group: ${{ github.workflow }}
cancel-in-progress: false

jobs:
dashboard:
runs-on:
labels: [ self-hosted, linux, x64 ]
group: light
steps:

- uses: actions/github-script@v7
id: dashboard-updater
env:
GITHUB_TOKEN: ${{ secrets.DASHBOARD_TEST_TOKEN }}
with:
result-encoding: string
retries: 3
retry-exempt-status-codes: 400,401
github-token: ${{ secrets.DASHBOARD_TEST_TOKEN }}
script: |
// This query deletes an item from the project board
const clearQuery = `mutation ClearBoard($boardID: ID!, $itemID: ID!) {
deleteProjectV2Item(input: {projectId: $boardID, itemId: $itemID}) {
deletedItemId
}
}`;

// This query gets a list of items on the project board
const getProjectData = `query GetProjectData($boardID: ID!, $cursor: String){node(id: $boardID) {
... on ProjectV2 {
items(first: 100, after: $cursor) {
pageInfo {
hasNextPage
endCursor
}
items: edges {
node {
id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we filter the query for what to remove to look for things closed or merged more than X days ago?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe though what I'm grabbing now is just 'everything on the board'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How long are we talking days wise?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nvm - this doesn't look possible, you can find the type (issue/pr etc) but not any further details and the only option I can see as to ordering is the order it appears on the project board itself

}
}
}
}
}}`;

// This query gets the ID of the project board
const board = `query {
organization(login: "SwanseaUniversityMedical") {
projectV2(number: 15) {
id
}
}
}`;

const boardData = await github.graphql(board);
const boardID = boardData.organization.projectV2.id;

// Clear the contents of the project board
let cursor = null;
let hasNextPage = true;
const boardItems = [];

while (hasNextPage) {
const response = await github.graphql(getProjectData, { boardID, cursor });
boardItems.push(...response.node.items.items.map(item => item.node.id));
hasNextPage = response.node.items.pageInfo.hasNextPage;
cursor = response.node.items.pageInfo.endCursor;
}

for (let i = 0; i < boardItems.length; i++) {
await github.graphql(clearQuery, { boardID, itemID: boardItems[i] });
}


// This query gets a list of repositories in the organization
const query = `query Search($cursor: String) {
search(query: "org:SwanseaUniversityMedical", type: REPOSITORY, first: 100, after: $cursor) {
pageInfo {
hasNextPage
endCursor
},
repos: edges {
node {
... on Repository {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this ... on Repository { part doing? Not familiar with the syntax.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It forces graphql to make sure that the node is the correct type iirc, it also has other functions in graphql depending on how you use it, on line 103 I think it helps expand the response and acts like a for loop while if used in other places it can make blocks reusable: https://graphql.org/learn/queries/#fragments

name
isPrivate
}
}
}
}
}`;

// Build a list of repositories for the organization
cursor = null;
hasNextPage = true;
const repositories = [];

while (hasNextPage) {
const response = await github.graphql(query, { cursor });
repositories.push(...response.search.repos.map(repo => repo.node.name));
hasNextPage = response.search.pageInfo.hasNextPage;
cursor = response.search.pageInfo.endCursor;
}

// This query gets a list of PRs in the organization
const PRQuery = `query PRs($cursor: String, $repo: String!) {
organization(login: "SwanseaUniversityMedical") {
repository(name: $repo) {
pullRequests(first: 100, after: $cursor, states: [OPEN]) {
pageInfo {
hasNextPage
endCursor
}
PRs: edges {
node {
id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we limit it to only authored by marvin-serp-bot (by variable).

Copy link
Contributor Author

@rdash99 rdash99 Oct 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't limit by author unfortunately - it was one of the first things I tried to do

}
}
}
}
}
}`;

// This query gets a list of issues in the organization
const IssueQuery = `query Issues($cursor: String, $repo: String!) {
organization(login: "SwanseaUniversityMedical") {
repository(name: $repo) {
issues(first: 100, after: $cursor, states: [OPEN]) {
pageInfo {
hasNextPage
endCursor
}
issues: edges {
node {
id
rdash99 marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
}
}
}`;

// This mutation adds an item to the project board
const addToBoard = `mutation AddToBoard($boardID: ID!, $ID: ID!) {
addProjectV2ItemById(input: {projectId: $boardID, contentId: $ID}) {
item {
id
}
}
}`;

// Get all PRs and then add them to the project board
for (let i = 0; i < repositories.length; i++) {
let cursor = null;
let hasNextPage = true;
const PRS = [];

while (hasNextPage) {
const response = await github.graphql(PRQuery, { cursor, repo: repositories[i] });
PRS.push(...response.organization.repository.pullRequests.PRs.map(pr => pr.node.id));
hasNextPage = response.organization.repository.pullRequests.pageInfo.hasNextPage;
cursor = response.organization.repository.pullRequests.pageInfo.endCursor;
}

for (let j = 0; j < PRS.length; j++) {
await github.graphql(addToBoard, { boardID, ID: PRS[j] });
}
}

// Get all issues and then add them to the project board
for (let i = 0; i < repositories.length; i++) {
let cursor = null;
let hasNextPage = true;
const issues = [];

while (hasNextPage) {
const response = await github.graphql(IssueQuery, { cursor, repo: repositories[i] });
issues.push(...response.organization.repository.issues.issues.map(issue => issue.node.id));
hasNextPage = response.organization.repository.issues.pageInfo.hasNextPage;
cursor = response.organization.repository.issues.pageInfo.endCursor;
}

for (let j = 0; j < issues.length; j++) {
await github.graphql(addToBoard, { boardID, ID: issues[j] });
}
}
Loading