From 73f4f80ee7f34ecc2c34022df3a7064c71a32a8c Mon Sep 17 00:00:00 2001 From: Spyros Date: Sun, 29 Oct 2023 02:06:02 +0000 Subject: [PATCH] Merge staging (#439) * trigger first staging build * Inital Hack of NEO4j DB creation * Added: Neo4j docker run * Added NEO_DB Class * Create Inital Path API response * Build basic UI for testing * Mock score and WIP UI * implement scoring and basic tests * scoring passing tests * Update the UI * Updated: Dropdowns are now dynamic * Localise neo4j * Added Navigation method * Add share and nav links * readme improvement * Hide table on new search * Optermise query to remove relates to * Get duel running method working and show empty values * Refactor grouping & scoring code locations * Add colour to strength raiting * Add gap analysis tests * Short drop down list * Styling improvements and legends * rm scikit version pin * loosen requirement in sqlalchemy * more deps changes * fix deps * Disable auto db pop (#372) * Add gap analysis tests * Short drop down list * Disable CRE_Graph auto population on startup * revert conflict mess with tests * Migrate Neo4j population to seperate function * Add CLI population command for DB * linting fixes * fix pillow * Use python 3.11.4 for CI * Parse cre_dep object from neo4j Node * Standardise Gap Analysis API Object * Fix bad parse field * Use Standard display format and create GA type * GA: Show all strong links by default (or min of 3) * Better route display * GA add support for tools * Fix existing tests * Add parsing tests * make parse_node static * Use Enum types * Better route display * linting * Minimal GA styling * Fix: GA share link spaces * linting * Added: Direct link grade and removed external link icon * Fix: linked via standard was possible * Added: Per link score penalty * Fix: GA caching not taking queries (#412) * Make Direct Dark Green (#411) * Make Direct Dark Green * Update GapAnalysis.tsx Signed-off-by: John Harvey <10814889+john681611@users.noreply.github.com> --------- Signed-off-by: John Harvey <10814889+john681611@users.noreply.github.com> * GA Reduce related punishment (#416) * Rename gap to map (#419) * GA make strong status <=2 * Rename URL and UI from Gap to Map analysis * GA fix bad links (#417) Fix bad links in GA (use standard function) * Speed up CICD (#418) * Rename share link (#415) * Neo4j ORM (#408) * Make Direct Dark Green * Update GapAnalysis.tsx Signed-off-by: John Harvey <10814889+john681611@users.noreply.github.com> * Trial implementation of Neo4J ORM * Populate Neo4j DB using model * More progress in conversion * get ORM working * Fix tests * Remove old NeoDB driver connection * Fix: id issue * Fix: Tags strings splitting * linting fixes * Remove NeomodelPath reference * Revert rebase bug --------- Signed-off-by: John Harvey <10814889+john681611@users.noreply.github.com> * Remove BOLT from neo4j Setup requirement (#428) * pin neomodel to a decent version * relax six requirements * Sort Base standard alphabetically (#427) * Use allShortestPaths over shortestPath (#431) * Added Cross-cutting concerns to GA blacklist (#430) * Added Cross-cutting concerns to GA blacklist * GA page linting * Remove double brackets * Fix GA loading disappearing * rename variable * Revert "Use allShortestPaths over shortestPath (#431)" This reverts commit e33d13d9e1de2514f08fc4d8dadba0177fa4d665. * Precompute map analysis (#433) * add time metrics * progress * Revert "Revert "Use allShortestPaths over shortestPath (#431)"" This reverts commit 167828c40955cb0f470c137256cbfb2be2a3d307. * map analysis through redis * new reqs * add worker actually * abort if background job fails * lint * minor cleanup * move hash to utils and make db method write to redis optionally * make ga job results shared among clients * lint * Split ga response (#434) * add time metrics * map analysis through redis * move hash to utils and make db method write to redis optionally * lint * Added: Front end support for weaker links calls * Added: DB seperation for smaller inital response and cache extras * Added: DB side of part split response logic * Refactor and tests --------- Co-authored-by: Spyros * skip returning ids for failed jobs * add optional tls to redis, heroku requirement * add optional tls to redis, heroku requirement * add optional tls to redis, heroku requirement * add optional tls to redis, heroku requirement * add optional tls to redis, heroku requirement * add optional tls to redis, heroku requirement * use db to cache gap analysis results instead of redis (#435) * use db to cache gap analysis results instead of redis * lint * typo * fix 1 test, fix bugs, make cache key into its own function * lint * fix-mock * migration * Fix: bad ref * fix tests * lint * minor changes * lint --------- Co-authored-by: john681611 * Get e2e tests reporting correctly (#395) * try pip cacheing * try yarn cache * try background service method * e2e test fixes * tmp increase timeouts for e2e tests due to new neo4j experiments * fix e2e tests * make e2e workflow match makefile * Revert actions simplification * increase timing * balance timings --------- Signed-off-by: Spyros Co-authored-by: Spyros Co-authored-by: Spyros * minor fixes * fix-lint * fix caching * fix e2e? * add tabs * lint --------- Signed-off-by: John Harvey <10814889+john681611@users.noreply.github.com> Signed-off-by: Spyros Co-authored-by: john681611 Co-authored-by: John Harvey <10814889+john681611@users.noreply.github.com> --- application/frontend/src/pages/Search/components/BodyText.tsx | 3 +++ application/frontend/src/scaffolding/Header/Header.tsx | 4 ++++ application/frontend/src/test/basic-e2etest.ts | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/application/frontend/src/pages/Search/components/BodyText.tsx b/application/frontend/src/pages/Search/components/BodyText.tsx index 7e089c105..7347627b9 100644 --- a/application/frontend/src/pages/Search/components/BodyText.tsx +++ b/application/frontend/src/pages/Search/components/BodyText.tsx @@ -30,6 +30,9 @@ export const SearchBody = () => { into an AI model to create the world's first security-specialized chatbot. This ensures you get a more reliable answer, and also a reference to a reputable source. + + Use Map Analysis to find how any two Resources connect with eachother +

HOW?

OpenCRE links each section of a resource (like a standard or guideline) to a shared topic, known as a diff --git a/application/frontend/src/scaffolding/Header/Header.tsx b/application/frontend/src/scaffolding/Header/Header.tsx index f1d09813c..362c0176d 100644 --- a/application/frontend/src/scaffolding/Header/Header.tsx +++ b/application/frontend/src/scaffolding/Header/Header.tsx @@ -13,6 +13,10 @@ const getLinks = (): { to: string; name: string }[] => [ to: `/`, name: 'Open CRE', }, + { + to: `/chatbot`, + name: 'OpenCRE Chat', + }, { to: `/map_analysis`, name: 'Map analysis', diff --git a/application/frontend/src/test/basic-e2etest.ts b/application/frontend/src/test/basic-e2etest.ts index 6e5e855f7..e6dbe1caa 100644 --- a/application/frontend/src/test/basic-e2etest.ts +++ b/application/frontend/src/test/basic-e2etest.ts @@ -162,7 +162,7 @@ describe('App.js', () => { it('can smartlink', async () => { const response = await page.goto('http://127.0.0.1:5000/smartlink/standard/CWE/1002'); expect(response.url()).toBe('http://127.0.0.1:5000/node/standard/CWE/sectionid/1002'); - + const redirectResponse = await page.goto('http://127.0.0.1:5000/smartlink/standard/CWE/404'); page.waitForNavigation('networkidle2'); expect(redirectResponse.url()).toBe('https://cwe.mitre.org/data/definitions/404.html');