Skip to content

Commit

Permalink
AO-909 Update to docusaurus 3.x
Browse files Browse the repository at this point in the history
  • Loading branch information
kaggis committed Mar 11, 2024
1 parent c23a0ec commit 4bd1a6c
Show file tree
Hide file tree
Showing 12 changed files with 8,076 additions and 14,844 deletions.
4 changes: 2 additions & 2 deletions Jenkinsfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pipeline {
agent {
docker {
image 'node:buster'
image 'node:18-buster'
}
}
options {
Expand Down Expand Up @@ -82,4 +82,4 @@ pipeline {
}
}
}
}
}
3 changes: 2 additions & 1 deletion website/docs/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ slug: /

## Metrics Framework
<p align='justify'>
The Recommender System (RS) is a novel component of the EOSC Portal meant to improve user experience. It provides EOSC users with recommendations concerning resources that could be of their interest, based on a multi-focal perspective of the users. Measuring the success of such system is crucial to get valuable insights in many aspects that affect user experience. In this approach, an independent metrics framework as a service is being introduced to support the evaluation and adaptation of recommendation mechanisms. The use of additional diagnostic metrics and visualizations offers deeper and sometimes surprising insights into a model’s performance. The evaluation is quantitatively being performed by processing information such as <b>resources</b>, <b>user actions</b>, <b>ratings</b>, and <b>recommendations</b> in order to measure the impact of the AI-enhanced services and user satisfaction as well as to incorporate this feedback and improve the services provided, via a user-friendly API and dashboard UI.</p>
The Recommender System (RS) is a novel component of the EOSC Portal meant to improve user experience. It provides EOSC users with recommendations concerning resources that could be of their interest, based on a multi-focal perspective of the users. Measuring the success of such system is crucial to get valuable insights in many aspects that affect user experience. In this approach, an independent metrics framework as a service is being introduced to support the evaluation and adaptation of recommendation mechanisms. The use of additional diagnostic metrics and visualizations offers deeper and sometimes surprising insights into a model’s performance. The evaluation is quantitatively being performed by processing information such as <b>resources</b>, <b>user actions</b>, <b>ratings</b>, and <b>recommendations</b> in order to measure the impact of the AI-enhanced services and user satisfaction as well as to incorporate this feedback and improve the services provided, via a user-friendly API and dashboard UI.
</p>

![Flow](static/img/flow.png)

Expand Down
6 changes: 4 additions & 2 deletions website/docs/metrics_descriptions/accuracy.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,11 @@ Measures Recommendations' accuracy based on users' access to the services. A val

## Description
The accuracy $$(A$$ of the recommendations is based on users' access to the services. A value of 1, indicates that the RS model got all the predictions right, and a value of 0 indicates that the RS model did not make a single correct prediction. Generally, the Accuracy mathematical expression is defined as:
$$A=\frac{Number\;of\;correct\;predictions}{Total\;number\;of\;predictions}$$<p>
$$A=\frac{Number\;of\;correct\;predictions}{Total\;number\;of\;predictions}$$
<p>
In RS Metrics the computation is determined by the following formula:
$$Accuracy=\frac{Number\;of\;correctly\;recommended\;services}{Total\;number\;of\;services}$$where correctness is defined as if the service is both accessed by the user and also it is recommended by the RS</p>
$$Accuracy=\frac{Number\;of\;correctly\;recommended\;services}{Total\;number\;of\;services}$$where correctness is defined as if the service is both accessed by the user and also it is recommended by the RS
</p>

## Output

Expand Down
2 changes: 1 addition & 1 deletion website/docs/metrics_descriptions/catalog_coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ sidebar_position: 2
The percentage (%) of the division of the unique services found in recommendations to the total number of published services.

## Description
The Catalog Coverage is described by the formula: $$\frac{unique\_rec\_services}{services}$$
The Catalog Coverage is described by the formula: $\frac{unique\_rec\_services}{services}$

## Output

Expand Down
11 changes: 7 additions & 4 deletions website/docs/metrics_descriptions/click_through_rate.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ sidebar_position: 3
The number of user clicks through recommendations panels divided by the total times recommendation panels were presented to users.

## Description
The number of user clicks through recommendations panels divided by the total times recommendation panels were presented to users. Takes into account all historical data of user actions.<p>The metric is expressed by the formula: $$Click-Through Rate=\frac{clicks}{views}$$</p>
The number of user clicks through recommendations panels divided by the total times recommendation panels were presented to users. Takes into account all historical data of user actions.
<p>
The metric is expressed by the formula: $$Click-Through Rate=\frac{clicks}{views}$$
</p>

## Output

Expand All @@ -29,9 +32,9 @@ A value of 0 indicates that no clicks through recommendations panels occurred.
## Process Flow:
* ### Retrieve user actions with recommendation panel
Get only the user actions that present a recommendation panel to the user in the source page. Those are actions with the following source paths:
* /services
* /services
* /services/c/{any category name}
* `/services`
* `/services`
* `/services/c/{any category name}`
* ### Count user actions with recommendation panel
Count the items in the above list as they represent the times recommendations panels were presented to the users of the portal
* ### Filter list
Expand Down
6 changes: 4 additions & 2 deletions website/docs/metrics_descriptions/diversity_gini_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ sidebar_position: 4
Measures Recommendations' diversity. The index is 0 when all items are chosen equally often, and 1 when a single item is always chosen.

## Description
The diversity ($$G$$) of the recommendations according to Gini Index. The index is 0 when all items are chosen equally often, and 1 when a single item is always chosen (see [book](https://link.springer.com/10.1007/978-1-4939-7131-2_110158)). Generally, the Gini Index mathematical expression is defined as:<p>
The diversity ($$G$$) of the recommendations according to Gini Index. The index is 0 when all items are chosen equally often, and 1 when a single item is always chosen (see [book](https://link.springer.com/10.1007/978-1-4939-7131-2_110158)). Generally, the Gini Index mathematical expression is defined as:
<p>
$$G=\frac{1}{n-1}\sum_{j=1}^{n}(2j-n-1)p(i_j)$$, where $$i_1,\ldots,i_n$$ is the list of items ordered according to increasing $$p(i)$$ and each item $$i$$ accounts for a proportion $$p(i)$$ of user recommendations.
In RS Metrics the computation is determined by the following formula:
$$Diversity=\frac{1}{n-1}\sum_{j=1}^{n}(2j-n-1)\left(\frac{count(j)}{recommendations}\right)$$ </p>
$$Diversity=\frac{1}{n-1}\sum_{j=1}^{n}(2j-n-1)\left(\frac{count(j)}{recommendations}\right)$$
</p>

## Output

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,10 @@ sidebar_position: 5
Measures Recommendations' diversity. The entropy is 0 when a single item is always chosen or recommended, and log n when n items are chosen or recommended equally often.

## Description
The diversity ($$H$$) of the recommendations according to Shannon Entropy. The entropy is 0 when a single item is always chosen or recommended, and log(n) when n items are chosen or recommended equally often (see [book](https://link.springer.com/10.1007/978-1-4939-7131-2_110158)). Generally, the Shannon Entropy mathematical expression is defined as: $$H=-\sum_{i=1}^{n}p(i)\log_2 p(i)$$<p>
In RS Metrics the formula is determined as: $$Diversity=-\sum_{i=1}^{services}\left(\frac{count(i)}{recommendations}\right)\log_2 \left(\frac{count(i)}{recommendations}\right)$$</p>
The diversity ($$H$$) of the recommendations according to Shannon Entropy. The entropy is 0 when a single item is always chosen or recommended, and log(n) when n items are chosen or recommended equally often (see [book](https://link.springer.com/10.1007/978-1-4939-7131-2_110158)). Generally, the Shannon Entropy mathematical expression is defined as: $$H=-\sum_{i=1}^{n}p(i)\log_2 p(i)$$
<p>
In RS Metrics the formula is determined as: $$Diversity=-\sum_{i=1}^{services}\left(\frac{count(i)}{recommendations}\right)\log_2 \left(\frac{count(i)}{recommendations}\right)$$
</p>

## Output

Expand Down
6 changes: 5 additions & 1 deletion website/docs/metrics_descriptions/novelty.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@ sidebar_position: 7
The novelty metric expresses the ability of the system to recommend items not generally seen before by the population of users.

## Description
Calculating novelty of the recommender system based on the user actions using the following formula: $$Novelty=\frac{\sum\nolimits_{i \in R}-log(p(i))}{|R|}$$ For each service item $i$ belonging to the set of recommended services $R$ calculate the portion $p(i)$ of the times the service has been viewed to the total views of the services produced by the user actions data.
Calculating novelty of the recommender system based on the user actions using the following formula:
$$
Novelty=\frac{\sum\nolimits_{i \in R}-log(p(i))}{|R|}
$$
For each service item $i$ belonging to the set of recommended services $R$ calculate the portion $p(i)$ of the times the service has been viewed to the total views of the services produced by the user actions data.

## Output

Expand Down
6 changes: 4 additions & 2 deletions website/docs/use_cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ slug: /use_cases
## Monitoring the Recommender System of the EOSC Marketplace/Search Service

<p align='justify'>
The Recommender Metrics Framework currently monitors the Recommender System of the EOSC Marketplace/Search Service by reporting diagnostic metrics and visualizations. The Recommender System (RS) is a component of the EOSC Search Service designed to enhance the user experience by providing recommendations based on a multi-focal perspective of the users. The software calculates statistics, metrics, KPIs, and visualizations to measure the success of the RS that offer deeper insights into the performance of the system. The evaluation process involves quantitatively processing information such as resources, user actions, ratings, and recommendations to measure the impact of the AI-enhanced services and user satisfaction. The feedback obtained from the evaluation is incorporated to improve the services provided through a user-friendly API and dashboard UI.</p>
The Recommender Metrics Framework currently monitors the Recommender System of the EOSC Marketplace/Search Service by reporting diagnostic metrics and visualizations. The Recommender System (RS) is a component of the EOSC Search Service designed to enhance the user experience by providing recommendations based on a multi-focal perspective of the users. The software calculates statistics, metrics, KPIs, and visualizations to measure the success of the RS that offer deeper insights into the performance of the system. The evaluation process involves quantitatively processing information such as resources, user actions, ratings, and recommendations to measure the impact of the AI-enhanced services and user satisfaction. The feedback obtained from the evaluation is incorporated to improve the services provided through a user-friendly API and dashboard UI.
</p>


## Evaluate a third-party RS

<p align='justify'>
The Recommender Metrics Framework due to its design can be used as an analysis tool of the recommendation engine of a Recommender System. Specifically, if the RMF is fed with the necessary input information such as resources, user actions, ratings, and recommendations can carry out the preprocessing stage which is retrieving data from multiple sources through a connector module that claims and transforms the data, establishing connections between service-related knowledge, eliminating dummy or irrelevant data, tagging associations in the data such as registered or anonymous users and services, and generating statistical information. Subsequently, RMF will perform the next phase's steps which are: processing the data, computing the designated evaluation metrics, and uniformly presenting the resulting information. The results such as statistics, metrics, and KPIs will be displayed through a visually appealing UI/dashboard and a REST API, with comprehensive documentation on how metrics are calculated and the expected output range.</p>
The Recommender Metrics Framework due to its design can be used as an analysis tool of the recommendation engine of a Recommender System. Specifically, if the RMF is fed with the necessary input information such as resources, user actions, ratings, and recommendations can carry out the preprocessing stage which is retrieving data from multiple sources through a connector module that claims and transforms the data, establishing connections between service-related knowledge, eliminating dummy or irrelevant data, tagging associations in the data such as registered or anonymous users and services, and generating statistical information. Subsequently, RMF will perform the next phase's steps which are: processing the data, computing the designated evaluation metrics, and uniformly presenting the resulting information. The results such as statistics, metrics, and KPIs will be displayed through a visually appealing UI/dashboard and a REST API, with comprehensive documentation on how metrics are calculated and the expected output range.
</p>
12 changes: 6 additions & 6 deletions website/docusaurus.config.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
// @ts-check
// Note: type annotations allow type checking and IDEs autocompletion

const lightCodeTheme = require('prism-react-renderer/themes/github');
const darkCodeTheme = require('prism-react-renderer/themes/dracula');
const math = require('remark-math');
const katex = require('rehype-katex');
const lightCodeTheme = require('prism-react-renderer').themes.github;
const darkCodeTheme = require('prism-react-renderer').themes.dracula;
import remarkMath from 'remark-math';
import rehypeKatex from 'rehype-katex';

const copyrightMessage = `
<p xmlns:cc="http://creativecommons.org/ns#">
Expand Down Expand Up @@ -51,8 +51,8 @@ const config = {
/** @type {import('@docusaurus/preset-classic').Options} */
({
docs: {
remarkPlugins: [math],
rehypePlugins: [katex],
remarkPlugins: [remarkMath],
rehypePlugins: [rehypeKatex],
sidebarPath: require.resolve('./sidebars.js'),
// Please change this to your repo.
// Remove this to remove the "edit this page" links.
Expand Down
Loading

0 comments on commit 4bd1a6c

Please sign in to comment.