[WIP] Adding tWDA Solver

[IA3005]

No description provided.

[IA3005]

Add new solver : tWDA

[bruAristimunha]

Hey @IA3005 (cc. @tomMoral)

Can you give me a little more description?

[IA3005]

Hi @bruAristimunha ,
Sorry, I forgot to add the description. It is about a new classifier for BCI that is called tWDA.
@tomMoral suggested to me to add it in the benchmark.
Since it is my first full request in github, I'm not sure if there are some problems with the new solver as I had the notification "Some checks were not successful".

[bruAristimunha]

Hey @IA3005 =)

Okay, welcome! I can help you with this process because it is your first PR. Do you prefer a meet or GitHub assistance? By meeting, I can explain more things, and through GitHub, I will give you more general instructions.

I am available in the mattermost (my name), discord (aristimunha), or at the LISN (https://maps.app.goo.gl/XQfj55kihbVeGjRV9), anything that works for you.

[bruAristimunha]

Hey, @matthieutrs! I was wondering if could you please make a review for this PR on the BCI benchmark repository.

I think I would take a long time to review because of neurips and I think Tom is on leave.

[tomMoral]

Hello!

Sorry for the very long delay in reviewing your PR.
I was on leave for a while and it took a while to get back to it.

Here are a few first remarks. Could you share the result of a run on one of the dataset with one other method (for instance CSPLDA)?
using the command benchopt run -d bnci -s twda -s csplda

Note that we did a big clean up with #14, maybe it would be nice to rebase your change on the new version (which removes the augmented solvers which were too complex).

Also, it would be nice to add comments for the different functions that you provide, as it would help reviewing the code.

[IA3005]

Hi! I missed all your messages on GitHub. I'm very sorry for that. I saw that you have changed the solver's structure in the new version of benchmark_bci. I updated my version of the solver 'tWDA'; it is named now "Cov-tWDA" to be similar to the other existing solvers :) I took into account your previous comments.

[tomMoral]

Hello!
Thanks for taking the time to update this PR :)

• Could you fix the linting errors? you can check them locally by running flak8 . in the repo (it can be installed with pip install flake8).
• Could you share the result of one run with a concurrent method on BCI- competition dataset?
  thanks

[IA3005]

Hi! Thank you for reviewing.
I fixed the linting errors. For the results, here is the comparison of my solver tWDA with default parameters with MDM and CSP-LDA on the dataset BNCI-MI with the intra-session paradigm:

and on the simulated dataset with the inter-subject paradigm:

To assess the different settings for my solver, here is the result of the comparison of the default tWDA (wherein the dfs are estimated) vs tWDA with dfs=[10] vs WDA (wherein the dfs=['inf']) on the dataset BNCI-MI with the intra-session paradigm:

and on the simulated dataset with the inter-subject paradigm:

[IA3005]

Hi, The version of flake8 I used for the previous commit was 3.9.0 and thus, it was unable to detect the 2 linting errors indicated in the check. Now, after upgrading my flake8, I fixed all the linting errors.
concerning the failure of the check , it is caused by the library "pymanopt": in fact, it is needed for my solver. In the current version , I wrote in the Solver class requirements = ["pyriemann", "pip:pymanopt"] to ensure that this library is installed. I am not sure if this really did the job. I noticed that if "pymanopt" is already installed on the machine, the code works.

[tomMoral]

I think moving the method defintion in another file should solve the problem.

[IA3005]

I think that my solver is not the cause of the failure of the last check...