Investigate how to split CWA refactoring

[wg102]

This ticket is related to #831

Still feel like even if it's
maybe instead of starting straight from scratch, it might make more sense to take the refactor with our code.
if we maintain python, let's pull in
we pull in this jupyter-apis into our statcan/kubeflow, as is. with the goal of refactoring it to have the same structure of the python but in go.

Need to figure:

Most important is

• how the front-end with back-end and how common connects to it too.
• Look into the pluggability of front end, if they did a separation of concerns in a way to 'plug and play'
• Jupyter-api vs crud-web apps: If we are already recoding, does it make sense to do it at smae time, or cherry pick after
• Folder structure we will use: just add a go folder under backend?
• This will change our deployment of jupyter-apis (since it won't be jupyter-apis anymore)
• Sequence in which we want to do the work
  • This is needed to split efficiently the tasks to multiple people.
• Create all the ticket of all the tasks into this parent.
• Checking that all the validation is still valid. on a previous test, I noticed some of teh validation from jupyter got lost in CWA Jupyter....

Possible things to do:

• List all features coded in jupyter-apis that we need to port over
  • Evaluate how much of our changes are actually portable as is.
  • Have a good idea of what changes we made and why and the spots where it was implemented.
• List of feature in CWA that we want to block/remove/disable
• Decide visually how it will look, possibly include UX

Thing we want to make sure to do

• Co-authoring for features from other people.

Old ticket about 1.2 that might be useful to look into

• Move jupyter-apis (frontend) into StatCan/kubeflow jupyter-web-app kubeflow#44
• [EPIC] Pushing features upstream - Jupyter Web App kubeflow#45

[wg102]

It might be possible to use a tool to start most of the coding between python and golang.
See pythago
Usefull link to learn python to go : https://towardsdatascience.com/from-python-to-go-901cea46446f

We need to consider if we just write the go things that interest us, if we write the full golang code, compared to the original python code, and if we write the full golang code, including the rok sections.

[wg102]

Volumes

Following discussion of 27/01/22

• Some features might need to be removed for the volumes. Such features are things like browsing volumes, downloading files etc.
• Need to choose if we use their volumes, or ours.

An idea could be to have a 'setup' that disables the ui of them + and Authorization Policy that blocks it completelyu.

[wg102]

We might also want to include some tests to make sure the golang code works as expected.

[wg102]

Idea of splitting work:

• MVP (Minimal version, in pair programming)
• EVERYTHING into Go (need to divide up, maybe per folder/file)
• Kubecost
• Prob
• Volumes
• Validation (our customized extra stuff)
• i18n
• Checking the whole thing, including missing validation in their refactor

These items will be added to #831 (comment)

[skyeturriff]

Overview and rationale for proposed strategy:

First task will be an MVP which will be the setting up upstream's CWA folder structure in our kubeflow repo, and porting over the necessary setup/infrastructure code from our jupyter-api repo to just be able to access the Notebooks Server page (no additional functionality). This allows us to confirm a good base to continue development on.

Next task will be dividing up the coding of upstream's CWA into Go. This will give a good base structure for us to then add in our customizations, and also provide a good foundation of the app's new structure and design, which will help us make decisions when adding in our custom components. This also allows for iterative testing at this stage.

Once the base app is in Go, we can then tackle in parallel the addition of our custom components, and the testing of successful integration of each at this point.

[Jose-Matsuda]

Regarding the "EVERYTHING into Go (need to divide up, maybe per folder/file)" bullet we do not have a solid idea for how to divide this just yet as we are all inexperienced here.

The most important folder to tackle first is probably going to be the crud-web-app/common/backend/kubeflow/kubeflow/crud_backend/api as these get used by the files in jupyter/backend/apps.

These api files also resemble (at least in name) the current go files created by Zach in jupyter-apis so that may help in accelerating and easing us more into what / how things are done.

These files can be assigned with a file per person initially where say an 'equivalent' or Zach's work exists jupyter-apis first.

It may also be good to just have a folder named go-backend in places necessary so that we don't completely nuke the existing code, and may make it easier to upstream

[skyeturriff]

Closing as investigation is complete, with strategy going forward planned out as above. First step is the MVP ticket, which has been created and ready for next sprint. Outline for Go tickets has been established, and will be created during next sprint in time for the following sprint when work can begin after the MVP is in place.