Plan to support the design space comparison

[ZwwWayne]

Hi @rajprateek , @ir413 ,
Thanks for your team's great work, it provides many insights to the community. I am sure that the model zoos and the current codebase could inspire future research a lot.

I am also a little bit curious about the future plans of your codebase. So I want to ask that do you have any plans to support the design space comparison in this repo? For example, to allow users to sample & train models from different design spaces and compare these design spaces as described in the Sec. 3.1, as shown in Fig. 5, 7, and 9 in the paper. I think this feature could help the community to reproduce the comparison process and further improve this codebase's impact.

[rajprateek]

Hi @ZwwWayne - Thanks for the suggestion! We do have a plan to add tools to pycls to make it easy to run and analyze population-level experiments, which will help with design space comparisons. We don't have a concrete date yet but this is on our roadmap and we want to make this available to users, so it will happen in the coming weeks/months.

[ZwwWayne]

Hi @rajprateek ,
Awesome!
I am looking forward to these exciting features!

[tuggeluk]

Hey @rajprateek ,
I am planning to do some experiments based on the methodology in "Designing Network Design Spaces". So I just wanted to ask if you guys have an ETA on the features needed to do so.
Thanks, best
Lukas

[rajprateek]

Hi all, we've released tools to study model population stats! Please see https://github.com/facebookresearch/pycls/releases/tag/0.2