💡 We're looking for your creative genius to level up the Penpot challenge.
Share your groundbreaking ideas and let's redefine what's possible here ! 🔥
Due to the limited academic and practical work done on the component and page as SVG generation, we propose the bonus challenge.
Instead of generating long SVG text files for each component, we can utilize controllable image generation to generate the components or page visualization in the existing or target styles.
The system will allow a user to generate an image of the component or the page based on the input style or layout mockup.
| Technology readiness | Risks | Complexity |
|---|---|---|
🟡 Some elements exist but require adaptation |
🔴 Higher than average |
🟡 Moderately complex |
Step #1) Input mockup layout in the SVG format. It can be user-created or text-based generation using with Challenge #1 | Text-to-layout generator
Step #2) Based on the layout and text prompt, generate compatible images and texts via ControlNet [Github].
Step #3) User pick several inspiring options for further SVG re-implementation
- ML model:
- Input: text prompt
- Output: layouts and various complete designs
- Dataset:
[Research]
- Apple’21 proposed “parsing layout into components” [Website]
-
LayoutDM [Github]
-
BoostingGUI :first tries on controllable page-generation [Paper]
-
UI description [Paper]
-
Text-to-image generator:
-
Editing images:
[Business Solutions]
🟢 Pros
- An existing high-quality open-source solution for image generation
- More research-active area
🔴 Cons
- Requires diverse new dataset and additional pre-processing
- Generation quality should be usable enough for the design application