Merge pull request #126 from 4ms/add_firmware_docs

Start writing documentation for firmware development

README.md

@@ -1,11 +1,11 @@
-## Meta Module
+# Meta Module
 
 [![Build Simulator](https://github.com/4ms/metamodule/actions/workflows/build_simulator.yml/badge.svg)](https://github.com/4ms/metamodule/actions/workflows/build_simulator.yml)
 [![Build VCV Rack Plugin](https://github.com/4ms/metamodule/actions/workflows/build_vcv_plugin.yml/badge.svg)](https://github.com/4ms/metamodule/actions/workflows/build_vcv_plugin.yml)
 [![Run VCV unit tests](https://github.com/4ms/metamodule/actions/workflows/run_vcv_tests.yml/badge.svg)](https://github.com/4ms/metamodule/actions/workflows/run_vcv_tests.yml)
 [![Build Firmware](https://github.com/4ms/metamodule/actions/workflows/build_test_firmware.yml/badge.svg)](https://github.com/4ms/metamodule/actions/workflows/build_test_firmware.yml)
 
-### Start
+## Start
 
 First, clone this repo and `cd` into the new directory.
 
@@ -28,21 +28,23 @@ git submodule update --init --recursive
 
 Next, setup your development environment by [following the instructions on this page](./docs/Setup.md).
 
-### Next Steps
+## Next Steps
 
 The Meta Module environment is built using three separate components: The VCV Rack Plugin (which includes the Meta Module patch exporter module), the Firmware for the Meta Module hardware, and the Firmware Simulator that allows you to run the firmware locally to test changes.
 
-To build these components, please follow the separate build guides:
+For information about building and using these components, please follow the separate guides:
+
 - [VCV Rack Plugin](./vcv/README.md)
-- [Simulator](./simulator/Building.md)
+- [Simulator](./simulator/README.md)
 - [Firmware](./firmware/README.md)
 
-### Usage
+## Usage
 
 - [Creating Meta Module Patches With VCV](./docs/BasicVCVPatching.md)
+- [Updating Firmware](./docs/user-firmware-update.md)
 
 
-### Contributing
+## Contributing
 
 If you would like to port your own VCV modules to the Meta Module platform, please see the [Porting Guide](./docs/Porting.md).
 

docs/Firmware-Boot.md

@@ -0,0 +1,47 @@
+# Firmware Boot Procedure
+
+The chip's ROM code reads the `BOOT0_2` DIP switch and then:
+
+- either halts (if it's set to `Debug`)
+- or attempts to load a bootloader from the SD Card (if it's set to `SD`)
+- or attempts to load a bootloader from the QSPI NOR Flash chip (if it's set to
+  `Flash`)
+
+and then starts executing the bootloader.
+
+The bootloader (which is `mp1-boot`) initializes the DDR RAM and various system
+peripherals (security, MMU, clocks). After that it halts if Pin 2 and 4 of the
+"Control Expander" (`J102`) are shortened with a jumper.
+
+Otherwise (no jumper detected), it will check if the rotary encoder is pressed.
+If so, it loads an alternative firmware image from whatever media it booted
+from (SD or Flash). On the Flash chip, USB-DFU "second-stage bootloader" (SSBL)
+is preloaded. The SD card could also have an alternative firmware but this is
+not used yet.
+
+Otherwise (no jumper and rotary not pressed) it loads the firmware image from the
+default location on the selected boot media and then starts execution.
+
+### Secondary A7 core startup
+
+Both A7 cores startup at the same time, but Core 2 (aka secondary core) will go to 
+sleep immediately. The first core (primary, aka Core 1)
+will proceed with the procedure descrived above.
+
+The MetaModule firmware awakens the secondary core and tells it to start executing from
+the `aux_core_start` assembly code in 'firmware/lib/mdrivlib/target/stm32mp1_ca7/boot/startup_ca7.s`
+which does some basic stack setup and then jumps to
+the `aux_core_main()` function in `firmware/src/core_a7/aux_core_main.cc`. Here, the
+secondary core handles the screen updates and runs the LVGL tasks. It also gets interrupted 
+by the primary core to help with audio processing.
+
+
+### M4 startup
+
+The M4 firmware is bundled inside the A7 firmware. After power-on there is no
+M4 firmware running (there is no internal flash to store the M4 firmware), so
+the M4 core is just sitting idle. The A7 core does some early init and then
+loads the M4 firmware and starts that core.
+
+The M4 firmware is responsible for reading all controls and the gate jacks. It also
+handles all USB and SD Card transactions.

docs/Porting.md

@@ -1,9 +1,92 @@
-If you would like to port your VCV-compatible modules to the Meta Module platform, you will need to follow a few steps for exporting artwork and ensuring binary compatibility.
+### Instructions for adding a new module (WIP)
 
-### Converting assets
+1) First step is add the module code as a git submodule:
 
-(TODO)
+```bash
+git submodule https://github.com/<user>/<repo> firmware/vcv_ports/<Brand>
+```
+
+2) Create artwork folders:
+
+```bash
+mkdir -p firmware/src/gui/images/<Brand>/modules/
+mkdir -p firmware/src/gui/images/<Brand>/components/
+```
+
+3) Create a modules.cmake file: `firmware/vcv_ports/glue/<Brand>/modules.cmake` 
+Look at other brands examples and copy what's done there.
+
+`modules.cmake` must do a few things:
+
+- Create a list of module names call `<Brand>Modules`.
+  These names should be the VCV Rack module slugs. Ideally, these names are
+  also the names of the SVG faceplate files (without the .svg extension) and
+  also the names of the C++ source files (without the .cpp or .cc extension),
+  but if that's not the case you will just need to manually specify the SVGs
+  and source file names manually.
+
+- Invoke the module list filtering function on the list, like this:
+    ```cmake
+    include(${CMAKE_CURRENT_LIST_DIR}/../filter.cmake)
+    limit_modules_built("<Brand>" <Brand>Modules)
+    ```
+    This allows developers to speed up build times and reduce binary sizes by specifying a white-list of
+    modules to include in the build.
+
+- Set a list called `<BRAND>_FACEPLATE_SVGS` (<BRAND> is all caps), which
+  contains full paths to the faceplate .svg files. This is used when
+  (re-)generating all artwork files (convering SVG files to PNG and LVGL
+  format).
+
+
+4) Create a brand CMakeLists.txt file:  `firmware/vcv_ports/glue/<Brand>/CMakeLists.txt`
+This file tells CMake how to build your modules.
+
+- Typically at the top of the CMakeLists file, you will `include()` the
+  `modules.cmake` file, and use the list of module names to generate the
+  list of source files needed to compile, but that's not a strict requirement.
+
+- The CMakeLists.txt must create a CMake target OBJECT library named <Brand>Library, like this:
+  ```cmake
+  add_library(
+    <Brand>Library OBJECT
+    ${BRAND_SOURCE_PATHS}
+    ${OTHER_SOURCES}
+    more_source_files.cpp
+  )
+  ```
+  You can then use normal CMake commands like `target_include_directories`,
+  `target_compile_options`, etc to specify how your modules should be built.
+
+
+5) Add the brand name to `firmware/vcv_ports/brands.cmake`
+
+```cmake
+# List of brands
+set(brands
+  ...
+  <Brand>
+)
+```
+
+
+5) Add the plugin to the VCV whitelist (see `vcv/src/mapping/module_directory.hh`).
+
+6) Generate faceplate artwork. 
+   You can now run `make faceplate-image` to generate the faceplate artwork. This will 
+   also re-generate all other brands artwork, so it's recommend that you a
+   limit file (see [Firmware README.md](../firmware/README.md) ) to just
+   specify your brands' modules, or perhaps even a subset of them just to
+   start. This works by reading the cmake list you created `BRAND_FACEPLATE_SVGS`
+
+7) Generate component artwork.
+   Put all SVG files for components into a folder (from the repo root) `graphics/<Brand>/components/`.
+   Then run `make comp-images` to generate the component images. This will regenerate all brand's component images.
+   (TODO: allow limiting this). Please only commit actually changed images.
+
+8) Connect VCV Widgets to MetaModule Elements and LVGL images.
+(TODO). For an example, see [this commit](https://github.com/4ms/metamodule/pull/135/commits/3d5a721e7c9beea818e58401e82ba4faf5d52321)
+Also, see issue #47
+
 
-To create the artwork files from the SVGs, you must have Inkscape installed an on your PATH
 
-...To Be Continued...

docs/firmware-building.md

@@ -0,0 +1,166 @@
+## Building Firmware
+
+This requires `arm-none-eabi-gcc` version 12.2 or later installed on your PATH.
+Please see the [setup guide](../docs/Setup.md) for some important notes about this.
+
+Make sure you are in the right branch and you already updated the submodules.
+
+To prepare the build system: (only needs to be run once)
+
+```
+make configure
+```
+
+To compile, run:
+
+```
+make all
+```
+
+The `make configure` command is a shortcut for running:
+
+```
+# MacOS, Linux:
+cmake --fresh --preset base -GNinja
+
+# MinGW:
+cmake --fresh --preset base -G"Unix Makefiles"
+```
+
+(The work-around for MinGW is documented with [issue #78](https://github.com/4ms/metamodule/issues/78))
+
+The `make all` command is a shortcut for running:
+
+```
+cmake --build --preset base
+```
+
+### Limiting the modules built
+
+You also can limit the modules built to substantially reduce the compilation
+and link times, as well as binary size. Create a text file with the modules
+you want built, one per line. Each line should contain an
+entry in the form `Brand:Module`. For example:
+
+```
+echo "4ms:EnOsc" >> quickbuild.txt
+echo "Befaco:EvenVCO" >> quickbuild.txt
+echo "hetrickcv:PhasorGen" >> quickbuild.txt
+
+make limit quickbuild.txt
+```
+
+This would tell CMake to re-configure the project and just build those three modules.
+You can still open patches containing other modules, but their artwork won't be shown
+and you can't play them.
+
+
+### Using an SD Card
+
+*Optional*: If you plan to flash firmware to an SD Card, then you can specify the
+path the SD Card device to save time. If you don't do this, then the system
+will prompt you whenever you run one of the SD Card flashing scripts. The
+device path should be to the entire SD Card device (not just one partition).
+```
+cmake --preset base -DSD_DISK_DEV=/dev/disk4
+
+# Alternatively, set an environment variable:
+export SD_DISK_DEV=/dev/disk4
+```
+
+The firmware is built as `firmware/build/mp1corea7/medium/main.elf` and `main.uimg` 
+in the same directory. The .elf file is used when debugging, and the .uimg file
+is used when copying firmware to NOR Flash or an SD card.
+
+
+### Specifying the toolchain
+
+*Optional*: if you have multiple versions of the gcc arm toolchain installed and don't want to 
+change your PATH for this project, you can set the METAMODULE_ARM_NONE_EABI_PATH var like this:
+
+```
+# Put in your bashrc/zshrc for convenience:
+# Note the trailing slash (required)
+export METAMODULE_ARM_NONE_EABI_PATH=/path/to/arm-gnu-toolchain-12.x-relX/bin/
+```
+
+
+### Automatically generated materials
+
+Several files are automatically generated using python scripts, e.g. faceplate
+LVGL code. These generated files are already committed for a few reasons: 1)
+the conversion process uses some specific external programs (inkscape CLI, and
+a particular version of node); 2) generating all the assets takes a long time;
+3) the assets don't change very often (if ever) and are completely orthogonal
+to the code. Also conversion from SVG to PNG can generate a file that is
+visually the same but has a different binary representation, causing lots of
+noise in the git diffs. However if you wish to (re)-generate these files, the
+following commands can be run:
+
+```
+# Generating LVGL image files for components
+make comp-images
+
+# Generating LVGL image files for faceplates
+make faceplate-images
+
+# Updating/creating 4ms VCV artwork SVGs files from *_info.svg files
+make vcv-images
+
+# Updating/creating CoreModule *_info.hh files from *_info.svg
+make module-infos
+
+# All of the above
+make regenerate-all
+```
+
+If you just want to re-generate one image (that is, convert one SVG to a PNG and LVGL format), then you can invoke the python script directly.
+
+Here are some examples. These commands can be run from anywhere, but for these examples we'll show it from the firmware dir:
+
+```
+cd firmware/
+```
+
+
+Make sure the output directories exist first:
+
+```
+mkdir -p src/gui/images/BRANDNAME/components/
+mkdir -p src/gui/images/BRANDNAME/modules/
+```
+
+Convert a component SVG:
+(The DPI will be automatically detected and the output will be rescaled properly)
+
+```
+../shared/svgextract/svgextract.py convertSvgToLvgl \
+                                   path/to/newcomponent.svg  \
+                                   src/gui/images/BRANDNAME/components/newcomponent.c
+```
+
+Convert a faceplate SVG:
+This is the same as converting a component SVG, except for Alpha blending is disabled, and the height is fixed at 240px.
+
+```
+../shared/svgextract/svgextract.py createLvglFaceplate \
+                                   path/to/newfaceplate.svg \
+                                   src/gui/images/BRANDNAME/modules/newfaceplate.c
+```
+
+Optionally, you can only export one layer from a faceplate SVG file by specifying the layer name as a 3rd argument:
+
+```
+../shared/svgextract/svgextract.py createLvglFaceplate \
+                                   path/to/newfaceplate.svg \
+                                   src/gui/images/BRANDNAME/modules/newfaceplate.c \
+                                   artworklayer
+```
+
+The svgextract script can do more, to see its help and view available commands:
+
+```
+../shared/svgextract/svgextract.py 
+```
+
+