Write more documentation about low-level build system details

[asiekierka]

I found a post by @SArpnt which outlines some unintuitive aspects to the way BlocksDS is put together that could be resolved by documentation. Low priority, but I'll outline the gist as extracted from the post here; these could form the beginning of an FAQ:

• user question: "why are ELF files used for ndstool?"
  • objcopy binary files were used in the past, but this required passing the entrypoint address separately among other issues (what issues? it's been so long ago and I wasn't around for these discussions)
  • with the advent of the DSi, ARM7 and ARM9 binaries were split further into additional ARM7i and ARM9i binaries which contain code exclusive to the DSi (so that it doesn't take up valuable RAM space on the DS); this is resolved by marking these ELF sections out separately
  • debuggers (NO$GBA, gdb via DesMuME/melonDS) rely on DWARF debug information contained within ELF files to provide source-level debugging and breakpoints
  • for the above reasons, inventing a custom format would've required patches to binutils and probably other tools too, but ELF is already a standard for modern embedded development
• user question: "why does BlocksDS require a specific GCC and libc?"
  • a custom GCC is not required; the only patch Wonderful provides that is relevant to BlocksDS is one which builds libgcc and libstdc++ with ARMv4 and ARMv5 optimizations separately, and that's not mandatory (will lead to very minor performance/size drops on the latter)
  • however, a custom libc is required, and a GCC build may depend on a specific libc (especially once libstdc++ is introduced): libnds provides implementations of POSIX C functions, and these may not link correctly with a different libc than the one provided by Wonderful. the source code to this libc is provided here and the build flags are here.
  • add a note that for ease of bug reproduction, it is recommended to report bugs in BlocksDS using the official compiler stack
• user question: "why does BlocksDS require libnds even if I'm not using any functions from it?"
  • the crt0 startup code uses libnds to, among other things, initialize the memory protection unit (MPU) which is required to, among other things, properly map ITCM and DTCM to memory, which is itself required to copy data to those sections
  • it is possible to use BlocksDS without the crt0 by passing -nostartfiles to the linker
  • the libc may also require libnds for some POSIX API calls, such as filesystem I/O, exit(), etc
• user question: "where can I find information on low-level NDS development?"
  • GBATEK, GBATEK addendums, etc.
  • there are no better sources at this time
  • due to many reasons, such as:
    • the ARM9 MPU requiring hardware initialization to configure caches/ITCM/DTCM
    • cross-CPU code being required for half of all hardware access, necessitating a reliable FIFO/IPC interface
    • the touch screen requiring a surprisingly involved driver
    • libnds's license being very permissive
  • etc., it is not recommended to try writing NDS ROMs with no prior art/backing libraries
• user question: "what emulators can I use to debug NDS homebrew?"
  • https://blocksds.github.io/docs/usage/debugging/ covers this
• user question: "why does BlocksDS require passing .specs files to GCC?"
  • The .specs files' role is to provide configuration flags for GCC. Many compiler arguments have been moved here in order to allow changing them with GCC and BlocksDS updates without causing downstream breakage to Makefiles provided by end-user projects.

Some other questions which came up on Discord but shouldn't get stuck on Discord:

• user question: "are there plans for TWL DSP support?"
  • blocked by the lack of a production-quality compiler
  • there is an experimental LLVM port, but it's stuck on an old version of LLVM + requiring extensive LLVM patches to support a CHAR_BIT != 8 environment (chars are 16-bit on the Teak); see https://www.embecosm.com/2017/04/18/non-8-bit-char-support-in-clang-and-llvm/ for more information
  • link to the relevant GitHub issue
• user question: "are there plans for TWL WiFi support?"
  • requires a lot of work, no ETA
  • link to the relevant GitHub issue

[AntonioND]

A few more thoughts:

• A guide on how to build your first binary.
• As mentioned by @asiekierka, a lot of the questions here are good for a FAQ format.
• A short shell/powershell to install BlocksDS with the most common libraries for people that have issues following the installation guide.

[SArpnt]

i was going to try and make a fork and contribute properly, having you find out about it this way feels kinda bad. all the info here is very good, especially for the debugging.

i should probably say now that my actual overarching goal was to make ds homebrew with rust. most things supposed to be used for it didn't work and at that point i was very frustrated trying to pull apart the blocksds makefiles myself.

i do have some folders sitting around with varying levels of features and rust integration at this point, although i've forgotten the steps i took to get there and what's exactly in them. can't get them to build now. the two main things i know i did was build the rust as a static library to put in the rest of the build system, and build a tiny rom without libnds (kept writing to some random pointer that wasn't in the code though?)

i still want to do ds homebrew, i'll probably try to clean these up later. i'm also willing to contribute to wl-toolchain and blocksds if you think it would help (last i remember i think the build setup for the arch packages had some issues with directories? i'd need to refresh my memory on everything before i do anything else)

[AntonioND]

No worries! In general, if you want help and you can't find anything in the documentation, you can ask us directly in the gbadev Discord server (in the #nds and #nds-help channels): https://gbadev.net/ Don't be scared to ask anything there!

I've started a new page to document the build process in particular: #212

[asiekierka]

Note that Discord might not be an option for some users; unfortunately, we do not have the resources and time to maintain our own community or bridges to any other platform.

(Between that and the lack of discoverability of already answered questions on Discord, maybe we should consider enabling GitHub Discussions...)

[AntonioND]

Yeah, that's true, but it's the best option for quick support. There is also an IRC bridge that people can use if Discord isn't an option: https://gbadev.net/chat.html

However, GitHub is better for design discussions because of how it can encourage people to actually think about what they write and it helps being structured with the text. I've enabled it in case it is useful.

[asiekierka]

We might want to document #dsdev in our documentation too, then, as there is a niche of people who continue to refuse to use Discord, especially in niche FOSS-leaning spaces.

[AntonioND]

We might want to document #dsdev in our documentation too, then, as there is a niche of people who continue to refuse to use Discord, especially in niche FOSS-leaning spaces.

I forgot to mention it, I've added a note about this: d8e08b6