soc: xlnx: zynqmp: overhaul the Cortex-R MPU setup

[ibirnbaum]

To this day, the MPU region definitions provided for the ZynqMP/UltraScale+'s Cortex-R cores, which are pretty old code by now, cause various problems on actual hardware, which qemu_cortex_r5 tends to ignore:

• The RAM region is not associated with the device tree at all. The RAM base address is always hardcoded to 0, which obviously causes problems if the RAM base address isn't at that location according to the device tree.
• The RAM region's size is fixed, also not associated with the device tree.
• The RAM region's definition, if starting at address 0, doesn't just contain RAM, but also includes both the ATCM (with the exception vectors right at the start) and BTCM memory areas, whose availability isn't universally guaranteed, and which may have a 'black hole' between them depending on the operational mode of the R-Core cluster (with QEMU generously ignoring this). There are no facilities to explicitly locate data in the TCMs and use them in the intended fashion according to the technical reference manual. There's also no way to control which part of section .text ends up in the TCMs and from what point of the text section on, regular RAM is used. Half of some random function may end up at the very upper end of the BTCM, and the other half continues on from the start of RAM behind the BTCM.

For the first two items, the macros CONFIG_SRAM_BASE_ADDRESS and REGION_SRAM_SIZE exist, which haven't been referenced prior.

Also, XIP wasn't considered and local macro definitions for region properties were used although global ones are already provided.

Therefore, the proposed overhaul changes the following:

• Leave the sram0 declaration at base address 0x0 for all in-tree boards which already use it, but move the declaration over to the boards' device trees. If it works for them and the inclusion of ATCM and BTCM isn't an issue there, great.
• Remove the universal sram0 declaration from the SoC level, as having different offset addresses (beyond the TCMs) and sizes now actually becomes an option, enabling effective memory map layout specification for memory resource management / exclusion when operating in parallel with the A-cores. Specify sram0 at the board level as the ZynqMP doesn't come with integrated, always fixed-size RAM like the common microcontrollers, but with flexible RAM sizes instead, depending on the RAM chip attached on the target hardware.
• Reference the region property macros provided in arm_mpu_v7m.h instead of using local declarations
• Add a (potentially overlapping) region declaration which implements R/O for .text and .rodata
• Handle R/O and XN properties for the flash0 region depending on whether CONFIG_XIP is set or not.
• Modify the linker command script so that the region size bit mask for the ROM region is computed correctly.
• and as a minor extra for qemu_cortex_r5: so far, this target has only been able to provide 32MB of RAM with a non-zero offset address (must be a multiple of the region size, so (0x0 + 32MB)) due to only specifying a total ATCM + BTCM + RAM size of 64 MB starting at 0x0 in the device tree parsed by QEMU, so having 64MB @ (0x0 + 64MB) didn't work. Therefore, increase the RAM size in the QEMU device tree for qemu_cortex_r5.

I'm well aware that tying the RAM region size to the device tree directly contradicts #61008 which dates back to last year. However, I do believe that this was never a good idea to start with: generally opening up 2 GB of RAM address space in an SoC where the R-cores and the A-cores share the RAM just so that openamp SHMs could be effortlessly randomly placed at an address that may or may not be beyond the range actually used by the Zephyr image seems extremely risky when considering the possibility of parallel Linux operation on the A-cores. Also, this is a feature that is not generally used by everyone developing for the ZynqMP R-cores. Memory ranges for such specific use cases can be declared on a need-to-have basis in the respective target's device tree or in application-specifc overlays, the MPU setup will consider such additional region declarations automatically via mpu_configure_regions_from_dt(). If there's any board definitions or samples in-tree that make use of this feature on the ZynqMP, I can have a look on how to set up the SHM memory regions properly and add the fixes to this branch.

[michalsimek]

This is not upstream QEMU but AMD/Xilinx qemu fork. Is there any warning about it?
From my perspective having dtb as blob in source code is not a good idea. Isn't there a better way to put there dts and dtc will convert it as the part or build process.
Another reason is that it is hard to review what has been changed from between one dtb and another dtb.

[ithinuel]

You can refer to https://docs.zephyrproject.org/latest/contribute/bin_blobs.html for the policy around binary blobs in Zephyr.

[michalsimek]

You can refer to https://docs.zephyrproject.org/latest/contribute/bin_blobs.html for the policy around binary blobs in Zephyr.

Thanks.
"The Zephyr Project does not host binary blobs in its Git repositories or anywhere else."
In this case it is just DT file and source code can be seen but I know it doesn't look nice.

[ibirnbaum]

Should there be a warning regarding the fact that Xilinx's fork of QEMU is used? If there were any functional issues with this version, this would become obvious with pretty much every PR as the QEMU targets are tested as part of the CI. Admittedly, there seem to be a few hiccups in there (aside from the afforementioned not caring about the black hole between the ATCM and the BTCM that exists on actual hardware, for example, there's just nothing in the clock configuration registers within the Zynq-7000's SLCR), but I'd only see a benefit in switching over if those issues were proven fixed in mainline QEMU. As this version is, along with other QEMU flavors, shipped with the SDK, it's not a case of having accidentially downloaded the wrong flavor of QEMU.

Agreed, the case of QEMU only working with DTB's is inconvenient, as you can't easily diff. So adding the textual description that the RAM size was changed was the best that could be done here. However, I wouldn't see the DTB as a blob as in a firmware blob, for example. Your suggestion of having the device tree for QEMU in DTS format and converting it during the build process is rather something that I'd see in the area of the build framework / west integration, as QEMU is integrated all the way into (at least) west's run and debugserver commands. The link is likely board.cmake. I'd suggest writing an Enhancement Suggestion to the folks in charge of the build system.

[michalsimek]

I am new to this project and things around it but descriptions for AMD/Xilinx Qemu has never been pushed to my upstream projects. It is only kept in downstream repositories.
The reason was that maintainers wanted to use upstream Qemu only. That's the reason why I am pushing Microblaze V to QEMU source code too even we have internal QEMU working based on DT.
Is there any support for other emulators inside Zephyr? Because if yes, AMD Qemu could be consider as another emulator or it's flavor and integration could be done in a similar way.

[stephanosio]

QEMU source code too even we have internal QEMU working based on DT.
Is there any support for other emulators inside Zephyr? Because if yes, AMD Qemu could be consider as another emulator or it's flavor and integration could be done in a similar way.

We already have Xilinx QEMU in the Zephyr SDK and it is currently being used to test ZynqMP platforms in the upstream CI.