bigPULP

HERO is our open-source, FPGA-based, heterogeneous embedded SoC research platform that combines a fully-modifiable RISC-V manycore accelerator with an ARM Cortex-A host processor.

HERO consists of many different hard- and software components. This repository contains the bigPULP hardware platform, i.e., the sources and build scripts to generate the FPGA bitstream implementing the RISC-V manycore accelerator of HERO.

Being the big brother of the open-source, multicore, Parallel Ultra-Low Power (PULP) computing platform jointly developed by ETH Zurich and the University of Bologna, bigPULP is based on the same cluster architecture and sources. Depending on the target FPGA, bigPULP uses one or multiple PULP clusters that share an L2 instruction and data memory, a global interconnect, synchronization infrastructure, as well as the Remap Address Block (RAB) - a software-managed I/O memory management unit - which allows accelerator to coherently access the platform's main memory including support for shared virtual memory (SVM).

For further information about HERO, refer to our HERO paper at https://arxiv.org/abs/1712.06497 and the HERO website https://www.pulp-platform.org/hero/.

Detailed HOWTOs on how to build and use the HERO platform can be found here:

https://pulp-platform.org/hero/doc/

For more information about the PULP project in general, please refer to the official PULP platform website:

https://www.pulp-platform.org/

Getting Started

Preparation

Before being able to build a bitstream or simulate the FPGA design, get the latest version of the IP cores used inside bigPULP:

./update-ips

This will download all the required IPs, solve dependencies and generate the scripts by calling ./generate-scripts.

Then, enter the fpga directory and adjust the sourceme.sh script. Select the target board and adjust the path to the Vivado-specific simulation libraries.

Source the sourceme.sh script:

. sourceme.sh

Generate all the Xilinx IP cores used inside bigPULP:

make ips

Generating the FPGA bitstream

After having downloaded the PULP IP cores, having set up and sourced the sourceme.sh script and having generated the Xilinx IP cores as shown in the preparation section, you can start generating an FPGA bitstream. To this end, enter the fpga directory and execute

make synth-pulp_cluster

to start Xilinx Vivado and synthesize the cluster netlist. Afterwards, run

make synth-pulp_soc

to start Xilinx Vivado and synthesize the SoC containing possibly multiple clusters as well as SoC-level IP cores.

Finally, the top-level design containing the bigPULP SoC and the interfaces to the host can be generated and an FPGA bitstream can be generated. To this end, enter the folder bigpulp-z-70xx and run

make clean gui

to start Xilinx Vivado, synthesize the top-level netlist and generate the FPGA bitstream.

NOTE: When targeting other platforms such as the Xilinx Zynq UltraScale+ MPSoC or the Juno ARM Development Platform, enter the corresponding directory, i.e., bigpulp-zux or bigpulp, respectively.

How to create a bootable HERO system image using the generated FPGA bitstream is is shown in a detailed HOWTO at: https://pulp-platform.org/hero/doc/software

Simulating the FPGA design

To debug the bigPULP platform, this repository provides a set of simulation scripts and testbenches.

NOTE: The simulation platform only models the bigPULP subsystem. The host is not part of the simulation. Instead, the host is modeled using an AXI master plug driven by the testbench, and the shared main memory is modeled using Xilinx BRAM IP cores.

After having downloaded the PULP IP cores, having set up the sourceme.sh script and having generated the Xilinx IP cores as shown in the preparation section, you need to download the free AMBA4 AXI-Lite Verification IP from SysWip: http://syswip.com/axi4-lite-verification-ip and extract the archive.

Then, open the sourceme.sh script and adjust the AXI4LITE_VIP_PATH variable to point to where you just extracted the archive.

Source the sourceme.sh script:

. sourceme.sh

Next, copy the .slm files, i.e., memory initialization files for the application of interest, to the folder sim-bigpulp-z-70xx/tb/current/slm_files. These files are generated by the PULP SDK when compiling an application. They are typically found inside the build directory build/system-bigpulp*/.

Finally, enter the folder sim-bigpulp-z-70xx/vivado and run

make clean gui

to start Xilinx Vivado, compile the entire design and start the RTL simulation.

NOTE: When targeting other platforms such as the Xilinx Zynq UltraScale+ MPSoC or the Juno ARM Development Platform, enter the corresponding directory, i.e., sim-bigpulp-zux/vivado or sim-bigpulp/vivado, respectively.

bigPULP repository structure

After being fully setup as explained in the Getting Started section, this root repository is structured as follows:

• fe contains the front-end RTL code of bigPULP.

• fe/rtl contains the main platform RTL code including packages a include files.

• fe/ips contains all IPs downloaded by update-ips script.

• fpga contains all FPGA-specific files to build and simulate the RTL code including:

  • fpga/rtl: FPGA-specific RTL code.
  • fpga/ips: Vivado build scripts to generate the Xilinx IP cores instantiated in the design.
  • fpga/pulp_cluster: Vivado build scripts to generate the netlist of the PULP cluster IP instantiated in the design.
  • fpga/pulp_soc: Vivado build scripts to generate the netlist of the bigPULP SoC containing one or multiple PULP clusters as well as SoC-level IPs such as the RAB.
  • fpga/bigpulp-z-70xx: Vivado build scripts and top-level RTL files to generate the bigPULP bitstream and Xilinx SDK files for HERO based on Xilinx Zynq-7000 SoCs.
  • fpga/sim-bigpulp-z-70xx: Vivado scripts and testbenches to simulate bigPULP when targeting Xilinx Zynq-7000 SoCs.
  • fpga/bigpulp-zux: Vivado build scripts and top-level RTL files to generate the bigPULP bitstream, Xilinx SDK and PetaLinux input files for HERO based on Xilinx Zynq UltraScale+ MPSoCs.
  • fpga/sim-bigpulp-zux: Vivado scripts and testbenches to simulate bigPULP when targeting Xilinx Zynq UltraScale+ MPSoCs.
  • fpga/bigpulp: Vivado build scripts and support files to generate the multicluster bigPULP bitstream for HERO based on the ARM Juno Development Platform.
  • fpga/sim-bigpulp: Vivado scripts and testbenches to simulate bigPULP when targeting the ARM Juno Development Platform.

• ipstools contains the utilities to download and manage the IPs and their dependencies.

• ips_list.yml contains the list of IPs required directly by the platform. Notice that each of them could in turn depend on other IPs, so you will typically find many more IPs in the ips directory than are listed in this file.

• rtl_list.yml contains the list of places where local RTL sources are found (e.g. fe/rtl/components).

Requirements

The RTL platform has the following requirements:

• Relatively recent Linux-based operating system; we tested Ubuntu 16.04 and CentOS 7.
• Xilinx Vivado Design Suite version 2017.2.
• ModelSim in reasonably recent version (we tested it with version 10.6b).
• Python 3.4, with the pyyaml module installed (you can get that with pip3 install pyyaml).

Repository organization

The PULP platforms are highly hierarchical and the Git repositories for the various IPs follow the hierarchy structure to keep maximum flexibility. Most of the complexity of the IP updating system are hidden behind the update-ips and generate-scripts Python scripts; however, a few details are important to know:

• Do not assume that the master branch of an arbitrary IP is stable; many internal IPs could include unstable changes at a certain point of their history. Conversely, in top-level platforms (pulpissimo, pulp, bigPULP) we always use stable versions of the IPs. Therefore, you should be able to use the master branch of bigPULP safely.
• By default, the IPs will be collected from GitHub using HTTPS. This makes it possible for everyone to clone them without first uploading an SSH key to GitHub. However, for development it is often easier to use SSH instead, particularly if you want to push changes back. To enable this, just replace https://github.com with [email protected] in the ipstools_cfg.py configuration file in the root of this repository.

The tools used to collect IPs and create scripts for simulation have many features that are not necessarily intended for the end user, but can be useful for developers; if you want more information, e.g. to integrate your own repository into the flow, you can find documentation at https://github.com/pulp-platform/IPApproX/blob/master/README.md

External contributions

The supported way to provide external contributions is by forking one of our repositories, applying your patch and submitting a pull request where you describe your changes in detail, along with motivations. The pull request will be evaluated and checked with our regression test suite for possible integration. If you want to replace our version of an IP with your GitHub fork, just add group: YOUR_GITHUB_NAMESPACE to its entry in ips_list.yml or ips/pulp_soc/ips_list.yml. While we are quite relaxed in terms of coding style, please try to follow these recommendations: https://github.com/pulp-platform/ariane/blob/master/CONTRIBUTING.md

Known issues

Currently, the bigPULP platform cannot be built for the Juno ARM Development Platform from the sources in this repository since part of the required IP cores and top-level wrappers from ARM are not freely available. For internal development, download the juno-support repository from the internal GitLab server and configure JUNO_SUPPORT_PATH in sourceme.sh accordingly.

Support & Questions

For support on any issue related to this platform or any of the IPs, please add an issue to our tracker on https://github.com/pulp-platform/bigPULP/issues