[WIP] Data driven toolchain wrappers #2225
Conversation
probot-autolabeler
bot
added
lang-python
|
@mkurc-ant - I kind of generally like the idea but can you chat with @rodrigomelo9 about his attempt at doing something in their vein at https://github.com/PyFPGA/symbiflow_cli ? It is also worth thinking about how this compares / overlaps with edalize.
|
|
@olofk - I would like your thoughts here... |
|
@umarcor - Do you have any thoughts too? |
|
Hi @mkurc-ant the idea of symbiflow_cli is to replace So... I think that we are working on almost the same. As I said, it started as part of PyFPGA. There, I provide an unified API to use different EDA tools (ISE, Vivado, Quartus, Libero-SoC). I didn't added symbiflow (it is not true at all, there is support in the branch https://github.com/PyFPGA/pyfpga/tree/symbiflow-support) because it does not support ice40 devices. I believe that symbiflow must provide a tool to "drive" the flow. Then, tools such as edalize, PyFPGA and other, can use the simbiflow_cli API. Currently, it is being performed using the toolchain_wrappers, which are different for each platform, highly dependant on the OS, support for ice40 and ecp5 was not yet added, etc. You can found a discussion about the current symbiflow_cli at PyFPGA/symbiflow_cli#5 (WIP: I am modyfing/symplifying some CLI arguments based on Tim's suggestions).
|
|
I have symbiflow_cli at https://github.com/PyFPGA/symbiflow_cli because it is in an early stage. But the idea is, if the Symbiflow project is interested, donate (and move) it to the SymbiFlow organization. I think that is a good idea to have it in a separate repo, so if you agree, maybe we can define how it must be there, work there together, and have something better in less time :-D Let me clarify why I think that we need to solve the symbiflow's "flow" and then, tools such as edalize could use it. We can define today how the API looks like, which are the arguments of the CLI, and then, we can perform the changes that we want to support new platforms and tools. We can modify intermediate formats, add utilities for certain situations, etc, without changes at edalize. It will be complex, there will be a lot of tools combined in several ways here. I think that is a good idea to have a buffer, and do not depend on another layer between the low-level tools and the symbiflow's "flow".
|
|
@mithro When it comes to edealize I imagine it just using the wrapper scripts (in whatever form they'll have) as it does with all toolchains it supports. @rodrigomelo9 Thank you for shedding some light at your project. I have to take a more detailed look into it. I'll get back to you with some thoughts. |
|
Hi @rodrigomelo9. I've seen your implementation of symbiflow_cli - It is very promising. You've taken a wider approach and supported the flows outside symbiflow-arch-defs whereas I mostly focused on the latter. My goal is more towards providing a replacement for current toolchain wrapper scripts in symbiflow-arch-defs as the current ones have a conflict between Xc7 and QuickLogic archs hence those two cannot be used simultaneously. The set of toolchain wrappers I propose is not entirely data driven - it turned out to be more "plugin" driven. For each distinct architecture there is an individual set of python scripts responsible for invoking all the tools like Yosys and VPR plus other helper scripts. I propose that instead of integrating symbiflow_cli and symbiflow-arch-defs wrapper scrpts we make the former invoke the latter. By doing this we can move out all the intricacies necessary for symbiflow-arch-defs out of symbiflow_cli thus making it cleaner. This will also speed up development process as the only thing that we will have to agree on is the commandline parameters or a kind of a Python API. What are your thoughts on that? |
|
@mkurc-ant My first thought is... I don't know. Maybe, it depends? I saw the Files changed Tab at this PR. All the work was done under The synthesis of symbiflow_cli supports some features which are not solved here: VHDL support through GHDL and ghdl-yosys-plugin, specify an architecture (VHDL), specify defines (Verilog), specify generics/parameters, and Verilog include directories (I am not sure if none of them are currently supported by SymbiFlow). The alternatives are to move the functionality here or to create an alternative synthesis wrapper at symbiflow_cli. What about other platforms? (I saw some files around for iCE40 and ecp5 but not toolchain_wrappers). What about another backend? (nextpnr?). I am asking because you called them the flows outside symbiflow-arch-defs. On the other hand, symbiflow_cli allows using fine-grain containers from the hdl/containers project, so you only need to install docker/podman and the provided Python package. Moreover, you will download only what you need (if you never use VHDL or iCE40 devices, you will no download GHDL, nextpnr-ice40 or the icestorm tools). To avoid generating new containers (tool + scripts from this project) the engine must be supported in the invocation of the underlaying tool (yosys, vpr, etc). The alternative for that is to support containers here, create a lot of new containers with scripts from this project installed (is not the idea), or a new development (also, is not the idea). I think that we need to answer these questions (there will be several toolchain_wrappers? support for VHDL and specify arch/defines/parameters/includes? ice40/ecp5/netpnr? Containers support?) to detect if it has sense to work in the same project or in a producer/consume approach. What I am sure is that I wan't to provide a SymbiFlow alternative. I want to help with the SymbiFlow project. Of course, you know better than me what are the needs of the project, and you have the decision on how it is solved. |
|
Yes, this is WIP and the scripts are located temporarly under xc. But the plan is to add QuickLogic flows as soon as xc7 is ready. Then the scripts will be moved somewhere else. Your way of supporting VHDL via GHDL is indeed a nice to have feature. I'm not sure how to proceed with that as in arch-defs we require pretty customized synthesis flows and techmapping. We could think of a solution where eg. symbiflow_cli does the VHDL stuff, invokes As for the verilog include directories and other extended synthesis options - it is currently implemented for QuickLogic flow in the current wrappers. Basically there is a Python script which reads those command line switches and outputs a set of Yosys commands responsible for eg. loading cell libraries, setting include directories etc. See https://github.com/SymbiFlow/symbiflow-arch-defs/blob/master/quicklogic/common/utils/convert_compile_opts.py I haven't considered using any docker-like containers, for now in arch-defs it is assumed that the necessary tools are available in the system or installed via Conda. I'd need to check how we handle rr graph distribution (they are big). We should have separate conda packages for each device / device family. Certainly the wrapper scripts in arch-defs will not target nextpnr as backend as the whole arch-defs repo is devoted to architectures for VPR. This implies the need for additional techmaps for Yosys and utility scripts that are invoked in between VPR pack, place and route passes. For those reasons I would like those wrappers to support only the architectures supported in arch-defs. Since your symbiflow_cli aims at using both nextpnr and VPR I'd stick for now with symbiflow_cli invoking arch-defs wrappers whenever a device supported by arch-defs is targetted.
|
|
Hi @mkurc-ant first of all, thanks for providing insights into the project that I had not clear (basically, this repo is to be used with VPR). Yes, I saw that the Yosys synthesis is much more complex with VPR than with nextpnr (no worries, I know why).
A possible solution is to install the scripts at this repo in the containers provided with the hdl/containers project (which will be containers suited to handle things such as the rr graph). To know more, can I ask where are the rr graph used? I guess VPR? (remember, I have experience with ice40 and ecp5 flows, based on nextpnr).
Yes, I agree that seems a good idea, at least at the moment. Of course, I am always around :P so feel free to mention me if you think that I can help. I have not idea about reverse engineering an FPGA, but I know VHDL/Verilog, Python/Bash/Tcl, and I have experience with devices and tools from Xilinx, Intel, Microsemi and Lattice |
|
I just found this document from a long time ago --> https://docs.google.com/document/d/1lNYBP3OVc76WCDI--irPfANDvPdjZwkOuxVt_3PN31o/edit#heading=h.1d2m14ywud9p
|
|
You shared with me this document time ago... Maybe from here my "confusion"? about to support also nextpnr :-D |
|
I found the document when looking at other things, so thought I would link it from this thread so everyone could see it. |
… in in project flow
mkurc-ant
force-pushed
the
better_flow_wrappers
branch
from
517b526 to
d9a9185
Compare
|
The current symbiflow backend in Edalize uses the symbiflow_synth, _pack, place, etc entry points. This is not really optimal and is something I have been hoping to find time to fix. The problem is that the symbiflow* wrappers hide a lot of functionality from the underlying tools. Most notably when it comes to yosys where (at least last time I checked) you couldn't inject custom tcl scripts or set toplevel parameters. I think it would make more sense to have an Edalize backend that calls the EDA tools (yosys, VPR, nextpnr etc) directly and then build symbiflow convenience wrapper on top of this. |
|
Superseeded by chipsalliance/f4pga#530. |
This PR is meant to solve the issue of having multiple, conflicting platform-specific sets of wrapper scripts for tools used at different stages of source -> bitstream flow.
The idea is to encapsulate the tools in new, platform-agnostic python scripts and have a file describing a
flowfor each platform. This is called aplatform flowdefinition. The file lists stages of a flow, each one associated with a certain tool and given a configuration for that tool, such as dependencies that the tool take and produces as well as some named parameters.The main script
symbiflow.pycan be then used to request any target that can be generated using a flow for a given platform.Example:
Generates net target (packed netlist) for the platform
arty_100given project configuration described inflow.jsonThis is still an early work in progress