Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update QONNX parsing for 1.0 #979

Merged
merged 86 commits into from
Oct 31, 2024
Merged

Conversation

jmitrevs
Copy link
Contributor

@jmitrevs jmitrevs commented Mar 12, 2024

Description

This change updates the ONNX parser and adds support for QONNX. It replaces PR #832. It only supports ONNX that has been cleaned by the qonnx package, including converting convolutions to be channels-last and changing Gemm to MatMul and Add.

In QONNX Quant nodes can act on constants as well as the datapath. To make handling this easier, we explicitly put constants in the initial graph. There are also some helper nodes like MatMul and Conv that are introduced to support the explicit constant nodes. After the convert flow, no special ONNX nodes remain in the graph, though.

Generally Quant nodes that have power-of-2 scales and no zero-offset get converted to fixed data types either by setting the types of constants or adding a linear activation that is usually merged into preceding nodes. Non-power-of-2 scales result in ApplyAlpha nodes beings added to scale and unscale, with propagation across some layers. This can be further optimized and has generally been tested less.

This includes the changes from PR #855 with a few updates that will be backported and discussed there. Therefore, this PR needs to wait till that PR is merged, which is why I am making it draft.

Note: for the config_from_onnx_model I made the default granularity be "name" because that enables automatic precision inference, which you need for QONNX. The way that I did that is that I set config['Model']['Precision'] to the default (e.g. fixed<16,6>), but all the precisions filled by config['Model'] are auto. These can be overriden if, for example, the accumulator becomes too wide. In general, though, they are set by the infer_precision.py optimizer.

Binary networks are not yet supported.

More information can be found in this presentation:
https://www.icloud.com/keynote/025yxvgBx8IF2m3Iso6HosqPw#QONNX_Ingestion_0p1

Type of change

  • New feature (non-breaking change which adds functionality)
  • A new research paper code implementation

Tests

The pytest, test_qonnx.py, is the main test, building some models from the QONNX model zoo

Checklist

  • I have read the guidelines for contributing.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.
  • I have installed and run pre-commit on the files I edited or added.
  • I have added tests that prove my fix is effective or that my feature works.

@jmitrevs jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Sep 29, 2024
@jmitrevs jmitrevs marked this pull request as ready for review September 29, 2024 21:35
@jmitrevs
Copy link
Contributor Author

The sep_conv bug is fixed with #1069.

@jmitrevs jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Oct 1, 2024
@jmitrevs jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Oct 2, 2024
@jmitrevs
Copy link
Contributor Author

I added more tests from example models. Note--one is skipped because I think the qonnx channels-last conversion doesn't take into account enough activations, with fastmachinelearning/qonnx#148 opened for it.

@jmitrevs jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Oct 23, 2024
@jmitrevs jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Oct 24, 2024
@jmitrevs jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Oct 24, 2024
@JanFSchulte JanFSchulte added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Oct 24, 2024
@vloncar vloncar merged commit 2c17f66 into fastmachinelearning:main Oct 31, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
please test Trigger testing by creating local PR branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants