Skip to content

Port onnx float16 from onnxconverter-common #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Port onnx float16 from onnxconverter-common #86

wants to merge 3 commits into from

Conversation

@bjeffrey92
Copy link

A few weeks ago I created an issue on the onnx library repo enquiring about the state of the onnxconverter-common library, which appears to no longer be actively maintained. @justinchuby responded to me and suggested that the module of onnxconverter-common which I require, for converting a single precision onnx model to mixed precision, could be ported into this library.

Here, I have ported the relevant functions from the float16 module from onnxconverter-common (https://github.com/microsoft/onnxconverter-common/blob/master/onnxconverter_common/float16.py) to this repo at the location suggested by @justinchuby. I also copied the relevant tests across.

Note, that this is not originally my code, hence I have retained the attribution and the original MIT license information with the source code. The modifications I made were just to:

  • strip out some functions that are not needed for my use case
  • add type hinting throughout
  • apply formatting and linting rules specified in the ir-py repo
    I have not made any material change to the logic of any of the functions in the original module in onnxconverter-common.
@bjeffrey92 bjeffrey92 requested review from a team and titaiwangms as code owners June 17, 2025 12:21
@justinchuby
Copy link
Member

Thank you - for this pass we would want to replace onnx.helper usages with relavant onnx_ir apis. Could you do that?
@justinchuby
Copy link
Member

You can start by removing the imports onnx, helper and onnx_proto.
@justinchuby
Copy link
Member

You may also refer to the rest of the files in the passes directory for examples
@bjeffrey92
Copy link
Author

@justinchuby thanks for looking at this so quickly, but can you be a bit more specific please? I don't see any functions in the other modules of the passes directory which look like they replicate the onnx.helper and onnx.numpy_helper functions that are used here (onnx.helper.make_node, onnx.helper.make_tensor_value_info, onnx.numpy_helper.to_array)
@bjeffrey92
Copy link
Author

@justinchuby and when you say that you want me to remove the onnx_proto import, do you mean just import the protos from the top level of the onnx library, eg onnx.ModelProto rather onnx.onnx_pb.ModelProto?

Like how it is referenced here: https://github.com/onnx/ir-py/blob/main/src/onnx_ir/passes/common/shape_inference.py#L22
@justinchuby
Copy link
Member

Sorry I forgot to mention: the shape inference and checker passes are the only two not to reference because they need to directly use onnx apis. You many see https://github.com/onnx/ir-py/blob/main/src/onnx_ir/passes/common/constant_manipulation.py as an example.

The essential idea is this: onnx_ir provides a complete set of apis to manipulate an onnx graph so you don’t need to work directly with protobuf. Therefore any import from the onnx package in a graph transformation pass is not needed. I.e. onnx.numpy_helper, onnx.helper etc. that will directly generate protobuf objects should not be used.
@justinchuby
Copy link
Member

You may find the api documentation here: https://onnx.ai/ir-py/api/index.html as well as an ai generated version: https://deepwiki.com/onnx/ir-py
@justinchuby
Copy link
Member

Specifically you may use:

onnx.helper.make_node: ir.node()
onnx.helper.make_tensor_value_info: value.shape=..., value.dtype=...
onnx.numpy_helper.to_array: ir.tensor()
@bjeffrey92
ported module for converting onnx model to mixed pres from onnxconver…
63e08cc 
…ter-common Signed-off-by: bjeffrey92 <36240394+bjeffrey92@users.noreply.github.com>
@bjeffrey92
abide by all linting rules
9567865 
Signed-off-by: bjeffrey92 <36240394+bjeffrey92@users.noreply.github.com>
@bjeffrey92
add MIT license information to source code
365554f 
Signed-off-by: bjeffrey92 <36240394+bjeffrey92@users.noreply.github.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 20, 2025
View reviewed changes
src/onnx_ir/passes/common/onnx_float_16.py

import numpy as np
import numpy.typing as npt
import onnx

Check warning

Code scanning / lintrunner

RUFF/TID251

`onnx` is banned: Use onnx_ir methods and classes instead, or create an exception with `# noqa: TID251` if you need to use onnx directly. See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
import numpy.typing as npt
import onnx
import packaging.version as pv
from onnx import helper, numpy_helper

Check warning

Code scanning / lintrunner

RUFF/TID251

`onnx` is banned: Use onnx_ir methods and classes instead, or create an exception with `# noqa: TID251` if you need to use onnx directly. See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
import numpy.typing as npt
import onnx
import packaging.version as pv
from onnx import helper, numpy_helper

Check warning

Code scanning / lintrunner

RUFF/TID251

`onnx.helper` is banned: onnx helpers tend to be protobuf-y and slow. Consider using ir.tensor, ir.DataType and related methods instead. See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
import numpy.typing as npt
import onnx
import packaging.version as pv
from onnx import helper, numpy_helper

Check warning

Code scanning / lintrunner

RUFF/TID251

`onnx.numpy_helper` is banned: onnx numpy helpers tend to be slow. Consider using ir.tensor, ir.DataType and related methods instead. See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
import onnx
import packaging.version as pv
from onnx import helper, numpy_helper
from onnx import onnx_pb as onnx_proto

Check warning

Code scanning / lintrunner

RUFF/TID251

`onnx` is banned: Use onnx_ir methods and classes instead, or create an exception with `# noqa: TID251` if you need to use onnx directly. See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
np_array = np.where(
between(float("-inf"), np_array, -max_finite_val), -max_finite_val, np_array
)
return np.float16(np_array) # pyright: ignore[reportReturnType]

Check failure

Code scanning / lintrunner

MYPY/return-value

Incompatible return value type (got "floating[_16Bit]", expected "ndarray[tuple[Any, ...], dtype[floating[_16Bit]]]") To disable, use ` # type: ignore[return-value]`
src/onnx_ir/passes/common/onnx_float_16.py
func_infer_shape = None
if not disable_shape_infer and pv.Version(onnx.__version__) >= pv.Version("1.2"): # pyright: ignore[reportPrivateImportUsage]
try:
from onnx.shape_inference import infer_shapes

Check warning

Code scanning / lintrunner

RUFF/TID251

`onnx` is banned: Use onnx_ir methods and classes instead, or create an exception with `# noqa: TID251` if you need to use onnx directly. See https://docs.astral.sh/ruff/rules/banned-api
@codecov
Copy link

codecov bot commented Jun 23, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 7.69231% with 192 lines in your changes missing coverage. Please review.

Project coverage is 71.66%. Comparing base (e211825) to head (365554f).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/onnx_ir/passes/common/onnx_float_16.py 7.69% 192 Missing ⚠️
Additional details and impacted files 
@@ Coverage Diff @@ ## main #86 +/- ## ========================================== - Coverage 74.52% 71.66% -2.87%  ========================================== Files 38 39 +1 Lines 4687 4895 +208 Branches 957 1017 +60 ========================================== + Hits 3493 3508 +15  - Misses 841 1033 +192  - Partials 353 354 +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 23, 2025
View reviewed changes
src/onnx_ir/passes/common/onnx_float_16.py

import numpy as np
import numpy.typing as npt
import onnx

Check warning

Code scanning / lintrunner

RUFF/TID251 Warning

onnx is banned: Use onnx_ir methods and classes instead, or create an exception with # noqa: TID251 if you need to use onnx directly.
See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
import numpy.typing as npt
import onnx
import packaging.version as pv
from onnx import helper, numpy_helper

Check warning

Code scanning / lintrunner

RUFF/TID251 Warning

onnx is banned: Use onnx_ir methods and classes instead, or create an exception with # noqa: TID251 if you need to use onnx directly.
See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
import numpy.typing as npt
import onnx
import packaging.version as pv
from onnx import helper, numpy_helper

Check warning

Code scanning / lintrunner

RUFF/TID251 Warning

onnx.helper is banned: onnx helpers tend to be protobuf-y and slow. Consider using ir.tensor, ir.DataType and related methods instead.
See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
import numpy.typing as npt
import onnx
import packaging.version as pv
from onnx import helper, numpy_helper

Check warning

Code scanning / lintrunner

RUFF/TID251 Warning

onnx.numpy\_helper is banned: onnx numpy helpers tend to be slow. Consider using ir.tensor, ir.DataType and related methods instead.
See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
import onnx
import packaging.version as pv
from onnx import helper, numpy_helper
from onnx import onnx_pb as onnx_proto

Check warning

Code scanning / lintrunner

RUFF/TID251 Warning

onnx is banned: Use onnx_ir methods and classes instead, or create an exception with # noqa: TID251 if you need to use onnx directly.
See https://docs.astral.sh/ruff/rules/banned-api
src/onnx_ir/passes/common/onnx_float_16.py
np_array = np.where(
between(float("-inf"), np_array, -max_finite_val), -max_finite_val, np_array
)
return np.float16(np_array) # pyright: ignore[reportReturnType]

Check failure

Code scanning / lintrunner

MYPY/return-value Error

Incompatible return value type (got "floating[_16Bit]", expected "ndarray[tuple[Any, ...], dtype[floating[_16Bit]]]") To disable, use # type: ignore[return-value]
src/onnx_ir/passes/common/onnx_float_16.py
func_infer_shape = None
if not disable_shape_infer and pv.Version(onnx.__version__) >= pv.Version("1.2"): # pyright: ignore[reportPrivateImportUsage]
try:
from onnx.shape_inference import infer_shapes

Check warning

Code scanning / lintrunner

RUFF/TID251 Warning

onnx is banned: Use onnx_ir methods and classes instead, or create an exception with # noqa: TID251 if you need to use onnx directly.
See https://docs.astral.sh/ruff/rules/banned-api
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants

@bjeffrey92 @justinchuby