Conversion of trained model(.npz) to tflite

[maketo97]

@syoyo @sukhodolin @luomai @lgarithm @Yuyue Hi, I have ran the script of the export_tflite.py but meet the issue of 'BatchDataset' object is not callable. May I get your suggestion to solve this issue? Thanks.

2022-04-14 12:03:01.103757: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
C:\Users\tonyh\anaconda3\lib\site-packages\numpy_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\tonyh\anaconda3\lib\site-packages\numpy.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\Users\tonyh\anaconda3\lib\site-packages\numpy.libs\libopenblas.NOIJJG62EMASZI6NYURL6JBKM4EVBGM7.gfortran-win_amd64.dll
warnings.warn("loaded more than 1 DLL from .libs:"
2022-04-14 12:03:03.919437: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2022-04-14 12:03:03.920201: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2022-04-14 12:03:03.936544: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s
2022-04-14 12:03:03.936735: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2022-04-14 12:03:03.941809: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2022-04-14 12:03:03.949328: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2022-04-14 12:03:03.955103: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2022-04-14 12:03:03.957090: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2022-04-14 12:03:03.962445: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2022-04-14 12:03:03.965118: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2022-04-14 12:03:03.965988: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2022-04-14 12:03:03.966194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
[INFO]: Configuration initialized!
[MODEL] INFO: Setting Resnet18_backbone!
[MODEL] INFO: Enable model backbone pretraining:False
2022-04-14 12:03:04.587777: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-04-14 12:03:04.588737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s
2022-04-14 12:03:04.590697: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2022-04-14 12:03:04.592058: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2022-04-14 12:03:04.593235: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2022-04-14 12:03:04.594663: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2022-04-14 12:03:04.596044: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2022-04-14 12:03:04.597845: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2022-04-14 12:03:04.599085: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2022-04-14 12:03:04.600392: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2022-04-14 12:03:04.601523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2022-04-14 12:03:05.265789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-04-14 12:03:05.265974: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2022-04-14 12:03:05.267897: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2022-04-14 12:03:05.269206: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:02:00.0, compute capability: 8.6)
2022-04-14 12:03:05.272014: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
[MODEL] INFO: Using LightweightOpenpose model arch!

[INFO]: Welcome to Hyperpose Development Platform!

====================================================================================================
[INFO]: Variable Definition:
[INFO]: parts: the joints of human body, Enum class
[INFO]: limbs: the limbs of human body, List of tuple. example: [(joint index 1, joint index 2),...]
[INFO]: colors: the visualize color for each parts, List. example: [(0,0,255),...] (optional)
[INFO]: n_parts: number of human body joints, int. example: n_parts=len(parts)
[INFO]: n_limbs: number of human body limbs, int. example: n_limbs=len(limbs)
[INFO]: hin: height of the model input image, int. example: 368
[INFO]: win: width of the model input image, int. example: 368
[INFO]: hout: height of model output heatmap, int. example: 46
[INFO]: wout: wout of model output heatmap, int. example: 46

====================================================================================================
[INFO]: Object Definition:
[INFO]: config: a object contains all the configurations used to assemble the model, dataset, and pipeline. easydict object.
return by the Config.get_config function.

[INFO]: model: a neural network takes in the image and output the calculated activation map. BasicModel object.
have forward, cal_loss, infer(optional) functions.
custom: users could inherit the Model.BasicModel class for customization.
example: please refer to Model.LightWeightOpenPose class for details.

[INFO]: dataset: a dataset generator provides train and evaluate dataset. Base_dataset object.
have get_train_dataset and get_eval_dataset functions.
custom: users could inherit the Dataset.BasicDataset class for customizationn
example: please refer to Datset.CocoDataset class for details.

[INFO]: augmentor: a data augmentor that takes in the image, key point annotation, mask and perform affine transformation for data augmentation. BasicAumentor object.
have process and process_only_image functions.
custom: users could inherit the Model.BasicAugmentor class for customization.
example: please refer to Model.BasicAugmentor class for details.

[INFO]: preprocessor: a data preprocessor that takes in the image, key point annotation and mask to produce the target heatmap
for model to calculate loss and learn. BasicPreProcessor object.
have process function.
custom: users could inherit the Model.BasicPreProcessor class for customizationn
example: please refer to Model.openpose.PreProcessor class for details.

[INFO]: postprocessor: a data postprocessor that takes in the predicted heatmaps and infer the human body joints and limbs.
have process function. BasicPostProcessor object.
custom: users could inherit the Model.BasicPostProcessor class for customization
example: please refer to the Model.openpose.PostProcessor class for details.

[INFO]: visualizer: a visualizer that takes in the predicted heatmaps and output visualization images for train and evaluation.
have visualize and visualize_comapre functions. BasicVisualizer object.
custom: users could inherit the Model.BasicVisualizer class for customization.
example: please refer to the Model.openpose.Visualizer class for details.

====================================================================================================
[INFO]: Development platform basic usage:
1.Use the sets APIs of Config module to configure the pipeline, choose the algorithm type, the neural network
backbone, the dataset etc. that best fit your application scenario.
2.Use the get_model API of Model module to get the configured model, use get_dataset API of dataset module to
get the configured dataset, use the get_train API of Model module to get the configured train procedure. Then start
training! Check the loss values and sample training result images during training.
3.Use the get_eval API of Model module to get the configured evaluation procedure. evaluate the model you trained.
4.Eport the model to .pb, .onnx, .tflite formats for deployment.
[INFO]: Development platform custom usage:
Hyperpose enables users to custom model, dataset, augmentor, preprocessor, postprocessor and visualizer.
Users could inherit the corresponding basic class(mentioned above), and implement corresponding the member functions
required according to the function annotation, then use Config.set_custom_xxx APIs to set the custom component.
[INFO]: Additional features:
1.Parallel distributed training with Kungfu.
2.Domain adaption to leverage unlabled data.
3.Neural network backbone pretraining.
[INFO]: Currently all the procedures are uniformed to be channels_first data format.
[INFO]: Currently all model weights are saved in npz_dict format.

====================================================================================================
[DATA] INFO: Using MSCOCO dataset!
[DATA] INFO: Using MSCOCO dataset version:2017
[DATA] INFO: generating official evaluating data...
loading annotations into memory...
Done (t=0.22s)
creating index...
index created!
Overall get 5000 valid pose images from ./data\mscoco2017\val2017 and ./data\mscoco2017\annotations\person_keypoints_val2017.json
[DATA] INFO: total 5000 official evaluating data generated!
[DATA] INFO: filtering non-exist images and targets
[DATA] INFO: filtering finished! total 5000 images and targets left, 0 invalid found.
[DATA] INFO: eval dataset generation finished!
test rep_dataset:<BatchDataset shapes: , types: tf.float32>
2022-04-14 12:03:10.341282: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1
2022-04-14 12:03:10.341886: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2022-04-14 12:03:10.345371: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s
2022-04-14 12:03:10.345519: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2022-04-14 12:03:10.346483: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2022-04-14 12:03:10.347721: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2022-04-14 12:03:10.348919: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2022-04-14 12:03:10.350544: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2022-04-14 12:03:10.352452: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2022-04-14 12:03:10.353376: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2022-04-14 12:03:10.354696: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2022-04-14 12:03:10.357520: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2022-04-14 12:03:10.358586: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-04-14 12:03:10.359677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2022-04-14 12:03:10.360801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2022-04-14 12:03:10.361998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:02:00.0, compute capability: 8.6)
2022-04-14 12:03:10.363133: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2022-04-14 12:03:10.644173: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:928] Optimization results for grappler item: graph_to_optimize
function_optimizer: Graph size after: 1511 nodes (1303), 3016 edges (2807), time = 51.623ms.
function_optimizer: function_optimizer did nothing. time = 1.284ms.

2022-04-14 12:03:14.000137: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:316] Ignored output_format.
2022-04-14 12:03:14.000269: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:319] Ignored drop_control_dependency.
2022-04-14 12:03:14.125133: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s
2022-04-14 12:03:14.125440: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2022-04-14 12:03:14.128525: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2022-04-14 12:03:14.129714: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2022-04-14 12:03:14.130943: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2022-04-14 12:03:14.132017: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2022-04-14 12:03:14.133044: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2022-04-14 12:03:14.134062: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2022-04-14 12:03:14.135088: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2022-04-14 12:03:14.136167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2022-04-14 12:03:14.137155: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-04-14 12:03:14.138268: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2022-04-14 12:03:14.139320: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2022-04-14 12:03:14.140509: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:02:00.0, compute capability: 8.6)
2022-04-14 12:03:14.141507: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
INFO: Created TensorFlow Lite delegate for select TF ops.
2022-04-14 12:03:16.153224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s
2022-04-14 12:03:16.154270: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2022-04-14 12:03:16.156074: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2022-04-14 12:03:16.157709: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2022-04-14 12:03:16.158826: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2022-04-14 12:03:16.159961: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2022-04-14 12:03:16.161424: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2022-04-14 12:03:16.162684: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2022-04-14 12:03:16.163717: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2022-04-14 12:03:16.164830: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2022-04-14 12:03:16.165863: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-04-14 12:03:16.166728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2022-04-14 12:03:16.167719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2022-04-14 12:03:16.167826: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:02:00.0, compute capability: 8.6)
2022-04-14 12:03:16.168712: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
INFO: TfLiteFlexDelegate delegate: 1 nodes delegated out of 192 nodes with 1 partitions.

Traceback (most recent call last):
File "export_tflite.py", line 38, in
tflite_model_quant = converter.convert()
File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 1117, in convert
return super(TFLiteConverterV2, self).convert()
File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 942, in convert
return super(TFLiteFrozenGraphConverterV2,
File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 632, in convert
result = self._calibrate_quantize_model(result, **flags)
File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 459, in _calibrate_quantize_model
return calibrate_quantize.calibrate_and_quantize(
File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\optimize\calibrator.py", line 93, in calibrate_and_quantize
for sample in dataset_gen():
TypeError: 'BatchDataset' object is not callable