Pre-trained RNN-T .h5 to tflite Conversion Issue

[ksettaluri6]

Hello,

I am trying to take this pre-trained RNN-T model saved as h5 and convert it to tflite.

I have installed the required packages according to the requirements.txt, and have tried unsuccessfully to convert to tflite using various TensorFlow versions (for example tf 2.5 has tensorflow-text=2.5 and tensorflow-io=0.18). Specifically:

1. Using any tf versions >2.5 leads to a Failed to functionalize Control Flow V1 ops. Consider using Control Flow V2 ops instead. error. Though there are some resources online on how to resolve this, they suggest running those commands at the initial saving of the trained model. No solutions I found worked here, including tf.enable_control_flow_v2(). I specifically tried tf 2.5, tf 2.8, tf 2.9 and tf-nightly.
2. Some posted issues on the github mention using tf-nightly to get it to work, however tf-nightly does not save any versions < 2.9.0. I tried a tf-nightly version (tf 2.10) but the tensorflow-text module seems to only support up to tf2.9 - so it downgrades tf every time I try to install. Some solutions appeared to have used tf-nightly ~2.4. Looks like there are no nightly versions that date back that far.
3. tf2.4: it appears that others had a unique error that occurred here as well, specifically tensorflow.python.framework.errors_impl.InvalidArgumentError: Attempting to add a duplicate function with name: __inference_standard_lstm_11544 where the previous and current definitions differ. Previous definiton: signature. I've tried suggestions on the issue tracker say to go to nightly (doesn't work see 3), or tf 2.3.X.
4. tf 2.3.0 and tf 2.3.X: lead to the same error - tensorflow.lite.python.convert.ConverterError: input resource[0] expected type resource != float, the type of streaming_transducer_greedy_while_streaming_transducer_decoder_streaming_transducer_prediction_embedding_embedding_lookup_11637_0[0] , again solutions say to go to tf-nightly or tf2.4.

Is there a specific tensorflow, tensorflow-text, and tensorflow-io that can be used to load the pre-trained RNN-T h5 model? Or could you add tf.enable_control_flow_v2() when saving the h5?

[liuyibox]

Any updated solution here? I encountered the same issue here with tf2.9.

[liuyibox]

This bug has been resolved here. Basically, we just need to load model while are are loading the concrete function, i.e., change this line to converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func], model).

[yiqiaoc11]

I find another issue while deploy the tflite of 21h5 as below:
2022-11-26 00:51:46.362790: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Listening...
INFO: Created TensorFlow Lite delegate for select TF ops.
INFO: TfLiteFlexDelegate delegate: 10 nodes delegated out of 468 nodes with 2 partitions.

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Traceback (most recent call last):
File "pyaudio/main_.py", line 67, in listen_callback
interpreter.invoke()
File "C:\Users\vanil\anaconda3\lib\site-packages\tensorflow\lite\python\interpreter.py", line 917, in invoke
self.interpreter.Invoke()
RuntimeError: tensorflow/lite/kernels/range.cc:45 (start >= limit && delta < 0) || (start <= limit && delta > 0) was not true.Node number 454 (RANGE) failed to invoke.
Traceback (most recent call last):
File "pyaudio/main.py", line 251, in
main()
File "pyaudio/main_.py", line 100, in main
time.sleep(.5)
RuntimeError

//////////////////////////code ////////////////////////////////////

    interpreter.allocate_tensors()
    interpreter.set_tensor(input_details[0]['index'], pred_init)
    interpreter.set_tensor(input_details[1]['index'], tf.constant(0))
    interpreter.set_tensor(input_details[2]['index'], enc_init)
    interpreter.set_tensor(input_details[3]['index'], tf.constant([1.0]))

    interpreter.invoke()