Selectively build TFLite for iOS failure

[gaikwadrahul8]

Click to expand!

Issue Type

Build/Install

Have you reproduced the bug with TF nightly?

Yes

Source

source

Tensorflow Version

2.12.0

Custom Code

No

OS Platform and Distribution

MacOS Ventura 13.1

Mobile device

N/A

Python version

3.9.16

Bazel version

5.3

GCC/Compiler version

14.0.0

CUDA/cuDNN version

N/A

GPU model and memory

N/A

Current Behaviour?

This was a subsequence bug from another issue(https://github.com/tensorflow/tensorflow/issues/59853#issuecomment-1481642687) and @yishuangP asked me to file a separate issue for Tensorflow Tools.

I'm trying to build a tf framework from certain tflite model to reduce framework size. However I keep seeing below error. The command I'm using is `bash tensorflow/lite/ios/build_frameworks.sh --input_models=tflite_model_vocab_changes_run_18026681_full_dataset.tflite --target_archs=x86_64,armv7,arm64`


ERROR: /Users/ghe/projects/tensorflow/tensorflow/BUILD:1128:21: declared output 'tensorflow/libtensorflow_framework.2.dylib' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /Users/ghe/projects/tensorflow/tensorflow/BUILD:1128:21: Executing genrule //tensorflow:libtensorflow_framework.2.dylib_sym [for host] failed: not all outputs were created or valid
realpath: illegal option -- -
usage: realpath [-q] [path ...]
Target //tensorflow/lite/ios/tmp:TensorFlowLiteSelectTfOps_framework failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1046.736s, Critical Path: 138.53s
INFO: 3339 processes: 25 internal, 3314 local.
FAILED: Build did NOT complete successfully

### Standalone code to reproduce the issue

```shell
There is no code involved. To reproduce:

1. git fetch a new project
2. Copy the toy model I provided into its root directory
3. Run `bash tensorflow/lite/ios/build_frameworks.sh --input_models=tflite_model_vocab_changes_run_18026681_full_dataset.tflite --target_archs=x86_64,armv7,arm64` command

It will compile for a couple minutes, and fail finally.

Relevant log output

$ bash tensorflow/lite/ios/build_frameworks.sh --input_models=tflite_model_vocab_changes_run_18026681_full_dataset.tflite --target_archs=x86_64,armv7,arm64
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=314
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/Applications/Xcode.app/Contents/Developer/usr/bin/python3 --action_env PYTHON_LIB_PATH= /Library/Python/3.9/site-packages --python_path=/Applications/Xcode.app/Contents/Developer/usr/bin/python3
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/ghe/projects/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/ghe/projects/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:ios in file /Users/ghe/projects/tensorflow/.bazelrc: --apple_platform_type=ios --apple_bitcode=embedded --copt=-fembed-bitcode --copt=-Wno-c++11-narrowing --noenable_platform_specific_config --copt=-w --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --define=with_xla_support=false
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/55dd04f6bcf797b4ff20e74158377bcc912b9870.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
INFO: Analyzed target //tensorflow/lite/ios/tmp:TensorFlowLiteC_framework (144 packages loaded, 7602 targets configured).
INFO: Found 1 target...
Target //tensorflow/lite/ios/tmp:TensorFlowLiteC_framework up-to-date:
  bazel-out/applebin_ios-ios_x86_64-opt-ST-74bbdaf492e7/bin/tensorflow/lite/ios/tmp/TensorFlowLiteC_framework.zip
INFO: Elapsed time: 38.614s, Critical Path: 5.29s
INFO: 13 processes: 2 internal, 11 local.
INFO: Build completed successfully, 13 total actions
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=314
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/Applications/Xcode.app/Contents/Developer/usr/bin/python3 --action_env PYTHON_LIB_PATH= /Library/Python/3.9/site-packages --python_path=/Applications/Xcode.app/Contents/Developer/usr/bin/python3
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/ghe/projects/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/ghe/projects/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:monolithic in file /Users/ghe/projects/tensorflow/.bazelrc: --define framework_shared_object=false --define tsl_protobuf_header_only=false --experimental_link_static_libraries_once=false
INFO: Found applicable config definition build:macos in file /Users/ghe/projects/tensorflow/.bazelrc: --apple_platform_type=macos --copt=-DGRPC_BAZEL_BUILD --copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/55dd04f6bcf797b4ff20e74158377bcc912b9870.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
INFO: Build options --apple_bitcode, --apple_platform_type, --copt, and 1 more have changed, discarding analysis cache.
INFO: Analyzed target //tensorflow/lite/tools:list_flex_ops_no_kernel_main (7 packages loaded, 1231 targets configured).
INFO: Found 1 target...
Target //tensorflow/lite/tools:list_flex_ops_no_kernel_main up-to-date:
  bazel-bin/tensorflow/lite/tools/list_flex_ops_no_kernel_main
INFO: Elapsed time: 0.292s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
~/projects/tensorflow/tensorflow/lite/ios/tmp ~/projects/tensorflow
~/projects/tensorflow
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=314
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/Applications/Xcode.app/Contents/Developer/usr/bin/python3 --action_env PYTHON_LIB_PATH= /Library/Python/3.9/site-packages --python_path=/Applications/Xcode.app/Contents/Developer/usr/bin/python3
INFO: Reading rc options for 'build' from /Users/ghe/projects/tensorflow/.bazelrc:
  'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/ghe/projects/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/ghe/projects/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:ios in file /Users/ghe/projects/tensorflow/.bazelrc: --apple_platform_type=ios --apple_bitcode=embedded --copt=-fembed-bitcode --copt=-Wno-c++11-narrowing --noenable_platform_specific_config --copt=-w --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --define=with_xla_support=false
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/55dd04f6bcf797b4ff20e74158377bcc912b9870.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
INFO: Build options --apple_bitcode, --apple_platform_type, --copt, and 2 more have changed, discarding analysis cache.
INFO: Analyzed target //tensorflow/lite/ios/tmp:TensorFlowLiteSelectTfOps_framework (435 packages loaded, 36544 targets configured).
INFO: Found 1 target...
ERROR: /Users/ghe/projects/tensorflow/tensorflow/BUILD:1128:21: declared output 'tensorflow/libtensorflow_framework.2.dylib' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /Users/ghe/projects/tensorflow/tensorflow/BUILD:1128:21: Executing genrule //tensorflow:libtensorflow_framework.2.dylib_sym [for host] failed: not all outputs were created or valid
realpath: illegal option -- -
usage: realpath [-q] [path ...]
Target //tensorflow/lite/ios/tmp:TensorFlowLiteSelectTfOps_framework failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1046.736s, Critical Path: 138.53s
INFO: 3339 processes: 25 internal, 3314 local.
FAILED: Build did NOT complete successfully

[gaikwadrahul8]

This issue originally reported by @heguanyu has been moved to this dedicated repository for LiteRT to enhance issue tracking and prioritization. To ensure continuity, we have created this new issue on your behalf.

We appreciate your understanding and look forward to your continued involvement.

[pkgoogle]

Original Issue: tensorflow/tensorflow#60088