Build fails: Cython.Compiler.Errors.InternalError: Internal compiler error: '_common.pxi' not found

[yurivict]

===>  Building for py311-ta-lib-0.5.0
cd /usr/ports/finance/py-ta-lib/work-py311/ta_lib-0.5.0 && /usr/bin/env -i HOME=/usr/ports/finance/py-ta-lib/work-py311  MACHINE_ARCH=amd64  PWD="${PWD}"  __MAKE_CONF=/nonexistent OSVERSION=1401500 PATH=/usr/local/libexec/ccache:/usr/ports/finance/py-ta-lib/work-py311/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin TERM=xterm-256color XDG_DATA_HOME=/usr/ports/finance/py-ta-lib/work-py311  XDG_CONFIG_HOME=/usr/ports/finance/py-ta-lib/work-py311  XDG_CACHE_HOME=/usr/ports/finance/py-ta-lib/work-py311/.cache  HOME=/usr/ports/finance/py-ta-lib/work-py311 PATH=/usr/local/libexec/ccache:/usr/ports/finance/py-ta-lib/work-py311/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin PKG_CONFIG_LIBDIR=/usr/ports/finance/py-ta-lib/work-py311/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing  " CCACHE_DIR="/tmp/.ccache" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" /usr/local/bin/python3.11 -m build --no-isolation --wheel 
* Getting build dependencies for wheel...
* Building wheel...
running bdist_wheel
running build
running build_py
running egg_info
writing ta_lib.egg-info/PKG-INFO
writing dependency_links to ta_lib.egg-info/dependency_links.txt
writing requirements to ta_lib.egg-info/requires.txt
writing top-level names to ta_lib.egg-info/top_level.txt
reading manifest file 'ta_lib.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
adding license file 'AUTHORS'
writing manifest file 'ta_lib.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
    main()
  File "/usr/local/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
    return _build_backend().build_wheel(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/setuptools/build_meta.py", line 244, in build_wheel
    return self._build_with_temp_dir(['bdist_wheel'], '.whl',
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/setuptools/build_meta.py", line 229, in _build_with_temp_dir
    self.run_setup()
  File "/usr/local/lib/python3.11/site-packages/setuptools/build_meta.py", line 282, in run_setup
    self).run_setup(setup_script=setup_script)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/setuptools/build_meta.py", line 174, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 144, in <module>
    setup(
  File "/usr/local/lib/python3.11/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 177, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
    dist.run_commands()
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.11/site-packages/wheel/_bdist_wheel.py", line 387, in run
    self.run_command("build")
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.11/site-packages/setuptools/command/build.py", line 24, in run
    super().run()
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 131, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "setup.py", line 122, in build_extensions
    super(build_ext, self).build_extensions()
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 459, in build_extensions
    self._build_extensions_serial()
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 485, in _build_extensions_serial
    self.build_extension(ext)
  File "/usr/local/lib/python3.11/site-packages/Cython/Distutils/build_ext.py", line 130, in build_extension
    new_ext = cythonize(
              ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1010, in cythonize
    module_list, module_metadata = create_extension_list(
                                   ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 859, in create_extension_list
    kwds = deps.distutils_info(file, aliases, base).values
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 707, in distutils_info
    return (self.transitive_merge(filename, self.distutils_info0, DistutilsInfo.merge)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 716, in transitive_merge
    return self.transitive_merge_helper(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 722, in transitive_merge_helper
    deps = extract(node)
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 687, in distutils_info0
    cimports, externs, incdirs = self.cimports_externs_incdirs(filename)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Cython/Utils.py", line 129, in Cython.Utils.cached_method.wrapper
  File "/usr/local/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 574, in cimports_externs_incdirs
    for include in self.included_files(filename):
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Cython/Utils.py", line 129, in Cython.Utils.cached_method.wrapper
  File "/usr/local/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 556, in included_files
    include_path = self.context.find_include_file(include, source_file_path=filename)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/Cython/Compiler/Main.py", line 299, in find_include_file
    error(pos, "'%s' not found" % filename)
  File "/usr/local/lib/python3.11/site-packages/Cython/Compiler/Errors.py", line 178, in error
    raise InternalError(message)
Cython.Compiler.Errors.InternalError: Internal compiler error: '_common.pxi' not found

ERROR Backend subprocess exited when trying to invoke build_wheel
*** Error code 1

Version: 0.5.0
Python-3.11

[mrjbq7]

Dangit, looks like 0.5.0 didn't include the .pxi if you want to use your own cython to make the C file.

I made this patch 327a8ff and released 0.5.1, can you try again?

[yurivict]

0.5.1 fails like this:

In file included from talib/_ta_lib.c:1235:
In file included from /usr/local/lib/python3.11/site-packages/numpy/core/include/numpy/arrayobject.h:5:
In file included from /usr/local/lib/python3.11/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
In file included from /usr/local/lib/python3.11/site-packages/numpy/core/include/numpy/ndarraytypes.h:1929:
/usr/local/lib/python3.11/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with "          "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^
talib/_ta_lib.c:10083:13: warning: call to undeclared function 'PyDataType_ELSIZE'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 10083 |   __pyx_r = PyDataType_ELSIZE(__pyx_v_self);
       |             ^
talib/_ta_lib.c:10096:13: warning: call to undeclared function 'PyDataType_ALIGNMENT'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 10096 |   __pyx_r = PyDataType_ALIGNMENT(__pyx_v_self);
       |             ^
talib/_ta_lib.c:10113:15: warning: call to undeclared function 'PyDataType_FIELDS'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 10113 |   __pyx_t_1 = PyDataType_FIELDS(__pyx_v_self);
       |               ^
talib/_ta_lib.c:10113:13: error: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
 10113 |   __pyx_t_1 = PyDataType_FIELDS(__pyx_v_self);
       |             ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
talib/_ta_lib.c:10134:15: warning: call to undeclared function 'PyDataType_NAMES'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 10134 |   __pyx_t_1 = PyDataType_NAMES(__pyx_v_self);
       |               ^

py311-cython3-3.0.11 is used.

[mrjbq7]

Is that a failure? Looks like a warning.

If you are using numpy 2, then the 0.5 release is for you. (latest 0.5.1)

If you are using numpy 1, then the 0.4 release is for you. (latest 0.4.34)

[yurivict]

error: incompatible integer to pointer conversion assigning to 'PyObject *'

[mrjbq7]

I think PyDataType_ELSIZE is part of the numpy 2 ?

[mrjbq7]

https://numpy.org/devdocs/numpy_2_0_migration_guide.html#migration-c-descr

#if NPY_ABI_VERSION < 0x02000000
  #define PyDataType_ELSIZE(descr) ((descr)->elsize)
#endif

You probably want the ta-lib<=0.5 releases.

[yurivict]

py311-numpy-1.26.4_2 is used.

Perhaps we shouldn't upgrade the port until numpy-2 is available.

[mrjbq7]

I was resisting upgrading to numpy2 because we use cython to generate the C files and there didn't seem a clean way to make them work for both numpy1 and numpy2, so the approach someone recommended was to have an 0.4 series that supports numpy1 and and 0.5 series that supports numpy2.

I couldn't get the npy_2_compat.h approach to work cleanly...

[sanurielf]

Same issue here conda-forge/ta-lib-feedstock#49 But only for OSx builds. I will pin numpy to >= 2 on this conda builds and retry.

[mrjbq7]

Are you saying the “_common.pxi” issue is happening on 0.5.1 or the numpy issue? If numpy, you can choose to support numpy 1 with ta-lib 0.4 branch, or numpy 2 with the 0.5 branch.

[sanurielf]

Yep, I just merged version 0.5.1 (for conda package) with the pinned version of numpy >= 2 < 3. All builds are ok. https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1083545&view=results