Update to NDK 27 #20935

licy183 · 2024-07-22T14:30:50Z

Closes #20927

I'll update the flang toolchain this weekend.

licy183 · 2024-07-22T15:32:02Z

Emmm... Seems out of space on GitHub Action...

licy183 · 2024-07-22T16:28:52Z

Should wait for actions/runner-images#10314, otherwise CI will fail on the packages in big-pkg.list.

truboxl · 2024-07-23T03:05:53Z

Can -fopenmp -static-openmp be dropped?

finagolfin · 2024-07-23T05:02:32Z

Can -fopenmp -static-openmp be dropped?

I think some packages require it, so not dropped completely, but if you want to remove it from this central location, see which packages fail to build or run, then add it for just those, that's fine.

truboxl · 2024-07-23T10:53:32Z

Ok I will kick off a test build with NDK r26b with the openmp options removed...

IntinteDAO · 2024-07-23T16:33:16Z

Closes #20927

I'll update the flang toolchain this weekend.

Why can't we just update to ndk 29?

TomJo2000 · 2024-07-23T16:34:35Z

Why can't we just update to ndk 29?

I think you are confusing NDK version and API level.

IntinteDAO · 2024-07-23T17:02:27Z

Why can't we just update to ndk 29?

I think you are confusing NDK version and API level.

Possible. I just tried to build one app on Termux and it throws an error that the function appeared in Android 29. I assumed it was about NDK because it's a C++ app

TomJo2000 · 2024-07-23T17:10:41Z

Yeah you're probably talking about the API target.
See:

for additional information.

finagolfin · 2024-07-24T12:08:09Z

scripts/setup-android-sdk.sh

 	ANDROID_NDK_FILE=android-ndk-r${TERMUX_NDK_VERSION}-linux.zip
-	ANDROID_NDK_SHA256=ad73c0370f0b0a87d1671ed2fd5a9ac9acfd1eb5c43a7fbfbd330f85d19dd632
+	ANDROID_NDK_SHA256=2f17eb8bcbfdc40201c0b36e9a70826fcd2524ab7a2a235e2c71186c302da1dc
 elif [ "$TERMUX_NDK_VERSION" = 23c ]; then


Any reason this pull keeps 23c around, rather than switching to keeping 26b as the backup and 27 as the new default?

It is the last NDK to support -march=armv7-a -mfpu=vfpv3-d16

Some packages are still using it for that?

All existing arm packages are built with LDFLAGS -mfpu=neon so no. This is only kept to extend hardware support for Termux. I am pretty much the only maintainer that still uses it.

OK, what do you use 23c for, some old armv7 devices?

@licy183, would it make sense to keep 26b around also as a backup or no need?

Emmm... I think there is no need to keep NDK 26b.

Wasn't 23c useful for compiling some packages in termux-user-repository as well?

Hmm, github is removing everything older than NDK 26 from their CI images, actions/runner-images#10342.

finagolfin · 2024-07-24T12:15:52Z

Possible. I just tried to build one app on Termux and it throws an error that the function appeared in Android 29.

Android 29 is the version of the OS APIs, while NDK 27 is a new development toolchain that supports Android OS APIs 21 through 35, the latest OS API version that will be released next month.

We target Android OS API 24, partially for the reasons given in the links @TomJo2000 gave.

finagolfin · 2024-07-24T23:43:54Z

ndk-patches/27/stdio.h.patch

 #include <stdarg.h>
 #include <stddef.h>

+#if !defined(__swift__)


Needed this last commit for the upcoming Swift 6, shouldn't affect anything else.

Could you explain the reason for this change or what issues are fixed?

Sure, wasn't sure if anyone would be interested. 😄

Swift 6 adds a fully modularized android.modulemap, as opposed to the glibc.modulemap we reuse from linux in the current Swift 5.10 package. This change is primarily to help with the new C++ interoperability features of Swift, which require a modularized libc module map to go with the libc++ module.modulemap from the NDK that we currently ship.

The problem is these additional headers we patch in for Termux break that modularity, in ways I don't understand. So, I found that the Swift compiler sets __swift__ when importing these headers and undefined these when building for Swift, including the ctermid() inline function definition that needs that header.

truboxl · 2024-07-25T04:56:44Z

Ok I will kick off a test build with NDK r26b with the openmp options removed...

I have yet to see any build failure when openmp options are removed (currently up to 832 packages)

finagolfin · 2024-07-25T07:35:24Z

@Grimler91, do we need a full package rebuild after this?

Grimler91 · 2024-07-25T10:27:45Z

@finagolfin based on previous updates there should be no need. Also doesn't seem to be any large obvious changes that should affect us in changelog, but I'll try to run some builds to check for new errors affecting packages

Grimler91 · 2024-07-25T10:52:46Z

ndk-patches/27/stdlib.h.patch

 +#include <stdint.h>
+endif


Missing # here, #endif

@licy183, I didn't pull again before fixing this typo just now. Feel free to push your local branch again with my last commit added if I missed something.

finagolfin · 2024-07-25T11:04:16Z

I'm going to test my daily Swift CI for Android now with NDK 27, will add that cached NDK download step to this pull too if all goes well, so we won't have to wait on the github runner image to add NDK 27.

Grimler91 · 2024-07-25T21:06:07Z

Busybox fails to build due to missing symbols in static lib generation, so maybe several packages that internally use static libs will have similar errors:

networking/tc.c:236:20  AR      procps/lib.a
: error: use of undeclared identifier 'TCA_CBQ_MAX'
  236 |         struct rtattr *tb[TCA_CBQ_MAX+1];
      |                           ^
networking/tc.c:247:26: error: use of undeclared identifier 'TCA_CBQ_MAX'
  247 |         parse_rtattr_nested(tb, TCA_CBQ_MAX, opt);
      |                                 ^
  CC      coreutils/mktemp.o
networking/tc.c:249:9: error: use of undeclared identifier 'TCA_CBQ_RATE'
  249 |         if (tb[TCA_CBQ_RATE]) {
      |                ^
networking/tc.c:250:22: error: use of undeclared identifier 'TCA_CBQ_RATE'
  250 |                 if (RTA_PAYLOAD(tb[TCA_CBQ_RATE]) < sizeof(*r))
      |                                    ^
networking/tc.c:253:20: error: use of undeclared identifier 'TCA_CBQ_RATE'
  253 |                         r = RTA_DATA(tb[TCA_CBQ_RATE]);
      |                                         ^
networking/tc.c:255:9: error: use of undeclared identifier 'TCA_CBQ_LSSOPT'
  255 |         if (tb[TCA_CBQ_LSSOPT]) {
      |                ^
networking/tc.c:256:22: error: use of undeclared identifier 'TCA_CBQ_LSSOPT'
  256 |                 if (RTA_PAYLOAD(tb[TCA_CBQ_LSSOPT]) < sizeof(*lss))
      |                                    ^
networking/tc.c:256:47: error: invalid application of 'sizeof' to an incomplete type 'struct tc_cbq_lssopt'
  256 |                 if (RTA_PAYLOAD(tb[TCA_CBQ_LSSOPT]) < sizeof(*lss))
      |                                                             ^~~~~~
networking/tc.c:238:9: note: forward declaration of 'struct tc_cbq_lssopt'
  238 |         struct tc_cbq_lssopt *lss = NULL;
      |                ^
networking/tc.c:259:22: error: use of undeclared identifier 'TCA_CBQ_LSSOPT'
  259 |                         lss = RTA_DATA(tb[TCA_CBQ_LSSOPT]);
      |                                           ^
networking/tc.c:261:9: error: use of undeclared identifier 'TCA_CBQ_WRROPT'
  261 |         if (tb[TCA_CBQ_WRROPT]) {
      |                ^

Biswa96 · 2024-07-26T03:30:39Z

That's probably not related to NDK update, see https://www.mail-archive.com/busybox%40busybox.net/msg29286.html. There are similar issues reported about glibc or Linux kernel update (googlefu).

finagolfin · 2024-07-26T04:01:59Z

I'm seeing an issue with using CMAKE_SYSTEM_PROCESSOR with NDK 27, which doesn't affect Termux because the standalone toolchain does not pull in the NDK's CMake config. I'll ask them what's going on, but shouldn't be a stopper for us.

finagolfin · 2024-07-26T08:22:44Z

Heh, I first looked up if anyone had reported a CMAKE_SYSTEM_PROCESSOR issue with the NDK, and of course, the only one was a Termux user years ago about a different problem. 😄

Edit: I didn't read it closely enough, he's actually a non-Termux user complaining about how a change I made to CMake for Termux affects him. 😉

finagolfin · 2024-07-30T16:17:51Z

Alright, added a commit to download NDK 27 to the github CI for big packages and cache it for later, plus kicked off a build of libllvm using it.

finagolfin · 2024-07-30T16:52:46Z

Libllvm setup with the new NDK all looked fine, but failed only because I didn't update the flang version also, which is enforced to match. Kicked off a new build of zig instead, which won't take hours like libllvm.

finagolfin · 2024-07-31T06:55:15Z

Alright, Zig built fine with NDK 27. I didn't know it shipped with its own copy of the LLVM source now, so that builds fine too.

This pull now works for all packages and can be used as a base for any further CI testing wanted, before merging.

truboxl · 2024-07-31T15:18:25Z

NDK r27 fails to build current apt

Grimler91 · 2024-08-01T17:59:53Z

NDK r27 fails to build current apt

Can reproduce as well, error looks something like:

In file included from /home/builder/.termux-build/apt/src/apt-pkg/depcache.cc:12:
In file included from /home/builder/.termux-build/apt/build/include/apt-pkg/algorithms.h:32:
In file included from /home/builder/.termux-build/apt/build/include/apt-pkg/cachefilter.h:9:
In file included from /home/builder/.termux-build/apt/build/include/apt-pkg/pkgcache.h:78:
In file included from /home/builder/.termux-build/apt/build/include/apt-pkg/mmap.h:31:
In file included from /home/builder/.termux-build/_cache/android-r27-api-24-v0/sysroot/usr/include/c++/v1/string:625:
In file included from /home/builder/.termux-build/_cache/android-r27-api-24-v0/sysroot/usr/include/c++/v1/string_view:936:
In file included from /home/builder/.termux-build/_cache/android-r27-api-24-v0/sysroot/usr/include/c++/v1/algorithm:1847:
In file included from /home/builder/.termux-build/_cache/android-r27-api-24-v0/sysroot/usr/include/c++/v1/__algorithm/nth_element.h:15:
/home/builder/.termux-build/_cache/android-r27-api-24-v0/sysroot/usr/include/c++/v1/__algorithm/sort.h:319:44: error: invalid operands to binary expression ('const APT::Container_iterator<APT::VersionContainerInterface, std::vector<pkgCache::VerIterator>, APT::VersionContainer<std::vector<pkgCache::VerIterator>>>' and 'difference_type' (aka 'long'))
  319 |   for (_RandomAccessIterator __i = __first + difference_type(1); __i != __last; ++__i) {
      |                                    ~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~
/home/builder/.termux-build/_cache/android-r27-api-24-v0/sysroot/usr/include/c++/v1/__iterator/reverse_iterator.h:302:1: note: candidate template ignored: could not match 'reverse_iterator<_Iter>' against 'difference_type' (aka 'long')
  302 | operator+(typename reverse_iterator<_Iter>::difference_type __n, const reverse_iterator<_Iter>& __x) {
      | ^
/home/builder/.termux-build/_cache/android-r27-api-24-v0/sysroot/usr/include/c++/v1/__iterator/wrap_iter.h:194:1: note: candidate template ignored: could not match '__wrap_iter<_Iter1>' against 'difference_type' (aka 'long')
  194 | operator+(typename __wrap_iter<_Iter1>::difference_type __n, __wrap_iter<_Iter1> __x) _NOEXCEPT {
      | ^
[ ... ]

truboxl · 2024-08-02T14:28:47Z

Those are coming from upstream or were inserted by our Termux builds? Either way, it should be easy to remove that flag.

Termux builds. Unfortunately "too many" ncurses based packages fail to build on NDK r27. I think "all" the OpenMP options should be dropped and re-enabled per package.

The current sox package that was enabled together with toolchain wide OpenMP options in #6146 just dont expose --multi-threaded option anymore nor does $PREFIX/lib/libsox.so.3 has any OpenMP symbols. Both of these are true on Ubuntu. I am not going to check back whether is truly confirmed working back then or because of NDK upgrade stopped working since then.

Here is what I am thinking to do:

Drop all the OpenMP build options
Investigate sox and enable OpenMP properly
Write tests to check for OpenMP symbols when -fopenmp is specified, also fail build if libomp.so is added as dependency
Rebuild ncurses

However whether to do this before or after switching to NDK r27 is up for debate

finagolfin · 2024-08-03T11:56:03Z

Updated this pull to use NDK 27 from the CI image for non-Docker builds, now that it's deployed.

whether to do this before or after switching to NDK r27 is up for debate

After the switch IMO, as we should get this in soon.

finagolfin · 2024-08-07T03:13:14Z

Time to get this in? I've moved my Swift CI for Android to NDK 27 without a problem.

licy183 · 2024-08-07T15:35:46Z

Tests fine on all of my devices (Android9-MIX2s-arm, Android13-MIX4-aarch64, Android7-Bluestacks-i686, Android7-Bluestacks-x86_64). I'll merge this now.

…docker-build variable

pgaskin · 2024-08-08T14:49:03Z

/data/data/com.termux/files/usr/lib/pkgconfig/ncursesw.pc:17:Libs:  -L/data/data/com.termux/files/usr/lib -Wl,-rpath=/data/data/com.termux/files/usr/lib -fopenmp -static-openmp -fno-openmp-implicit-rpath -Wl,--enable-new-dtags -lncursesw
/data/data/com.termux/files/usr/lib/pkgconfig/curses.pc:17:Libs:  -L/data/data/com.termux/files/usr/lib -Wl,-rpath=/data/data/com.termux/files/usr/lib -fopenmp -static-openmp -fno-openmp-implicit-rpath -Wl,--enable-new-dtags -lncursesw
/data/data/com.termux/files/usr/lib/pkgconfig/ncurses.pc:17:Libs:  -L/data/data/com.termux/files/usr/lib -Wl,-rpath=/data/data/com.termux/files/usr/lib -fopenmp -static-openmp -fno-openmp-implicit-rpath -Wl,--enable-new-dtags -lncursesw

Those are coming from upstream or were inserted by our Termux builds?

They were added by the Termux build. They weren't present yesterday.

licy183 · 2024-08-09T01:58:37Z

They were added by the Termux build. They weren't present yesterday.

It will be fixed in #21038.

licy183 requested review from Grimler91 and finagolfin as code owners July 22, 2024 14:30

Grimler91 approved these changes Jul 22, 2024

View reviewed changes

licy183 force-pushed the ndk branch 2 times, most recently from e4ccc07 to 1c0b2a1 Compare July 22, 2024 16:25

licy183 force-pushed the ndk branch from 1c0b2a1 to 25561c6 Compare July 22, 2024 16:38

finagolfin reviewed Jul 24, 2024

View reviewed changes

licy183 force-pushed the ndk branch from 533acef to 5aa5ec3 Compare July 25, 2024 03:18

truboxl force-pushed the ndk branch from 5aa5ec3 to 15c0e54 Compare July 25, 2024 05:11

Grimler91 reviewed Jul 25, 2024

View reviewed changes

finagolfin force-pushed the ndk branch from 15c0e54 to bdf5073 Compare July 25, 2024 10:55

finagolfin force-pushed the ndk branch from bd6c346 to b00dbef Compare July 30, 2024 16:08

finagolfin force-pushed the ndk branch from b00dbef to 9b25e5d Compare July 30, 2024 16:49

finagolfin force-pushed the ndk branch from 9b25e5d to c5e73de Compare July 31, 2024 06:49

finagolfin force-pushed the ndk branch from c5e73de to ff4b6af Compare July 31, 2024 07:09

finagolfin force-pushed the ndk branch from ff4b6af to c1e0c68 Compare August 3, 2024 09:00

termux deleted a comment from codenamedpkt Aug 7, 2024

truboxl approved these changes Aug 7, 2024

View reviewed changes

licy183 and others added 8 commits August 7, 2024 23:35

scripts(toolchain): update to ndk-r27

c41dda9

bump(main/libandroid-stub): bump to 27

0079ef3

bump(main/libc++): bump to 27

24b3062

bump(main/vulkan-loader-android): bump to 27

af57934

bump(main/ndk-sysroot): bump to 27

71a759f

bump(main/ndk-multilib): bump to 27

b0132d6

scripts(toolchain): fix Bionic modularization for the upcoming Swift 6

0510788

ci(packages): Consolidate a handful of big-pkgs checks into a single …

8b99075

…docker-build variable

licy183 force-pushed the ndk branch from c1e0c68 to 8b99075 Compare August 7, 2024 15:35

licy183 merged commit 5e90782 into master Aug 7, 2024
5 checks passed

licy183 deleted the ndk branch August 7, 2024 16:18

pgaskin mentioned this pull request Aug 8, 2024

enhance(main/cmus): Add AAudio output plugin #21069

Merged

Update to NDK 27 #20935

Update to NDK 27 #20935

Conversation

licy183 commented Jul 22, 2024 • edited Loading

licy183 commented Jul 22, 2024

licy183 commented Jul 22, 2024 • edited Loading

truboxl commented Jul 23, 2024

finagolfin commented Jul 23, 2024

truboxl commented Jul 23, 2024

IntinteDAO commented Jul 23, 2024

TomJo2000 commented Jul 23, 2024

IntinteDAO commented Jul 23, 2024

TomJo2000 commented Jul 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

truboxl Jul 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

finagolfin commented Jul 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

truboxl commented Jul 25, 2024

finagolfin commented Jul 25, 2024

Grimler91 commented Jul 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

finagolfin commented Jul 25, 2024

Grimler91 commented Jul 25, 2024

Biswa96 commented Jul 26, 2024

finagolfin commented Jul 26, 2024

finagolfin commented Jul 26, 2024 • edited Loading

finagolfin commented Jul 30, 2024

finagolfin commented Jul 30, 2024

finagolfin commented Jul 31, 2024

truboxl commented Jul 31, 2024

Grimler91 commented Aug 1, 2024

truboxl commented Aug 2, 2024

finagolfin commented Aug 3, 2024

finagolfin commented Aug 7, 2024

licy183 commented Aug 7, 2024

pgaskin commented Aug 8, 2024

licy183 commented Aug 9, 2024

licy183 commented Jul 22, 2024 •

edited

Loading

licy183 commented Jul 22, 2024 •

edited

Loading

truboxl Jul 24, 2024 •

edited

Loading

finagolfin commented Jul 26, 2024 •

edited

Loading