add android-ndk r21d #3004

a4z · 2020-09-25T14:15:27Z

Specify library name and version: android-ndk r21d

I've read the guidelines for contributing.
I've followed the PEP8 style guides for Python code in the recipes.
I've used the latest Conan client version.
I've tried at least one configuration locally with the
conan-center hook activated.

This is a mixture of what has been at bincrafters, and changes @jgsogo has in his branch, but make it working (for me)

There are some questions, like , and option for cmake (android can build today with system cmake or any other)
But for a base of discussion, I add this now
Then, together with #1137, everything for doing mobile packaging basically exists, (except some docs on how to use profiles to do this, but I am working on that)

Note, this only works with the new 2 profile (build / host) syntax

conan-center-bot · 2020-09-25T14:16:14Z

Failure in build 1 (7b086cb5322306cd897c7bebc5ffed039475ea9b):

android-ndk/r21d
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONAN CENTER INDEX URL (KB-H027)] The attribute 'url' should point to: https://github.com/conan-io/conan-center-index (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H027)
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONANDATA.YML FORMAT (KB-H030)] Additional entry ['Windows'] not allowed in 'sources':'r21d' of conandata.yml (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H030)
  - [HOOK - conan-center.py] pre_export(): ERROR: [NO FINAL ENDLINE (KB-H041)] File '/home/conan/w/cci_PR-3004/recipes/android-ndk/all/../config.yml' does not end with an endline (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H041)

conan-center-bot · 2020-09-25T14:20:30Z

Failure in build 2 (435c295464e631fa2bb5690fb55317840a131612):

android-ndk/r21d
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONANDATA.YML FORMAT (KB-H030)] Additional entry ['Windows'] not allowed in 'sources':'r21d' of conandata.yml (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H030)
  - [HOOK - conan-center.py] pre_export(): ERROR: [NO FINAL ENDLINE (KB-H041)] File '/home/conan/w/cci_PR-3004/recipes/android-ndk/all/../config.yml' does not end with an endline (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H041)

madebr · 2020-09-25T22:18:35Z

recipes/android-ndk/all/conanfile.py

+    def source(self):
+        tarballs = self.conan_data["sources"][self.version]
+        tools.get(**tarballs[str(self.settings.os)])
+        extracted_dir = self.name + "-" + self.version
+        os.rename(extracted_dir, self._source_subfolder)


I think the downloading of the binaries should move to build,
because the following will fail:

conan create android-ndk/r21d@ -s os=Windows conan create android-ndk/r21d@ -s os=Linux -ks

I do not understand
If you want to download this for a operating system you are not on, this package will not make any sense

This would make it possible for a devops guy to create android-ndk packages on his Linux machine for all platforms.
This is possible now too, but there is a possibility that, by executing the commands in my previous comment, a wrong package is generated.

This is actually a good point I never thought about because I have all 3 OSes just available,
but I sometimes found it annoying to change the computer
So I will investigate in this, thanks!

madebr · 2020-09-25T22:22:27Z

recipes/android-ndk/all/conanfile.py

+        if os.name == 'posix':
+            os.chmod(filename, os.stat(filename).st_mode | 0o111)
+
+    def package_info(self):


Should this package have a toolchain method?
See https://docs.conan.io/en/latest/creating_packages/toolchains/cmake.html

needs further investigation since I never used that methode, but a good point!

also, there is actually a toolchain file included in, /build/cmake/android.toolchain.cmake
need to check if this could simplify the cmake wrapper

I'm also interested in knowing if this package will be compatible with the new toolchain features.
For instance here https://docs.conan.io/en/latest/integrations/cross_platform/android.html#use-built-in-conan-toolchain they use a profile:host with android_ndk_installer/r20@bincrafters/stable and conan_toolchain.cmake gets filled with all the information.

Is the current package a drop-in replacement for the bincrafters recipe?

@Hopobcn , this NDK package makes much more, it allows also building packages that do not use CMake, like openssl or curl

Why creating an own toolchain if you can get one from the NDK, that is for sure the most complete one?
If you create your toolchain file, it should just include that one from the NDK anyway

It is not a drop in replacement, because it works with the (not so) new 2 profile approach, the bincrafter one used the one profile way with os_buid and os_arch
The future is the 2 profile way.

recipes/android-ndk/all/conanfile.py

recipes/android-ndk/config.yml

Co-authored-by: Anonymous Maarten <[email protected]>

conan-center-bot · 2020-09-26T18:40:36Z

Failure in build 3 (f78f67f165a6471e10b81c2e4ad6c6c87f9ebacd):

android-ndk/r21d
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONANDATA.YML FORMAT (KB-H030)] Additional entry ['Windows'] not allowed in 'sources':'r21d' of conandata.yml (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H030)
  - [HOOK - conan-center.py] pre_export(): ERROR: [NO FINAL ENDLINE (KB-H041)] File '/home/conan/w/cci_PR-3004/recipes/android-ndk/all/../config.yml' does not end with an endline (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H041)

Co-authored-by: Anonymous Maarten <[email protected]>

conan-center-bot · 2020-09-26T18:43:54Z

Failure in build 4 (5b7170713647dcd34bfbf90119c54f073108b3da):

android-ndk/r21d
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONANDATA.YML FORMAT (KB-H030)] Additional entry ['Windows'] not allowed in 'sources':'r21d' of conandata.yml (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H030)

Croydon · 2020-09-27T03:43:45Z

Bincrafters repo: https://github.com/bincrafters/conan-android_ndk_installer

At Bincrafters the Docker image bincrafters/docker-centos-gcc48 was used for building this recipe for Linux. Was there a specific reason for it? Or asked differently: If it works with our regular Docker images, which all use newer compiler versions and operating systems, why do we have a special image only for this single recipe?

a4z · 2020-09-27T19:40:46Z

I do not think that there is any image to use, since this package is 'just' re-packaging files and providing some setup for consumption. The only dependencyis to repack the right package for the platform.
I also have used the original bincrafter recipe that uses os_build and os_arch on all 3 platforms to build some packages for android and I can not remember on any problems, so no idea why there should be a specialised image, maybe due to the size of the package (nearly 1 GB download / 1 GB package size, think its about 850 mb)

recipes/android-ndk/all/conanfile.py

conan-center-bot · 2020-10-06T17:39:46Z

Failure in build 5 (ba854f40638769cece130291fde55951fbc43bb8):

android-ndk/r21d
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONANDATA.YML FORMAT (KB-H030)] Additional entry ['Windows'] not allowed in 'sources':'r21d' of conandata.yml (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H030)

recipes/android-ndk/all/conanfile.py

stale · 2020-12-01T13:05:09Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

a4z · 2020-12-01T13:42:52Z

I am against that!

dmn-star · 2020-12-08T11:46:52Z

Please make android-ndk public available on CCI. :-)

SSE4 · 2020-12-11T08:36:46Z

hello @a4z , do you need any help with this PR to make it green again, so we can review, approve and merge?
please tell us, if you need any help with testing, debugging, etc. I have Android set up on several machines, as well as real devices to test on, so feel free to ask any questions.

dmn-star · 2021-01-03T11:13:11Z

I don't think I really understand the logic behind cci bot. :-) Why is the pull request still marked as blocked? All blocked issues are merged now. Why do you have to manually retrigger the bot by closing/reopening the pull request? Why is the pull still not merged after 14 hours?
🤨

prince-chrismc · 2021-01-03T19:11:59Z

@dmn-star

Why is the pull request still marked as blocked?

Human error!

Why do you have to manually retrigger the bot by closing/reopening the pull request?

CCI is still very young, sometimes the things go wrong... ideal it should automatically build ever commit and only faily due to human error... that's not the case yet

Why is the pull still not merged after 14 hours?

I takes one Conan team member and two community reviews to merge the PR. #2857 has more details

recipes/android-ndk/all/conanfile.py

dmn-star · 2021-01-03T20:11:14Z

@prince-chrismc I work in highly regulated industry but the 3 whitelisted reviewers that is really strong. :-) Anyway good to know, I will not waste my time with reviews anymore.

PS: we should push https://github.community/t/cross-reference-issues-with-blocks-and-blocked-by/1437 to reduce human errors.

SSE4 · 2021-01-04T04:40:25Z

this is still EAP (consider it more or less a beta), so please don't expect much stability right away. given the fact CI is complicated and depends on several external resources (e.g. github repositories, docker registries, cloud running agents, bintray etc.) any instability in these external resources may cause an unexpected error even if CI itself is super stable. but CI itself is still under heavy development, we still have to figure out how to do things right (e.g. rebuild downstream packages that become outdated after new builds of the upstream), so don't expect a coherent behavior from the CI itself or its bots out of the box.

a4z · 2021-01-04T07:57:13Z

... I will not waste my time with reviews anymore.

A review is not just about giving an OK, @dmn-star , but also to find places of improvements in PRs, or confirm it looks good.
So even if your review does not count for the merge bot, it is not a waste of time and appreciated if a review is seriously done.

dmn-star · 2021-01-04T10:26:28Z

@SSE4
Understand. Maybe you can improve the cci-bot so that it automatically rebuilds the PRs after an "unexpected error" is fixed (here is still lots of "unexpected error" labels n PR list now)

A review is not just about giving an OK

@a4z In many cases it is. Simple bugfixes, trivial recipe changes, new library versions and so on. But here is not the right place for the feedback.

I have cloned new android-ndk and get an error (on macOS):
conan create recipes/android-ndk/all android-ndk/r21d@ --build

android-ndk/r21d (test package): Running test()
/bin/sh: ndk-build: command not found
ERROR: android-ndk/r21d (test package): Error in test() method, line 13
self.run("ndk-build --version", run_environment=True)
ConanException: Error 127 while executing DYLD_LIBRARY_PATH="" DYLD_FRAMEWORK_PATH="" ndk-build --version

a4z · 2021-01-04T11:18:16Z

It runs on my OSX, and on the cci builds,
so chances are good there is something wrong on you machine,
but without more info there is not help.
(this seems als not to be the best place for help)

but check if your whole output looks something like that one
https://c3i.jfrog.io/c3i/misc/logs/pr/3004/12/android-ndk/r21d/46f53f156846659bf39ad6675fa0ee8156e859fe/create_stdout.txt

and if not, check where it is different that could cause that problem

dmn-star · 2021-01-04T12:00:13Z

After I called conan create ... second time sucess. 👍 Strange, I think I should check my env.

prince-chrismc · 2021-01-04T15:05:33Z

I will not waste my time with reviews anymore.

We need more reviewers! Please do not back off, I didn't mean to be discouraging!

Those of us on that short list were/are actively reviewing before that existed, so being active in the community is the fast track to be being added yourself.

❤️ join us!

dmn-star · 2021-01-05T11:54:26Z

I will try my best. :-)

chgans · 2021-05-28T08:27:25Z

Is there official documentation on how to use this?
The one from here https://docs.conan.io/en/latest/systems_cross_building/cross_building.html#linux-windows-macos-to-android doesn't mention how to use this new android-ndk package.

a4z · 2021-05-28T08:32:37Z

you use 2 profiles, profile:build and profile:host

p:b is your current machine, p:h the android target, eg armv8
in p:h you have the ndk as build requirement, that's it

chgans · 2021-05-28T09:28:17Z

I've reported #5666, and answer was the same: Use 2 profiles.
Is there any documentation somewhere?
I'm new to conan and would like to learn how to use it for deps management on Android.
I would like to contribute too, as some of my deps are not available.

chgans · 2021-05-28T10:19:12Z

This doc (https://docs.conan.io/en/latest/integrations/cross_platform/android.html#using-docker-images) says that when using docker images "it should just work".
So I created this host profile based on the doc:

conan@c832a96ac876:/home/chgans/Projects/neon/conan-get-started$ conan profile show host
Configuration for profile host:

[settings]
os=Android
os.api_level=21
compiler=clang
compiler.version=8
compiler.libcxx=libc++
arch=armv8
[options]
[build_requires]
[env]
conan@c832a96ac876:/home/chgans/Projects/neon/conan-get-started$ conan profile show default
Configuration for profile default:

[settings]
os=Android
os_build=Linux
arch=armv8
arch_build=x86_64
compiler=clang
compiler.version=8
compiler.libcxx=libc++
build_type=Release
os.api_level=21
[options]
[build_requires]
[env]

the default came with the container.

It has unblocked me for some package, but now libbacktrace fails, it's a deps for boot:

libbacktrace/cci.20210118: Calling:
 > source_subfolder/configure '--disable-shared' '--enable-static' '--prefix=/home/conan/.conan/data/libbacktrace/cci.20210118/_/_/package/6396ba2f98098bdc12034cbfd81e345010d25f43' '--bindir=${prefix}/bin' '--sbindir=${prefix}/bin' '--libexecdir=${prefix}/bin' '--libdir=${prefix}/lib' '--includedir=${prefix}/include' '--oldincludedir=${prefix}/include' '--datarootdir=${prefix}/share' 
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for gcc... /android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... configure: error: in `/home/conan/.conan/data/libbacktrace/cci.20210118/_/_/build/6396ba2f98098bdc12034cbfd81e345010d25f43':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
libbacktrace/cci.20210118: 
libbacktrace/cci.20210118: ERROR: Package '6396ba2f98098bdc12034cbfd81e345010d25f43' build failed
libbacktrace/cci.20210118: WARN: Build folder /home/conan/.conan/data/libbacktrace/cci.20210118/_/_/build/6396ba2f98098bdc12034cbfd81e345010d25f43
ERROR: libbacktrace/cci.20210118: Error in build() method, line 85
        autotools = self._configure_autotools()
while calling '_configure_autotools', line 78
        self._autotools.configure(configure_dir=self._source_subfolder, args=args)
        ConanException: Error 1 while executing source_subfolder/configure '--disable-shared' '--enable-static' '--prefix=/home/conan/.conan/data/libbacktrace/cci.20210118/_/_/package/6396ba2f98098bdc12034cbfd81e345010d25f43' '--bindir=${prefix}/bin' '--sbindir=${prefix}/bin' '--libexecdir=${prefix}/bin' '--libdir=${prefix}/lib' '--includedir=${prefix}/include' '--oldincludedir=${prefix}/include' '--datarootdir=${prefix}/share'

I wonder if the default profile is for the host, and build profile should have native compiler. It's not clear from the doc

a4z · 2021-05-28T10:50:52Z

In the context of this cci package, the docs provide nothing, but they mention the 2 profile way
https://docs.conan.io/en/latest/systems_cross_building/cross_building.html?highlight=build%20host#conan-v1-24-and-newer

everything else you find will use the old way, using 1 profile, and you can forget about that

your host profile looks kind of ok,
but misses this ndk package as build requirement

the build/default one should be your machine you build one, so that is wrong since it says its Android/arm
just create a new one, conan profile new --detect --force default and use that one as your --profile:build

chgans · 2021-05-28T10:57:01Z

Alright, looks like that was it:

conan@c832a96ac876:/home/chgans/Projects/neon/conan-get-started$ conan profile show build
Configuration for profile build:

[settings]
os=Linux
compiler=clang
compiler.version=8
compiler.libcxx=libc++
arch=x86_64
[options]
[build_requires]
[env]
conan@c832a96ac876:/home/chgans/Projects/neon/conan-get-started$ conan profile show default
Configuration for profile default:

[settings]
os=Android
os_build=Linux
arch=armv8
arch_build=x86_64
compiler=clang
compiler.version=8
compiler.libcxx=libc++
build_type=Release
os.api_level=21
[options]
[build_requires]
[env]

I got lot of pre-built packages, and was able to build others.
Have only issue with libtorrent (#5666) and some weird problem with libusb (#5675)

Thanks for the link, it made me realised that i was looking at some old documentation. 👍

chgans · 2021-05-28T10:59:11Z

BTW, i started learning Conan this morning! 🤣

a4z · 2021-05-28T11:36:07Z

in your host profile, what you seem to have as default, add

[build_requires]
android-ndk/r21d

so this package will be picked up and gets used to compile for Android

add android-ndk

7b086cb

conan-center-bot added the Failed label Sep 25, 2020

point url to cci

435c295

madebr reviewed Sep 25, 2020

View reviewed changes

recipes/android-ndk/all/conanfile.py Outdated Show resolved Hide resolved

madebr reviewed Sep 25, 2020

View reviewed changes

recipes/android-ndk/all/conanfile.py Show resolved Hide resolved

madebr reviewed Sep 25, 2020

View reviewed changes

recipes/android-ndk/config.yml Outdated Show resolved Hide resolved

speeling in comment

f78f67f

Co-authored-by: Anonymous Maarten <[email protected]>

remove trailing spaces

5b71707

Co-authored-by: Anonymous Maarten <[email protected]>

a4z mentioned this pull request Oct 6, 2020

[package] libcurl/7.71.1: build on Android started to fail with on of the last (few) commits #3121

Closed

a4z commented Oct 6, 2020

View reviewed changes

recipes/android-ndk/all/conanfile.py Outdated Show resolved Hide resolved

improve (fix) exit condition in package info

ba854f4

madebr reviewed Oct 6, 2020

View reviewed changes

recipes/android-ndk/all/conanfile.py Show resolved Hide resolved

a4z mentioned this pull request Oct 13, 2020

[bug] build requirement packages should have the possibility to influence the package id conan-io/conan#7876

Closed

a4z mentioned this pull request Nov 11, 2020

[bug] cross compilation with build requirements is total broken conan-io/conan#7566

Closed

jgsogo mentioned this pull request Nov 24, 2020

[package] xmlsec/1.2.30: Android build failed #3627

Closed

stale bot added the stale label Dec 1, 2020

stale bot removed the stale label Dec 1, 2020

madebr approved these changes Jan 2, 2021

View reviewed changes

SSE4 requested a review from uilianries January 2, 2021 12:17

SSE4 approved these changes Jan 2, 2021

View reviewed changes

dmn-star approved these changes Jan 2, 2021

View reviewed changes

SSE4 removed the blocked Affected by an external issue and waiting until it is solved label Jan 3, 2021

github-actions bot mentioned this pull request Jan 3, 2021

Pull Requests Ready to be Reviewed prince-chrismc/conan-center-index-pending-review#1

Open

prince-chrismc reviewed Jan 3, 2021

View reviewed changes

recipes/android-ndk/all/conanfile.py Show resolved Hide resolved

danimtb approved these changes Jan 4, 2021

View reviewed changes

conan-center-bot merged commit 4893a6f into conan-io:master Jan 4, 2021

Croydon mentioned this pull request Jan 4, 2021

Android NDK Installer conan-io/wishlist#111

Closed

add android-ndk r21d #3004

add android-ndk r21d #3004

Conversation

a4z commented Sep 25, 2020

conan-center-bot commented Sep 25, 2020

conan-center-bot commented Sep 25, 2020

madebr Sep 25, 2020 • edited Loading

Choose a reason for hiding this comment

a4z Sep 26, 2020

Choose a reason for hiding this comment

madebr Sep 26, 2020

Choose a reason for hiding this comment

a4z Sep 26, 2020

Choose a reason for hiding this comment

madebr Sep 25, 2020 • edited Loading

Choose a reason for hiding this comment

a4z Sep 26, 2020

Choose a reason for hiding this comment

Hopobcn Jan 6, 2021

Choose a reason for hiding this comment

a4z Jan 6, 2021

Choose a reason for hiding this comment

conan-center-bot commented Sep 26, 2020

conan-center-bot commented Sep 26, 2020

Croydon commented Sep 27, 2020

a4z commented Sep 27, 2020

conan-center-bot commented Oct 6, 2020

stale bot commented Dec 1, 2020

a4z commented Dec 1, 2020

dmn-star commented Dec 8, 2020 • edited Loading

SSE4 commented Dec 11, 2020

dmn-star commented Jan 3, 2021

prince-chrismc commented Jan 3, 2021 • edited Loading

dmn-star commented Jan 3, 2021

SSE4 commented Jan 4, 2021

a4z commented Jan 4, 2021

dmn-star commented Jan 4, 2021 • edited Loading

a4z commented Jan 4, 2021

dmn-star commented Jan 4, 2021

prince-chrismc commented Jan 4, 2021

dmn-star commented Jan 5, 2021

chgans commented May 28, 2021

a4z commented May 28, 2021

chgans commented May 28, 2021

chgans commented May 28, 2021

a4z commented May 28, 2021

chgans commented May 28, 2021

chgans commented May 28, 2021

a4z commented May 28, 2021

madebr Sep 25, 2020 •

edited

Loading

madebr Sep 25, 2020 •

edited

Loading

dmn-star commented Dec 8, 2020 •

edited

Loading

prince-chrismc commented Jan 3, 2021 •

edited

Loading

dmn-star commented Jan 4, 2021 •

edited

Loading